8.2 Exploratory Multidimensional IRT#
The mirt package allows us to fit all dichotomous and polytomous IRT models
presented above in a multidimensional exploratory or confirmatory fashion. The
main function is mirt(). Through the itemtype argument, a particular IRT model
can be specified. In this section, we focus on exploratory specifications, that is,
each item is free to load on each factor.
Fit the models#
Let us illustrate an exploratory multidimensional 2-PL model fit using all binary ZAREKI addition and subtraction items described in Sect. 4.2.1. First, we compute an exploratory Princals in order to get an idea of the structure and dimensionality.
We load the dataset zareki and inspect it. Then, extract the first 16 columns, name this subset itzareki. Next, fit the model using the princals function, name it przar.
ro.r('data("zareki")')
# Convert to Python
zareki = pandas2ri.rpy2py(ro.globalenv['zareki'])
# Eliminate first item (misfit)
itzareki = zareki.loc[:, zareki.columns[:16]]
print(itzareki.head())
# Put data into R
ro.globalenv['itzareki'] = itzareki
# Fit the model
ro.r("przar <- princals(itzareki)")
addit1 addit2 addit3 addit4 addit5 addit6 addit7 addit8 subtr1 \
1 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0
2 1.0 1.0 1.0 1.0 0.0 1.0 1.0 0.0 1.0
3 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
4 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
5 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
subtr2 subtr3 subtr4 subtr5 subtr6 subtr7 subtr8
1 1.0 1.0 1.0 1.0 1.0 0.0 1.0
2 1.0 1.0 1.0 1.0 1.0 0.0 1.0
3 1.0 0.0 1.0 1.0 0.0 0.0 1.0
4 1.0 0.0 1.0 1.0 0.0 1.0 1.0
5 1.0 0.0 1.0 1.0 0.0 0.0 1.0
%%R
plot(przar)
%%R
plot(przar, "screeplot")
The scree plot suggests that two dimensions should be
sufficient. From the loadings plot in the left panel, we see that the dimensions are
not addition vs. subtraction, as we might have expected.
Let us fit two models: a unidimensional 2-PL and a two-dimensional 2-PL.
Note that we use a 2-PL model as the data is dichotomous. If we would want to apply the mirt() function to a polytomous dataset we could for example use a graded response model by setting the itemtype argument to 'graded' (see below).
Fit one 1D model and one 2D model using the mirt function and the extracted subset. Note the we want to fit 2-PL model, therefore we should set itemtype = "2PL Remember the syntax? If not, check the hint! Name your models zar1d and zar2d.
ro.r('zar1d <- mirt(itzareki, 1, itemtype = "2PL")')
ro.r('zar2d <- mirt(itzareki, 2, itemtype = "2PL")')
Iteration: 20, Log-Lik: -2252.281, Max-Change: 0.00008
Iteration: 133, Log-Lik: -2235.476, Max-Change: 0.00010
Compare and investigate the models#
Use the anova function to compare the two models.
print(ro.r('anova(zar1d, zar2d)'))
AIC SABIC HQ BIC logLik X2 df p
zar1d 4568.562 4589.671 4617.415 4691.182 -2252.281
zar2d 4564.952 4595.957 4636.706 4745.051 -2235.476 33.61 15 0.004
For further fit examination of the 1D and the 2D model, let us compute the M2 statistic including the CFA/SEM fit indices.
print(ro.r("M2(zar1d)"))
print(ro.r("M2(zar2d)"))
M2 df p RMSEA RMSEA_5 RMSEA_95 SRMSR
stats 129.675 104 0.04483401 0.02694632 0.004455123 0.04072114 0.05232611
TLI CFI
stats 0.9757999 0.9790266
M2 df p RMSEA RMSEA_5 RMSEA_95 SRMSR TLI
stats 94.11394 89 0.3350678 0.013 0 0.03274862 0.04382372 0.9943675
CFI
stats 0.9958225
The output of the 1D model suggest slight misfit (significant M2 p-value) and some criteria (e.g. TLI and CFI) suggest a worse fit of the 1D model compared to the 2D model.
For the 2D model, we get a low RMSEA and a high CFI, and the M2 p-value is not significant. The 2D model fits well. Note that the 1D model does not provide a bad fit but we can assume that the 2D model provides a slightly better fit, therefore we continue using the 2D model.
Itemfit statistics can be computed using the corresponding function in mirt.
Let us still search for potentially bad fitting items. For that, lets first print out itemfit statistics using mirt::itemfit.
The function only uses the model object as an input. Remember, we want to investigate the 2D model (zar2d). Assign the output to ifit2D2pl.
ro.r("ifit2D2pl <- mirt::itemfit(zar2d)")
# Convert to Python
ifit2D2pl = pandas2ri.rpy2py(ro.globalenv['ifit2D2pl'])
ifit2D2pl.head()
| item | S_X2 | df.S_X2 | RMSEA.S_X2 | p.S_X2 | |
|---|---|---|---|---|---|
| 1 | addit1 | 1.588226 | 6.0 | 0.000000 | 0.953420 |
| 2 | addit2 | 15.986000 | 8.0 | 0.054185 | 0.042581 |
| 3 | addit3 | 7.655657 | 7.0 | 0.016598 | 0.363934 |
| 4 | addit4 | 5.685148 | 7.0 | 0.000000 | 0.576955 |
| 5 | addit5 | 6.581252 | 7.0 | 0.000000 | 0.473740 |
As we don’t want to go through all items manually, lets do this in a clever way. Look at the output from the head function, notice the p.S_X2? Lets use this p-value (column 4 of ifit2D2pl) as a cut-off. If an item has a significant (p < .05) S_X2 value we should take a closer look at it. Think about how to you can print the items (rows) of the ifit2D2pl object which have a p-value (column 5) smaller than .05.
misfitting_items = ifit2D2pl[ifit2D2pl.iloc[:, 4] < 0.05]
print(misfitting_items)
item S_X2 df.S_X2 RMSEA.S_X2 p.S_X2
2 addit2 15.986000 8.0 0.054185 0.042581
12 subtr4 20.871019 8.0 0.068789 0.007498
The subtr4 item and the subtr2 could be eliminated since they show some misfit, and then the model needs to be refitted again. However, since the global model fit suggested a well-fitting solution, let us keep them.
The factor analytic parameterization can be obtained through the summary call. We can also request to apply an orthogonal or non-orthogonal rotation for better interpretability.
print(ro.r("summary(zar2d, rotate = 'varimax')"))
print(ro.r("summary(zar2d, rotate = 'oblimin')"))
Rotation: varimax
Rotated factor loadings:
F1 F2 h2
addit1 0.4128 0.2238 0.220
addit2 0.0114 0.5467 0.299
addit3 0.2524 0.7828 0.677
addit4 0.4719 0.4551 0.430
addit5 0.5499 0.4741 0.527
addit6 0.2206 0.6760 0.506
addit7 0.2945 0.3930 0.241
addit8 0.1309 0.7139 0.527
subtr1 0.5203 0.1779 0.302
subtr2 0.9047 0.0903 0.827
subtr3 0.4172 0.4279 0.357
subtr4 0.4412 0.3204 0.297
subtr5 0.2362 0.2261 0.107
subtr6 0.5291 0.3940 0.435
subtr7 0.5490 0.4833 0.535
subtr8 0.5142 0.4090 0.432
Rotated SS loadings: 3.271 3.448
Factor correlations:
F1 F2
F1 1
F2 0 1
None
Rotation: oblimin
Rotated factor loadings:
F1 F2 h2
addit1 0.3784 0.1509 0.220
addit2 -0.1638 0.6057 0.299
addit3 0.0232 0.8112 0.677
addit4 0.3684 0.3943 0.430
addit5 0.4474 0.3969 0.527
addit6 0.0229 0.6998 0.506
addit7 0.1948 0.3673 0.241
addit8 -0.0872 0.7634 0.527
subtr1 0.5106 0.0742 0.302
subtr2 0.9584 -0.1147 0.827
subtr3 0.3174 0.3770 0.357
subtr4 0.3783 0.2517 0.297
subtr5 0.1849 0.1955 0.107
subtr6 0.4505 0.3127 0.435
subtr7 0.4435 0.4073 0.535
subtr8 0.4294 0.3328 0.432
Rotated SS loadings: 2.594 3.205
Factor correlations:
F1 F2
F1 1.000
F2 0.477 1
None
Plot results#
The mirt package provides several plotting options for two-dimensional models. Lets plot the Item characteristic
surface (ICS) for the 3rd item in 2D. For that, use the itemplot function. It takes the model (zar2d), the item (3) and a list of rotation coordinates as inputs. The latter is already specified, you only need to complete the remaining 2 arguments. Note: main = "ICS addit3" sets the title of the plot.
%%R
itemplot(zar2d, 3, main = "ICS addit3", rot = list(xaxis = -70, yaxis = 50, zaxis = 10))
Compute person scores, one for each dimension#
Finally, we can compute the person parameters, one for each dimension (only first six persons shown here).
ro.r("fscores(zar2d)")
array([[ 0.31629087, 0.08593608],
[-0.13045086, -0.26424651],
[-0.31739614, 0.07663827],
[ 0.12578337, 0.45310387],
[-0.31739614, 0.07663827],
[-0.43184699, -0.89689762],
[ 0.07289139, 0.35176114],
[-1.23422604, -2.06142719],
[-1.48519979, -0.09390775],
[-2.20962536, -2.5090477 ],
[ 0.10344465, 0.06438149],
[ 0.55527341, 0.85361562],
[-0.76972592, -1.50475692],
[-0.4801597 , -0.51822168],
[ 0.72721598, 0.52640777],
[-1.2854217 , -1.59494078],
[-1.67145428, -0.71656631],
[-1.79202968, -1.27932313],
[ 0.56545172, 0.76491233],
[-2.49302421, -2.77576318],
[-0.69890551, -1.05061595],
[-0.21921476, -1.28373925],
[-1.63795304, -2.83945932],
[-0.92519626, -1.78262542],
[-1.68028061, -1.08794529],
[-2.09102447, -2.0604506 ],
[-0.30693942, -1.12662113],
[-0.1059822 , 0.11204094],
[-0.33548998, -0.33346133],
[-0.24373398, -0.86165832],
[-1.07831403, -1.43365507],
[-0.83126225, -1.113698 ],
[-0.48453543, -0.96212669],
[-0.38830706, -1.6593653 ],
[-0.69344402, -1.70025597],
[ 0.9174705 , 1.11044424],
[ 0.50900022, -0.16574271],
[ 0.9174705 , 1.11044424],
[-1.06422707, 0.35957528],
[ 0.9174705 , 1.11044424],
[-0.46637761, -1.07607498],
[-0.95197992, -0.77033224],
[-0.05260196, -0.13968572],
[-0.17957017, -0.65667029],
[-0.58603181, -0.95332006],
[-0.3476692 , -0.42992366],
[-0.08581317, -0.18269061],
[-2.2115383 , -2.24248433],
[-0.47073835, -0.54961539],
[-1.03267395, -1.45757591],
[ 0.07289139, 0.35176114],
[-1.01231809, -1.61741005],
[-2.84185615, -2.73793199],
[-1.28213343, -1.23402229],
[-1.48519979, -0.09390775],
[-1.41578829, -0.48481619],
[-0.1249879 , -0.84853256],
[-2.83884494, -2.0422829 ],
[-2.04189706, -2.39343061],
[-1.53062957, -2.22649691],
[ 0.9174705 , 1.11044424],
[ 0.31848336, 0.09603713],
[-0.46769188, -0.48512361],
[-1.51204401, -2.63734787],
[-0.43184699, -0.89689762],
[ 0.2290386 , 0.00475948],
[-0.81563303, -1.18248914],
[ 0.27969167, -0.28546417],
[-1.02944083, -1.02153823],
[-0.90186335, -0.78959596],
[-1.1582214 , -2.06851941],
[-0.2731009 , -1.31251902],
[-2.25732548, -1.65028726],
[-2.91191983, -2.71528996],
[-0.48474388, -0.12442353],
[ 0.9174705 , 1.11044424],
[-0.68090233, -0.68359398],
[-0.03372734, -0.9612118 ],
[ 0.65815109, -0.03957003],
[-0.46471432, -0.59687325],
[-0.40933088, 0.0739482 ],
[-0.51193207, -1.79059538],
[-0.31739614, 0.07663827],
[ 0.49175192, 0.18200285],
[ 0.72721598, 0.52640777],
[ 0.31848336, 0.09603713],
[ 0.81693192, 0.48753743],
[-0.29173265, -0.36012477],
[ 0.14087079, 0.44338171],
[-0.75584311, -1.71895867],
[-0.43880482, -0.3165532 ],
[-0.66106387, -0.51233311],
[-2.23029461, -1.55565258],
[-1.19323044, -0.21282941],
[ 0.9174705 , 1.11044424],
[ 0.03795838, -0.14467632],
[ 0.02682711, -0.18847165],
[-1.89210637, -0.94685356],
[-0.29102782, 0.23845545],
[-1.47493342, -0.10367837],
[-0.50764168, -1.02667006],
[-0.03372734, -0.9612118 ],
[-0.98273796, -1.81078753],
[-1.7741221 , -1.2169966 ],
[-2.3825909 , -2.16608327],
[-2.45067451, -1.65299843],
[-1.45832419, -2.24650401],
[-0.76395735, -1.59926112],
[-0.73311149, -0.32604547],
[-2.00032567, -1.45579337],
[ 0.35915409, 0.49892787],
[-0.31558659, 0.08649215],
[ 0.19087937, -0.39228382],
[-0.22130506, 0.15890564],
[ 0.00593164, 0.08789477],
[ 0.23200277, 0.53403623],
[-0.62039684, -1.13969898],
[ 0.22832747, 0.51605585],
[-0.34921673, -0.43913142],
[-1.06216903, -1.19681617],
[ 0.35575409, -0.31540359],
[-1.12499824, -1.33705072],
[-1.90893587, -0.56130057],
[ 0.9174705 , 1.11044424],
[ 0.12578337, 0.45310387],
[-0.48151543, -1.07589332],
[ 0.56545172, 0.76491233],
[-1.79757209, -1.81286225],
[ 0.43973996, 0.76453059],
[-0.91653912, -1.32725902],
[ 0.9174705 , 1.11044424],
[ 0.18093649, 0.4109574 ],
[-1.23105041, -1.02640108],
[ 0.12578337, 0.45310387],
[ 0.12578337, 0.45310387],
[ 0.81693192, 0.48753743],
[-2.05524971, -1.5029458 ],
[ 0.81693192, 0.48753743],
[-0.30338962, 0.06707731],
[ 0.68871016, 0.82013614],
[ 0.23200277, 0.53403623],
[ 0.23373031, 0.13524887],
[ 0.49175192, 0.18200285],
[ 0.9174705 , 1.11044424],
[ 0.56545172, 0.76491233],
[ 0.7437468 , 0.95053287],
[ 0.28450157, 0.22363539],
[ 0.38031754, 0.65567252],
[ 0.40516365, 0.22096236],
[ 0.40516365, 0.22096236],
[ 0.9174705 , 1.11044424],
[ 0.45543686, 0.75465866],
[ 0.81693192, 0.48753743],
[ 0.9174705 , 1.11044424],
[ 0.22832747, 0.51605585],
[-0.57295236, -0.12596692],
[-0.17824907, 0.22845386],
[ 0.38031754, 0.65567252],
[-0.58547276, -0.11008516],
[ 0.31629087, 0.08593608],
[ 0.9174705 , 1.11044424],
[ 0.9174705 , 1.11044424],
[ 0.40701153, 0.29289162],
[-1.19742751, -1.30541474],
[-0.73106459, -0.99683979],
[ 0.9174705 , 1.11044424],
[-0.08435726, -0.65486994],
[-0.05323364, -0.07223895],
[ 0.9174705 , 1.11044424],
[-0.05974462, -0.21506342],
[ 0.65815109, -0.03957003],
[ 0.23156471, 0.12512306],
[ 0.52594762, 0.67201928],
[ 0.9174705 , 1.11044424],
[ 0.43973996, 0.76453059],
[ 0.40516365, 0.22096236],
[-0.027278 , 0.34629397],
[ 0.81693192, 0.48753743],
[-1.60256154, -0.70867637],
[ 0.0435718 , 0.28437489],
[-0.55866784, -1.46290799],
[ 0.30669717, -0.51943738],
[ 0.05652706, -0.50125517],
[ 0.21630893, -0.43329327],
[ 0.07289139, 0.35176114],
[ 0.9174705 , 1.11044424],
[-0.6501304 , 0.67137374],
[ 0.72721598, 0.52640777],
[-0.58665373, 0.42840877],
[-0.61110514, -0.61770814],
[-0.01475972, 0.39358564],
[ 0.72721598, 0.52640777],
[ 0.9174705 , 1.11044424],
[-1.47194898, -0.43502362],
[ 0.36984048, 0.18429962],
[ 0.72721598, 0.52640777],
[ 0.18359755, 0.39813733],
[ 0.7437468 , 0.95053287],
[ 0.9174705 , 1.11044424],
[ 0.59142945, 0.93253449],
[ 0.25203763, 0.69321948],
[-0.56279968, -0.99239631],
[ 0.9174705 , 1.11044424],
[-0.50764168, -1.02667006],
[-1.49173257, -0.06110069],
[ 0.21061369, -0.77575483],
[-2.02080538, -1.39741242],
[-0.55145399, -1.50131701],
[ 0.56545172, 0.76491233],
[ 0.40516365, 0.22096236],
[ 0.81693192, 0.48753743],
[ 0.21630893, -0.43329327],
[ 0.81693192, 0.48753743],
[ 0.9174705 , 1.11044424],
[ 0.9174705 , 1.11044424],
[ 0.56545172, 0.76491233],
[ 0.32501537, -0.39311614],
[-0.42941972, -0.05211917],
[-0.23396059, -0.31035179],
[-0.4271287 , -0.7445533 ],
[ 0.49175192, 0.18200285],
[-1.11128966, -0.87209925],
[-0.25377303, -0.39039731],
[-0.31739614, 0.07663827],
[-1.7451197 , -0.27792817],
[-1.68707879, -0.24902624],
[ 0.9174705 , 1.11044424],
[ 0.65649111, 0.34691109],
[-0.46567916, -0.47797599],
[ 0.49175192, 0.18200285],
[ 0.9174705 , 1.11044424],
[-0.16388518, 0.21880663],
[ 0.81693192, 0.48753743],
[ 0.9174705 , 1.11044424],
[-0.654101 , -1.11693876],
[ 0.9174705 , 1.11044424],
[-0.37489603, 0.07062787],
[-0.4860387 , 0.92540777],
[ 0.9174705 , 1.11044424],
[ 0.9174705 , 1.11044424],
[-0.09446739, -0.72204249],
[ 0.34337877, 0.05270238],
[ 0.9174705 , 1.11044424],
[ 0.31629087, 0.08593608],
[ 0.38300018, 0.66675728],
[ 0.17331921, -0.03941228],
[ 0.9174705 , 1.11044424],
[ 0.9174705 , 1.11044424],
[-0.43993925, -0.47222596],
[ 0.14087079, 0.44338171],
[ 0.75901255, 0.56735305],
[ 0.7437468 , 0.95053287],
[-0.05990198, -0.02550023],
[ 0.9174705 , 1.11044424],
[ 0.07289139, 0.35176114],
[-0.50764168, -1.02667006],
[ 0.40516365, 0.22096236],
[ 0.38031754, 0.65567252],
[ 0.18359755, 0.39813733],
[-0.84438236, -0.43578473],
[ 0.56545172, 0.76491233],
[-1.39079777, 0.01753639],
[-0.25814499, -0.37274596],
[ 0.52115017, 0.26840281],
[-1.46660201, -0.62567255],
[ 0.81693192, 0.48753743],
[-0.82858914, -0.86309381],
[ 0.56545172, 0.76491233],
[ 0.9174705 , 1.11044424],
[ 0.49175192, 0.18200285],
[ 0.9174705 , 1.11044424],
[ 0.38031754, 0.65567252],
[-1.04118259, 0.5244688 ],
[ 0.14087079, 0.44338171],
[-0.72330745, -0.18208846],
[ 0.07289139, 0.35176114],
[ 0.01842247, -0.61776648],
[ 0.49175192, 0.18200285],
[ 0.9174705 , 1.11044424],
[-0.24157428, -1.33867331],
[-0.57224663, -0.11957698],
[-2.11912947, -0.81290132],
[-0.87843793, 0.53730007],
[ 0.49175192, 0.18200285],
[ 0.35915409, 0.49892787],
[-0.68238385, -0.57238811],
[ 0.81693192, 0.48753743],
[ 0.9174705 , 1.11044424],
[ 0.68871016, 0.82013614],
[ 0.9174705 , 1.11044424],
[-0.04197563, 0.35600356],
[ 0.72721598, 0.52640777],
[ 0.23821845, 0.50095111],
[ 0.02682711, -0.18847165],
[ 0.10787313, 0.00702052],
[ 0.18906442, -0.40169025],
[ 0.9174705 , 1.11044424],
[-2.3373127 , -1.44905895],
[ 0.07289139, 0.35176114],
[-0.43513287, -0.38422045],
[ 0.9174705 , 1.11044424],
[-0.27809503, -0.25960194],
[ 0.48271777, 0.95087299],
[-0.21922961, -0.36057522],
[ 0.17029944, -0.63349831],
[-2.227724 , -1.25874833],
[ 0.10787313, 0.00702052],
[ 0.28452728, 0.62126426],
[ 0.0404032 , 0.26721341],
[ 0.22832747, 0.51605585],
[ 0.9174705 , 1.11044424],
[ 0.9174705 , 1.11044424],
[-1.21499934, -0.70402104],
[ 0.38031754, 0.65567252],
[ 0.07545082, -0.02758408],
[ 0.9174705 , 1.11044424],
[ 0.38031754, 0.65567252],
[ 0.9174705 , 1.11044424],
[-1.59210666, -0.14579451],
[ 0.9174705 , 1.11044424],
[-0.01065988, -0.47196257],
[ 0.9174705 , 1.11044424],
[ 0.38545942, 0.17495772],
[ 0.9174705 , 1.11044424],
[-0.03181092, -0.22247048],
[ 0.9174705 , 1.11044424],
[-0.03372734, -0.9612118 ],
[ 0.60887004, 0.22965779],
[-1.33143194, -0.06973491],
[ 0.26424032, -0.27643894],
[-0.49009137, -0.54572067],
[ 0.9174705 , 1.11044424],
[ 0.7437468 , 0.95053287],
[ 0.56545172, 0.76491233],
[ 0.72721598, 0.52640777],
[-1.1832441 , -1.25799326],
[ 0.9174705 , 1.11044424],
[ 0.23156471, 0.12512306],
[ 0.81693192, 0.48753743],
[-0.87550883, -1.22018193],
[-0.49512862, -0.54959239]])