R Under development (unstable) (2024-02-02 r85855 ucrt) -- "Unsuffered Consequences" Copyright (C) 2024 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > ################################################################ > # Example 1 # > # The GDI method to validate Q-matrix # > ################################################################ > set.seed(123) > > library(Qval) Qval R Package (version 0.1.4; 2024-02-01) > > ## generate Q-matrix and data > K <- 4 > I <- 20 > example.Q <- sim.Q(K, I) > IQ <- list( + P0 = runif(I, 0.0, 0.2), + P1 = runif(I, 0.8, 1.0) + ) > example.data <- sim.data(Q = example.Q, N = 500, IQ = IQ, model = "GDINA", distribute = "horder") distribute = horder model = GDINA number of attributes: 4 number of items: 20 num of examinees: 500 average of P0 = 0.085 average of P1 = 0.898 theta_mean = -0.06 , theta_sd = 0.996 a = 1.5 1.5 1.5 1.5 b = 0.5 1.5 -1.5 -0.5 > > ## simulate random mis-specifications > example.MQ <- sim.MQ(example.Q, 0.1) rate of mis-specifications = 0.1 rate of over-specifications = 0.01 rate of under-specifications = 0.09 > > ## using MMLE/EM to fit CDM model first > example.CDM.obj <- CDM(example.data$dat, example.MQ) Iter = 1 Max. abs. change = 0.48443 Deviance = 11093.48 Iter = 2 Max. abs. change = 0.13558 Deviance = 9693.66 Iter = 3 Max. abs. change = 0.08718 Deviance = 9644.69 Iter = 4 Max. abs. change = 0.04002 Deviance = 9631.28 Iter = 5 Max. abs. change = 0.03262 Deviance = 9624.26 Iter = 6 Max. abs. change = 0.02697 Deviance = 9619.79 Iter = 7 Max. abs. change = 0.02181 Deviance = 9616.73 Iter = 8 Max. abs. change = 0.01687 Deviance = 9614.64 Iter = 9 Max. abs. change = 0.01236 Deviance = 9613.25 Iter = 10 Max. abs. change = 0.00698 Deviance = 9612.39 Iter = 11 Max. abs. change = 0.00503 Deviance = 9611.89 Iter = 12 Max. abs. change = 0.00372 Deviance = 9611.64 Iter = 13 Max. abs. change = 0.00271 Deviance = 9611.51 Iter = 14 Max. abs. change = 0.00196 Deviance = 9611.45 Iter = 15 Max. abs. change = 0.00142 Deviance = 9611.42 Iter = 16 Max. abs. change = 0.00102 Deviance = 9611.40 Iter = 17 Max. abs. change = 0.00087 Deviance = 9611.39 Iter = 18 Max. abs. change = 0.00054 Deviance = 9611.38 Iter = 19 Max. abs. change = 0.00038 Deviance = 9611.38 Iter = 20 Max. abs. change = 0.00027 Deviance = 9611.38 Iter = 21 Max. abs. change = 0.00020 Deviance = 9611.38 Iter = 22 Max. abs. change = 0.00014 Deviance = 9611.38 Iter = 23 Max. abs. change = 0.00010 Deviance = 9611.38 Iter = 24 Max. abs. change = 0.00008 Deviance = 9611.38 > > ## using the fitted CDM.obj to avoid extra parameter estimation. > Q.GDI.obj <- validation(example.data$dat, example.MQ, example.CDM.obj, method = "GDI") GDI method with PAA in test level iteration ... Iter=1/1, 6 items have changed, delta PVAF=0.82173 > > > ## also can validate the Q-matrix directly > Q.GDI.obj <- validation(example.data$dat, example.MQ) GDI method with PAA in test level iteration ... Iter = 1 Max. abs. change = 0.48443 Deviance = 11093.48 Iter = 2 Max. abs. change = 0.13558 Deviance = 9693.66 Iter = 3 Max. abs. change = 0.08718 Deviance = 9644.69 Iter = 4 Max. abs. change = 0.04002 Deviance = 9631.28 Iter = 5 Max. abs. change = 0.03262 Deviance = 9624.26 Iter = 6 Max. abs. change = 0.02697 Deviance = 9619.79 Iter = 7 Max. abs. change = 0.02181 Deviance = 9616.73 Iter = 8 Max. abs. change = 0.01687 Deviance = 9614.64 Iter = 9 Max. abs. change = 0.01236 Deviance = 9613.25 Iter = 10 Max. abs. change = 0.00698 Deviance = 9612.39 Iter = 11 Max. abs. change = 0.00503 Deviance = 9611.89 Iter = 12 Max. abs. change = 0.00372 Deviance = 9611.64 Iter = 13 Max. abs. change = 0.00271 Deviance = 9611.51 Iter = 14 Max. abs. change = 0.00196 Deviance = 9611.45 Iter = 15 Max. abs. change = 0.00142 Deviance = 9611.42 Iter = 16 Max. abs. change = 0.00102 Deviance = 9611.40 Iter = 17 Max. abs. change = 0.00087 Deviance = 9611.39 Iter = 18 Max. abs. change = 0.00054 Deviance = 9611.38 Iter = 19 Max. abs. change = 0.00038 Deviance = 9611.38 Iter = 20 Max. abs. change = 0.00027 Deviance = 9611.38 Iter = 21 Max. abs. change = 0.00020 Deviance = 9611.38 Iter = 22 Max. abs. change = 0.00014 Deviance = 9611.38 Iter = 23 Max. abs. change = 0.00010 Deviance = 9611.38 Iter = 24 Max. abs. change = 0.00008 Deviance = 9611.38 Iter=1/1, 6 items have changed, delta PVAF=0.82173 > > ## item level iteration > Q.GDI.obj <- validation(example.data$dat, example.MQ, method = "GDI", iter.level = "item", maxitr = 150) GDI method with PAA in item level iteration ... Iter = 1 Max. abs. change = 0.48443 Deviance = 11093.48 Iter = 2 Max. abs. change = 0.13558 Deviance = 9693.66 Iter = 3 Max. abs. change = 0.08718 Deviance = 9644.69 Iter = 4 Max. abs. change = 0.04002 Deviance = 9631.28 Iter = 5 Max. abs. change = 0.03262 Deviance = 9624.26 Iter = 6 Max. abs. change = 0.02697 Deviance = 9619.79 Iter = 7 Max. abs. change = 0.02181 Deviance = 9616.73 Iter = 8 Max. abs. change = 0.01687 Deviance = 9614.64 Iter = 9 Max. abs. change = 0.01236 Deviance = 9613.25 Iter = 10 Max. abs. change = 0.00698 Deviance = 9612.39 Iter = 11 Max. abs. change = 0.00503 Deviance = 9611.89 Iter = 12 Max. abs. change = 0.00372 Deviance = 9611.64 Iter = 13 Max. abs. change = 0.00271 Deviance = 9611.51 Iter = 14 Max. abs. change = 0.00196 Deviance = 9611.45 Iter = 15 Max. abs. change = 0.00142 Deviance = 9611.42 Iter = 16 Max. abs. change = 0.00102 Deviance = 9611.40 Iter = 17 Max. abs. change = 0.00087 Deviance = 9611.39 Iter = 18 Max. abs. change = 0.00054 Deviance = 9611.38 Iter = 19 Max. abs. change = 0.00038 Deviance = 9611.38 Iter = 20 Max. abs. change = 0.00027 Deviance = 9611.38 Iter = 21 Max. abs. change = 0.00020 Deviance = 9611.38 Iter = 22 Max. abs. change = 0.00014 Deviance = 9611.38 Iter = 23 Max. abs. change = 0.00010 Deviance = 9611.38 Iter = 24 Max. abs. change = 0.00008 Deviance = 9611.38 Iter=1/150, 1 items have changed, delta PVAF=0.82173 Iter = 1 Max. abs. change = 0.47567 Deviance = 10962.21 Iter = 2 Max. abs. change = 0.09715 Deviance = 9599.43 Iter = 3 Max. abs. change = 0.05036 Deviance = 9559.00 Iter = 4 Max. abs. change = 0.02984 Deviance = 9550.49 Iter = 5 Max. abs. change = 0.01871 Deviance = 9546.53 Iter = 6 Max. abs. change = 0.01408 Deviance = 9544.18 Iter = 7 Max. abs. change = 0.01348 Deviance = 9542.66 Iter = 8 Max. abs. change = 0.01300 Deviance = 9541.60 Iter = 9 Max. abs. change = 0.01266 Deviance = 9540.82 Iter = 10 Max. abs. change = 0.01253 Deviance = 9540.21 Iter = 11 Max. abs. change = 0.01260 Deviance = 9539.71 Iter = 12 Max. abs. change = 0.01280 Deviance = 9539.27 Iter = 13 Max. abs. change = 0.01300 Deviance = 9538.86 Iter = 14 Max. abs. change = 0.01298 Deviance = 9538.45 Iter = 15 Max. abs. change = 0.01244 Deviance = 9538.04 Iter = 16 Max. abs. change = 0.01118 Deviance = 9537.64 Iter = 17 Max. abs. change = 0.00927 Deviance = 9537.29 Iter = 18 Max. abs. change = 0.00361 Deviance = 9537.02 Iter = 19 Max. abs. change = 0.00195 Deviance = 9536.86 Iter = 20 Max. abs. change = 0.00097 Deviance = 9536.78 Iter = 21 Max. abs. change = 0.00061 Deviance = 9536.74 Iter = 22 Max. abs. change = 0.00049 Deviance = 9536.72 Iter = 23 Max. abs. change = 0.00038 Deviance = 9536.71 Iter = 24 Max. abs. change = 0.00030 Deviance = 9536.70 Iter = 25 Max. abs. change = 0.00023 Deviance = 9536.70 Iter = 26 Max. abs. change = 0.00018 Deviance = 9536.70 Iter = 27 Max. abs. change = 0.00026 Deviance = 9536.70 Iter = 28 Max. abs. change = 0.00011 Deviance = 9536.70 Iter = 29 Max. abs. change = 0.00008 Deviance = 9536.70 Iter=2/150, 1 items have changed, delta PVAF=0.52167 Iter = 1 Max. abs. change = 0.47244 Deviance = 10869.81 Iter = 2 Max. abs. change = 0.09805 Deviance = 9526.83 Iter = 3 Max. abs. change = 0.05008 Deviance = 9482.95 Iter = 4 Max. abs. change = 0.02872 Deviance = 9474.58 Iter = 5 Max. abs. change = 0.01736 Deviance = 9470.69 Iter = 6 Max. abs. change = 0.01419 Deviance = 9468.37 Iter = 7 Max. abs. change = 0.01357 Deviance = 9466.84 Iter = 8 Max. abs. change = 0.01318 Deviance = 9465.77 Iter = 9 Max. abs. change = 0.01294 Deviance = 9464.98 Iter = 10 Max. abs. change = 0.01288 Deviance = 9464.36 Iter = 11 Max. abs. change = 0.01301 Deviance = 9463.84 Iter = 12 Max. abs. change = 0.01321 Deviance = 9463.39 Iter = 13 Max. abs. change = 0.01334 Deviance = 9462.95 Iter = 14 Max. abs. change = 0.01311 Deviance = 9462.52 Iter = 15 Max. abs. change = 0.01224 Deviance = 9462.10 Iter = 16 Max. abs. change = 0.01062 Deviance = 9461.71 Iter = 17 Max. abs. change = 0.00628 Deviance = 9461.38 Iter = 18 Max. abs. change = 0.00287 Deviance = 9461.15 Iter = 19 Max. abs. change = 0.00156 Deviance = 9461.03 Iter = 20 Max. abs. change = 0.00076 Deviance = 9460.97 Iter = 21 Max. abs. change = 0.00055 Deviance = 9460.94 Iter = 22 Max. abs. change = 0.00044 Deviance = 9460.92 Iter = 23 Max. abs. change = 0.00034 Deviance = 9460.91 Iter = 24 Max. abs. change = 0.00027 Deviance = 9460.91 Iter = 25 Max. abs. change = 0.00021 Deviance = 9460.90 Iter = 26 Max. abs. change = 0.00016 Deviance = 9460.90 Iter = 27 Max. abs. change = 0.00030 Deviance = 9460.90 Iter = 28 Max. abs. change = 0.00010 Deviance = 9460.90 > > ## search method > Q.GDI.obj <- validation(example.data$dat, example.MQ, method = "GDI", search.method = "ESA") GDI method with ESA in test level iteration ... Iter = 1 Max. abs. change = 0.48443 Deviance = 11093.48 Iter = 2 Max. abs. change = 0.13558 Deviance = 9693.66 Iter = 3 Max. abs. change = 0.08718 Deviance = 9644.69 Iter = 4 Max. abs. change = 0.04002 Deviance = 9631.28 Iter = 5 Max. abs. change = 0.03262 Deviance = 9624.26 Iter = 6 Max. abs. change = 0.02697 Deviance = 9619.79 Iter = 7 Max. abs. change = 0.02181 Deviance = 9616.73 Iter = 8 Max. abs. change = 0.01687 Deviance = 9614.64 Iter = 9 Max. abs. change = 0.01236 Deviance = 9613.25 Iter = 10 Max. abs. change = 0.00698 Deviance = 9612.39 Iter = 11 Max. abs. change = 0.00503 Deviance = 9611.89 Iter = 12 Max. abs. change = 0.00372 Deviance = 9611.64 Iter = 13 Max. abs. change = 0.00271 Deviance = 9611.51 Iter = 14 Max. abs. change = 0.00196 Deviance = 9611.45 Iter = 15 Max. abs. change = 0.00142 Deviance = 9611.42 Iter = 16 Max. abs. change = 0.00102 Deviance = 9611.40 Iter = 17 Max. abs. change = 0.00087 Deviance = 9611.39 Iter = 18 Max. abs. change = 0.00054 Deviance = 9611.38 Iter = 19 Max. abs. change = 0.00038 Deviance = 9611.38 Iter = 20 Max. abs. change = 0.00027 Deviance = 9611.38 Iter = 21 Max. abs. change = 0.00020 Deviance = 9611.38 Iter = 22 Max. abs. change = 0.00014 Deviance = 9611.38 Iter = 23 Max. abs. change = 0.00010 Deviance = 9611.38 Iter = 24 Max. abs. change = 0.00008 Deviance = 9611.38 Iter=1/1, 6 items have changed, delta PVAF=0.82184 > > ## cut-off point > Q.GDI.obj <- validation(example.data$dat, example.MQ, method = "GDI", eps = 0.90) GDI method with PAA in test level iteration ... Iter = 1 Max. abs. change = 0.48443 Deviance = 11093.48 Iter = 2 Max. abs. change = 0.13558 Deviance = 9693.66 Iter = 3 Max. abs. change = 0.08718 Deviance = 9644.69 Iter = 4 Max. abs. change = 0.04002 Deviance = 9631.28 Iter = 5 Max. abs. change = 0.03262 Deviance = 9624.26 Iter = 6 Max. abs. change = 0.02697 Deviance = 9619.79 Iter = 7 Max. abs. change = 0.02181 Deviance = 9616.73 Iter = 8 Max. abs. change = 0.01687 Deviance = 9614.64 Iter = 9 Max. abs. change = 0.01236 Deviance = 9613.25 Iter = 10 Max. abs. change = 0.00698 Deviance = 9612.39 Iter = 11 Max. abs. change = 0.00503 Deviance = 9611.89 Iter = 12 Max. abs. change = 0.00372 Deviance = 9611.64 Iter = 13 Max. abs. change = 0.00271 Deviance = 9611.51 Iter = 14 Max. abs. change = 0.00196 Deviance = 9611.45 Iter = 15 Max. abs. change = 0.00142 Deviance = 9611.42 Iter = 16 Max. abs. change = 0.00102 Deviance = 9611.40 Iter = 17 Max. abs. change = 0.00087 Deviance = 9611.39 Iter = 18 Max. abs. change = 0.00054 Deviance = 9611.38 Iter = 19 Max. abs. change = 0.00038 Deviance = 9611.38 Iter = 20 Max. abs. change = 0.00027 Deviance = 9611.38 Iter = 21 Max. abs. change = 0.00020 Deviance = 9611.38 Iter = 22 Max. abs. change = 0.00014 Deviance = 9611.38 Iter = 23 Max. abs. change = 0.00010 Deviance = 9611.38 Iter = 24 Max. abs. change = 0.00008 Deviance = 9611.38 Iter=1/1, 6 items have changed, delta PVAF=0.82122 > > ## check QRR > print(getQRR(example.Q, Q.GDI.obj$Q.sug)) [1] 0.925 > > proc.time() user system elapsed 16.20 0.62 16.81