A Simplified Method for Predicting Pattern Match Ratio

Cognitive diagnostic test design (CDTD) has a direct impact on the pattern match ratio (PMR) of the classification of examinees. It is more helpful to know the quality of a test during the stage of the test design than after the examination is taken. The theoretical construct validity (TCV) is an index of the test quality that can be calculated without testing, and the relationship between the PMR and the TCV will be revealed. The TCV captures the three aspects of the appeal of the test design as follows: (1) the TCV is a measure of test construct validity, and this index will navigate the processes of item construction and test design toward achieving the goal of measuring the intended objectives, (2) it is the upper bound of the PMR of the knowledge states of examinees, so it can predict the PMR, and (3) it can detect the defects of test design, revise the test in time, improve the efficiency of test design, and save the cost of test design. Furthermore, the TCV is related to the distribution of knowledge states and item categories and has nothing to do with the number of items.


INTRODUCTION
Cognitive diagnosis (CD) has received much attention, providing diagnostic information of knowledge or skills (often called "attributes" in the CD literature) to the examinees (de la Torre and Douglas, 2004;de la Torre, 2008;DeCarlo, 2011;Liu et al., 2012;Kang et al., 2017;Huebner et al., 2018). It is critical to ensure that high-quality cognitive diagnostic tests can accurately diagnose the knowledge state (KS, i.e., the latent cognitive states) of examinees. The set of KSs is represented by the Q S matrix. In fact, cognitive diagnostic test design (CDTD) is the design of a Q matrix, called Q t , i.e., rows representing attributes and columns representing attribute vectors, namely, items. By anchoring the items with attribute vectors, proposition experts and measurement experts transform items into measurable forms and then diagnose examinees. In a word, the design of the Q t matrix is the problem of how to match the attribute vectors to achieve a certain predetermined goal.
Until present, the studies on the CDTD methods are still relatively weak, and they focus on the following two aspects: (1) CDTD based on the perfect Q matrix The so-called "perfect Q matrix" refers to the Q t matrix that makes the ideal response pattern (IRP) and KS correspond one to one. If the Q matrix in tests is a perfect Q matrix, the pattern match ratio (PMR) improves no matter whether the CDTD is either dichotomous or polytomous.
(i) Examples of dichotomous CDTD: For the four attribute hierarchies of Leighton (Leighton et al., 2004), if the Q t matrix is a Boolean matrix, and there is no compensation between the attributes, then the reachable matrix (or it is equivalent classes) acts as the submatrix of Q t which can achieve a one-to-one correspondence between the set of IRPs and the set of KSs. The more reachable matrices in the Q t matrix, the higher the PMR (Ding et al., 2010. Ding et al. (2010) called such a Q t matrix a sufficient and necessary matrix, i.e., a perfect Q matrix . The results are similar to those of Chiu et al. (2009), DeCarlo (2011, and Madison and Bradshaw (2015) on independent structures. With the independent structure and four attributes, Samejima (1995) believed that when the Q t matrix was the identity matrix (i.e., the identity matrix of independent structure is a reachable matrix), all of the KSs would not be misjudged. Chiu et al. (2009) also found that the Deterministic Input Noisy "AND" Gate (DINA) model and the Deterministic Input Noisy Output "OR" gate (DINO) model could diagnose all potential attribute mastery patterns when the Q t matrix included the identity matrix. Similar results have been addressed in other studies (DeCarlo, 2011;Madison and Bradshaw, 2015). (ii) Examples of polytomous CDTD: To achieve the one-to-one correspondence between the set of KSs and the set of IRPs, the rooted tree structure, the independent structure, and the perfect Q matrices of the rhombus structure are introduced under the item score rule that one ideal score is added if mastering one attribute adhering to the item (Ding et al., 2014a). In the initial stage of CD-CAT, each attribute can be diagnosed by using the reachable matrix (Tu et al., 2013). In CD-CAT, the higher the percentage of the examinees is, whose testing items are (or contain) the reachable matrix according to the selection strategy, the higher the PRM is.
(2) CDTD based on the index The Cognitive Diagnostic Index (CDI) (Henson and Douglas, 2005) and the Attribute-level Discrimination Index (ADI) (Henson et al., 2008) are based on the level of items and attributes for CD. Kuo et al. (2016) indicated that each attribute in the test must be measured at least three times to attain better correct attribute classification, so they proposed modified CDIs and ADIs, namely, MCDI and MADI. The Shannon's entropy (Xu et al., 2003) and posterior-weighted Kullback-Leibler (PWKL) (Cheng, 2009) were introduced in CD-CAT. Cheng (2010) believed that adequate coverage of each attribute could improve the validity of the test scores, and then the attribute-balancing index was proposed. Subsequently, the index was further improved (Yu et al., 2011;Liu et al., 2018;Sun et al., 2019). Adaptive multigroup testing method for cognitive diagnosis (CD-AMGT) (Luo et al., 2018), which selects a group of appropriate items in different diagnosis stages, has the advantages of uniform use of item bank and less time to calculate.
The PMR is the main evaluation index for cognitive diagnostic tests. In CDTD, the pretest evaluation of the PMR is more positive than the posttest evaluation because the designed test can be modified quickly, the designer can make up for possible errors before testing, and material resources and time will be saved. At present, the PMR is the posttest estimation based on the data measured or simulated, so it is impossible to calculate PMR immediately during the design process. Furthermore, it is meaningful to discuss the maximum PMR for the pretest, and the maximum PMR is related to the matching degree between the designed test and the cognitive model, as well as the quality and length of the test.
The rest of the study is organized as follows: First, the TCV used in this study is briefly described. Second, the theoretical proof of the relationships between the TCV and the PMR is introduced in detail. The TCV is then evaluated in a simulation study. The end of the study is the discussion and conclusion.

Cognitive Diagnosis
The cognitive model is a prerequisite for CD. It is represented by an attribute hierarchy, which specifies the psychological ordering of the attributes required to solve test items. Attributes are those basic cognitive processes or skills required to solve test items correctly. There are five forms of basic hierarchical structures (Leighton et al., 2004;Cheng, 2010), namely, A, B, C, D and E (Figure 1).
Attribute 1 is considered a prerequisite to other attributes, and attribute 5 depends on some attributes in models except the independent model. The adjacency (A), reachability (R), incidence (Q), and reduced incidence (Q r ) matrices are specified by Tatsuoka (1995). The columns of the Q r matrix indicate that all possible items must be created to reflect the relationships among the attributes in the hierarchy. The possible latent cognitive states (i.e., KS), which is all the columns of the incidence matrix, possess cognitive attributes that are consistent with the hierarchy (when the hierarchy is based on cognitive considerations), and they apply these attributes systematically (when the hierarchy is based on procedural considerations) (Gierl et al., 2007). Let q j = q j1 , q j2 , · · · , q jK T j = 1, · · · , m denote the jth dichotomous column vector (i.e., the jth category item) of the Q r matrix. All KSs are represented by column vectors: α i = (α i1 , α i2 , · · · , α iK ) T , where α ik = 1(k = 1, · · · , K) indicates that the ith category examinee has mastered attribute k, and α ik = 0 otherwise. K is the total number of attributes measured by the test. Let the Q s matrix denote all KSs, in fact, including zero vector (denoted as0, i.e., this kind of examinee does not master any attribute) and the Q r matrix for cognitive attribute consistency. Thus, α i and q j are all K-dimensional vectors. The Q t matrix consists of some column vectors of the Q r matrix. Based on the cognitive model (including attributes and hierarchy among them), the Q r and Q s matrix can be obtained, that is, all possible items and KSs can be obtained. On the contrary, if the Q t matrix is known, some KSs can be obtained through the augment algorithm Yang et al., 2008), and the cognitive model can be derived by comparing the rows (Tatsuoka, 1995). In general, it is impossible for some items (i.e., the Q t matrix) to replace all the items (i.e., the Q r matrix), which express the cognitive structure, so some cognitive structures extracted from the Q t matrix may be inconsistent with the theoretical one.

The DINA Model
Cognitive diagnostic models have been proposed for many years, including the rule space model (Tatsuoka, 1983), the "Noisy Input Deterministic ' AND' Gate" (NIDA) model (Maris, 1999), the fusion model (Hartz, 2002), the reduced reparameterized unified model (R-RUM; Hartz, 2002), and the DINA model (Haertel, 1989). The DINA model is completely noncompensatory. The DINA model treats slipping and guessing at the item level. Parameter s j indicates the probability of "slipping, " and parameter g j denotes the probability of "guessing." The item response function, therefore, can be written as follows: When n ij = 1, the ith examinee should be able to answer item j correctly, unless he/she "slips." Similarly, when n ij = 0, the ith examinee should not be able to answer item j correctly, unless he/she is a lucky guesser (Cheng, 2010).

Theoretical Construct Validity
Theoretical construct validity (TCV) is used to measure the degree of consistency between the theoretical cognitive model and the cognitive model implied in the Q t matrix (Ding et al., 2012).
Definition 1 Let α 1 , α 2 , · · · , α N 1 denote N 1 KS of the theoretical cognitive model given by experts, β 1 , β 2 , · · · , β N 2 denote N 2 KS derived from the Q t matrix, and γ 1 , γ 2 , · · · , γ N 3 = β 1 , β 2 , · · · , β N 2 ∩ α 1 , α 2 , · · · , α N 1 denote N 3 KS. when γ k = α i , the TCV for the Q t matrix can be written as follows: where p i represents the probability of the ith category examinees, that is, the ratio of such examinees whose KS is α i in the total population. In particular, when all KS ratios in the total population are equal, then In fact, the TCV is a measure of the degree to which the Q t matrix represents the theoretical cognitive model (Ding et al., 2012). The observed response pattern (ORP) and the CDM are necessary for the set of the estimation of KSs of the examinees. The set of IRPs is determined by the set of KSs, the test Q matrix, the element value of the Q t matrix (the dichotomous or the polytomous), the calculation method of the ideal score, the compensation between attributes, and so on. The ORP is related not only to the above mentioned factors but also to the item quality and random factors. Thus, if there is no random factor, the better the item quality, the closer the ORP is to the IRP. Due to the slipping and the guessing in the answering process of examinees, the PMR of the set of KSs estimated by the ORP is not higher than that estimated by the IRP, that is, PMR ORP ≤ PMR IRP . The PMR IRP acts as the maximum PMR ORP , and the smaller the slipping and the guessing, the more accurate the KSs based on the ORP. How to get the PMR IRP quickly is an interesting problem.
To clearly solve the interesting problem, a theoretical explanation that makes sense of the complexity is firmly couched within the examples.
Definition 2 Define the relationship between two attribute vectors α i and q j as α i ≥ q j if and only if α ik ≥ q jk , for k = 1, 2, . . . , K. Strict inequality between the attribute vectors is involved (i.e., α i > q j ) if α ik > q jk for at least one k (de la Torre, 2011). α i ≤ q j and α i < q j can be defined similarly as mentioned earlier. If the relationship does not exist, then α i has nothing to do with q j . The definition of comparison between column vectors also applies to row vectors.

Examples
The theoretical cognitive model is an independent structure of three attributes, according to the methods suggested by Tatsuoka for calculating the adjacency (A), reachability (R), incidence (Q), and reduced incidence (Q r ) matrices; then, adding zero vector to the Q r matrix, there are 2 3 = 8 possible KSs, that is, N 1 is 8. The Q s matrix is represented by a 3 × 8 matrix as follows: where α i (i = 1, · · · , 8) is the ith category examinees. Test items, represented by a 3 × 3 matrix, can be written as follows: where q j is the jth item when items are not duplicated, otherwise it represents the jth category item.

Calculation of TCV
A new matrix, called the Q + t matrix, is made of the Q t matrix and the two new columns. The two new columns based on the augment algorithm Yang et al., 2008) are generated from the Q t matrix, while the non-zero vectors (0, 1, 1) T and (0, 0, 1) T in the Q s matrix cannot be generated as follows: Five KSs in the Q + t matrix are derived from the Q t matrix, that is, N 2 is 5. There are five same possible latent cognitive states between the theoretical cognitive model and the cognitive model implied in the test design, that is, {γ 1 , γ 2 , γ 3 , γ 4 , γ 5 } = {α 2 , α 3 , α 5 , α 6 , α 8 }, N 3 is 5 (N 1 , N 2 , and N 3 are the same as Definition 1), when adding zero vector (0 = (0, 0, 0) T ).

Calculation of PMR IRP
Ideal response (IR) depends on the relationship between α i and q j . Let IR α i , q j = α i o q j = K k = 1 (α ik ) q jk = 1 denote that the ith examinee responses correctly on the jth item, and IR α i , q j = 0 otherwise. Clearly, IR α 1 , q 1 = IR α 1 , q 2 = IR α 1 , q 3 = 0 due to 0 ≤ α 1 < q 1 < q 3 and 0 ≤ α 1 < q 2 ; IR α 2 , q 1 = 1, IR α 2 , q 2 = IR α 2 , q 3 = 0 due to q 1 ≤ α 2 < q 3 ; and α 2 having nothing to do with q 2 . Similarly, the set of IRPs of the Q s matrix with respect to the Q t matrix is represented by a 3 × 8 matrix as follows:   In Equation 8, the row represents the item, and the column represents α ′ i s IRP. There are six different IRPs, that is, six KS can be correctly estimated without taking the slipping and the guessing into account. In essence, the estimated five KSs based on five IRPs are the same as vectors in the Q + t matrix (five different categories), and adding estimated zero vector (because the IRP is zero vector), there are six categories. α 4 = (0, 0, 1) T and α 7 = (0, 1, 1) T are the same categories to zero vector (α 1 = 0) and α 3 = (0, 1, 0) T , respectively; thus, no new categories are generated.
The whole process of dividing the Q s matrix can be vividly described as follows: the Q s matrix is similar to a line, and five vectors in the Q + t matrix are similar to five dots that classify the line into six categories in which only one KS can be estimated correctly; therefore, PMR IRP = 6 8 = 3 4 . From calculations 1 and 2, it can be known that TCV = PMR IRP .
Examples of other structures are shown in Table 1.
Although structures are different, convergent, or divergent, the result of the relationship between the TCV and the PMR IRP is the same: the number of vectors of the Q + t matrix in the convergent structure was 3, and then the Q s matrix could be classified into four categories; the number of vectors of the Q + t matrix in the divergent structure was 5 due to two new columns derived from the Q t matrix, and then the Q s matrix could be classified into six categories. For the linear structure and the unstructured, the results are similar.
Notably, all items of the Q t matrix are different because the repetition of items does not increase the "coverage" of the cognitive model by the Q t matrix. Repeated items only reduce random errors; thus, in the following discussion, it is not necessary to consider the repeated items in the Q t matrix.
Theoretical Derivation of TCV = PMR IRP Let R denote reachable matrix, the Q r matrix is a set of all possible items that can be written as follows: In fact, Q t ⊆ Q r , Qs = 0, Q r .
For every α i (i = 1, · · · , n) (except for zero vector) in the Q s matrix, there should be a q ′ j (∈ Q r ) corresponding to it, that is, Based on the augment algorithm, the Q + t matrix can be defined as follows: where V represents the Boolean union operation, p ∈ Q means that p is the item (column) of the Q matrix, and Q ⊆ Q t indicates that the Q matrix is a subset of the Q t matrix and contains one or more items. New columns of the Q + t matrix can be obtained by the Boolean union of two or more items in the Q t matrix.
There are m columns in the Q + t matrix, adding zero vector, m+1 categories of the KSs are derived from the Q t matrix in total. n is the number of the set of KSs derived from the theoretical cognitive model, that is, n columns in the Qs matrix, so the TCV can be calculated as follows: The maximum lower bound of α i can be found in the Q + t matrix by comparing α i with q j in the Q + t matrix, and it can be defined as follows: q j ′ = max q j q j ≤ α i , q j ∈ Q + t , α i ∈ Qs, i = 1, 2, . . . , n; j = 1, 2, . . . , m In fact, j ′ is the subscript of the maximum item, that is, j ′ = argmax q j , j ′ ∈ {1, 2, · · · , m }. Let q j ′ denote a set of α i with the same maximum lower bound q j ′ : If q j ′ does not exist, then let 0 denote α i set as follows: 0 = α i q j > α i or α i has nothing to do with q j All the α ′ i s with the same IRP will be classified into one category by comparing α i with all p items in the Q t matrix: based on the definition of q j ′ , if q j ′ exists, it means that q j = p∈Q p ≤ q j ′ ≤ α i , so the IRs between α i and p (p ≤ α i ) are 1, that is, IR(α i , p) = α i o p = 1, the IRs between α i and the rest of p in the Q t matrix are 0, that is, IR(α i , p) = α i p = 0. Therefore, all the α ′ i s in q j ′ have the same IR, and these α ′ i s belong to one category. If q j ′ does not exist, for all p items in the Q t matrix, α i <p or α i has nothing to do with p, the IRs between α i and p is 0, IR is the same with zero vector 0 , and thus, these α ′ i s are the same category as zero vector.
Proposition 1: All α i s in the Q s matrix are classified into q j ′ (q j ′ ∈Q + t ) or 0 (i = 1, . . . ,n; j ′ = 1, . . . ,m;m≤n). First, there must be existed a α i for every q j ′ in the Q + t matrix, so that α i is equal to q j ′ , so q j ′ is the maximum lower bound of α i , α i is an element of a set q j ′ . m α ′ i s are divided into m sets q j ′ .
Second, for the remaining n-m α ′ i s, (1) For every p in the Q t matrix, if α i < p or α i has nothing to do with p, then q j ′ does not exist, so α i belongs to set 0 ; (2) If p ≤ α i , there must be existed q j ′ acted as the maximum lower bound of α i , so α i belongs to set q j ′ .
Combining (1) and (2), Proposition 1 is proved. Proposition 2: If the number of q j in the Q + t matrix is m, all α ′ i s in the Q s matrix are classified into m+1 categories.
From Proposition 1, the conclusion is clearly true, that is, m+1 categories of the set of KSs can be estimated correctly. Thus, PMR IRP = m+1 n . The result of TCV = PMR IRP shows that the TCV is equal to the PMR estimated by the set of IRPs. For PMR ORP ≤ PMR IRP = TCV, the TCV is the upper bound of the PMR estimated by the ORP. When k is smaller, such as k ≤ 5, the TCV can be calculated by pen, otherwise, it is easily derived by using a computer.

SIMULATION STUDY
A simulation study was carried out to evaluate the relationships between the TCV and the PMR.
Five attribute hierarchical structures were studied, namely, independent, linear, convergent, divergent, and unstructured. The number of attributes was set at 4, that is, K = 4. The study needed to consider the influence of the distribution of examinees, item attribute vector, and their proportions on the TCV. Two kinds of distribution of the KSs of examinees were discussed as follows: the average distribution (30 persons for every KS) and the normal distribution. In particular, the standard multivariate normal distributions in the independent structure were investigated. The total number of examinees was the same. In contrast, there were six Q t matrices for each structure, items would be selected from the Q r matrix, and its proportions were different. The test length was 20. The descriptive statistics of the examinees and the Q t matrices are reported in Table 2.
To compare the effects of different slips on the TCV and the PMR, the slips were 0.15 and 0.02, respectively. The set of IRPs was obtained by the items of the Q t matrix and the set of KSs of the Q s matrix. Let x denoted the IR score of an examinee on an item, r randomly generated from Uniform (0, 1), if r> 1 − s, x (x was dichotomous) would be changed to 1-x, and x otherwise.
The DINA model and the maximum-likelihood estimation method were used to estimate the KS. Considering the differences in the distribution of examinees, the Q t matrix, and the slips, there were 116 levels in total, and each level was tested 30 times. The final PRM was an average of 30 PMRs.
The PMR index can be defined as follows: where N is the number of examinees. α i−correct = 1 represents that the ith examinee is estimated correctly. Table 3 compares the TCV and the PMR obtained from the linear structure. The first column shows the different distribution of examinees, and the other columns show the results of the different Q t matrices.

RESULTS
Clearly, the TCV was superior: the TCV was uniformly higher than the PMR regardless of the distribution of examinees and the Q t matrices. Although the repetition of items in the Q t matrices, the TCV was not changed when the distribution of examinees and the category of items in the Q t matrices remain unchanged. Therefore, this helped in explaining why repeated items were not necessary to count. As is known to all, the smaller the slip is, the higher the PMR is. But the TCV had nothing to do with the slip, so the smaller the slip, the smaller the gap between the TCV and the PMR. For all the attribute structures, when the  (1011) The normal distribution TCV was low, the PMR was also low and vice versa. Notably, the more the item categories were, the larger the TCV would be. In particular, if the Q t matrix contained the reachable matrix that could augment all possible item categories, then TCV = 1, regardless of the distribution of examinees. In other words, when the reachable matrix was a submatrix of the Q t matrix, the PMR would be higher than that of the Q t matrix that did not include the reachable matrix if the other conditions were the same. From Tables 4-7, the data of other structures show the same results as linear. In addition, the lesser the structure, the greater the difference between the TCV and the PMR.

DISCUSSION AND CONCLUSION
Guided by a cognitive model, the CD can detect how well the examinees have mastered certain knowledge or skills. All CDTDs aim at diagnosing examinees as much as possible, and the main evaluation index is the PMR. The higher the accuracy rate of the KSs, the higher the test construct validity. It is more meaningful to be able to calculate the PMR during CDTD. Tatsuoka (2009, p. 78-79) believed that the sufficient Q matrix can improve the test construct validity. However, how to measure the construct validity? Inspired by the  evaluation of the sufficient Q matrix by Tatsuoka (1995Tatsuoka ( , 2009, an evaluation index for cognitive diagnostic test (design) was developed, i.e., TCV, which made up for the defects of Tatsuoka's idea (Tatsuoka, 1995(Tatsuoka, , 2009). This study proposes a simplified method for predicting the PMR, namely, the TCV method for CD. The TCV intuitive meaning is as follows: the set of KSs is derived from the Q t matrix through the augment algorithm (i.e., this design can inspire some latent cognitive states), and if the probability distribution of the examinees in the population is known, then TCV = j p j . In particular, when the probability distribution of the set of KSs in the total population is discrete uniform, the TCV is equal to the sum, which is the number of categories of the set of KSs derived from the Q t matrix plus 1, divided by the number of categories of the set of KSs in the population. In general, the TCV measures the degree of consistency between the cognitive  model derived from matrix Q t and the theoretical cognitive model (Ding et al., 2012).
As the proof and the simulation showed, PMR ORP ≤ PMR IRP = TCV. Therefore, the TCV can be used to predict the PMR. Notably, the TCV is related to the distribution of examinees and item category, not related to the proportion of items. In other words, when calculating the TCV, repeated items should be treated as one item.
The TCV is numerically equal to the PMR based on the set of IRPs, and the factors that affect the set of IRPs are as follows: the cognitive model (e.g., the number of attributes, attribute hierarchy, and compensation between attributes), the composition of the test matrix (e.g., Boolean matrix and multivalued Q matrix), the item score (e.g., 0-1 score or multilevel score). Whatever has an effect on the set of IRPs influences the TCV. When the test Q matrix (Q t ) is a Boolean matrix, the score is 0 or 1, and the IR is 1 if and only if α i ≥ q j , the TCV is the upper bound of the PMR. The TCV has nothing to do with the CDM (i.e., classification method); therefore, the TCV is calculated by CDM-free. Thus, the conclusion is the same for the DINA model, the AHM (Attribute Hierarchy Method, Gierl et al., 2007)  The number of attributes has an effect on the TCV. For example, independent structure, if the probability distribution of the set of KSs in the total population is equal, different items containing only two attributes are selected, then when the number of attributes K is 3, and the TCV is 5/8; when K is 4, the TCV is 3/4; and when K is 5, the TCV is 27/32. However, under the same conditions, the number of attributes does not affect the conclusion that the TCV is the upper bound of the PMR at all (as shown by the proof). Furthermore, the lower the number of attributes, the higher the PMR. Therefore, the simulation study selected fewer attributes (K = 4). Similarly, the smaller the random in the ORP is, the higher the PMR is. To prove that the TCV is the upper bound of the PMR, in the simulation study, the random is relatively small (s = 0.02). According to the abovementioned logic, the result that TCV is the upper bound of the PMR is also true when the random is larger.
An interesting question arises as follows: the TCV is not equal to the PMR, why the TCV is useful for predicting the PMR? There are three reasons: First, the most important reason is that the TCV can be obtained during CDTD, which is instructive to adjust selected items at any time and to timely judge the test quality. Second, the TCV is the upper bound of the PMR, the smaller the slip, the smaller the gap between the TCV and the PMR. The TCV does not change with the slip. If the TCV is high, the PMR is also higher; therefore, it is feasible to use the TCV as an index of the PMR to predict the test quality. Third, the TCV is easy to calculate according to the formula.
The TCV can be used not only to predict the PMR but also, more importantly, to detect the defects of CDTD. By using the augment algorithm, the set of KSs can be derived from the Q t matrix, and then, the TCV can be calculated. Under the same conditions, if the TCV value is lower, it means that there are fewer kinds of attribute vectors (i.e., items) of the reachable matrix in the Q t matrix, and thus, the more KSs cannot be accurately estimated. At this time, test designers can modify the test Q matrix (i.e., the Q t matrix) before testing (not posttest evaluation), that is, modify the test (such as filling the columns of the reachable matrix or filling the columns expanded by the reachable matrix through the augment algorithm). Adjusting the selected items according to the TCV value at any time is not only beneficial to evaluate the test quality in time in CDTD but also can save cost and improve efficiency, which has the effect of two times the result with half the effort. This method undoubtedly has great advantages in CDTD.
If the test contains the reachable matrix, the cognitive model derived from the test is consistent with the theoretical cognitive model, and the TCV is 1. At this time, as long as the item quality is good (i.e., the slip is low) and attributes are measured a certain number of times, then the PMR is relatively high. In most cases, however, the PMR is not equal to 1 because the test is short, the quality of the items is poor, or the examinees do not answer carefully. At this time, although the result is rough when the TCV is used to predict the PMR, even so, under the same cases, the test, which contained the reachable matrix (in this case, the Q t matrix is complete Q matrix, Cai et al., 2018), has the higher PMR.
Although this study shows that the TCV method works successfully with CD, it has limitations in several aspects: (1) Since the TCV is determined by the Q t matrix, the Q t matrix must be complete and reliable, which is the premise of using the TCV. In some cases, this condition may be quite harsh. But the RUM model allows the Q t matrix to be incomplete, and the conclusion of this study cannot be applied. Furthermore, the complete and accurate calibration of the Q t matrix is still a very difficult problem. (2) If the score is 0 or 1 and IR is 1 if α i ≥ q j , other IR rules are not applicable in this case. Nor does it apply if there is compensation between attributes. (3) Only the dichotomous and non-compensable attributes are considered, a natural question that arises is how to get the TCV when the scoring is polytomous and attributes are compensable. These will be the interesting topics for future studies.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.