Abstract
Background: Screening for post-stroke cognitive impairment (PSCI) is necessary because stroke increases the incidence of and accelerates premorbid cognitive decline. The Quick Mild Cognitive Impairment (Qmci) screen is a short, reliable and accurate cognitive screening instrument but is not yet validated in PSCI. We compared the diagnostic accuracy of a Chinese version of the Qmci screen (Qmci-CN) compared with the widely-used Chinese versions of the Montreal Cognitive Assessment (MoCA-CN) and Mini-Mental State Examination (MMSE-CN).
Methods: We recruited 34 patients who had recovered from a stroke in rehabilitation unit clinics in 2 university hospitals in China: 11 with post-stroke dementia (PSD), 15 with post-stroke cognitive impairment no dementia (PSCIND), and 8 with normal cognition (NC). Classification was made based on clinician assessment supported by a neuropsychological battery, independent of the screening test scores. The Qmci-CN, MoCA-CN, and MMSE-CN screens were administered randomly by a trained rater, blind to the diagnosis.
Results: The mean age of the sample was 63 ± 13 years and 61.8% were male. The Qmci-CN had statistically similar diagnostic accuracy in differentiating PSD from NC, an area under the curve (AUC) of 0.94 compared to 0.99 for the MoCA-CN (p = 0.237) and 0.99 for the MMSE-CN (p = 0.293). The Qmci-CN (AUC 0.91), MoCA-CN (AUC 0.94), and MMSE-CN (AUC 0.79) also had statistically similar accuracy in separating PSD from PSCIND. The MoCA-CN more accurately distinguished between PSCIND and normal cognition than the Qmci-CN (p = 0.015). Compared to the MoCA-CN, the administration times of the Qmci-CN (329s vs. 611s, respectively, p < 0.0001) and MMSE-CN (280 vs. 611s, respectively, p < 0.0001) were significantly shorter.
Conclusion: The Qmci-CN is accurate in identifying PSD and separating PSD from PSCIND in patients post-stroke following rehabilitation and is comparable to the widely-used MoCA-CN, albeit with a significantly shorter administration time. The Qmci-CN had relatively poor accuracy in identifying PSCIND from NC and hence may lack accuracy for certain subgroups. However, given the small sample size, the study is under-powered to show superiority of one instrument over another. Further study is needed to confirm these findings in a larger sample size and in other settings (countries and languages).
Introduction
Post-stroke cognitive impairment (PSCI) is increasingly prevalent among community-dwelling older adults, due to the increasing incidence and the decreasing mortality associated with stroke (, ). This trend is also evident in China (). According to the degree of cognitive decline and impairment in activities of daily living (ADL), patients with PSCI can be divided into 2 groups: post-stroke cognitive impairment no dementia (PSCIND), those with slight cognitive impairment with or without subtle impairment in instrumental ADL (IADL) and those with established post-stroke dementia (PSD) with definite impairment in basic ADL (BADL) (). PSCI is common in China, affecting ~80% of stroke survivors of which PSCIND and PSD represent around 49% and 32% of cases, respectively (). Once established cognition may decline rapidly (). Despite this, care usually focuses on post-stroke physical disability with less attention paid to cognitive decline (–). PSCI has a significant impact on independence and on the potential to return to work after stroke (). Given these points, international guidelines recommend cognitive screening for those at risk of PSCI (, ).
As yet, no single screening and assessment tool is recommended, although tools should be selected according to the population being evaluated, the stage of progression based on functional status, the preferences of patients and families, as well as the resources available (). Studies show that both the Montreal Cognitive Assessment (MoCA) and Mini Mental State Examination (MMSE) are both widely-used to detect cognitive impairment after stroke, though in a recent systematic review comparing the accuracy and utility of instruments, the MoCA was shown to be the most valid and clinically feasible to identify PSCI (). The Chinese version of the MoCA (MoCA-CN) shows high sensitivity and specificity for PSCI () as well as other types of vascular cognitive impairment (), although the Chinese version of MMSE (MMSE-CN) remains the most widely-used screening instrument in China ().
The Quick mild cognitive impairment (Qmci) screen is a new, short cognitive screening instrument originally designed to identify mild cognitive impairment (MCI), which is highly accurate in differentiating between normal cognitive function, subjective cognitive disorders, MCI and early dementia (, ). The Qmci screen is more accurate than MMSE in detecting early cognitive changes (–) and when compared to the MoCA, it has similar accuracy and better specificity but a shorter administration time (–). The Chinese version of the Qmci (Qmci-CN) has recently been validated in a Chinese general rehabilitation outpatient clinic sample (). To date, the Qmci screen has not been examined in patients post-stroke. Given this point, the aim of this research was to investigate the diagnostic accuracy of the Qmci-CN in detecting cognitive impairment by comparing it to Chinese versions of the standardized MMSE (MMSE-CN) and MoCA (MoCA-CN) in patients recovering from stroke attending Chinese rehabilitation unit clinics.
Methods
Participants
Consecutive participants were recruited by therapists from rehabilitation clinics in two hospitals in Guangzhou, China between August 2017 and April 2018. All patients were aged over 18 years of age and had a stroke diagnosed by a neurologist confirmed with brain imagining (CT or MRI). Only patients consented by their attending doctor participated. The study excluded patients who were unable to communicate verbally in Chinese or had communication problems that could influence the performance of cognitive testing such as severe dysphasia, apraxia, visual impairment, hearing loss, or altered consciousness. Patients with known dementia prior to the stroke or those with existing neurological, psychiatric disorders or other comorbidities that potentially affect cognitive function, e.g., Parkinson's disease were also excluded. Further, patients who couldn't read and write, those with a recent history of alcohol or drug abuse and those who declined to participate were excluded. All those included signed informed written consent. This study received ethical approval from The Six Affiliated Hospital of Sun Yat-sen University, reference number 2019ZSLYEC-110.
The Qmci-CN
The Qmci screen has 6 subtests covering 5 cognitive domains (). The first subtest examines orientation to place and time. The second, five-word registration, examines attention and working memory. The third, a clock drawing test, examines visuospatial/executive function and attention. The fourth subtest is delayed recall examining episodic memory (5-word recall). The next subtest is categorical verbal fluency tests sematic memory and language (e.g., naming of animals in 1 min). The final subtest is logical memory, immediate verbal recall of a short story, which also examines episodic memory. Each subtest is time-limited with an administration time of ~5 min for the Qmci screen (). The total possible score is 100 points with higher scores suggesting normal cognition. The optimal cut-off score for cognitive impairment (MCI and dementia vs. normal controls) for the Qmci screen (original English language version) is ≤62/100 (). However, the optimal cut-off for the Chinese version (Qmci-CN) is lower at ≤55/100 (). This difference is considered to be multi-factorial but most likely relates to the small sample of Chinese patients included in the single validation study to date ().
Data Collection
Patients' characteristics including their age, sex, educational level, and other demographic and clinical information were collected. The Qmci-CN, MoCA-CN, and MMSE-CN were administered in random order by a trained rater, who was blind to the diagnosis. The interval between administration of the scales was ~5 min. Following screening, patients were administered a battery of neuropsychologist tests and the Lawton Instrumental Activities of Daily Living (IADL) scale () scale to support a clinician to make a diagnosis and classify patients into those with normal cognition, PSCIND, and PSD. The diagnostic classification was made independent of the results of the cognitive screening instruments. The following neuropsychological tests were used to assess cognitive function: (1) The Auditory Verbal Learning Test–H tested memory including immediate and delayed recall (); (2) The Rey-Osterrieth Complex Figure Test (RCFT) was used to examine the visuospatial function domain (, ); (3) Animal Fluency Test was used to assess language ();(4) and the Chinese modified version of the Trail Making Test (TMT-A, TMT-B) was used to examine executive function (). The caregiver burden inventory was also administered to the family or nurse of the patient to assess carer strain (). These widely-used scales were chosen as they have good reliability and validity. PSCIND was diagnosed among those with decline in at least 1 cognitive domain (memory, visuospatial function, language and executive function) but normal or subtle functional IADL decline related to cognitive decline rather than stroke-related (). PSD was diagnosed if at least 2 out of 4 cognitive function domains were impaired on the neuropsychologist tests with ADL impairment attributable to cognitive decline rather than stroke-related physical disability.
Statistical Analyses
Statistical analyses were performed with SPSS 20.0 and MEDCALC® 19.2.0. The Chi-squared (χ2) test was used to investigate differences between categorical variables among groups. Normality was tested with Shapiro-Wilk test. If variables were normally distributed, then Levene's test for homogeneity of variance was performed. Variables with homogeneity of variance, were compared using a student t-test or one-way analysis of variance (ANOVA) for differences between three or more groups (NC, PSCIND and PSD). If data weren't normally distributed, the Kruskal-Wallis H test was used to test for differences between the 3 diagnostic groups. Significance was set a level of 0.05. Receiver operating characteristic (ROC) curve analysis was used to assess diagnostic accuracy based on the area under the curve (AUC). ROC curves were compared using the DeLong method (). Accuracy was excellent if AUC results were between 0.90 and 1.0, good if they were between 0.8 and 0.9, fair if between 0.7 and 0.8, poor between 0.6 and 0.7, and where values were found to be between 0.50 and 0.60 they were regarded as a fail. The optimal cut-off for each test was determined from Youden's Index (J = sensitivity + specificity−1). We estimated sample size using a precision-based calculation. Here as the expected prevalence of cognitive impairment (PSD or PSCIND) was ~70% based on existing studies () and the sensitivity and specificity of the Qmci-CN for detecting cognitive impairment is ~85% [based on existing studies of the instrument ()], it was estimated that between 70 and 164 patients would be required at a precision of 0.1 (10%) at a significance of 0.05 (α).
Results
In all, 230 patients with stroke were screened for the study, 125 patients with stroke were excluded as they met exclusion criteria, while 54 declined to participate. Hence, 51 patients with stroke were recruited. Among these, 17 patients did not complete the full assessment. Of the final sample of 34 patients included, 8 had normal cognition (NC), 15 were classified as having PSCIND and 11 patients were diagnosed with PSD. Patient characteristics are presented in Table 1. In all, 13 (38.2%) participants were female. The mean age of the total sample was 63 years, with a standard deviation (SD) of 13 years. The mean (±SD) age of those in the NC, PSCIND, and PSD group were 58 (± 13), 65 (±11), and 65 (±16) years, respectively. The mean (±SD) number of years in education of all participants was 13(±4) years, with a mean of 14(±5), 13(±3), and 10(±5) among the NC, PSCIND and PSD groups, respectively. There were no statistically significant differences in age (p = 0.338), sex (p = 0.254), or educational level (p = 0.306) between the 3 diagnostic groups. The interval between presentation with stroke and the assessment varied from 2 weeks up to 13 years.
Table 1
| Characteristics | All (n = 34) N (%) or Mean ± SD [Range] | NC (n = 8) N (%) or Mean ± SD [Range] | PSCIND (n = 15) N (%) or Mean ± SD [Range] | PSD (n = 11) N (%) or Mean ± SD [Range] | P = x |
|---|---|---|---|---|---|
| Gender | |||||
| Female | 13 (38.2%) | 1(12.5%) | 5(33.3%) | 6(54.5%) | 0.254 |
| Male | 21 (61.8%) | 7(87.5%) | 9(60%) | 5(45.5%) | |
| Age (years) | 63.38 ± 13.07 [31–85] | 57.75 ± 12.78 [31–71] | 65.4 ± 11.12 [45–79] | 64.73 ± 15.58 [43–85] | 0.338 |
| Education (years) | 12.56 ± 4.25 [0–19] | 14.25 ± 4.56 [6–19] | 13 ± 3.29 [6–17] | 10.74 ± 4.86 [0–19] | 0.306 |
| Living arrangements | |||||
| Living with family | 26 (76.5%) | 8 (100%) | 12 (80%) | 8 (90.9%) | 0.188 |
| Living with a formal carer | 6 (17.6%) | 0 (0%) | 3 (20%) | 2 (18.2%) | |
| Living alone | 2 (5.9%) | 0 (0%) | 0 (0%) | 1 (9.1%) | |
| Work intensity | |||||
| Low | 11 (32.4%) | 3 (37.5%) | 6 (40%) | 1 (9.1%) | 0.110 |
| Medium | 5 (14.7%) | 3 (37.5%) | 6 (40%) | 7 (63.6%) | |
| High | 5 (14.7%) | 2 (25%) | 2 (13.3%) | 1 (9.1%) | |
| Other (none, volunteer or not provided) | 13 (38.2%) | 0 (0%) | 1 (6.7%) | 2 (18.2%) | |
| Hypertension (proportion with) | 20 (58.8%) | 4 (50%) | 12 (80%) | 10 (90.0%) | 0.021* |
| Hyperglycaemia (proportion with) | 13 (38.2%) | 3 (37.5%) | 5 (33.3%) | 4 (36.4%) | 0.204 |
| Hyperlipemia (proportion with) | 9 (26.5%) | 1 (12.5%) | 8 (53.3%) | 1 (9.1%) | 0.034* |
| Dyssomnia (proportion with) | 7 (20.6%) | 2 (25%) | 4 (26.7%) | 1 (9.1%) | 0.202 |
Characteristics of the participants included in total (n = 34) and according to diagnosis classification.
NC, normal cognitive; PSCIND, post-stroke cognitive impairment no dementia; PSD, post-stroke dementia; SD, standard deviation.
Statistically significant (p < 0.05).
Cognitive Test Scoring and Administration
The median scores and administration times for each diagnostic group with their interquartile range (IQR) including comparisons between all 3 diagnostic groups and pair-wise comparisons are presented in Table 2. We found statistically significant differences in total median test scores between all 2 diagnostic groups (NC, PSCIND, and PSD) (p < 0.01). Analyses showed that all 3 diagnostic groups were different from each other, and higher tests scores related with higher level of cognitive ability. While the MoCA showed a clear gradient in median scores from NC to PSCIND and PSD (decreasing from 27 to 22 to 13), the Qmci-CN had similar median scores for NC and PSCIND. The MMSE-CN had similar values for PSCIND and PSD. Comparisons between administration times for the cognitive screens are presented in Table 3. The median (± IQR) administration time for the Qmci-CN was 328.50 ± 50.25 s vs. 610.87 ± 116.75 for the MoCA-CN, a statistically significant difference (p < 0.0001). The median administration time for the MMSE-CN was 280.17 ± 43.75, which was also shorter than the MoCA-CN (p < 0.0001). Examining administration times by diagnostic classification group showed that there were no significant differences for either the Qmci-CN (p = 0.144), MoCA-CN (p = 0.333), or MMSE-CN (p = 0.173). A moderate gradient effect was seen with the Qmci-CN (r = 0.52, p = 0.771); those with better cognition had higher scores, albeit these were not statistically significantly shorter.
Table 2
| Cognitive test (score/administration time) | Total sample (n = 34) | NC (n = 8) | PSCIND (n = 15) | PSD (n = 11) | Kruskal–Wallis H-test comparing all 3 groups (P = x) | NC vs. PSCIND (P = x) | NC vs. Post-stroke CI (PSCIND & PSD) (P = x) | PSCIND vs. PSD (P = x) |
|---|---|---|---|---|---|---|---|---|
| Qmci-CN scores (median ± IQR) | 55.50 ± 14.25 [0–69] | 57.50 ± 8.25 [48–66] | 57.00 ± 7.00 [47–69] | 37.00 ± 35.00 [4–57] | P < 0.001* | P = 0.548 | P = 0.074 | P < 0.001* |
| MoCA-CN scores (median ± IQR) | 22.00 ± 8.50 [1–30] | 27.00 ± 5.50 [22–30] | 22.00 ± 3.00 [15–27] | 13.00 ± 9.00 [1–22] | P < 0.001* | P = 0.007* | P < 0.001* | P < 0.001* |
| MMSE-CN scores (median ± IQR) | 25.00 ± 3.79 | 27.00 ± 1.00 | 24.00 ± 3.71 | 23.00 ± 17.00 | P = 0.001* | P = 0.040 | P = 0.003* | P = 0.011 |
| [0–30] | [25–29] | [23–30] | [0–25] | |||||
| Qmci-CN time (s) (median ± IQR) | 328.50 ± 50.25 [150–408] | 273.00 ± 91.00 [230–382] | 331.00 ± 41.00 [298–406] | 335.00 ± 45.00 [150–408] | P = 0.144 | P = 0.034 | P = 0.082 | P = 0.878 |
| MoCA-CN time (s) (median ± IQR) | 610.87 ± 116.75 [176–1023] | 580.50 ± 154.65 [357–633] | 617.00 ± 186.00 [454–791] | 610.87 ± 286.00 [176–1023] | P = 0.333 | P = 0.076 | P = 0.110 | P = 1.000 |
| MMSE-CN time(s) (median ± IQR) | 280.17 ± 43.75 | 255.00 ± 59.38 | 280.17 ± 15.00 | 280.17 ± 70.00 | P = 0.173 | P = 0.076 | P = 0.110 | P = 0.799 |
| [178–510] | [178–326] | [193–329] | [182–510] |
Test scores and administration times for the Chinese versions of the quick mild cognitive impairment screen (Qmci-CN), montreal cognitive assessment (MoCA-CN) and mini mental state examination (MMSE-CN) by diagnostic group.
CI, cognitive impairment; IQR, interquartile range; NC, normal cognition; PSCIND, post-stroke cognitive impairment no dementia; PSD, post-stroke dementia.
Statistically significant (p < 0.01).
Table 3
| Variable | Group 1 | Group 2 | Mean difference | Std. error | P-value | 95% Confidence interval | |
|---|---|---|---|---|---|---|---|
| Lower bound | Upper bound | ||||||
| Time taken | Qmci-CN | MoCA-CN | −291.753 | 30.567 | <0.0001 | −368.33 | −215.18 |
| Qmci-CN | MMSE-CN | 38.951 | 18.562 | 0.123 | −7.48 | 85.38 | |
| MoCA-CN | MMSE-CN | 330.704 | 33.434 | <0.0001 | 247.84 | 413.57 | |
ANOVA Post-hoc pair-wise analysis comparing administration times for the Qmci-CN screen, MoCA-CN, and MMSE-CN.
MMSE-CN, mini mental state examination; MoCA-CN, montreal cognitive assessment; Qmci-CN, quick mild cognitive impairment screen.
Screening for Post-stroke Cognitive Impairment (PSCIND and PSD)
Measures of diagnostic accuracy including optimal cut-offs for each instrument are presented in Table 4. Cross cross-tabulated results are provided as Supplementary Material. Comparisons between instruments are presented in Table 5. ROC curve analyses showed that the Qmci-CN, MoCA-CN and MMSE-CN had similar accuracy in separating PSD from NC; the MoCA-CN (AUC 0.994) had slightly higher accuracy but this was not statistically greater than the Qmci-CN (AUC 0.938) or MMSE-CN (AUC 0.983). At their optimal cut-offs, all 3 tests showed excellent sensitivity but poor specificity for the ability to separate PSD from NC. The Qmci-CN had an optimal cut-off score of ≤47 vs. ≤19.5 for the MoCA-CN and ≤26 for the MMSE-CN. The results also showed that the Qmci-CN, MMSE-CN, and MoCA-CN had statistically similar accuracy in separating PSCIND from PSD. The Qmci-CN had an AUC of 0.906 compared to an AUC of 0.936 for the MoCA-CN and an AUC of 0.791 for the MMSE-CN.
Table 4
| Diagnostic classification | Cognitive screen | AUC [95% CI] | Optimal cut-off point (youden index) | Sensitivity and specificity [%] |
|---|---|---|---|---|
| PSCIND vs. NC | Qmci-CN | 0.583 [0.336–0.831] | ≤55.5 | Sensitivity = 88%, Specificity = 53% |
| MoCA-CN | 0.842 [0.672–1.000] | ≤26 | Sensitivity = 63%, Specificity = 7% | |
| MMSE-CN | 0.763 [0.561–0.964] | ≤26.5 | Sensitivity = 88%, Specificity = 27% | |
| PSD vs. NC | Qmci-CN | 0.938 [0.833–1.000] | ≤47 | Sensitivity = 100%, Specificity = 18% |
| MoCA-CN | 0.994 [0.972–1.000] | ≤19.5 | Sensitivity = 100%, Specificity = 9% | |
| MMSE-CN | 0.983 [0.935–1.000] | ≤26 | Sensitivity = 88%, Specificity = 0% | |
| PSD vs. PSCIND | Qmci-CN | 0.909 [0.783–1.000] | ≤46.5 | Sensitivity = 100%, Specificity = 18% |
| MoCA-CN | 0.936 [0.839–1.000] | ≤18.5 | Sensitivity = 93%, Specificity = 9% | |
| MMSE-CN | 0.791 [0.613–0.969] | ≤21 | Sensitivity = 100%, Specificity = 55% | |
| Post stroke CI (PSCIND/PSD) vs. NC | Qmci-CN | 0.733 [0.558–0.909] | ≤55.5 | Sensitivity = 88%, Specificity = 39% |
| MoCA-CN | 0.906 [0.800–1.000] | ≤21.5 | Sensitivity = 100%, Specificity = 39% | |
| MMSE-CN | 0.856 [0.727–0.984] | ≤26.5 | Sensitivity = 88%, Specificity = 15% | |
| PSD vs. Post stroke CI (NC/PSCIND) | Qmci-CN | 0.919 [0.808–1.000] | ≤46.5 | Sensitivity = 100%, Specificity = 18% |
| MoCA-CN | 0.957 [0.889–1.000] | ≤18.5 | Sensitivity = 96%, Specificity = 9% | |
| MMSE-CN | 0.858 [0.730–0.985] | ≤23.64 | Sensitivity = 74%, Specificity = 27% |
Area Under the Curve (AUC) values and optimal cut-offs for the Chinese versions of the quick mild cognitive impairment (Qmci-CN) screen, montreal cognitive assessment (MoCA-CN), and mini mental state examination (MMSE-CN).
CI, cognitive impairment; NC, normal cognition; PSCIND, post-stroke cognitive impairment no dementia; PSD, post stroke dementia.
Table 5
| Diagnostic classification | Comparison group (area under the curve) | Z-statistic | P-value | |
|---|---|---|---|---|
| PSCIND vs. NC | Qmci-CN | MoCA-CN | 2.442 | 0.015* |
| (0.583) | (0.842) | |||
| Qmci-CN | MMSE-CN | 1.457 | 0.145 | |
| (0.583) | (0.763) | |||
| MoCA-CN | MMSE-CN | 0.747 | 0.455 | |
| (0.842) | (0.763) | |||
| PSD vs. NC | Qmci-CN | MoCA-CN | 0.517 | 0.237 |
| (0.938) | (0.994) | |||
| Qmci-CN | MMSE-CN | 1.818 | 0.293 | |
| (0.938) | (0.983) | |||
| MoCA-CN | MMSE-CN | 1.866 | 0.406 | |
| (0.994) | (0.983) | |||
| PSD vs. PSCIND | Qmci-CN | MoCA-CN | 0.517 | 0.605 |
| (0.909) | (0.936) | |||
| Qmci-CN | MMSE-CN | 1.818 | 0.069 | |
| (0.909) | (0.791) | |||
| MoCA-CN | MMSE-CN | 1.866 | 0.062 | |
| (0.936) | (0.791) | |||
| Post stroke CI (PSCIND/PSD) vs. NC | Qmci-CN | MoCA-CN | 2.489 | 0.013* |
| (0.733) | (0.906) | |||
| Qmci-CN | MMSE-CN | 1.594 | 0.111 | |
| (0.733) | (0.856) | |||
| MoCA-CN | MMSE-CN | 0.842 | 0.400 | |
| (0.906) | (0.856) | |||
| PSD vs. NC/PSCIND | Qmci-CN | MoCA-CN | 0.795 | 0.426 |
| (0.919) | (0.957) | |||
| Qmci-CN | MMSE-CN | 1.216 | 0.224 | |
| (0.919) | (0.858) | |||
| MoCA-CN | MMSE-CN | 1.821 | 0.069 | |
| (0.957) | (0.858) | |||
Pair-wise comparisons between the diagnostic accuracy of the Chinese versions of the quick mild cognitive impairment (Qmci-CN) screen, montreal cognitive assessment (MoCA-CN) and mini mental state examination (MMSE-CN).
CI, cognitive impairment; NC, normal cognition; PSCIND, post-stroke cognitive impairment no dementia; PSD, post-stroke dementia.
Statistically Significant.
ROC analysis showed that the Qmci-CN had an AUC of 0.733 compared with AUCs of 0.906 and 0.856 for the MoCA-CN and MMSE-CN, respectively, in separating PSCI (either PSCIND/PSD) from NC. The Qmci-CN was less accrate than the MoCA-CN (p = 0.013) but was similar to the MMSE-CN (p = 0.11) in differentiating PSCI (either PSCIND/PSD) from NC. The Qmci-CN was also less accurate in its ability to distinguish PSCIND from NC (AUC 0.583) compared with the MoCA-CN (AUC of 0.842), (p = 0.015). ROC curves are presented in Figure 1.
Figure 1
Discussion
This study compared the diagnostic accuracy of the newly-developed Qmci-CN for its ability to screen for PSCI with the widely-used MoCA-CN and MMSE-CN in a rehabilitation population in China. This is, to our knowledge, the first study validating the Qmci screen in a post-stroke population in any language. The results showed that all three of these scales are equally able to differentiate PSD from NC and PSD from PSCIND. The Qmci-CN had excellent accuracy (AUC > 0.90) in separating both diagnostic groups but poor accuracy in distinguishing PSCIND from NC. While the MoCA-CN was significantly more accurate in separating post-stroke CI (either PSCIND or PSD) from normal and in turn PSCIND from NC than the Qmci-CN, it had a statistically significantly longer median administration time. The MoCA-CN took almost twice as long to score. This is a consistent finding in studies comparing the two instruments (). It also had similar accuracy to the MMSE-CN but again longer administration times. The Qmci-CN and MMSE-CN had similar administration times in this study.
These results are different from previous research examining the diagnostic accuracy of other language versions of the Qmci, especially the English version, which show that the Qmci screen is more accurate in discriminating MCI (equivalent of PSCIND) from both normal controls and dementia than the MoCA and the MMSE (–). These studies also generally show that the MMSE is less accurate than both the MoCA and Qmci screen (). The reasons for why this study differs from others is most likely related to its very small sample size. Sample size calculations based on previous studies suggest that this study was not powered adequately to show superiority of one test over the other. The sample size in some of the diagnostic groups, particularly the NC group (N = 8), is too small to interpret the findings reliably. Further, those included in this study were a different cohort with all patients recovering from an acute stroke. These may have cognitive impairment in multiple other cognitive domains, while retaining memory (). The Qmci-CN is better able to detect amnestic type cognitive impairment () compared with other short screens as it is more heavily weighted toward memory. This said, the goal of this study was not to show superiority of the Qmci-CN but to examine its utility and psychometric properties in patients post-stroke compared with more-commonly used screening tests.
This study also examines the optimal cut-off points for the 3 instruments to identify NC, PSCIND, and PSD. At the established cut-off for cognitive impairment (≤26) rather than post-stroke cognitive impairment (≤22) (), the MoCA had excellent sensitivity but poor specificity. While the MMSE may lack sensitivity for single domain cognitive impairment (), using the optimal cut-off points identified using Youden's Index, this study found that the MMSE had a similarly high sensitivity but low specificity. The optimal cut-off point for the Qmci-CN in separating PSCIND from NC was ≤55.5 compared with ≤47 to differentiate PSD from normal. While lower than the results in Irish () and Canadian () cohorts in patients attending memory clinics, these are more similar to results in Turkey (<53 for separating MCI from normal cognition) (, ) and recently in Greece (<51 for separating MCI from normal cognition) (). Possible reasons for this difference might include the background of participants including their educational level, albeit the sample included here had a similar median number of years in education to those in North America and Western Europe and higher than those in Turkey and Greece. Again the setting may have influenced the cut-offs as all patients here were recruited from rehabilitation clinics having had a recent acute stroke. This may have influenced the performance of patients in testing across several cognitive domains resulting in lower total scores, irrespective of diagnostic classification. However, given that this is a small sample, care should be taken when interpreting and applying these cut-offs. The authors caution that normative data are required to develop accurate cut-off scores for the Qmci-CN and to determine if these are comparable with the English language version of the Qmci screen.
Limitations
This study has a number of limitations as follows. First, the definitions of PSCI in the literature and in clinical practice are variable, differing from study to study (). In our research, we defined PSCI as a post-stroke neuropsychological syndrome with any cognitive impairment, which develops following a stroke event, including the subgroups of PSD and PSCIND. This definition does not suggest any particular underlying neuropathological process. Other definitions include time limits on the onset of cognitive decline, though we did not apply this. This is a limitation. For example, a national Korean study examining the epidemiology of vascular cognitive defined PSD as any major cognitive impairment evident more than 3 months after a stroke (). Others suggest the final diagnosis of PSD should be delayed until at least 6 months as stroke commonly results in delirium or transient reductions in cognition that do not persist. Hence, making the diagnosis too early may result in a higher prevalence of PSCI than would be expected (). Here no time interval was pre-specified and we recruited patients having completed a stroke at least 2 weeks but up to 13 years after the event. Hence, to ensure homogeneity and comparability, further study applying pre-specified time-based definitions of PSCI are therefore needed. Second, as the diagnostic criteria for PSCI incorporates ADL function and many patients with stoke have related physical impairment, it can be difficult to assess whether the changes in ADLs are truly related to cognitive impairment (). This may increase the heterogeneity among the 3 groups and is a potential confounder. Thirdly, as patients were all recruited from the department of rehabilitation medicine, most of these may have completed cognitive assessment before, with either the MoCA-CN and the MMSE-CN, potentially creating learning effects and introducing bias. This said, none of the patients would have been tested with the Qmci-CN while inpatients. Finally, the sample here is small and likely prone to selection bias, reducing the generalisability of the results and under-powering the study to show superiority of one screening instrument over another based on our sample size calculation, increasing the chance of type II errors. Only those who received rehabilitation were available and included, potentially reducing the generalisability of the findings. Hence, further studies examining these instruments in a broader sample of patients post-stroke including those who did not receive formal rehabilitation or were unlikely to benefit from rehabilitation should be recruited. As the sample size was small, recruitment in multiple sites over a longer period of time is also needed to increase the reliability, generalisability and ability to confirm superiority of one instrument over another.
Conclusion
In conclusion, the Qmci-CN was accurate in screening for PSD in a post-stroke rehabilitation clinic and had excellent accuracy in separating PSD from PSCIND and PSD from NC. The administration time of the Qmci-CN was significantly lower than the MoCA-CN, suggesting it is a reasonable alternative in busy clinics. However, the Qmci-CN had poor accuracy in separating PSCIND from NC and PSCI from NC, suggesting it may lack accuracy for certain subgroups even if its administration time is shorter. The MoCA-CN and MMSE-CN both had similar accuracy in PSCI, which given the shorter administration time of the MMSE-CN suggests that it may be the better instrument to use in this setting. However, because this was a small study, under-powered to show superiority of one instrument over another, with several limitations, further study is now needed before recommending the routine use of the Qmci-CN in order to confirm these findings by recruiting a larger sample size and externally validating the Qmci-CN in other samples and populations.
Statements
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by this study received ethical approval from The Sixth Affiliated Hospital of Sun Yat-sen University, reference number 2019ZSLYEC-110. The patients/participants provided their written informed consent to participate in this study.
Author contributions
YX and LY: design, concept, data collection, analysis, writing, and revising manuscript. YW: design, concept, and supervision. YL, SP, WW, WL, PC, WZ, YD, SG, and LS: concept and data collection. YX and LY: statistical analysis and supervision of statistics. DM: design and concept. RO'C: design, concept, analysis, writing, and revising manuscript. All authors contributed to the article and approved the submitted version.
Funding
The work described in this manuscript was fully supported by a grant from the Guangzhou Municipal Technological Major Tackling Plan Modern Industrial Technology Project of China (Grant No. 201802010039), the Guangdong Hopson-Pearl River Education Development Foundation (Grant No. H20190116202012724) and two grants from the National Natural Science Foundation of China (Grant Nos. 81601981 and 81472155).
Conflict of interest
DM and RO'C are co-copyright holders of the Qmci screen. DM, RO'C, and YX are copyright holders of the translation into Chinese, the Qmci-CN screen. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2021.608188/full#supplementary-material
References
1.
FeiginVLLawesCMBennettDABarker-ColloSLParagV. Worldwide stroke incidence and early case fatality reported in 56 population-based studies: a systematic review. Lancet Neurol. (2009) 8:355–69. 10.1016/S1474-4422(09)70025-0
2.
PendleburySTRothwellPM. Prevalence, incidence, and factors associated with pre-stroke and post-stroke dementia: a systematic review and meta-analysis. Lancet Neurol. (2009) 8:1006–18. 10.1016/S1474-4422(09)70236-4
3.
LiuMWuBWangWZLeeLMZhangSHKongLZ. Stroke in China: epidemiology, prevention, and management strategies. Lancet Neurol. (2007) 6:456–64. 10.1016/S1474-4422(07)70004-2
4.
Chinese Stroke Association. Post-stroke cognitive impairment management expert committee. Expert consensus on Post-stroke cognitive impairment management. Chin J Stroke. (2017) 12:519–31. 10.3969/j.issn.1673-5765.2017.06.011
5.
QuYZhuoLLiNHuYChenWZhouYet al. Prevalence of post-stroke cognitive impairment in China: a community-based, crosssectional study. PLoS ONE. (2015) 10:e0122864. 10.1371/journal.pone.0122864
6.
LevineDAGaleckiATLangaKMUnverzagtFWKabetoMUGiordaniBet al. Trajectory of cognitive decline after incident stroke. JAMA. (2015) 314:41–51. 10.1001/jama.2015.6968
7.
LeesRFearonPHarrisonJKBroomfieldNMQuinnTJ. Cognitive and mood assessment in stroke research: focused review of contemporary studies. Stroke. (2012) 43:1678–80. 10.1161/STROKEAHA.112.653303
8.
McKevittCFudgeNRedfernJSheldenkarACrichtonSRuddARet al. Self-Reported long-term needs after stroke. Stroke. (2011) 42:1398–403. 10.1161/STROKEAHA.110.598839
9.
PollockASt GeorgeBFentonMFirkinsL. Top ten research priorities relating to life after stroke. Lancet Neurol. (2012) 11:209. 10.1016/S1474-4422(12)70029-7
10.
FrideYAdamitTMaeirABen AssayagEBornsteinNMKorczynADet al. What are the correlates of cognition and participation to return to work after first ever mild stroke?Top Stroke Rehabil. (2015) 22:317–25. 10.1179/1074935714Z.0000000013
11.
GoASMozaffarianDRogerVLBenjaminEJBerryJDet al. Heart disease and stroke statistics−2013 update: a report from the American heart association. Circulation. (2012) 127:e6–245. 10.1161/CIR.0b013e31828124ad
12.
BurtonLTysonS. Screening for cognitive impairment after stroke: a systematic review of psychometric properties and clinical utility. J Rehabil Med. (2015) 47:193–203. 10.2340/16501977-1930
13.
ChenKLXuYChuAQDingDLiangXNNasreddineZSet al. Validation of the Chinese version of montreal cognitive assessment basic for screening mild cognitive impairment. J Am Geriatr Soc. (2016) 64:e285–90. 10.1111/jgs.14530
14.
GhafarMZAAMiptahHNO'CaoimhR. Cognitive screening instruments to identify vascular cognitive impairment: a systematic review. Int J Geriatr Psychiatry. (2019) 34:1114–27. 10.1002/gps.5136
15.
O'CaoimhRMolloyDW. The quick mild cognitive impairment screen (Qmci). In: LarnerAJ, editor. Cognitive Screening Instruments. Cham: Springer. 10.1007/978-3-319-44775-9_12
16.
O'CaoimhRGaoYGallagherPFEustaceJMcGladeCMolloyDW. Which part of the quick mild cognitive impairment screen (Qmci) discriminates between normal cognition, mild cognitive impairment and dementia?Age Ageing. (2013) 42:324–30. 10.1093/ageing/aft044
17.
O'CaoimhRGaoYMcGladeCHealyLGallagherPTimmonsSet al. Comparison of the quick mild cognitive impairment (Qmci) screen and the SMMSE in screening for mild cognitive impairment. Age Ageing. (2012) 41:624–9. 10.1093/ageing/afs059
18.
BuntSO'CaoimhRKrijnenWPMolloyDWGoodijkGPvan der SchansCPet al. Validation of the dutch version of the quick mild cognitive impairment screen (Qmci-D). BMC Geriatr. (2015) 15:115. 10.1186/s12877-015-0113-1
19.
ClarnetteRGohMBharadwajSRyanJEllisSSvendrovskiAet al. Screening for cognitive impairment in an Australian aged care assessment team as part of comprehensive geriatric assessment. Aging Neuropsychol Cog. (2019) 26:336–47. 10.1080/13825585.2018.1439447
20.
O'CaoimhRTimmonsSMolloyDW. Screening for mild cognitive impairment: comparison of “MCI specific” screening instruments. J Alz Dis. (2016) 51:619–29. 10.3233/JAD-150881
21.
ClarnetteRO'CaoimhRAntonyDNSvendrovskiAMolloyDW. Comparison of the quick mild cognitive impairment (Qmci) screen to the montreal cognitive assessment (MoCA) in an Australian geriatrics clinic. Int J Geriatr Psychiatry. (2016) 32:643–9. 10.1002/gps.4505
22.
YavuzBBVaranHDO'CaoimhRKizilarslanogluMCKilicMKMolloyDWet al. Validation of the turkish version of the quick mild cognitive impairment screen. Am J Alzheimer's Dis Other Demen. (2017) 32:145–56. 10.1177/1533317517691122
23.
XuYLinYYiLLiZLiXYuYet al. Screening for cognitive frailty using short cognitive screening instruments: comparison of the Chinese versions of the MoCA and Qmci screen. Front Psychol. (2020) 11:558. 10.3389/fpsyg.2020.00558
24.
Wen-JiaJJingSJing-NianNI. Activities of daily living rating for differentiating mild cognitive impairment and dementia. Chin J Gerontol. (2014) 34:865–8. 10.3969/j.issn.1005-9202.2014.04.001
25.
QinjieLIYaMYuanZGeriatricsDO. Auditory verbal learning test-huashan version in the diagnosis of amnestic mild cognitive impairment. Geriatr Health Care. (2016) 16:282–285. 10.3969/j.issn.1008-8296.2016.05.06
26.
GuoQHLvCZHongZ. Application of rey-osterrieth complex figure test in Chinese normal old people. Chin J Clin Psychol. (2000) 8:205–8. 10.16128/j.cnki.1005-3611.2000.04.003
27.
YanZJun-ChaoLUQi-HaoG. Rey-osterriche complex figure test used to identify mild alzheimer's disease. Chin J Clin Neurosci. (2006) 14:501–4. 10.3969/j.issn.1008-0678.2006.05.010
28.
GuoQHYuanJHongZ. Application of eight executive tests in participants at shanghai communities. Chin J Behav Med Sci. (2007) 16:628–31. 10.3760/CMA.J.ISSN.1005-8559.2007.07.022
29.
ChouKRJiann-ChyunLChuH. The reliability and validity of the Chinese version of the caregiver burden inventory. Nurs Res. (2002) 51:324–31. 10.1097/00006199-200209000-00009
30.
DeLongERDeLongDMClarke-PearsonDL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837–45. 10.2307/2531595
31.
ErkinjunttiTGauthierS. Diagnosing vascular cognitive impairment and dementia: concepts and controversies. In: WahlundL-O, editor. Vascular Cognitive Impairment in Clinical Practice. Cambridge University Press. (2009). p. 3–10. 10.1017/CBO9780511575976.002
32.
GlynnKCoenRLawlorBA. Is the quick mild cognitive impairment screen (QMCI) more accurate at detecting mild cognitive impairment than existing short cognitive screening tests? A systematic review of the current literature. Int J Geriatr Psychiatr. (2019) 34:1739–46. 10.1002/gps.5201
33.
LeesRSelvarajahJFentonCPendleburySTLanghornePStottDJet al. Test accuracy of cognitive screening tests for diagnosis of dementia and multidomain cognitive impairment in stroke. Stroke. (2014) 45:3008–18. 10.1161/STROKEAHA.114.005842
34.
O'CaoimhRGaoYGallagherPEustaceJMolloyDW. Comparing approaches to optimize cut-off scores for short cognitive screening instruments in mild cognitive impairment and dementia. J Alzheimers Dis. (2017) 57:123–33. 10.3233/JAD-161204
35.
VaranHDKizilarslanogluMCBalciCDenizOCoteliSDogrulRTet al. Comparison of the accuracy of short cognitive screens among adults with cognitive complaints in Turkey. Alzheimer Dis Assis Disord. (2020) 34:350–6. 10.1097/WAD.0000000000000391
36.
MessinisLO'DonovanMRMolloyDWMougiasANasiosAGPapathanasopoulosPet al. Comparison of the Greek version of the quick mild cognitive impairment screen and standardised mini-mental state examination. Arch Clin Neuropsychol. (2020). 10.1093/arclin/acaa062. [Epub ahead of print].
37.
RitchieCWTerreraGQuinnTJ. Dementia trials and dementia tribulations: methodological and analytical challenges in dementia research. Alzheimer Res Ther. (2015) 7:31. 10.1186/s13195-015-0113-6
38.
YuKHChoSJOhMSJungSLeeJHShinJHet al. Cognitive impairment evaluated with vascular cognitive impairment harmonization standards in a multicenter prospective stroke cohort in Korea. Stroke. (2013) 44:786–8. 10.1161/STROKEAHA.112.668343
39.
Ihle-HansenHThommessenBBruunWyller TEngedalKØksengårdARStensetVet al. Incidence and Subtypes of MCI and dementia 1 year after first-ever stroke in patients without pre-existing cognitive impairment. Dement Geriatr Cogn Disord. (2011) 32:401–7. 10.1159/000335361
Summary
Keywords
cognition screen, mild cognitive impairment, post-stroke dementia, China, stroke, Qmci-CN
Citation
Xu Y, Yi L, Lin Y, Peng S, Wang W, Lin W, Chen P, Zhang W, Deng Y, Guo S, Shi L, Wang Y, Molloy DW and O'Caoimh R (2021) Screening for Cognitive Impairment After Stroke: Validation of the Chinese Version of the Quick Mild Cognitive Impairment Screen. Front. Neurol. 12:608188. doi: 10.3389/fneur.2021.608188
Received
30 September 2020
Accepted
11 February 2021
Published
05 March 2021
Volume
12 - 2021
Edited by
Robert Zivadinov, University at Buffalo, United States
Reviewed by
Jelle Demeestere, University Hospitals Leuven, Belgium; Dejan Jakimovski, Buffalo Neuroimaging Analysis Center, United States
Updates
Copyright
© 2021 Xu, Yi, Lin, Peng, Wang, Lin, Chen, Zhang, Deng, Guo, Shi, Wang, Molloy and O'Caoimh.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yuling Wang wangyul@mail.sysu.edu.cnRónán O'Caoimh rocaoimh@hotmail.com
This article was submitted to Stroke, a section of the journal Frontiers in Neurology
†These authors share first authorship
‡These authors share senior authorship
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.