Modeling heterogeneity in cognitive trajectories in the Framingham Heart Study

Introduction: The prevalence of cognitive impairment in the population is growing; however, there is substantial heterogeneity in the rate of decline across different cognitive domains. Harmonized factor scores measuring memory, executive function, and language domains have been created in the Framingham Heart Study (FHS).

Methods: This work identified FHS participants with two or more repeated factor scores after age 60 and fitted latent class mixed models (LCMM) to cluster cognitive trajectories within each domain. Non-linear shapes of trajectories were modeled piecewise linearly, followed by stepwise selections to select cluster-specific change points.

Results: We identified different latent classes of participants with early cognitive decline, compared to late decliners, for each domain. Ten-fold cross-validation yielded stable subgroupings. Our findings show latent-class-related differential patterns in cognitive aging in the FHS. We also investigated the association between identified latent classes with existing protein biomarkers of cognitive aging in a subsample of the study and found elevated levels of CD40L and CD14 were associated with a higher risk of early decline in memory and executive function domain, respectively.

Discussion: In summary, our study advances the understanding of cognitive decline heterogeneity among FHS participants and sets the stage for further investigations into early intervention strategies and personalized approaches to mitigate cognitive aging risks.

1 Introduction

The number of adults aged 65 and older in the United States is expected to more than double in the next 40 years (Vincent, 2010). This demographic transition raises concerns about a significant rise in cognitive impairment and dementia among this population. Maintaining good cognitive health stands as a pivotal determinant of successful aging, self-sufficiency, and well-being (Gorelick et al., 2017). The objective evaluation of cognitive function involves the utilization of neuropsychological (NP) assessments, facilitating the longitudinal tracking of cognitive performance. Modeling of the longitudinal trajectories of cognitive performance with aging has been well documented in existing literature (Bangen et al., 2019; Hu et al., 2019; Liu et al., 2013; Proust-Lima et al., 2013; Steinerman et al., 2010; van der Willik et al., 2020). Although it is conventionally acknowledged that advanced age generally corresponds to a decline in cognitive function (Harada et al., 2013; Salthouse, 2019), there is a noteworthy degree of heterogeneity in the rate of cognitive decline observed within cohorts of older adults of the same age (Wilson et al., 2002). Substantial heterogeneity in the pace of cognitive decline across different cognitive domains has been observed as well (Goh et al., 2012; Lövdén et al., 2005; Small et al., 2011). Variability in cognitive decline within and across different cognitive domains has been related to different likelihoods of progression to Alzheimer’s disease (AD) (Cloutier et al., 2015). While numerous statistical/machine learning techniques have been utilized to discover risk stratification or disease progression subgroups based on participants’ demographic and biomarker profiles and subsequently offering post-hoc summaries of cognitive trajectories within each subgroup (Dong et al., 2017; Racine et al., 2016; Young et al., 2018), limited efforts have been directed toward directly clustering the trajectories of cognitive functions.

The inherent patterns of change in specific cognitive domains might suggest potential subtypes of AD and other dementias. The identification of latent classes in cognitive trajectories with aging thus may provide insights into early disease course modification before dementia onset or prior to clinical symptoms (Howlett et al., 2021). In addition, by clustering the cognitive trajectories directly, individuals can be placed into more homogenous subgroups, which may also facilitate the recognition of latent groups with an elevated risk of progressing to AD dementia. Such latent subgroupings could serve as a preclinical substitute for cognitive outcomes or offer valuable insights for participant categorization in AD prevention trials (Ritchie et al., 2016; Watts, 2018). Recently, latent class analysis has been directly applied to cognitive trajectories, revealing distinct subgroups of participants, each characterized by a unique pattern of cognitive function changes (Geifman et al., 2018; Howlett et al., 2021; Sebastiani et al., 2020; Villeneuve et al., 2019). While some prior studies have employed extensive neuropsychological batteries and constructed domain-level composite or factor scores (Geifman et al., 2018; Sebastiani et al., 2020), many have still focused on one or two cognitive domains, included smaller sizes, or relied on just two measurements per individual, which could oversimplify the modeling of cognitive trajectories by assuming a linear cognitive decline with age.

Identifying data sets with comprehensive longitudinal cognitive function assessment and choosing appropriate statistical analyses have represented the two major challenges in the pattern discovery of cognitive decline (Sebastiani et al., 2020). Various longitudinal cohorts have utilized distinct NP test batteries and protocols, which makes the combination of NP performance data across cohorts non-trivial. The Framingham Heart Study (FHS) is a large community-based study with multiple sub-cohorts, which has comprehensively phenotyped cognitive outcomes through repeated NP assessments conducted by trained examiners under strict quality control (Au et al., 2010). Recently, separate harmonized factor scores that measure memory, executive function and language domains have been established using structural equation modeling (Scollard et al., 2023). These scores facilitate the integration of longitudinal cognitive profiles across multiple FHS cohorts that had a growing test battery over time.

In this study, we aim to leverage the comprehensive set of NP tests administered to the FHS cohorts and implement the newly developed harmonized cognitive factor scores for memory, executive function, and language domains. Here we describe an innovative, flexible method for modeling longitudinal cognitive profiles within the FHS. It accommodates group-specific and domain-specific changes in cognitive decline rates at different ages, while clustering participants into latent subgroups based on their cognitive factor score trajectories. We characterize participants in each identified subgroup based on their distinct demographic profiles and examine the correlation between the identified subgroups and existing aging biomarkers, aiming to provide a more accurate and comprehensive approach to assessing cognitive aging.

2 Materials and methods

2.1 Study sample

The FHS is a community-based prospective cohort study, which recruited 5,209 participants as the Original cohort (Gen 1) in 1948 to investigate the risk of cardiovascular disease (CVD) (Dawber and Kannel, 1966). The Offspring cohort (Gen 2), recruited in 1971, consists of 5,124 participants who have at least one parent in the Original cohort and some of their spouses (Feinleib et al., 1975; Kannel et al., 1979). In 1994 the FHS further recruited 506 multi-ethnic participants as the Omni 1 cohort to reflect the greater racial and ethnic diversity in the town of Framingham (Tsao and Vasan, 2015). All cohorts have received routine examinations for CVD and related risk factors. The Gen 1 participants were invited to take a battery of neuropsychological (NP) tests between examinations 14 and 15 (1976–1978) (Elias et al., 1997). The initial NP battery administered to Gen 1 participants was at a core visit, while starting in 1981, NP batteries were administered during ancillary exams (Elias et al., 1997). The Gen 2 and Omni 1 cohorts were also invited to undergo NP testing starting 1999 and 2000, between their regular examinations 7–8 and 2–3, respectively, and repeatedly around every 5 years during ancillary exams (Au et al., 2010). However, the protocol until 1999 was to only administer the NP battery to those flagged as potentially cognitively impaired (Scollard et al., 2023), therefore, the Gen 1 participants who experienced cognitive decline were expected to have more frequent NP tests. All FHS participants were invited to take regular follow-up NP tests to help detect changes in their cognitive functioning but not all chose to attend the additional visits (Au et al., 2010; Satizabal et al., 2016; Scollard et al., 2023).

In this study, we included 2,339 FHS participants, 539 from Gen 1, 1,708 from Gen 2, and 92 from Omni 1, to investigate subgroups of differential patterns in the longitudinal cognitive function changes. We included data obtained after the age of 60 years from each participant and excluded participants with only one NP test visit after age 60, with a prevalent dementia diagnosis at their baseline visit for this study (first visit at or after age 60), or those without education data. The resulting total number of time points to be studied is 7,939 from 2,339 participants. Details of sample selection can be found in the flow chart in Figure 1. All participants provided written informed consent at each attended examination and at each NP testing; FHS exams are reviewed by the Institutional Review Board at Boston University Medical Center.

FIGURE 1

Figure 1. Sample selection flowchart.

2.2 Overview of the neuropsychological tests and domain factor scores in the FHS

All participants from the FHS cohorts were invited to undergo a comprehensive battery of NP tests although not all chose to attend. The NP tests were carried out by trained psychometricians following standard administration protocols. This initiative was undertaken as part of a larger study with the primary objective of establishing foundational measures of brain structure and cognition (Au et al., 2010). Detailed information concerning the NP test batteries employed in the FHS cohorts could be found in the existing literature (Au et al., 2010; Elias et al., 1997; Satizabal et al., 2016; Scollard et al., 2023).

Broadly, NP tests included in the battery are representative of the following four different cognitive domains: verbal and visual episodic memory, attention and executive function, language, and visuospatial abilities. Monitoring NP test performance provided a way of measuring and monitoring cognitive changes. However, different cohorts in the FHS went through different waves of NP test batteries. The NP battery in the FHS has grown over time. The initial battery administered to the Gen 1 cohort included fewer tests than the batteries the Gen 2, Omni 1 cohorts underwent, leading to difficulties in combining or comparing across the FHS cohorts (Scollard et al., 2023). To address this problem, calibrated and harmonized factor scores for the memory, executive function, and language domains were developed based on NP test batteries and items from Mini-Mental State Examination (MMSE) or Consortium to Establish a Registry for Alzheimer’s Disease, using data across all FHS NP test visits (Scollard et al., 2023). The NP tests and MMSE components included in developing each harmonized domain factor score for the memory, executive function, and language domains can be found in the developing paper (Scollard et al., 2023). Due to insufficient NP tests in the visuospatial domain, no harmonized factor score was composed for this domain. These calibrated and harmonized domain factor scores (referred to as the “factor scores” hereafter), quantify cognitive performance on the same scale across the cohorts, and minimize ceiling effects observed in raw NP test performance, facilitating pooled analyses across the FHS cohorts (Scollard et al., 2023).

2.3 Statistical analysis

2.3.1 The piecewise linear latent-class mixed effect models

For each of the three cognitive domains (memory, executive function, and language) with available longitudinal factor scores, we identified distinct trajectory classes using piecewise linear latent-class mixed effect models (LCMMs). The linear mixed-effect models (LMM) (Laird and Ware, 1982) is a standard statistical procedure to model trajectories of changes while allowing for both within-subject and between-subject variation. The LCMM is a finite mixture of LMMs, where the responses are assumed to be from a mixture of Gaussian distributions with group means described by LMM regression functions (Muthén and Shedden, 1999; Proust and Jacqmin-Gadda, 2005). As a model-based clustering framework, an LCMM can classify participants based on the shape of the trajectories of interest, and has been recently applied to model the heterogeneities in cognitive decline, dementia, and preclinical Alzheimer’s Diseases in several longitudinal cohorts (Geifman et al., 2018; Gonzales et al., 2020; Howlett et al., 2021; Proust-Lima et al., 2016; Villeneuve et al., 2019) and to study differential progression patterns in other diseases (Fang et al., 2023).

Cognitive decline is a complex process and typically shows a non-linear trend along age or time in the study, hence only fitting linear effects with age or time would over-simplify the model. While LCMMs are a flexible tool for identifying heterogeneous longitudinal patterns, caution is necessary when the trajectories are non-linear. One common practice is to assume a polynomial function of time (Fang et al., 2023; Proust-Lima et al., 2016). However, the polynomial-time-function term may also force the mean trajectory profile shapes to not make practical sense. Additionally, LCMMs typically suffers from computational convergence issues, especially when the number of model parameters to be estimated is large. To address these two issues, we proposed a piecewise linear LCMM to allow for group-specific change points and assume a linear change in outcome over the time segments with different slopes before and after the change points. We used a small, predefined change points sets to reduce the number of parameters to estimate and avoid the computational convergence issue. These age thresholds were selected based on clinical knowledge and prior literature suggesting inflection points in cognitive aging and preclinical Alzheimer’s disease progression (Salthouse, 2019; Sperling et al., 2011).

2.3.2 Model-building and selection

For each of the three cognitive domains, we followed the same procedure to build a piecewise linear LCMM. Mathematical details of the construction of the model can be found in Supplementary material section S1. Details for model specification of the piecewise linear LCMMs can be found in Supplementary material section S2. Briefly, we built models for the longitudinal factor scores with two sets of predefined change points, one at ages of 65, 75, and 85 years and another at ages of 70, 80, and 90 years. We included group-specific fixed effects for age, sex, and education level in all models. The age terms were broken down to piecewise linear with changes of slopes before and after the prespecified change points. We included individual random effects to allow individual trajectories to vary from the group-specific mean. We fit models for G = 1–4 latent groups, yielding a total of 8 models (G groups times change points at the two sets of ages). Models with more than 4 groups often failed to converge or yielded small subgroups with insufficient samples for downstream characterization and comparison and unstable parameter estimates, which limited their interpretability. For each of the 8 complete models, we used a backward selection algorithm, with details described in Supplementary material section S3, to remove unneeded change points.

LCMMs are finite mixtures of regression models, and the true number of components G is not known. We used the Bayesian Information Criterion (BIC) (Schwarz, 1978) to select the final best model from among the 8 models. The flexibility introduced by combining group-specific fixed effects and individual random effects with the backward selection method ultimately allows for the selection of group-specific change points for different cognitive domains.

2.3.3 Sensitivity analysis: 10-fold cross-validation for model robustness

To evaluate the robustness of the selected change points, the piecewise linear structure, and the identified number of groups, we performed a sensitivity analysis using 10-fold cross-validation for each of the three cognitive function domains. We randomly partitioned the study participants into 10 subsamples. We used 9 portions as the training data to fit a piecewise linear LCMM described above, obtained the model with the best BIC; then, the left over 1 portion was treated as the testing data and the best model identified in the training data were applied to predict the latent classes. This training-testing split was repeated for 10 iterations until each subsample was treated as the testing data once. After the 10 training-testing model fittings and class predictions, the number of groups selected for the training data sets, as well as the median predicted soft classifications (posterior probability of belonging to the predicted classes) were summarized to determine whether the models selected by BIC were stable. We summarized the average adjusted Rand Index (ARI) (Hubert and Arabie, 1985) across the 10 testing sets comparing class membership predicted by the training model and the original model fitted using the whole dataset to determine on the robustness of the model fitted.

2.3.4 Association with existing plasma biomarkers

To validate that the latent classes identified using the proposed piecewise linear LCMM can be applied as an alternative cognitive outcome, we further investigated the association between the identified latent classes and previously measured plasma protein inflammatory biomarkers in a subsample of the Offspring cohort. At the Offspring cohort routine exam seven, 85 plasma proteins were measured using the Systems Approach to Biomarker Research (Ho et al., 2018). A previous study from our group established associations of several inflammatory protein biomarkers from these 85 with cognitive test performance, brain MRI volumes, and incident dementia (Fang et al., 2022). Here, we also examine the association between the inflammatory protein biomarkers and the identified latent classes in cognitive factor scores across the three domains using logistic regression models. We identified a sample of 1,617 Offspring cohort participants who attended the seventh routine examination, where the protein biomarkers were profiled. For each cognitive domain, the latent classes were grouped into a dichotomous response variable representing early fast decline (coded as 1) vs. steady or late fast decline (coded as 0). The protein biomarker levels were normalized to mean 0 and standard deviation 1. We employed a logistic regression model, adjusted for age at the cognitive test visit closest to exam 7 after age 60, the time difference between this visit and exam 7 in years, sex, education level, and APOE ϵ4 carrier status. A robust standard error estimate was used to account for familial correlation. We tested 5 protein biomarkers that showed significant association with cross-sectional cognitive scores in our previous study (Fang et al., 2022): monocyte differentiation antigen (CD14), CD5 molecule-like (CD5L), soluble CD40 ligand (CD40L), soluble receptor for advanced glycation end products (sRAGE), and myeloperoxidase (MPO). Additionally, a sensitivity analysis was performed on a subsample of 906 participants, whose cognitive test closest to exam 7 after age 60 was within 2 years after exam 7, using the same model and covariates.

2.3.5 Analysis software

All analyses were carried out using the statistical software R version 4.0.2. The piecewise linear LCMMs were fitted with the hlme function from package lcmm version 1.9.3. The backward selection algorithm was written and implemented in R. ARIs were calculated using the adjustedRandIndex function from the mclust package version 5.4.10. Associations between protein biomarkers and the latent classes were fitted using the glm function from the stats package together with the sandwich package version 3.1-0 and the lmtest package version 0.9-40 for the robust standard error estimation. All figures from the model were created using ggplot2 package version 3.3.6.

3 Results

3.1 Latent classes identified from clustering the trajectories of three cognitive domain factor scores

3.1.1 Latent classes identified from the memory factor score trajectories

Overall participant demographics are summarized in Table 1. Our piecewise linear LCMM model identified 4 latent subgroups based on the memory domain factor score trajectories. Figure 2 illustrates the predicted mean profile for each subgroup superposed on the actual trajectory of individuals within the corresponding subgroup. Each of the four identified latent subgroups exhibited distinct patterns, characterized by changes in rates of decline in the memory domain factor scores around the age of 70 (class 3, early decline group), 80 (class 2 late decline group), and 90 (class 1, latest decline group) years. In contrast, participants in class 4 (earliest decline group) demonstrated a steep decline starting around the age of 60, which is the baseline of the study.

TABLE 1

Table 1. Participant demographics for all participants.

FIGURE 2

Figure 2. Predicted mean trajectories superposed on observed trajectories for each latent subgroup identified in the memory domain. The gray shaded trajectories are the true observations, whereas the colored lines are the mean predicted trajectories from each subclass.

Table 2 provides a summary of the participant demographics for each identified latent class. Average baseline ages are similar for classes 1 and 3 (age 69 and 68, respectively), whereas the average age at the first exam was earlier for the earliest decline group class 4 (age 64) and later for class 2 (75 years). Cognitive status as measured by the MMSE was similar among all participants at the baseline exam. However, the proportions of APOE ϵ4 carriers, and of incident dementia and MCI in the two early decline groups (classes 3 and 4) were greater over the entire follow up period compared to the later decline groups. Participants from class 4, the earliest decline latent group identified, also shows on average the shortest time in years between each NP tests during the full follow-up, which is expected due to the protocol that participants experiencing cognitive impairment were tested more frequently.

TABLE 2

Table 2. Demographic for subgroups identified in all three domains.

Among the two later decline groups (classes 1 and 2), class 2 identified participants with generally older age at baseline, paired with a higher proportion of participants experiencing dementia or MCI onset during follow-up. The mean predicted decline rate for class 2 is similar to those observed for class 3 but occurred at a later age. The late decline group class 2 contains participants with the oldest baseline age and the lowest proportion of participants attended college (54%), which is an indication that it grouped more Original cohort participants. On the other hand, class 1, the largest group, included participants whose memory domain cognitive function remained relatively stable until the age of 90. This group is also characterized by the lowest proportion of APOE ϵ4 carriers, dementia or MCI onset, and the longest time between each NP visit.

3.1.2 Latent classes identified from the executive function factor score trajectories

Our modeling identified two latent classes based on the factor score trajectories from the executive function domain. Predicted mean trajectories for each latent subgroup identified overlaid on the observed actual individual trajectories in the executive function domain are visualized in Figure 3. Participant demographics for each latent subgroup are summarized in Table 2.

FIGURE 3

Figure 3. Predicted mean trajectories superposed on observed trajectories for each latent subgroup identified in the executive function domain. The gray shaded trajectories are the true observations, whereas the colored lines are the mean predicted trajectories from each subclass.

Class 1 (n = 2,302) was characterized by a relatively consistent decline in overall executive function performance, marked by subtle alterations in the rate of decline around ages of 65 and 85 years. In contrast, the much smaller class 2 (n = 36) had a rapid decline from the baseline age of 60 years. The average age of the earlier, rapid decline class 2 at baseline was 3 years younger than the later, slower decline class 1 group. The two classes had similar baseline MMSE scores, and other baseline characteristics of the participants between these two groups are also similar. However, the earlier, faster decline group (class 2) had a higher proportion of APOE ϵ4 carriers and greater proportion of incidence of dementia and MCI during follow-up. Additionally, participants identified within the early decline latent subgroup experienced more deaths during follow up, with 36% surviving to the end of 2019 (end of the follow up), compared to 58% of the slower decline group. Similar to observed in the memory domain, the group with older baseline age also included a lower overall education level.

3.1.3 Latent classes identified from the language factor score trajectories

Our modeling identified three latent subgroups when clustering based on the language domain factor score trajectories. Figure 4 visually represents the predicted mean trajectories and observed trajectories for each identified latent subgroup in the language domain, while participant demographics for each subgroup are summarized in Table 2.

FIGURE 4

Figure 4. Predicted mean trajectories superposed on observed trajectories for each latent subgroup identified in the language domain. The gray shaded trajectories are the true observations, whereas the colored lines are the mean predicted trajectories from each subclass.

Participants identified within the latent class 1 (n = 2,196) exhibit a consistent decline throughout the entire follow-up with small increases in decline rate around the ages of 70 and 90. Class 3 (n = 50) exhibited an early steep decline starting at the beginning of the observed age range (age 60 years) followed by a steeper decline after age 80 years. The participants in class 2 (n = 92) had no decline until age 80, and then an increased decline rate after age 90 years.

Baseline characteristics of participants in the three latent groups differed. The late decline group (class 2) had the lowest proportion of APOE ϵ4 carriers (14%) and the early decline group (class 3) had the highest proportion (22%), but these differences between classes were smaller than for the memory and executive function domains. Consistent with findings in other two domains, the early decline group has the highest proportion of participants who developed dementia or MCI during follow-up. However, the late decline group (class 2) had a higher rate of incident MCI and dementia than the consistent decline group. Different from observations from the memory and executive function domain, where higher proportions of attended college appears more in groups with younger baseline age, the late fast decline group in the language domain identified higher proportions of participants who attend college but with the oldest baseline age as compared to the other groups.

3.1.4 10-fold cross-validation

We employed a 10-fold cross-validation approach to assess the robustness of the identified subgroups within each cognitive domain. Results of the 10-fold cross-validation for the memory, executive function, and language domains are detailed in Supplementary Tables S1–S3, separately.

In the memory domain, 6 out of 10 rounds of fitting the training data resulted in a 4-group solution, consistent with the findings from fitting the entire dataset. In these 6 rounds, when applying the model obtained from the training data to the testing data, the predicted cluster membership closely aligned with the whole-data model, with ARIs close to 1. In the remaining 4 rounds, where fitting the training data did not select a 4-component model, the best model favored 3 subgroups. Across all 10 rounds, the median posterior probability for all observations exceeded 0.74, indicating a robust and definite classification.

In the executive function domain, the 2-component piecewise linear LCMM model identified when fitting the entire dataset demonstrated high robustness. Among the 10 cross-validation rounds, 9 yielded a 2-component model when fitting the training data. In 8 out of these 9 rounds, the predicted class memberships showed ARIs close to 1 when comparing the class labels obtained by applying training data model to the testing data and the class labels predicted from the whole-data model. The median posterior probabilities were consistently close to 1 across all rounds, suggesting a definitive classification. The single round where fitting training data did not select a 2-component model did not prefer other models either; however, computational issues prevent the convergence even with a one-component piecewise linear LMM.

In the language domain, the latent classes identified when fitting the full dataset also proved to be robust. All 10 cross-validation rounds result in 3-component models when fitting the training data. Among these rounds, 8 models obtained from the training data predict the testing data class labels similar to the full-data model with ARIs above 0.86 and an average close to 0.9. In addition, the median posterior probabilities are also close to 0.9, indicating high confidence in the classification. For the remaining 2 rounds, although the best model selected when fitting the training data are 3-group solutions, they differed from the 3-group model obtained when fitting the entire dataset. The ARI between classifications on the testing data are only around 0.3, indicating poor alignment and the median posterior probabilities also drop from around 0.9 to approximately 0.7.

3.1.5 Comparison of latent classes across cognitive domains

For each of the three cognitive domains we investigated, the proposed piecewise linear functions identified two overarching patterns of cognitive decline among the FHS participants. For each cognitive domain, one or two latent subgroups were characterized by an early rapid decline (class 2 and 3 in the memory domain, class 2 in the executive function domain, and class 3 in the language domain), while the other classes exhibited either steady decline or late-onset rapid decline (classes 1 and 4 in memory, class 1 in executive function, and classes 1 and 2 in language).

Table 3 presents crosstabulations of the identified latent subgroups across three cognitive domains. In all three comparisons, the majority of the late decline groups comprised the same participants across the domains. In contrast, the early decliners in each domain are not consistently the same participants. Specifically, about 61% of the early decliners in executive function domain also exhibited early signs of fast decline in their memory domain, and 44% of the early decliners in language cognitive function also similarly showed early signs of decline in their memory domain. However, only 10 participants demonstrated early decline in both executive function (28%) and language (20%).

TABLE 3

Table 3. Predicted class membership across memory, executive function, and language domains: a) memory domain vs. executive function domain; b) memory domain vs. language domain; c) executive function domain vs. language domain.

3.2 Association of clinical outcomes with the latent classes

3.2.1 Association of early vs. late decline with incident MCI and dementia

Table 4 summarizes the distribution of individuals across early decline groups (0, 1, 2, or all 3 domains) and compares this with the clinical diagnoses of MCI and dementia during follow up through 2019. A total of 2,221 participants (about 95% from the study sample) were not clustered into the early decline group for any domain. Of these participants, 77% did not develop MCI or dementia. Among the 81 participants who were identified as early decliners for one cognitive domain, 70% developed either MCI (19%) or dementia (51%) during follow-up. For the 27 participants clustered into early decline groups for 2 domains, 82% developed dementia, and 7% developed MCI without dementia. Only 9 participants across all domains were clustered into early decline groups for all three domains, and all these individuals were diagnosed with dementia during follow up.

TABLE 4

Table 4. The number of domains for which an individual was clustered into an early decline group vs. clinical diagnosis of MCI or dementia.

3.2.2 Association with existing plasma inflammatory protein biomarkers

In the Offspring cohort subsample of this study, we investigated the association between a set of 5 protein biomarkers that were significantly associated with NP performance in our previous study (Ho et al., 2018) and the dichotomized latent classes (early rapid decline vs. late or steady decline) for each of the three cognitive domains adjusting for age, sex, education level, and APOE ϵ4 carrier status. Tables 5, 6 summarize the estimated odds ratios, standard errors, and p-values for the associations between the 5 protein biomarkers and the subclass membership for the memory domain and the executive function domain, respectively. Results for the language domain are summarized in Supplementary Table S4.

TABLE 5

Table 5. Association of the protein biomarkers with the piecewise linear LCMM identified subclasses as early decline vs. late decline in the memory domain.

TABLE 6

Table 6. Association of the protein biomarkers with the piecewise linear LCMM identified subclasses as early decline vs. late decline in the executive function domain.

For the memory domain, higher CD40L was nominally associated with higher odds of being clustered into the early decline group (estimated odds ratio 1.42). This trend persisted but was not significant in the smaller sensitivity analyses sample with ages closer to those at exam 7. For the executive function domain, while no signal was observed from the main analysis sample, higher CD14 exhibited a nominally significant positive association in the subsample closer in age to exam 7, indicating higher odds of being clustered into the early decline group as compared to the late decline group (estimated odds ratio as 1.57). In addition, although not significant, higher CD40L showed a trend of association with lower odds (estimated odds ratio 0.56, p-value = 0.06) of being classified as early decliners in the executive function domain. No other significant associations were identified except for CD14 and CD40L.

4 Discussion

In this study, we described a flexible way of modeling the heterogeneity in cognitive decline among FHS participants from multiple cohorts. By utilizing the newly developed cognitive domain factor scores in the FHS, we combined participants across different FHS cohorts, ensuring a large sample size. The proposed method leverages the LCMM to cluster participants into more homogeneous groups based on their longitudinal profiles of cognitive factor scores. The piecewise linear component, together with change point selection, provides flexibility to account for group-specific and domain-specific non-linearity in cognitive decline. The identified subgroups are stable and robust, illustrated by 10-fold cross-validation.

Each identified subgroup exhibited distinct patterns of cognitive decline across memory, executive function, and language domains, characterized by unique participant demographics, including baseline age, education level, and APOE ϵ4 carrier status. Specifically, higher education level is enriched in the subgroup of participants with a late decline in their language domain cognitive functions. This aligns with the existing findings that education has positive effects on cognitive function and individuals with higher educational attainment are likely to experience less cognitive decline as they age (Lövdén et al., 2020; Zahodne et al., 2015).

While most literature studying heterogeneity in cognitive aging discovered subgroups with different rates of cognitive function decline (Howlett et al., 2021), ages of dementia or AD onset (Villeneuve et al., 2019), or studies of which cognitive domains are disproportionately affected, it is challenging to bring all aspect together. Our study is distinguished from the literature by its identification of subgroups with different time points at which rapid cognitive decline began. Moreover, different participants were identified for early or late decline subgroups across different cognitive domains, highlighting the heterogeneity of the process of cognitive decline. Importantly, the cognitive trajectory classes capture information about decline that is related to, but not the same as, the clinical diagnoses: despite the absence of a perfect alignment between the identified early or late decline groups and the diagnoses of dementia and MCI, higher incident dementia and incident MCI rates were observed when the number of domains in which an individual was clustered into the early decline group increased.

Understanding the long preclinical phase of AD is crucial for early intervention opportunities (Sperling et al., 2011). Linking heterogeneity in the extended preclinical phase with existing physical or biological markers of AD or related dementia may advance the knowledge of multiple pathways of cognitive aging. Our proposed flexible model, which results in the subgrouping of individuals based on their cognitive function trajectories, could provide a potential proxy measure for the onset of cognitive decline. Particularly, the early decline group identified through this model may function as a significant risk factor for preclinical AD or related dementia. Aiming to demonstrate the broader utility of these subgroupings, we also established connections with existing peripheral inflammatory biomarkers of cognitive aging. Our previous study observed cross-sectional correlations between higher levels of peripheral CD14, CD40L, and MPO; lower levels of peripheral sRAGE with lower scores in the WAIS-IV Similarities subtest, which was commonly used to measure cognitive performance in the executive function domain (Ho et al., 2018); and CD5L associated with brain volume biomarkers. In our current analysis, with the addition of longitudinal data and cognitive domain scores that harmonize information from separate tests, we have identified nominally significant additional associations: elevated levels of CD14 are associated with a higher risk of early cognitive decline in the executive function and CD40L are associated with higher risk for early decline in the memory domain, respectively.

Our study benefits from the large amount of longitudinal data the community-based FHS cohorts provide, and their comprehensive neuropsychological assessments together with the calibrated and harmonized domain global cognitive function scores. However, there are several limitations as well. First, the FHS participants are predominantly white. Therefore, our results may not be generalizable to wider populations. However, although accounting for a small proportion of study samples, our study includes participants from the multiethnic Omni 1 cohort. In addition, the modeling framework we chose, LCMM, typically suffers from computational convergence issues, especially when the number of model parameters to be estimated is large. Therefore, the determination of time points at which the rate of decline changes is pre-specified based on expert opinions and hence not identified among the data, which may affect the model’s generalizability as well. Moreover, while LCMM is designed to handle missing data under the assumption of missing at random, it does not explicitly account for dropout related to worsening cognitive function. However, the trajectories observed in our data show significant declines in cognitive scores over time, suggesting that this issue is not universal. Last, the modeling framework our study exploited is not prognostic but instead only exploratory. We investigated the heterogeneity in the longitudinal cognitive function profiles first, then summarized participants’ demographics and investigated the associations with existing biomarkers in a post-hoc manner. Therefore, in this post-hoc association analysis, the small sample size in the early decline groups, especially for the executive function domain, may lead to higher variability in effect size estimate and hence reduce the estimate stability and hinder the power of detecting significant association. Additionally, although we have selected the closest cognitive test time points included in the modeling of trajectory subgroups to the biomarker acquisition exam, the time differences between these two instances could be large and hence introduce additional variability. Due to the exploratory nature of our study, we did not apply multiple testing corrections to our association analyses with the five proteins, and reported nominally significant (p < 0.05) associations. Additional studies will be needed to confirm the reported associations and further validate our findings.

Recently, Li et al. (2022) proposed regularized finite mixture regression models to perform clustering for the identification of heterogeneity among the samples and enabled variable selections for the regression model components simultaneously, hence, the source of the heterogeneity can be identified. An interesting future direction of exploration could be to extend the regularized finite mixture regression model to incorporate both fixed effects and random effects for better handling longitudinal data and identifying sources of heterogeneity. This could also help build a prognostic model so that the groups characterized by unique cognitive decline patterns can be predicted using the selected corresponding features like participants demographics, APOE genotype, cardiovascular risk factors, and other existing AD or dementia biomarkers. Another future direction includes developing a framework for changepoint detection while clustering the time-course profiles in the meantime so that the computational burden from the traditional LCMM can be lifted and the change points can be identified from a data-driven approach.

In summary, our study offers a comprehensive exploration of cognitive decline heterogeneity among FHS participants, employing a flexible, robust, and reproducible model that clusters individuals based on longitudinal cognitive factor scores. Our study identifies class-related differential patterns in cognitive aging in the FHS, creates a proxy classification of cognitive decline subtypes among the FHS participants which are available for the wider scientific community to use in other projects. Overall, our study advances the understanding of cognitive decline heterogeneity and sets the stage for further investigations into early intervention strategies and personalized approaches to mitigate cognitive aging risks.

Data availability statement

Publicly available datasets from the Framingham Heart Study were analyzed in this study. This data can be found here: https://biolincc.nhlbi.nih.gov/studies/fhs/.

Ethics statement

The studies involving humans were approved by the Institutional Review Board at Boston University Medical Center. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

YF: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing. JC: Writing – review & editing. JM: Writing – review & editing. CS: Writing – review & editing. MA: Writing – review & editing. WQ: Writing – review & editing. MD: Funding acquisition, Writing – review & editing. JMM: Conceptualization, Funding acquisition, Writing – review & editing. KL: Conceptualization, Funding acquisition, Methodology, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This work was supported by the National Institute on Aging (R01AG06745) and the National Institute of Health (contract number U19-AG06875). The National Heart, Lung, and Blood Institute and the Boston University Chobanian and Avedisian School of Medicine, Framingham Heart Study (contract number 75N92019D00031).

Acknowledgments

We thank the Framingham Heart Study participants, as well as the study team for their contributions.

Conflict of interest

The author declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer CX declared a shared affiliation with the author(s) JC, JM, CS, MA, WQ, JMM, and KL to the handling editor at the time of review.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnagi.2025.1471154/full#supplementary-material

References

Au, R., Seshadri, S., Wolf, P. A., Elias, M., Elias, P., Sullivan, L., et al. (2010). New norms for a new generation: Cognitive performance in the framingham offspring cohort. Exp. Aging Res. 30, 333–358. doi: 10.1080/03610730490484380

PubMed Abstract | Crossref Full Text | Google Scholar

Bangen, K. J., Armstrong, N. M., Au, R., and Gross, A. L. (2019). Metabolic syndrome and cognitive trajectories in the framingham offspring study. J. Alzheimers Dis. 71, 931–943. doi: 10.3233/JAD-190261

PubMed Abstract | Crossref Full Text | Google Scholar

Cloutier, S., Chertkow, H., Kergoat, M. J., Gauthier, S., and Belleville, S. (2015). Patterns of cognitive decline prior to dementia in persons with mild cognitive impairment. J. Alzheimers Dis. 47, 901–913. doi: 10.3233/JAD-142910

PubMed Abstract | Crossref Full Text | Google Scholar

Dawber, T. R., and Kannel, W. B. (1966). The Framingham study. An epidemiological approach to coronary heart disease. Circulation 34, 553–555. doi: 10.1161/01.cir.34.4.553

PubMed Abstract | Crossref Full Text | Google Scholar

Dong, A., Toledo, J. B., Honnorat, N., Doshi, J., Varol, E., Sotiras, A., et al. (2017). Heterogeneity of neuroanatomical patterns in prodromal Alzheimer’s disease: Links to cognition, progression and biomarkers. Brain 140, 735–747. doi: 10.1093/brain/aww319

PubMed Abstract | Crossref Full Text | Google Scholar

Elias, M. F., Elias, P. K., D’Agostino, R. B., Silbershatz, H., and Wolf, P. A. (1997). Role of age, education, and gender on cognitive performance in the framingham heart study: Community-based norms. Exp. Aging Res. 23, 201–235. doi: 10.1080/03610739708254281

PubMed Abstract | Crossref Full Text | Google Scholar

Fang, Y., Doyle, M. F., Chen, J., Alosco, M. L., Mez, J., Satizabal, C. L., et al. (2022). Association between inflammatory biomarkers and cognitive aging. PLoS One 17:e0274350. doi: 10.1371/journal.pone.0274350

PubMed Abstract | Crossref Full Text | Google Scholar

Fang, Y., McDonald, C. M., Clemens, P. R., Gordish, H. D., Illei, K., Hoffman, E. P., et al. (2023). Modeling early heterogeneous rates of progression in boys with duchenne muscular dystrophy. J. Neuromuscul. Dis. 10, 349–364. doi: 10.3233/JND-221527

PubMed Abstract | Crossref Full Text | Google Scholar

Feinleib, M., Kannel, W. B., Garrison, R. J., McNamara, P. M., and Castelli, W. P. (1975). The framingham offspring study. design and preliminary data. Prev. Med. 4, 518–525. doi: 10.1016/0091-7435(75)90037-7

PubMed Abstract | Crossref Full Text | Google Scholar

Geifman, N., Kennedy, R. E., Schneider, L. S., Buchan, I., and Brinton, R. D. (2018). Data-driven identification of endophenotypes of Alzheimer’s disease progression: Implications for clinical trials and therapeutic interventions. Alzheimers Res. Ther. 10:4. doi: 10.1186/s13195-017-0332-0

PubMed Abstract | Crossref Full Text | Google Scholar

Goh, J. O., An, Y., and Resnick, S. M. (2012). Differential trajectories of age-related changes in components of executive and memory processes. Psychol. Aging 27, 707–719. doi: 10.1037/a0026715

PubMed Abstract | Crossref Full Text | Google Scholar

Gonzales, M. M., Wang, C. P., Quiben, M., MacCarthy, D., Seshadri, S., Jacob, M., et al. (2020). Joint trajectories of cognition and gait speed in Mexican American and European American older adults: The San Antonio longitudinal study of aging. Int. J. Geriatr. Psychiatry 35, 897–906. doi: 10.1002/gps.5310

PubMed Abstract | Crossref Full Text | Google Scholar

Gorelick, P. B., Furie, K. L., Iadecola, C., Smith, E. E., Waddy, S. P., Lloyd-Jones, D. M., et al. (2017). Defining optimal brain health in adults: A presidential advisory from the american heart association/american stroke association. Stroke 48, e284–e303. doi: 10.1161/STR.0000000000000148

PubMed Abstract | Crossref Full Text | Google Scholar

Harada, C. N., Natelson Love, M. C., and Triebel, K. L. (2013). Normal cognitive aging. Clin. Geriatr. Med. 29, 737–752. doi: 10.1016/j.cger.2013.07.002

PubMed Abstract | Crossref Full Text | Google Scholar

Ho, J. E., Lyass, A., Courchesne, P., Chen, G., Liu, C., Yin, X., et al. (2018). Protein biomarkers of cardiovascular disease and mortality in the community. J. Am. Heart Assoc. 7:e008108. doi: 10.1161/JAHA.117.008108

PubMed Abstract | Crossref Full Text | Google Scholar

Howlett, J., Hill, S. M., Ritchie, C. W., and Tom, B. D. M. (2021). Disease modelling of cognitive outcomes and biomarkers in the European prevention of Alzheimer’s dementia longitudinal cohort. Front. Big Data 4:676168. doi: 10.3389/fdata.2021.676168

PubMed Abstract | Crossref Full Text | Google Scholar

Hu, X., Gu, S., Sun, X., Gu, Y., Zhen, X., Li, Y., et al. (2019). Cognitive ageing trajectories and mortality of Chinese oldest-old. Arch. Gerontol. Geriatr. 82, 81–87. doi: 10.1016/j.archger.2019.01.018

PubMed Abstract | Crossref Full Text | Google Scholar

Hubert, L., and Arabie, P. (1985). Comparing partitions. J. Classif. 2, 193–218. doi: 10.1007/BF01908075

Crossref Full Text | Google Scholar

Kannel, W. B., Feinleib, M., McNamara, P. M., Garrison, R. J., and Castelli, W. P. (1979). An investigation of coronary heart disease in families. the framingham offspring study. Am. J. Epidemiol. 110, 281–290. doi: 10.1093/oxfordjournals.aje.a112813

PubMed Abstract | Crossref Full Text | Google Scholar

Laird, N. M., and Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics 38, 963–974. doi: 10.2307/2529876

Crossref Full Text | Google Scholar

Li, Y., Yu, C., Zhao, Y., Yao, W., Aseltine, R. H., and Chen, K. (2022). Pursuing sources of heterogeneity in modeling clustered population. Biometrics 78, 716–729. doi: 10.1111/biom.13434

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, W., Zhang, B., Zhang, Z., and Zhou, X. H. (2013). Joint modeling of transitional patterns of Alzheimer’s disease. PLoS One 8:e75487. doi: 10.1371/journal.pone.0075487

PubMed Abstract | Crossref Full Text | Google Scholar

Lövdén, M., Bergman, L., Adolfsson, R., Lindenberger, U., and Nilsson, L. G. (2005). Studying individual aging in an interindividual context: Typical paths of age-related, dementia-related, and mortality-related cognitive development in old age. Psychol. Aging 20, 303–316. doi: 10.1037/0882-7974.20.2.303

PubMed Abstract | Crossref Full Text | Google Scholar

Lövdén, M., Fratiglioni, L., Glymour, M. M., Lindenberger, U., and Tucker-Drob, E. M. (2020). Education and cognitive functioning across the life span. Psychol. Sci. Public Interest. 21, 6–41. doi: 10.1177/1529100620920576

PubMed Abstract | Crossref Full Text | Google Scholar

Muthén, B., and Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics 55, 463–469. doi: 10.1111/j.0006-341x.1999.00463.x

PubMed Abstract | Crossref Full Text | Google Scholar

Proust, C., and Jacqmin-Gadda, H. (2005). Estimation of linear mixed models with a mixture of distribution for the random effects. Comput. Methods Prog. Biomed. 78, 165–173. doi: 10.1016/j.cmpb.2004.12.004

PubMed Abstract | Crossref Full Text | Google Scholar

Proust-Lima, C., Amieva, H., and Jacqmin-Gadda, H. (2013). Analysis of multivariate mixed longitudinal data: A flexible latent process approach. Br. J. Math. Stat. Psychol. 66, 470–487. doi: 10.1111/bmsp.12000

PubMed Abstract | Crossref Full Text | Google Scholar

Proust-Lima, C., Dartigues, J. F., and Jacqmin-Gadda, H. (2016). Joint modeling of repeated multivariate cognitive measures and competing risks of dementia and death: A latent process and latent class approach. Stat. Med. 35, 382–398. doi: 10.1002/sim.6731

PubMed Abstract | Crossref Full Text | Google Scholar

Racine, A. M., Koscik, R. L., Berman, S. E., Nicholas, C. R., Clark, L. R., Okonkwo, O. C., et al. (2016). Biomarker clusters are differentially associated with longitudinal cognitive decline in late midlife. Brain 139(Pt 8), 2261–2274. doi: 10.1093/brain/aww142

PubMed Abstract | Crossref Full Text | Google Scholar

Ritchie, C. W., Molinuevo, J. L., Truyen, L., Satlin, A., Van der Geyten, S., Lovestone, S., et al. (2016). Development of interventions for the secondary prevention of Alzheimer’s dementia: The European Prevention of Alzheimer’s Dementia (EPAD) project. Lancet Psychiatry 3, 179–186. doi: 10.1016/S2215-0366(15)00454-X

PubMed Abstract | Crossref Full Text | Google Scholar

Salthouse, T. A. (2019). Trajectories of normal cognitive aging. Psychol. Aging 34, 17–24. doi: 10.1037/pag0000288

PubMed Abstract | Crossref Full Text | Google Scholar

Satizabal, C. L., Beiser, A. S., Chouraki, V., Chêne, G., Dufouil, C., and Seshadri, S. (2016). Incidence of dementia over three decades in the framingham heart study. N. Engl. J. Med. 374, 523–532. doi: 10.1056/NEJMoa1504327

PubMed Abstract | Crossref Full Text | Google Scholar

Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6, 461–464. doi: 10.1214/aos/1176344136

PubMed Abstract | Crossref Full Text | Google Scholar

Scollard, P., Choi, S. E., Lee, M. L., Mukherjee, S., Trittschuh, E. H., Sanders, R. E., et al. (2023). Ceiling effects and differential measurement precision across calibrated cognitive scores in the Framingham Study. Neuropsychology 37, 383–397. doi: 10.1037/neu0000828

PubMed Abstract | Crossref Full Text | Google Scholar

Sebastiani, P., Andersen, S. L., Sweigart, B., Du, M., Cosentino, S., Thyagarajan, B., et al. (2020). Patterns of multi-domain cognitive aging in participants of the long life family study. Geroscience 42, 1335–1350. doi: 10.1007/s11357-020-00202-3

PubMed Abstract | Crossref Full Text | Google Scholar

Small, B. J., Dixon, R. A., and McArdle, J. J. (2011). Tracking cognition-health changes from 55 to 95 years of age. J. Gerontol. B Psychol. Sci. Soc. Sci. 66, (Suppl. 1), i153–i161. doi: 10.1093/geronb/gbq093

PubMed Abstract | Crossref Full Text | Google Scholar

Sperling, R. A., Aisen, P. S., Beckett, L. A., Bennett, D. A., Craft, S., Fagan, A. M., et al. (2011). Toward defining the preclinical stages of Alzheimer’s disease: Recommendations from the national institute on Aging-Alzheimer’s association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 280–292. doi: 10.1016/j.jalz.2011.03.003

PubMed Abstract | Crossref Full Text | Google Scholar

Steinerman, J. R., Hall, C. B., Sliwinski, M. J., and Lipton, R. B. (2010). Modeling cognitive trajectories within longitudinal studies: A focus on older adults. J. Am. Geriatr. Soc. 58, (Suppl. 2), S313–S318. doi: 10.1111/j.1532-5415.2010.02982.x

PubMed Abstract | Crossref Full Text | Google Scholar

Tsao, C. W., and Vasan, R. S. (2015). Cohort profile: The Framingham Heart Study (FHS): Overview of milestones in cardiovascular epidemiology. Int. J. Epidemiol. 44, 1800–1813. doi: 10.1093/ije/dyv337

PubMed Abstract | Crossref Full Text | Google Scholar

van der Willik, K. D., Hauptmann, M., Jóźwiak, K., Vinke, E. J., Ruiter, R., Stricker, B. H., et al. (2020). Trajectories of cognitive function prior to cancer diagnosis: A population-based study. J. Natl. Cancer Inst. 112, 480–488. doi: 10.1093/jnci/djz178

PubMed Abstract | Crossref Full Text | Google Scholar

Villeneuve, S. C., Houot, M., Cacciamani, F., Verrijp, M., Dubois, B., Sikkes, S., et al. (2019). Latent class analysis identifies functional decline with Amsterdam IADL in preclinical Alzheimer’s disease. Alzheimers Dement (N Y). 5, 553–562. doi: 10.1016/j.trci.2019.08.009

PubMed Abstract | Crossref Full Text | Google Scholar

Vincent, G. K. (2010). The next four decades: The older population in the United States: 2010 to 2050. Washington, D.C: US Department of Commerce, Economics and Statistics Administration, US Census Bureau.

Google Scholar

Watts, G. (2018). Prospects for dementia research. Lancet 391:416. doi: 10.1016/S0140-6736(18)30190-9

PubMed Abstract | Crossref Full Text | Google Scholar

Wilson, R. S., Beckett, L. A., Barnes, L. L., Schneider, J. A., Bach, J., Evans, D. A., et al. (2002). Individual differences in rates of change in cognitive abilities of older persons. Psychol. Aging 17, 179–193. doi: 10.1037/0882-7974.17.2.179

Crossref Full Text | Google Scholar

Young, A. L., Marinescu, R. V., Oxtoby, N. P., Bocchetta, M., Yong, K., Firth, N. C., et al. (2018). Uncovering the heterogeneity and temporal complexity of neurodegenerative diseases with Subtype and Stage Inference. Nat. Commun. 9:4273. doi: 10.1038/s41467-018-05892-0

PubMed Abstract | Crossref Full Text | Google Scholar

Zahodne, L. B., Stern, Y., and Manly, J. J. (2015). Differing effects of education on cognitive decline in diverse elders with low versus high educational attainment. Neuropsychology 29, 649–657. doi: 10.1037/neu0000141

PubMed Abstract | Crossref Full Text | Google Scholar