Digital neuropsychological measures by defense automated neurocognitive assessment: reference values and clinical correlates

Introduction Although the growth of digital tools for cognitive health assessment, there’s a lack of known reference values and clinical implications for these digital methods. This study aims to establish reference values for digital neuropsychological measures obtained through the smartphone-based cognitive assessment application, Defense Automated Neurocognitive Assessment (DANA), and to identify clinical risk factors associated with these measures. Methods The sample included 932 cognitively intact participants from the Framingham Heart Study, who completed at least one DANA task. Participants were stratified into subgroups based on sex and three age groups. Reference values were established for digital cognitive assessments within each age group, divided by sex, at the 2.5th, 25th, 50th, 75th, and 97.5th percentile thresholds. To validate these values, 57 cognitively intact participants from Boston University Alzheimer’s Disease Research Center were included. Associations between 19 clinical risk factors and these digital neuropsychological measures were examined by a backward elimination strategy. Results Age- and sex-specific reference values were generated for three DANA tasks. Participants below 60 had median response times for the Go-No-Go task of 796 ms (men) and 823 ms (women), with age-related increases in both sexes. Validation cohort results mostly aligned with these references. Different tasks showed unique clinical correlations. For instance, response time in the Code Substitution task correlated positively with total cholesterol and diabetes, but negatively with high-density lipoprotein and low-density lipoprotein cholesterol levels, and triglycerides. Discussion This study established and validated reference values for digital neuropsychological measures of DANA in cognitively intact white participants, potentially improving their use in future clinical studies and practice.


Introduction
Alzheimer's disease (AD) is a debilitating neurodegenerative disorder characterized by progressive cognitive decline.It is a major public health concern, affecting millions of individuals worldwide (1).Unfortunately, to date, there is no definitive cure or highly effective treatment for AD.Given the lack of effective therapeutic options, early detection of the disease is of paramount importance.Timely diagnosis allows for interventions and strategies that can potentially slow down disease progression and improve the quality of life for affected individuals.As such, there is a growing recognition in the scientific and medical communities of the critical need for accurate and early detection methods in the fight against AD.
Traditional neuropsychological (NP) testing has been the primary method for measuring cognitive function, but digital tools have emerged as a promising and convenient alternative (2).Among these tools, the Defense Automated Neurobehavioral Assessment (DANA) stands out as an innovative mobile application developed for the evaluation of neurobehavioral functioning in military personnel (3).DANA was conceptualized to address the pressing need for efficient, objective, and reliable assessment methods capable of monitoring cognitive performance and detecting subtle changes in neurobehavioral functioning over time.Its early deployments within military contexts have yielded valuable insights into the impact of stress-related mental health issues, including conditions such as posttraumatic stress disorder (4), post-concussive symptoms (5), and the effects of blast wave exposure resulting from the use of heavy weapon systems (6).However, DANA's versatility quickly became apparent, leading to its adoption in a wide range of applications across clinical and research domains.Beyond its original military focus, DANA could be applied in diverse scenarios, including the assessment of traumatic brain injury (7), caregiver burden (8), hypoxic burden at high altitudes (9), and cognition changes in mental health treatments (10).The psychometric properties of the DANA test batteries have been thoroughly assessed and documented, confirming DANA as a reliable and valid tool for neuropsychological assessment (3,(11)(12)(13)(14).The adaptability of DANA has transformed it from a specialized military tool into a versatile instrument capable of investigating a wide spectrum of neurocognitive aspects.This expanded utility underscores its significance in addressing various cognitive health-related challenges and outcomes across different clinical and research settings.
One notable challenge is the specialized expertise required to effectively interpret the digital neuropsychological measures obtained through tools like DANA.The outputs of digital tools typically comprise raw numerical values or complex algorithms.Consequently, it necessitates individuals well-versed in neurocognitive functioning and skilled in statistical analysis to extract meaningful insights from these results.This level of expertise is crucial to ensure accurate interpretation and prevent misinterpretation of the findings.Another challenge is the potential absence of established norms or benchmarks for these digital neuropsychological measures.In contrast to traditional neuropsychological tests, which benefit from established norms based on factors such as age and sex, digital neuropsychological measures often lack such reference values.This absence makes the interpretation of results and the establishment of benchmarks for future assessments a complex endeavor.As technology continues to advance and new assessment tools emerge, the development of standardized reference values may require time and concerted effort.Furthermore, the exploration of clinical risk factors linked to these digital neuropsychological measures remains largely unexplored within clinically characterized populations.The absence of established norms and clinical correlates can pose difficulties in determining the clinical implications of the digital measures obtained, emphasizing the importance of addressing this challenge as digital cognitive assessment tools become increasingly prevalent.
Therefore, the primary objective of this study is to establish reference values and investigate the association of clinical risk factors with digital neuropsychological measures derived from DANA in the Framingham Heart Study (FHS).This cohort has undergone a series of DANA tasks and comprehensive characterization of clinical risk factors.We further validated the reference values in participants from the Boston University Alzheimer's Disease Research Center (BU ADRC).

Study population
The FHS is a long-standing community-based prospective cohort study that initiated in 1948.Its primary objective is to identify risk factors of cardiovascular disease as well as dementia within the community.Remarkably, this study has successfully enrolled participants from three distinct generations (15), marking its enduring commitment to advancing our understanding of these critical health issues.In the FHS, cognitive status of participants is determined through a comprehensive dementia review process.This involves a specialized panel, consisting of a neurologist and a neuropsychologist, who meticulously evaluate each case of potential cognitive decline and dementia.Their assessment incorporates a variety of sources, including serial neurologic and neuropsychological evaluations, interviews with caregivers conducted over the phone, medical records, neuroimaging data, and autopsy results when available.The details of dementia review process can be referred to previous studies (16,17).In this study, 932 cognitively intact white participants from the Framingham Heart Study, including the Offspring, Generation 3, and New Offspring Spouse sub-cohorts, formed the reference cohort.
Additionally, a secondary cohort of 57 cognitively intact white participants who have completed at least one assessment of DANA tasks was recruited from the BU ADRC.These participants were diagnosed based on the National Alzheimer's Coordinating Center diagnostic procedures (18).Situated in the urban area of Boston, the BU ADRC focuses on older adults living in the community.It is one of 33 ADRCs funded by the National Institute on Aging.These centers contribute data to the National Alzheimer's Coordinating Center with the aim of fostering collaborative research on AD.More detailed description of the BU ADRC is available in previous publications (19,20).

Digital neuropsychological measures
This study utilized eight digital measures from three DANA tasks, including the Code Substitution task, the Go-No-Go task, and the Simple Reaction Time task.The Code Substitution task, gauges cognitive flexibility, reflecting how well an individual can transition between tasks or ideas.The Go-No-Go task evaluates inhibitory control, i.e., the ability to inhibit automatic or overlearned responses.The Simple Reaction Time task is a straightforward assessment of reaction speed, focusing on the response time to a given stimulus.For more information on these digital cognitive assessment methods, please refer to the study details (3).Participants will be asked to complete these tasks on a tablet provided to them by study staff during the in-clinic study visit.The details of study design and participation rate can be found in prior study (2).All touch screen and stylus responses will be recorded through the Linus Health app installed on the tablet.The tasks are composed of several trials during each session.Distinct files are produced for each task, encapsulating the various trials that make up one complete round of the cognitive assessment.The FHS and BU ADRC operate independent infrastructure pipelines while utilizing a shared database server, where data is stored in separate databases.For each task, through structured query language, data from individual trials is converted into aggregated examination datasets, with each dataset representing the mean value of the trials.This transformation includes recalculating metrics, conducting quality control, deriving extra metrics, and ensuring uniformity between the two cohorts.For each participant, the first examination of each DANA test was included in this study.

Statistical analyses
We derived reference values in the FHS cohort and stratified participants by sex and age groups: younger than 60 years old (<60), between 60 and 69 years (60-69), and aged 70 years or older (≥70).We employed t-tests to examine sex differences in digital neuropsychological measures.Additionally, one-way analysis of variance (ANOVA) was used to investigate the variations in these measures across different age groups.The reference values for each digital neuropsychological measure were defined as the median (2.5th and 97.5th percentiles) in each stratified group (21,22).We also examined the association between digital cognitive measures and clinical factors by a backward elimination approach.These clinical factors included age, sex, body mass index, Mini-Mental State Examination (MMSE), smoking, hypertension treatment, systolic blood pressure (SBP), diastolic blood pressure (DBP), total cholesterol (TC), low-density lipoprotein cholesterol (LDL), high-density lipoprotein cholesterol (HDL), the TC to HDL ratio, blood glucose levels, triglyceride levels, diabetes treatment, as well as prevalent cardiovascular disease, myocardial infarction (MI), congestive heart failure (CHF), and stroke.Our analytical process initiated with the inclusion of all these clinical factors in the linear regression model.We then assessed the association of each factor with digital measures.Subsequently, we identified and removed the factor with the least significant association, followed by a reassessment of the relationships between the remaining clinical factors and digital measures.This stepwise elimination procedure was repeated until all remaining clinical factors achieved at least a nominal level of significance (p < 0.05).In all our models, we forced age and sex as covariates to account for their potential confounding effects on the observed associations.

Cohort descriptive
In this study, we included 932 participants from the FHS as the reference cohort (mean age 63.0 ± 9.4 years, range 39.0-88.0years, 57.3% women).Their clinical characteristics detailed in Table 1.Of these participants, 66.6% had received college education or higher.In addition, we included 57 individuals from BU ADRC as the validation cohort (mean age 74.5 ± 8.7 years, range 52.1-92.5 years, 64.9% women).Among them, 80.7% had a college education or higher.

Reference values of digital neuropsychological measures
We established reference values for digital neuropsychological measures across three DANA tasks.These values, including the upper  S1-S3.Furthermore, we categorized the participants into two education groups: those without a college degree and those with college or higher education.Significant differences were found in digital measures between these groups (t-test, all p < 0.05).The reference values of these two groups can be found in Supplementary Tables S4-S9.Skewness estimates corresponding to partitions in Tables 2-4 are provided in the Supplementary Table S10.Participants with college and higher degree have faster reaction time than the participants with no college degree.Figure 4 represents the proportion of participants from the BU ADRC validation cohort who fall within the normal range (between the 2.5th and 97.5th percentiles) of each digital cognitive measure.The heatmap is stratified by sex and age groups.While the reference values for some subgroups' digital measures do not fully encompass the validation cohort, most of the measures are within the range of reference values.

Association of digital neuropsychological measures with clinical risk factors
The same digital cognitive measures observed across all three DANA tasks demonstrate distinct clinical correlates.Notably, for the average response time, it exhibited distinct associations within each task.In the Code Substitution task, the average response time for all correct test trials (ART_correct) was positively associated with diabetes and total cholesterol, and negatively associated with LDL, HDL, and triglycerides.Conversely, in Go-No-Go task, ART_ correct was positively associated with blood glucose, and negatively associated with SBP and MMSE.Lastly, in the Simple Reaction Time task, it exhibited a positive association with DBP and the prevalence of MI and stroke, while negatively associated with the prevalence of CHF.In the Code Substitution task, three digital neuropsychological measures demonstrated significant associations with four or more clinical factors.The Go-No-Go task revealed that two digital measures were significantly linked to three clinical risk factors.Meanwhile, the Simple Reaction Time task had seven digital measures significantly associated with three or more clinical risk factors.For a comprehensive breakdown of the clinical factors significantly related to each digital neuropsychological measure within the three DANA tasks, refer to Tables 5-7.The model's Akaike information criterion (AIC) value for each iteration is depicted in Supplementary Figures S1-S3.

Discussion
In this study, we established the reference values for digital neuropsychological measures for three DANA tasks in a large community-based cohort.These reference values were categorized by three age intervals, separately for men and women.Our findings provide new insights into the digital cognitive performance of individuals in different age groups.Our results align with previous research in the field of cognitive assessment.Similar to our observations, studies have reported sex-based differences in cognitive function, where women often outperform men in tasks involving verbal memory and attention (23,24), such as the Code Substitution task (3,25).A possible explanation for this difference between sex is organization of memory, with is more bilateral in females, but left-lateralized in the male brain (26).It is suggested that women may have higher connectivity between social motivation, attention, and memory subnetworks (27).However, these gender differences tend to diminish or reverse in tasks requiring spatial processing and reaction time (28, 29), such as the Go-No-Go and Simple Reaction Time tasks (30,31).It is proposed that males perform better in spatial processing or orientation tasks due to a preference for Euclidian strategies, or two-dimensional geometry (28).Other potential contributing factors include environmental context.For instance, males were more inclined to have greater familiarity with 3D computer simulations used to assess spatial processing, possibly owing to their more prevalent exposure to video games (28, 29).Furthermore, our study adds to the existing body of research by providing comprehensive age-specific reference values.These findings are consistent with prior investigations that have highlighted the impact of aging on cognitive performance (32, 33).The observed decline in reaction times with advancing age echoes the welldocumented age-related cognitive changes seen in both sexes (34, 35).
Our study also identified clinical risk factors associated with digital neuropsychological measures across three DANA tasks, revealing noteworthy distinctions among them.Particularly, the average response time, a key neuropsychological measure, exhibited distinct clinical correlates within each task.Our findings regarding the associations between average response time and lipid profile (including total cholesterol) in Code Substitution task were in line with prior studies that have explored the relationship between cholesterol levels and cognitive function (36,37).Notably, higher total cholesterol levels have been linked to cognitive impairment, emphasizing the importance of cardiovascular health in cognitive outcomes.The positive association between average response time and Frontiers in Neurology 07 frontiersin.orgthe prevalence of myocardial infarction and stroke in the Simple Reaction Time task underscores the intricate interplay between cardiovascular health and cognitive processing speed (38,39).These findings aligned with studies emphasizing the impact of cardiovascular risk factors on cognitive function.Moreover, our analysis revealed that different digital cognitive measures were significantly associated with varying numbers of clinical risk factors across the three DANA tasks.These results emphasize the multifaceted nature of cognitive assessment and highlight the need for a comprehensive evaluation of clinical risk factors in understanding cognitive performance.In the Code Substitution task, there was a negative correlation observed between the average response time for all correct trials and levels of LDL and triglycerides.The relationship between LDL, triglycerides, and cognitive performance has yielded mixed outcomes in various studies.While several research findings indicate a link between high LDL cholesterol and a heightened risk of dementia or reduced cognitive function (40)(41)(42)(43), other studies have reported contrary effects (44,45), or found no conclusive correlation (46).In the case of  The values are presented as coefficient, 95% confidence interval, p-value.Blank cells indicate that the corresponding clinical factors were not statistically significant and were therefore removed from the final backward models.Regression coefficients are presented per unit for continuous variables or as the presence of a condition for binary variables.Age and sex were forced as covariates in all the models.HDL, high-density lipoprotein cholesterol; LDL, low-density lipoprotein cholesterol; SBP, systolic blood pressure.
Heatmap illustrating the percentage of BU ADRC participants who scored within the normal range on digital measures, segmented by sex and age and confined to the 2.5th to 97.5th percentile range.A deep red color signifies that 100% (a proportion of 1) of the participants in that particular demographic and digital measures are within the normal range, set between the 2.5th and 97.5th percentiles.Conversely, a white color denotes a 0% proportion, meaning that none of the participants in the group fell within the normal range for that specific digital measure.Shades of lighter red indicate proportions between 0 and 1, reflecting the varying percentages of participants within the normal range.

0.00
The values are presented as coefficient, 95% confidence interval, p-value.Blank cells indicate that the corresponding clinical factors were not statistically significant and were therefore removed from the final backward models.Regression coefficients are presented per unit for continuous variables or as the presence of a condition for binary variables.Age and sex were forced as covariates in all the models.DBP, diastolic blood pressure; MI, myocardial infarction; CHF, congestive heart failure; RatioTC-HDL, the total cholesterol to high-density lipoprotein cholesterol ratio; SBP, systolic blood pressure.triglycerides, and cognitive function, pointing to the need for more comprehensive data in future research to unravel these mechanisms.This study has several advantages.First, the study employed eight digital measures from each of the three DANA tasks in a large wellcharacterized cohort, offering a comprehensive digital assessment of participants' decision-making and reaction times.These measures provided a detailed and multifaceted view of cognitive function, allowing for a more nuanced analysis.An additional cohort was used as an external validation cohort to validate the reference values.Second, a comprehensive list of clinical risk factors has been considered.The study utilized advanced statistical methods, including a backward elimination approach, to investigate clinical correlates.These robust analytical techniques enhance the accuracy and reliability of the study's results.
We also acknowledge several limitations in this study.The study exclusively involved White participants due to the limited availability of DANA data on other ethnicities from the FHS and BU ADRC.Future efforts should aim to establish reference values for digital cognitive assessments across a more diverse range of ethnic groups.While the study categorizes participants into three age groups (<60, 60-69, and ≥70), this grouping may not capture more nuanced age-related cognitive changes.As more data becomes available, a finer age stratification could provide a more detailed insight of human performance on digital cognitive assessment.Our findings indicate that education level has an influence on individuals' performance in DANA tasks.To establish reliable reference values stratified by education, further studies with larger sample sizes will be essential.While our research offers valuable insights into cognitive health assessments using the DANA tool, it is crucial to acknowledge that these measurements exclude assessments of episodic memory-a key cognitive domain impacted by AD.This limitation implies that while our findings contribute to understanding specific aspects of cognitive health, they may not comprehensively cover the cognitive landscape affected by AD.Regarding participant recruitment for our study, we extended invitations to all cognitively intact individuals within the FHS cohort.The essential criteria for participation included owning a smartphone, being proficient in English, and having WiFi access.We recognized that the decision to participate could be influenced by the participants' cognitive and socioeconomic status.Those who chose to partake in the study may have had a relatively better cognitive condition, enabling them to engage with the study requirements.This scenario raises the possibility of selection bias, suggesting that our sample may not entirely represent the broader population of individuals with intact cognition.
In summary, this study provides valuable insights into digital neuropsychological measures, their reference values, and clinical correlates in well-established cohorts.These insights contribute to the growing body of knowledge in the field of digital cognitive assessment and offer valuable guidance for future research and clinical practice.

FIGURE 1
FIGURE 1Box-and-whisker plots showing distributions of digital measures of Code Substitution task by age group for men and women.Each box encompasses the middle 50% of the dataset, specifically the second and third quartiles.The reference value (median) is indicated by the internal horizontal line.Whiskers extend from the boxes to the highest and lowest values within 1.5 times the IQR from the upper and lower quartiles, respectively.

FIGURE 2
FIGURE 2Box-and-whisker plots showing distributions of digital measures of Go-No-Go by age group for men and women.Each box encompasses the middle 50% of the dataset, specifically the second and third quartiles.The reference value (median) is indicated by the internal horizontal line.Whiskers extend from the boxes to the highest and lowest values within 1.5 times the IQR from the upper and lower quartiles, respectively.

FIGURE 3
FIGURE 3Box-and-whisker plots showing distributions of digital measures of Simple Reaction Time by age group for men and women.Each box encompasses the middle 50% of the dataset, specifically the second and third quartiles.The reference value (median) is indicated by the internal horizontal line.Whiskers extend from the boxes to the highest and lowest values within 1.5 times the IQR from the upper and lower quartiles, respectively.

TABLE 1
Demographics of the reference samples in FHS and BU ADRC cohorts.

TABLE 2
Reference values of digital measures in Code Substitution task.
ART_all, average response time for all trials (ms); ART_correct, average response time for all correct test trials (ms); SDRT_correct, standard deviation of response time for correct test trials (ms); ART_test, average response time for all test trials (ms); MRT_test, median response time for all test trials (ms); SDRT_test, standard deviation of response time for all test trials (ms); CE, cognitive efficiency value (measure of both speed and accuracy); Percent_correct, percent of trials with correct responses within allocated time.

TABLE 3
Reference values of digital measures in Go-No-Go task.
ART_all, average response time for all trials (ms); ART_correct, average response time for all correct test trials (ms); SDRT_correct, standard deviation of response time for correct test trials (ms); ART_test, average response time for all test trials (ms); MRT_test, median response time for all test trials (ms); SDRT_test, standard deviation of response time for all test trials (ms); CE, cognitive efficiency value (measure of both speed and accuracy); Percent_correct, percent of trials with correct responses within allocated time.

TABLE 4
Reference values of digital measures in Simple Reaction Time task.
ART_all, average response time for all trials (ms); ART_correct, average response time for all correct test trials (ms); SDRT_correct, standard deviation of response time for correct test trials (ms); ART_test, average response time for all test trials (ms); MRT_test, median response time for all test trials (ms); SDRT_test, standard deviation of response time for all test trials (ms); CE, cognitive efficiency value (measure of both speed and accuracy); Percent_correct, percent of trials with correct responses within allocated time.

TABLE 5
Multiple regression coefficients for digital measures of Code Substitution task.

TABLE 7
Multiple regression coefficients for digital measures of Simple Reaction Time task.

TABLE 6
Multiple regression coefficients for digital measures of Go-No-Go task.The values are presented as regression coefficient, 95% confidence interval, p-value.Blank cells indicate that the corresponding clinical factors were not statistically significant and were therefore removed from the final backward models.Regression coefficients are presented per unit for continuous variables or as the presence of a condition for binary variables.Age and sex were forced as covariates in all the models.SBP, systolic blood pressure; MMSE: Mini-Mental State Examination; RatioTC-HDL, the total cholesterol to high-density lipoprotein cholesterol ratio.