Validation of Strategic Memory Advanced Reasoning Training as an efficient and effective approach to treating warfighters with persistent cognitive complaints associated with mild traumatic brain injury

Introduction: This study directly compared the relative effectiveness of Strategic Memory Advanced Reasoning Training (SMART), which focuses on metacognitive strategies, to a traditional cognitive rehabilitation (CR) program previously developed and validated for the Study of Cognitive Rehabilitation Effectiveness study (SCORE), in treating warfighters with a history of mild traumatic brain injury (mTBI) and persistent post-concussive symptoms (PCS).

Methods: A total of 148 active-duty service members (SMs) were recruited for this randomized controlled trial (RCT). Participants were randomly assigned to either the SMART (n = 80) or SCORE (n = 68) intervention arms. Outcome measures were administered at the start (T1) and end of treatment (T2), and at 3 months post-treatment (T3). Only participants with data from all timepoints and adequate performance validity (SMART: n = 51; SCORE: n = 43) were used in analyses. The primary outcome measure was the Global Deficit Scale (GDS), a composite of seven different objective measures of cognitive performance. Secondarily, participants completed the Neurobehavioral Symptom Inventory (NSI) and Key Behaviors Change Inventory (KBCI) self-report measures of post concussive symptoms (PCS). Lastly, a cost effectiveness analysis (CEA) was performed directly comparing the relative efficiencies of the two CR interventions.

Results: Mixed Analysis of Variance (ANOVA) showed a significant decrease in GDS scores from T1 to T3 (p < 0.001, η_p² = 0.217), irrespective of intervention type (p = 0.986, η_p² = 0.000). The greatest improvement occurred between T1 (SMART: M = 0.70, SD = 0.79; SCORE: M = 0.70, SD = 0.72) and T2 (SMART: M = 0.29, SD = 0.58; SCORE: M = 0.29, SD = 0.40), with scores plateauing at T3 (SMART: M = 0.28, SD = 0.52; SCORE: M = 0.29, SD = 0.57). Similarly, there was a significant decrease in NSI scores over the same period (p < 0.001, η_p² = 0.138), regardless of intervention type (p = 0.412, η_p² = 0.010). Additionally, treatment improved patient perceived functionality (KBCI) from T1 to T2 and these gains remained stable at T3 (p < 0.001, η_p² = 0.377). CEA revealed SMART represented a 60% reduction in cost compared to SCORE.

Discussion: This study demonstrates that SMART is an effective strategy for reducing cognitive deficits and PCS in SMs with a history of mTBI, producing comparable outcomes to a traditional CR program in less time and with improved cost efficiencies.

Introduction

Traumatic brain injury (TBI) and warfighter brain health remains a major health concern for the United States Military Health System (1, 2). The Department of Defense (DoD) has recognized the critical importance of addressing these issues through programs like the Warfighter Brain Health Initiative, which aims to optimize brain health and improve outcomes following combat TBI among service members (SMs). This concern is underscored by the significant number of TBIs reported among warfighters, with nearly 505,896 cases documented between 2000 and early 2024, of which approximately 82% were classifiable as ‘mild’ (3). Mild traumatic brain injury (mTBI) is the most common type of brain injury, and is generally associated with good recovery; however, a percentage of individuals will have persistent symptoms, with cognitive complaints being among the most common (4). Individuals with chronic post-concussive symptoms (PCS), particularly in military and veteran populations, frequently report cognitive symptoms (5). Unfortunately, many SMs have a history of multiple head injuries as well as psychological co-morbidities, which can negatively influence recovery and increase the likelihood of long-term cognitive deficits (6, 7, 8). Moreover, PCS may limit a warfighter’s ability to complete their duties and, on a larger scale, limit overall force readiness.

Due to heightened awareness and the large number of SMs experiencing chronic PCS, there is increased importance placed on identifying or developing effective treatment options. An effective rehabilitation program to remediate cognitive deficits in injured SMs would have direct impact to force readiness and may decrease the long-term health utilization costs from both active duty SMs and veterans. When determining optimal treatment for SMs, it is important to recognize that a tenant of military medicine is to return injured warfighters to full-unrestricted duty status. In terms of mTBI, rehabilitation of cognitive deficits should be focused on the complex mental tasks necessary for high-pressure combat situations and other military operations. The expectation is that warfighters be cognitively ready, meaning they need to be mentally prepared to handle the demanding, unpredictable, and high-stress situations they may encounter in the field. Thus, cognitive readiness is crucial for mission success, as well as the overall health and safety of SMs. The recognized core components of cognitive readiness are aspects of higher-order cognitive reasoning: situational awareness, problem solving, metacognition, decision-making, adaptability, and creative thinking. Importantly, all military personnel must be able to translate the techniques and strategies learned in treatment to novel environments, including adapting to new technological capabilities, and making split-second potentially life-threating decisions, all of which are characteristic of both combat and training environments (9).

Another challenge is treatment generalizability from civilian and veteran communities to active duty SMs. This may pose some unique and unexplored factors, as the goal of treatment is not remediation of common activities of daily living (e.g., remembering appointments) but military readiness needed for worldwide deployability. Current cognitive rehabilitation (CR) strategies focus primarily on application of techniques originally developed for basic and instrumental activities of daily living that are impaired after more severe brain injury. CR has typically focused on compensating for subjective and functional cognitive complaints (10, 11). These treatments have shown relatively modest success in management of symptoms in SMs, who primarily have mild brain injuries and are requesting assistance with higher level cognitive functions (e.g., selective attention and complex decision-making) associated with military operations (12). The Study of Cognitive Rehabilitation Effectiveness (SCORE) randomized clinical trial (13) used various therapies in an active-duty population: (i) psychoeducation, (ii) independent self-administered computer-based CR, (iii) therapist-directed manualized CR, and (iv) therapist-directed CR integrated with cognitive behavioral therapy (CBT) psychotherapy. They found that treatment arms that included therapist-directed CR had superior outcomes to the other treatment arms as measured by self-reported day-to-day cognitive functioning. Though promising, the results of the SCORE trial suggest that traditional therapist-based CR interventions are far from optimal. In the SCORE therapist-based CR intervention, only 23.3% of the sample had a meaningful reduction in symptoms after extensive treatment. Another limitation of traditional therapist-directed CR treatment is duration of treatment. For example, the therapist-directed manualized CR used in the SCORE randomized clinical trial included 10 h a week of treatment for 6 weeks (13). A similar CR treatment (14) lasted 10 weeks in duration. Many published CR programs are very time-consuming which affects the time of return to duty and may hinder its ability to be executed in military medical settings with limited resources or in military populations who are frequently relocating geographical sites (e.g., Special Forces).

Strategic Memory Advanced Reasoning Training (SMART) represents a possible alternative to traditional CR, with the potential to overcome many of the limitations of traditional therapist-based CR when applied to treatment of mTBI in active duty SMs. SMART is an evidence-based, manualized cognitive training protocol focused on enhancing top-down executive functioning via three metacognitive strategies: strategic attention, integrated reasoning, and innovation (15). Numerous randomized clinical trials with various populations have demonstrated improved cognitive performance and emotional wellbeing using SMART strategies, which focus not on compensatory strategies for daily living but more advanced complex reasoning skills that are essential for warfighter readiness. SMART has shown efficacy in improving cognitive functioning in both neurologically injured individuals as well as healthy adults (16). In randomized control trials, SMART has been shown to enhance complex reasoning, working memory and innovative cognition in healthy adults (15, 17, 18). In these studies, compared to two different control conditions, SMART was linked to improved neural gains in both the fronto-parietal and cingulo-opercular brain networks, areas involved in speed based reasoning and essential for warfighter cognitive functioning in unpredictable combat situations. Multiple clinical trials have demonstrated that SMART improves mental agility, focus, psychological wellbeing, and functional life outcomes in individuals with mild cognitive deficits following TBI, as well as healthy individuals (15, 16, 19–21). Notably, SMART also has a shorter duration of treatment time than current CR treatments and may be administered by a wide array of rehabilitation therapists (e.g., speech therapists, occupational therapists, psychologists), which are already employed across the military health system. Perhaps most importantly to SMs, there is encouraging evidence that training with SMART may translate to gains on the battlefield. In a translation study involving military personnel, investigators showed significant improvement in cognitive functioning, evidenced by better performance on measures of integrated reasoning and innovation (22). Furthermore, this improved performance occurred following a shortened intervention period consisting of just 6–10 h over a 4-week period (22).

Several studies have demonstrated that SMART can influence some of the neurophysiology underpinning higher order cognitive functioning. A study using structural and resting-state functional MRI found that SMART increased cortical thickness in four right prefrontal regions and decreased resting-state connectivity in these areas in individuals with chronic TBI, compared to controls (19). Another study demonstrated that training with SMART induced changes in whole-brain neural networks, and that this neuroplasticity was associated with improved functioning (23). Using a graph-theoretical approach, investigators showed that SMART was associated with a reorganization of modular brain networks in individuals with a history of TBI. Specifically, they were able to show that SMART reduced modularity due to increased connectivity between modules, yielding increased global and local processing efficiencies (23). SMART has also been associated with increased blood flow to the brain. In a study of reasoning training in veterans and civilians with TBI and persistent (mild) functional deficits, investigators found an association between increased cerebral blood flow (CBF) and improved reasoning following SMART (24). This increased CBF was observed in the left inferior frontal region, the left insula, and the bilateral anterior cingulate cortex, areas associated with cognitive control performance and executive function (24). These same investigators also showed increased delivery of regional cerebral blood flow (rCBF) bilaterally in the precuneus (24), a region previously shown to have hypometabolism linked to increased severity of TBI (25). Based on the neurovascular coupling principle, the results of an increase in CBF following SMART in the bilateral precuneus are suggestive of improved neural health in this key region linked to psychological health. Notably, reduced neuronal activity in the precuneus has been reported in individuals with comorbid TBI and post-traumatic stress disorder (26). Lastly, SMART was shown to reduce depressive symptoms in TBI regardless of whether the training directly targeted specific psychiatric symptoms. Notably, investigators observed that decreased Beck Depressive Inventory scores within the depressive-symptoms group was associated with improvements in scores for PTSD, as well as TBI symptom awareness and functional status (27).

Given SMART’s effectiveness (as demonstrated by reduced cognitive deficits, improved performance on cognitive tasks, and enhanced physiological functioning), reduced time of treatment compared to traditional CR, and the potential to improve cognitive readiness for warfighters, a detailed exploration of SMART’s effectiveness in rehabilitating active duty SMs with mTBI is warranted. The primary objective of this study was to evaluate SMART treatment for rehabilitation of active duty SMs with chronic cognitive deficits following a mTBI. The key question was whether a cognitive-control, top-down training protocol can achieve better cognitive health, psychological health, and functional life outcomes than traditional cognitive rehabilitation. We hypothesized that SMART would result in a larger reduction in cognitive deficits as measured by the Global Deficit Scores (GDS) compared to a traditional (CR) program (SCORE). Secondarily, participants completed the Neurobehavioral Symptom Inventory (NSI) and Key Behaviors Change Inventory (KBCI) self-report measures of post concussive symptoms (PCS). Finally, a cost effectiveness analysis (CEA) comparing the relative costs and outcomes of SMART and SCORE was performed. This approach to evaluating rehabilitation programs can help guide decisions on resource allocation by identifying the most efficient program for achieving desired cognitive outcomes. We hypothesized that the differences in treatment duration and patient throughput of these independent cognitive rehabilitation programs would impact the overall cost of treatment.

Materials and methods

Design

This prospective randomized clinical trial (RCT) compared the effectiveness of two cognitive rehabilitation approaches—SMART and SCORE—among active-duty military personnel with persistent cognitive complaints following mTBI.

Participants

A total of 148 active-duty service members (SMs) were recruited from a large military treatment facility. Participants had a history of at least one mTBI based on the DoD diagnostic criteria (28). Participants had to be greater than 3 months from injury and have ongoing chronic cognitive complaints defined as moderate or greater severity on one of the four cognitive symptoms from the Neurobehavioral Symptom Inventory (NSI). TBI diagnosis was confirmed with the Ohio State University Traumatic Brain Injury Identification Method (OSU TBI-ID) (29–31) as well as documentation in electronic medical records (e.g., AHLTA/MHS GENESIS).

Figure 1 shows the flow of participants through the study protocol. Participants were grouped into cohorts and each cohort was randomized into one of the two targeted interventions (SMART and SCORE). The SMART intervention was able to accommodate cohorts of 2–6 participants and the SCORE intervention was able to accommodate 2–4 participants. Ultimately, 80 participants were randomized to SMART and 68 participants were randomized to SCORE. After 4 weeks of the SMART treatment, 90% (n = 72) of participants completed the immediate post-treatment assessment (T2). After 6 weeks of the SCORE treatment, 82.4% (n = 56) of participants completed the immediate post-treatment assessment (T2). The final post-assessment (T3) was completed 3 months after the treatment intervention; 77.5% (n = 62) of SMART participants completed T3 while 75% (n = 51) of SCORE participants completed T3.

Figure 1

Figure 1. (A) Flow of participants through the study protocol. After screening and consent, a pre-treatment evaluation was completed to determine baseline functioning. For each treatment, evaluations were completed immediately post-treatment (at approximately 4 weeks from baseline for SMART and 6 weeks from baseline for SCORE) and at 3 months post-treatment. (B) Details the study procedure timeline.

Performance validity testing (PVT)

To ensure cognitive assessments were valid and suitable for interpretation, this study relied on PVT designed to detect atypical cognitive performance that is not indicative of known and/or legitimate cognitive impairment. This was done to ensure that patient performance on tasks was reflective of their true abilities and not attributable to other factors (e.g., malingering, lack of effort). PVT included the Rey-15 Item Test and Recognition Trial with a threshold of <26 (32), the Delis Kaplan Executive Function System (DKEFS) Color Naming and Word Reading combined score with a cut off of <6 (33), as well as DKEFS Visual Scanning, Number Sequencing, and Letter Sequencing tests with a combined score of <15 (34). Participants were excluded from the study if they failed two of the three tests at any of the assessment time points. A total of 19 participants were excluded based on failed PVT testing.

Intervention arms

Strategic Memory Advanced Reasoning Training (SMART)

SMART is a proprietary, manualized cognitive training program focused on three core metacognitive strategies: Strategic Attention, Integrated Reasoning, and Innovation. Table 1 provides an overview of the SMART module topics, session structure, and example activities. Although the full session-by-session content is proprietary, this overview clarifies how we delivered SMART in a four-week timeframe (total of ~20 h). Week 1 focused on Strategy Education with five 1-h sessions introducing the core strategies. Weeks 2–4 focused on Strategy Application through a combination of group sessions and individual sessions that guided participants in applying the strategies to real-life tasks, with emphasis on military-relevant scenarios. Over the 4-week period, there were 14 h of group training and 6 h of individual training for a total of 20 h. All treatment sessions were completed by a licensed speech language pathologist (SLP). Each participant was provided a treatment manual with worksheets and educational activities. The SMART manual includes a guide to strategy application and goal setting prompts for individual and group activities and implementation assignments.

Table 1

Table 1. SMART training delivery.

Study of Cognitive Rehabilitation Effectiveness (SCORE)

The traditional CR program was a clinician-directed, manualized intervention previously developed and validated for the SCORE trial (13). In this functionally oriented program, performance is improved through repetition, errorless learning, and gradually increasing task difficulty and complexity in a structured systematic approach. The intervention lasted 6 weeks, at 10 h per week, for a total of 60 h. Each week, the intervention included 5 one-hour individual sessions, two of which were focused on compensatory strategies and three of which were focused on restorative strategies. There were also two 1-h weekly group therapy sessions focused on compensatory strategies, and 3 h of weekly computer-based work with the Attention Process Training-3 program (APT-3; (35)). All computer-based sessions were proctored by clinic staff who recorded performance and provided positive reinforcement of participation and effort. A licensed SLP conducted treatment. Each participant received a treatment manual comprised of educational materials and integrated individual and group activities and assignments.

Outcome measures

Global Deficit Score (GDS)

A GDS was calculated using methods adapted from previously published work (36, 37). GDS was calculated from 7 cognitive measures: Hopkins Verbal Learning Test-Revised Total Recall (HVLT-TR) and Delayed Recall (HVLT-DR) (38); Delis Kaplan Executive Functioning System Color Word Conditions 3 and 4 and Trail Making Conditions 4 (39); Paced Auditory Serial Addition Test (PASAT) (40, 41); and the Symbol Digit Modality Test (SDMT) (42). By aggregating deficits across multiple tests, the composite score (i.e., GDS) potentially enhances the detection of subtle cognitive deficits (particularly important in this population) that might not be apparent in individual assessments.

Demographically corrected t-scores (M = 50, SD = 10) for each cognitive measure were converted to deficit scores. T-score to deficit score conversions were as follows: >40 = 0, 35–39 = 1, 30–34 = 2, 25–29 = 3, 20–24 = 4, <20 = 5. The 7 deficit scores were averaged to calculate GDS. A score of 0 represents no impairment and higher scores represented greater cognitive impairment; based on prior studies, scores ≥0.5 signify impaired cognitive performance.

Neurobehavioral Symptom Inventory (NSI)

The NSI is a 22 item self-report questionnaire that assesses PCS (43). The questionnaire is a preferred measure by the DoD and Veterans Affairs. It is well-validated in research studies on TBI and PTSD in military populations, it has known reliability and validity, as well as guidelines for assessment of change (44–46). The NSI categorizes symptoms into four primary domains or clusters: physical, cognitive, affective, and sensory (46).

Key Behaviors Change Inventory (KBCI)

The KBCI is a self-reported measure of functional deficits that are associated with a TBI. It is a 64-item questionnaire that assesses behavioral areas such as lack of motivation, difficulties communicating, lack of insight into difficulties, and relationship problems (47). The scale measures eight domains of functioning (inattention, impulsivity, interpersonal, apathy, somatic, unawareness, communication, and emotional) and the total score which is an average of the scales; all scores represent a standardized t-score based on previously published normative data. The instrument has good content and construct validity and an internal consistency reliability of 0.82–0.91.

Statistical analysis

The differences between treatment groups sample characteristics including demography, military information and co-variates were evaluated using Students ‘t’ and chi-square tests for continuous and categorical variables, respectively. The Mann–Whitney U test was used for between groups comparisons where continuous variables were not normally distributed. Mixed-effect analysis of variance (ANOVA) was conducted for the primary hypothesis with treatment modality (SMART vs. TCR) as the independent variable. This approach took into consideration between-subject variance based on randomized group, as well as within-subject variance for repeated measures. Primary analyses were based on intention to treat, and we leveraged statistical power of multilevel modeling to handle missing follow-up data. Outcomes were assessed as change in GDS over time. The same procedure was completed for the NSI total score and KBCI total score. A significant p-value of 0.05 was used to assess the main and interaction effects (partial eta squared). Where analyses consisted of timepoints 1 and 2 only, within-group related samples t tests were performed, and Cohen’s d effect sizes reported.

Cost effectiveness analysis (CEA)

We completed a cost effectiveness analysis (CEA) comparing SMART and SCORE. CEA is a way to evaluate both the costs and health outcomes of one or more interventions by estimating how much it costs to gain a unit of a health outcome, in this case, improved cognitive and behavioral functioning. As outcomes were equivalent, the CEA focused on costs alone. The largest direct cost of the interventions was the time of the speech language pathologists (SLPs) and the warfighters who were treated while on duty. All other direct costs (e.g., equipment) were similar between the two treatments. Time costs for the SLP included individual and group therapy; for group treatment hours, the time was divided by number of participants in each group to find the mean SLP time spent in group therapy per participant. Warfighter time costs included individual and group therapy as well as homework and transportation. Mean total costs per participant were calculated as (mean hourly wage of a SLP multiplied by mean hours spent on that treatment per participant) plus (mean hourly wage of warfighter multiplied by mean hours spent on that treatment). The hourly mean wage of a SLP based in California where the study was executed was derived from the U.S. Bureau of Labor Statistics (BLS). The hourly mean wage of the warfighter was based on participant’s rank and the DoD’s monthly basic pay chart. All costs are reported in 2022 United States (U.S.) dollars.

Results

Demographics and TBI history

Demographic and injury history data are provided in Tables 2, 3, respectively. There were no statistical differences between the two treatment groups in terms of age, gender, race, ethnicity, education level, marital status, or current living situation (Table 2). For both groups, the participants were typically white/Caucasian, married men, in their mid-thirties (Table 2). In terms of TBI History, the treatment groups did not differ on any of the key variables from the Ohio State University TBI History; with the exception that the SMART condition had more years participating in activities related to repetitive head injuries (e.g., playing contact sports) (Table 3). Both samples had a median of 4 mTBIs in their lifetime with median time since last mTBI being more than 5 years (Table 3). Pre-treatment (T1) the groups did not differ in their neurobehavioral symptom severity (NSI), TBI related functional difficulties (KBCI), depression (Patient Health Questionnaire-8), sleep quality (Pittsburgh Sleep Quality Index-PSQI), or post-traumatic stress levels (Post-Traumatic Stress Disorder Checklist-PCL-M), nor did the groups differ in terms of their number of past combat deployments, combat exposure severity (Combat Exposure Scale-CES), lifetime blast exposure (Blast Exposure Threshold Study-BETS), or premorbid functioning (Test of Premorbid Functioning) (Table 3).

Table 2

Table 2. Demographics and injury history by intervention.

Table 3

Table 3. TBI factors and clinical factors by intervention pre-treatment.

The number of hours each participant spent with a clinician is detailed in Table 4. Consistent with the study design, the participants who completed SMART had significantly fewer hours in treatment than those in SCORE. For SCORE, the participants completed on average 26.6 of 30 h (88.7%) of individual treatment, 10.9 of 12 h (90.8%) of group treatment, and 11.0 of 12 h (91.7%) of homework for a total of 48.5 of 54 treatment hours (89.8%). The average clinician-directed activities in SCORE were 36.1 (SD = 3.5) hours compared to 18.9 h in SMART. The average time a participant spent with a licensed clinician in SMART was half (51.8%) of that for the SCORE condition.

Table 4

Table 4. Descriptive statistics detailing treatment hours per participant.

Impact on cognition and GDS

Table 5 shows the results of a mixed-effect ANOVA examining change in GDS by treatment condition, which revealed a large within group effect insofar as all participants showed a significant change in GDS from T1 to T3 (p < 0.001, η_p² = 0.217). There was no significant interaction between condition and rate of change (p = 0.986, η_p² < 0.001). The greatest decline occurred between T1 (SCORE: M = 0.70, SD = 0.72; SMART: M = 0.70, SD = 0.79) and T2 (SCORE: M = 0.29, SD = 0.40; SMART: M = 0.29, SD = 0.58), with these treatment effects remaining at T3 (SCORE: M = 0.29, SD = 0.57; SMART: M = 0.28, SD = 0.52).

Table 5

Table 5. Change in GDS by intervention-T1–T3.

To maximize power, a secondary analysis was conducted that included any participant that completed T1 and T2 (Table 6). This increased the sample size for both SCORE (n = 49) and SMART (n = 65). The results of the repeated measures ANOVA were consistent with the primary analysis and again showed a large within group effect in terms of GDS improvement (p < 0.001, η_p² = 0.311) but no interaction (p = 0.400, η_p² = 0.008). The specific neuropsychological tests comprising GDS were also examined in this sample to describe the effects across domains. Table 6 shows that for the SMART intervention, there was a significant improvement in all cognitive measures with large effects (d > 0.50). For participants who completed SCORE there were significant improvements on all measures except the DKEFS Trail Making Condition 4, a measure of mental flexibility, but this approached significance (p = 0.058). The effect sizes associated with the SCORE intervention were large (d > 0.50) for 5 of 7 measures.

Table 6

Table 6. Change in cognitive measures by intervention-T1–T2.

Impact on self-reported neurobehavioral symptoms

Table 7 shows the results of a mixed-effect ANOVA examining change in NSI total score by treatment condition revealed a significant within group effect where the overall sample had a reduction in NSI total score post-treatment (<0.001, η_p² = 0.138); however, the change was not impacted by the type of treatment (p = 0.412, η_p² = 0.010) (Table 7). The within group effect size was large with the change occurring from T1 to T2 and maintained at T3 (Table 7). Examination of the entire sample who completed T1 and T2 (Table 8) demonstrated that both interventions resulted in reduction of symptoms related to cognitive and vestibular functioning. The SCORE intervention but not the SMART intervention resulted in a reduction of affective symptoms. Neither intervention resulted in a reduction of somatosensory symptoms.

Table 7

Table 7. Change in NSI by intervention, T1–T3.

Table 8

Table 8. Change in NSI summary and subscale scores by intervention-T1-T2.

Impact on self-reported functioning

Table 9 shows the results of a mixed-effect ANOVA examining change in KBCI total score by treatment condition revealed again a large within group effect (<0.001, η_p² = 0.377). Treatment improved patients’ perceived level of functioning from T1 to T2 and these gains remained stable at 3 months post-treatment. In addition, there was a significant interaction with a moderate effect size between change in KBCI total scores and treatment condition (p = 0.017, η_p² = 0.086). Further analysis revealed that while both interventions led to a reduction in KBCI total scores between T1 and T2, only individuals who received the SCORE treatment showed continued decrease in self-perceived functional difficulties from T2 To T3, while those individuals receiving SMART showed a modest regression. This change in either direction was approximately one standardized T Score point, and further analysis revealed that this difference was not statistically significant (t(92) = 1.97, p = 0.052, d = 0.407).

Table 9

Table 9. Change in KBCI total scores by intervention, T1-T3.

To maximize sample size, we examined all participants who completed T1 and T2 (Table 10). A medium sized effect was observed on the KBCI total for both interventions (0.652 vs. 0.710). For SMART all 8 scales that comprise the KBCI had a significant reduction with medium effect sizes. SCORE resulted in a significant reduction in 7 of 8 scales (the 8th scale, Impulsivity, approached significance).

Table 10

Table 10. Change in summary and subscale KBCI scores by intervention-T1-T2.

Relative cost effectiveness

Shown in Table 11 is the mean treatment time for each intervention as detailed above (SCORE: 42.4 h, SMART: 18.9 h). SLPs spent on average 10.6 h per participant in SCORE and 3.15 h per participant in SMART. Based on the most up-to-date labor statistics, we estimated that the hourly mean wage of a SLP was $52.39, while the hourly mean wage of the warfighter in this study was $34.06 (Table 12). We found that the mean cost of SCORE treatment was $1,999.47 compared to the mean cost of SMART treatment of $808.76, representing a cost savings of $1,190.71 per participant, or 60% reduction in cost (Table 12).

Table 11

Table 11. Expected and mean observed time (hours) per treatment for SLPs and warfighters.

Table 12

Table 12. Mean time cost per treatment.

Discussion

This study directly compared two CR approaches for managing cognitive complaints in active-duty SMs with a history of mTBI and PCS-a novel metacognitive-focused treatment, “Strategic Memory Advanced Reasoning Training” (SMART), and a previously validated traditional program, the “Study of Cognitive Rehabilitation Effectiveness” (SCORE). The results of this prospective study showed that the two interventions had equivalent efficacy rehabilitating warfighters who had a history of mTBI with PCS. We showed that both interventions had a large effect on improving warfighter cognitive performance. The gains in cognitive performance, and an overall reduction in cognitive deficits, were observed immediately after the intervention and remained at 3 months post-intervention. In total, the results showed that completing cognitive rehabilitation reduced the rates of cognitive deficits in the sample by half. This improvement in cognitive performance was observed in terms of global functioning as well as on most individual cognitive tasks. This translates directly to warfighter cognitive readiness by helping diminish cognitive deficits that could impact the warfighter’s ability to complete their duties safely and effectively.

In addition to cognitive performance, the results also show that both CR programs were effective in reducing participants’ overall symptom burden as well as reducing their self-reported functional deficits. The improvements in self-reported functioning spanned all domains of interest including those directly related to cognitive abilities (i.e., inattention and impulsivity) as well as those indirectly related to military duties (i.e., interpersonal skills, communication with others, and emotional health). The one instance where we observed an interaction effect was on the KBCI, a measure of self-perceived functional difficulties. Notably, while both interventions reduced KBCI total scores between T1 and T2, those receiving the SCORE intervention showed continued improvement compared to participants in the SMART group, who showed a small regression toward baseline. However, when interpreting these results, we would caution that the change in either direction was only one T Score point, and the difference in change scores from T2 to T3 was not statistically significant suggesting this interaction is probably of limited meaningfulness.

Although the two treatments resulted in similar outcomes across multiple metrics, as anticipated, they varied dramatically in terms of treatment duration. This is important for two primary reasons. First, the quicker treatment can be completed, the less time the warfighter is away from their command, resulting in expedited return to duty. Second, shorter treatment time impacts the cost of healthcare. Cost effectiveness analysis showed SMART saved $1,190.71 per participant, representing a 60% reduction in cost. This streamlining of services has the potential to not only impact budget costs but is also likely to translate into better access to treatment for warfighters in need. This should be considered when identifying optimal treatments to improve and maintain warfighter brain health.

This study has some notable limitations. The design did not include a passive, no intervention control condition, which limits the generalizability of the findings. Inclusion of an additional active control (e.g., computerized CR program) would have also strengthened the conclusion drawn from this study. However, given the extant literature on cognitive rehabilitation, the improvements observed are believed to be directly related to the intervention and not external non-treatment factors. Particularly, the improvements on the cognitive tasks are generally larger than what is observed from practice effects. The fact that cognitive improvements were observed in conjunction with self-reported changes on two independent measures also supports the veracity of the results. Lastly, the 60-h SCORE protocol used in this study was derived from (13), which was current at the time of this study’s inception. However, since that time, it is likely that efforts have been made to revise and streamline SCORE, as well as other CR programs. Future research should focus on evaluating the effectiveness of shorter or more intensive versions of CR protocols to determine whether they can produce comparable cognitive and behavioral outcomes in SMs with mTBI and PCS.

The military has a longstanding desire to raise mental agility and acumen in its military operators, with a key goal for warfighters to return to duty. Persistent symptoms of concussion may delay a SM’s return to full active-duty status. Importantly, it is worth mentioning that mild or absence of impairment on neuropsychological assessments does not always correlate with patients’ symptoms, particularly in cases of mTBI/concussion. Consequently, detection and treatment of such subtle impairments associated with mTBI continues to be a challenge for clinicians. Despite some successful CR interventions many SMs continue to present with deficiencies in daily functioning (whether treated or not) and psychological wellbeing secondary to the effects of mTBI. Given SMART’s effectiveness (as demonstrated by reduced cognitive deficits, improved performance on cognitive tasks) and reduced time of treatment compared to traditional CR in this study, SMART is a cost effective and feasible alternative to traditional CR programs for treating PCS in active duty SMs with the goal of improving overall cognitive fitness/readiness for warfighters.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors without undue reservation.

Ethics statement

The studies involving humans were approved by Institutional review board at the Naval Medical Center, San Diego (NMCSD). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AD: Formal analysis, Visualization, Writing – original draft, Writing – review & editing. AB: Writing – review & editing, Project administration. JR: Formal analysis, Writing – review & editing. MC: Data curation, Methodology, Writing – review & editing. JL: Methodology, Writing – review & editing. JZ: Methodology, Writing – review & editing. EV: Methodology, Writing – review & editing. IB: Writing – review & editing. SC: Conceptualization, Writing – review & editing. JB: Conceptualization, Funding acquisition, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. Funding provided by the Congressionally Directed Medical Research Programs (CDMRP) Award No. W81XWH1910864.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Lindberg, MA, Sloley, SS, Ivins, BJ, Marion, DW, and Moy Martin, EM. Military TBI-what civilian primary care providers should know. J Family Med Prim Care. (2021) 10:4391–7. doi: 10.4103/jfmpc.jfmpc_98_21

PubMed Abstract | Crossref Full Text | Google Scholar

2. Lee, KM, Khatri, TL, and Fudge, ER. US Department of defense warfighter brain health initiative: maximizing performance on and off the battlefield. J Am Assoc Nurse Pract. (2020) 32:720–8. doi: 10.1097/JXX.0000000000000532

PubMed Abstract | Crossref Full Text | Google Scholar

3. TBICoE. 2000–2024 Q1 DoD worldwide numbers for TBI. Available online at: https://health.mil/Military-Health-Topics/Centers-of-Excellence/Traumatic-Brain-Injury-Center-of-Excellence/DOD-TBI-Worldwide-Numbers (Accessed December 2024).

Google Scholar

4. Schwab, K, Terrio, HP, Brenner, LA, Pazdan, RM, McMillan, HP, Mac Donald, M, et al. Epidemiology and prognosis of mild traumatic brain injury in returning soldiers: a cohort study. Neurology. (2017) 88:1571–9. doi: 10.1212/WNL.0000000000003839

PubMed Abstract | Crossref Full Text | Google Scholar

5. Helmick, KMembers of Consensus, C. Cognitive rehabilitation for military personnel with mild traumatic brain injury and chronic post-concussional disorder: results of April 2009 consensus conference. Neuro Rehabil. (2010) 26:239–55. doi: 10.3233/NRE-2010-0560

PubMed Abstract | Crossref Full Text | Google Scholar

6. Lange, RT, Brickell, TA, Kennedy, JE, Bailie, JM, Sills, C, Asmussen, S, et al. Factors influencing postconcussion and posttraumatic stress symptom reporting following military-related concurrent polytrauma and traumatic brain injury. Arch Clin Neuropsychol. (2014) 29:329–47. doi: 10.1093/arclin/acu013

PubMed Abstract | Crossref Full Text | Google Scholar

7. Iverson, GL, Echemendia, RJ, Lamarre, AK, Brooks, BL, and Gaetz, MB. Possible lingering effects of multiple past concussions. Rehabil Res Pract. (2012) 2012:316–575. doi: 10.1155/2012/316575

PubMed Abstract | Crossref Full Text | Google Scholar

8. Spira, JL, Lathan, CE, Bleiberg, J, and Tsao, JW. The impact of multiple concussions on emotional distress, post-concussive symptoms, and neurocognitive functioning in active duty United States marines independent of combat exposure or emotional distress. J Neurotrauma. (2014) 31:1823–34. doi: 10.1089/neu.2014.3363

PubMed Abstract | Crossref Full Text | Google Scholar

9. Fletcher, JD, and Wind, AP. The evolving definition of cognitive readiness for military operations In: HF O’Neil, editor. Teaching and measuring cognitive readiness. New York: Springer Science+Business Media (2014)

Google Scholar

10. Drag, LL, Spencer, RJ, Walker, SJ, Pangilinan, PH, and Bieliauskas, LA. The contributions of self-reported injury characteristics and psychiatric symptoms to cognitive functioning in OEF/OIF veterans with mild traumatic brain injury. J Int Neuropsychol Soc. (2012) 18:576–84. doi: 10.1017/S1355617712000203

PubMed Abstract | Crossref Full Text | Google Scholar

11. Spencer, RJ, Drag, LL, Walker, SJ, and Bieliauskas, LA. Self-reported cognitive symptoms following mild traumatic brain injury are poorly associated with neuropsychological performance in OIF/OEF veterans. J Rehabil Res Dev. (2010) 47:521–30. doi: 10.1682/JRRD.2009.11.0181

PubMed Abstract | Crossref Full Text | Google Scholar

12. Cooper, DB, Bunner, AE, Kennedy, JE, Balldin, V, Tate, DF, Eapen, BC, et al. Treatment of persistent post-concussive symptoms after mild traumatic brain injury: a systematic review of cognitive rehabilitation and behavioral health interventions in military service members and veterans. Brain Imaging Behav. (2015) 9:403–20. doi: 10.1007/s11682-015-9440-2

PubMed Abstract | Crossref Full Text | Google Scholar

13. Cooper, DB, Bowles, AO, Kennedy, JE, Curtiss, G, French, LM, Tate, DF, et al. Cognitive rehabilitation for military SMs with mild traumatic brain injury: a randomized clinical trial. J Head Trauma Rehabil. (2017) 32:E1–E15. doi: 10.1097/HTR.0000000000000254

PubMed Abstract | Crossref Full Text | Google Scholar

14. Storzbach, D, Twamley, EW, Roost, MS, Golshan, S, Williams, RM, O'Neil, M, et al. Compensatory cognitive training for operation enduring freedom/operation Iraqi freedom/operation new Dawn veterans with mild traumatic brain injury. J Head Trauma Rehabil. (2017) 32:16–24. doi: 10.1097/HTR.0000000000000228

PubMed Abstract | Crossref Full Text | Google Scholar

15. Chapman, SB, Spence, JS, Aslan, S, and Keebler, MW. Enhancing innovation and underlying neural mechanisms via cognitive training in healthy older adults. Front Aging Neurosci. (2017) 9:314. doi: 10.3389/fnagi.2017.00314

PubMed Abstract | Crossref Full Text | Google Scholar

16. Chapman, SB, and Mudar, RA. Enhancement of cognitive and neural functions through complex reasoning training: evidence from normal and clinical populations. Front Syst Neurosci. (2014) 8:69. doi: 10.3389/fnsys.2014.00069

PubMed Abstract | Crossref Full Text | Google Scholar

17. Chapman, SB, Aslan, S, Spence, JS, Hart, JJ Jr, Bartz, EK, Didehbani, N, et al. Neural mechanisms of brain plasticity with complex cognitive training in healthy seniors. Cereb Cortex. (2015) 25:396–405. doi: 10.1093/cercor/bht234

PubMed Abstract | Crossref Full Text | Google Scholar

18. Chapman, SB, Aslan, S, Spence, JS, Keebler, MW, DeFina, LF, Didehbani, N, et al. Distinct brain and behavioral benefits from cognitive vs. physical training: a randomized trial in aging adults. Front Hum Neurosci. (2016) 10:338. doi: 10.3389/fnhum.2016.00338

PubMed Abstract | Crossref Full Text | Google Scholar

19. Han, K, Davis, RA, Chapman, SB, and Krawczyk, DC. Strategy-based reasoning training modulates cortical thickness and resting-state functional connectivity in adults with chronic traumatic brain injury. Brain Behav. (2017) 7:e00687. doi: 10.1002/brb3.687

PubMed Abstract | Crossref Full Text | Google Scholar

20. Vas, AK, Chapman, SB, and Cook, LG. Language impairments in traumatic brain injury: a window into complex cognitive performance. Handb Clin Neurol. (2015) 128:497–510. doi: 10.1016/b978-0-444-63521-1.00031-5

Crossref Full Text | Google Scholar

21. Venza, EE, Chapman, SB, Aslan, S, Zientz, JE, Tyler, DL, and Spence, JS. Enhancing executive function and neural health in bipolar disorder through reasoning training. Front Psychol. (2016) 7:1676. doi: 10.3389/fpsyg.2016.01676

PubMed Abstract | Crossref Full Text | Google Scholar

22. Young, LR, Zientz, JE, Spence, JS, Krawczyk, DC, and Chapman, SB. Efficacy of cognitive training when translated from the laboratory to the real world. Mil Med. (2021) 186:176–83. doi: 10.1093/milmed/usaa501

PubMed Abstract | Crossref Full Text | Google Scholar

23. Han, K, Chapman, SB, and Krawczyk, DC. Cognitive training reorganizes network modularity in traumatic brain injury. Neurorehabil Neural Repair. (2020) 34:26–38. doi: 10.1177/1545968319868710

PubMed Abstract | Crossref Full Text | Google Scholar

24. Vas, AK, Chapman, S, Aslan, S, Spence, J, Keebler, M, Rodriguez-Larrain, G, et al. Reasoning training in veteran and civilian traumatic brain injury with persistent mild impairment. Neuropsychol Rehabil. (2016) 26:502–31. doi: 10.1080/09602011.2015.1044013

PubMed Abstract | Crossref Full Text | Google Scholar

25. Soman, S, Cheng, J, Chao, S, Kong, J, Adamson, M, Ashford, JW, et al. Mid life hypometabolism in the precuneus may be associated with prior history of traumatic brain injury (TBI) (P 5.329). Neurology. (2014) 82:P5.329. doi: 10.1212/WNL.82.10_supplement.P5.329

Crossref Full Text | Google Scholar

26. Yan, X, Brown, AD, Lazar, M, Cressman, VL, Henn-Haase, C, Neylan, TC, et al. Spontaneous brain activity in combat related PTSD. Neurosci Lett. (2013) 547:1–5. doi: 10.1016/j.neulet.2013.04.032

PubMed Abstract | Crossref Full Text | Google Scholar

27. Han, K, Martinez, D, Chapman, SB, and Krawczyk, DC. Neural correlates of reduced depressive symptoms following cognitive training for chronic traumatic brain injury. Hum Brain Mapp. (2018) 39:2955–71. doi: 10.1002/hbm.24052

PubMed Abstract | Crossref Full Text | Google Scholar

28. Department of Veteran Affairs. VA/DoD clinical practice guidelines. VADoD clin pract guidel. (2016). Available at: http://www.healthquality.va.gov/policy/index.asp

Google Scholar

29. Bogner, J, and Corrigan, JD. Reliability and predictive validity of the Ohio State University TBI identification method with prisoners. J Head Trauma Rehabil. (2009) 24:279–91. doi: 10.1097/HTR.0b013e3181a66356

PubMed Abstract | Crossref Full Text | Google Scholar

30. Corrigan, JD, and Bogner, J. Initial reliability and validity of the Ohio State University TBI identification method. J Head Trauma Rehabil. (2007) 22:318–29. doi: 10.1097/01.HTR.0000300227.67748.77

PubMed Abstract | Crossref Full Text | Google Scholar

31. Corrigan, JD, and Bogner, J. Screening and identification of TBI. J Head Trauma Rehabil. (2007) 22:315–7. doi: 10.1097/01.HTR.0000300226.67748.3e

Crossref Full Text | Google Scholar

32. Green, CM, Kirk, JW, Connery, AK, Baker, DA, and Kirkwood, MW. The use of the Rey 15-item test and recognition trial to evaluate noncredible effort after pediatric mild traumatic brain injury. J Clin Exp Neuropsychol. (2014) 36:261–7. doi: 10.1080/13803395.2013.879096

PubMed Abstract | Crossref Full Text | Google Scholar

33. Eglit, GML, Jurick, SM, Delis, DC, Filoteo, JV, Bondi, MW, and Jak, AJ. Utility of the D-KEFS color word interference test as an embedded measure of performance validity. Clin Neuropsychol. (2020) 34:332–52. doi: 10.1080/13854046.2019.1643923

PubMed Abstract | Crossref Full Text | Google Scholar

34. Jurick, SM, Eglit, GML, Delis, DC, Bondi, MW, and Jak, AJ. D-KEFS trail making test as an embedded performance validity measure. J Clin Exp Neuropsychol. (2022) 44:62–72. doi: 10.1080/13803395.2022.2073334

PubMed Abstract | Crossref Full Text | Google Scholar

35. Zickefoose, S, Hux, K, Brown, J, and Wulf, K. Let the games begin: a preliminary study using attention process training-3 and luminosity brain games to remediate attention deficits following traumatic brain injury. Brain Inj. (2013) 27:707–16. doi: 10.3109/02699052.2013.775484

PubMed Abstract | Crossref Full Text | Google Scholar

36. Carey, CL, Woods, SP, Rippeth, JD, Gonzalez, R, Moore, DJ, Marcotte, TD, et al. Initial validation of a screening battery for the detection of HIV-associated cognitive impairment. Clin Neuropsychol. (2004) 18:234–48. doi: 10.1080/13854040490501448

PubMed Abstract | Crossref Full Text | Google Scholar

37. Heaton, RK, Miller, SW, Taylor, MJ, and Grant, I. Revised comprehensive norms for an expanded Halstead-Reitan battery: demographically adjusted neuropsychological norms for African American and Caucasian adults. Lutz: Psychological Assessment Resources, Inc. (2004).

Google Scholar

38. Brandt, J, and Benedict, RH. Hopkins verbal learning test-revised. Florida: Psychological Assessment Resources (1998).

Google Scholar

39. Delis, DC. Delis-Kaplan executive function system (D-KEFS). San Antonio, TX: Psychological Corporation (2001).

Google Scholar

40. Gronwall, DM. Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot Skills. (1977) 44:367–73. doi: 10.2466/pms.1977.44.2.367

PubMed Abstract | Crossref Full Text | Google Scholar

41. Tombaugh, TN. A comprehensive review of the paced auditory serial addition test (PASAT). Arch Clin Neuropsychol. (2006) 21:53–76. doi: 10.1016/j.acn.2005.07.006

PubMed Abstract | Crossref Full Text | Google Scholar

42. Smith, A. Symbol digits modalities test: manual. Los Angeles: Western Psychological Services (2007).

Google Scholar

43. Cicerone, KD, and Kalmar, K. Persistent postconcussion syndrome: the structure of subjective complaints after mild traumatic braininjury. J Head Trauma Rehabil. (1995) 10:1–17. doi: 10.1097/00001199-199510030-00002

Crossref Full Text | Google Scholar

44. Belanger, HG, Lange, RT, Bailie, J, Iverson, GL, Arrieux, JP, Ivins, BJ, et al. Interpreting change on the neurobehavioral symptom inventory and the PTSD checklist in military personnel. Clin Neuropsychol. (2016) 30:1063–73. doi: 10.1080/13854046.2016.1193632

PubMed Abstract | Crossref Full Text | Google Scholar

45. Soble, JR, Silva, MA, Vanderploeg, RD, Curtiss, G, Belanger, HG, Donnell, AJ, et al. Normative data for the neurobehavioral symptom inventory (NSI) and post-concussion symptom profiles among TBI, PTSD, and nonclinical samples. Clin Neuropsychol. (2014) 28:614–32. doi: 10.1080/13854046.2014.894576

PubMed Abstract | Crossref Full Text | Google Scholar

46. Vanderploeg, RD, Silva, MA, Soble, JR, Curtiss, G, Belanger, HG, Donnell, AJ, et al. The structure of postconcussion symptoms on the neurobehavioral symptom inventory: a comparison of alternative models. J Head Trauma Rehabil. (2015) 30:1–11. doi: 10.1097/HTR.0000000000000009

PubMed Abstract | Crossref Full Text | Google Scholar

47. Kolitz, BP, Vanderploeg, RD, and Curtiss, G. Development of the key behaviors change inventory: a traumatic brain injury behavioral outcome assessment instrument. Arch Phys Med Rehabil. (2003) 84:277–84. doi: 10.1053/apmr.2003.50100

PubMed Abstract | Crossref Full Text | Google Scholar