Cognitive Training for Post-Acute Traumatic Brain Injury: A Systematic Review and Meta-Analysis

Objective: To quantitatively aggregate effects of cognitive training (CT) on cognitive and functional outcome measures in patients with traumatic brain injury (TBI) more than 12-months post-injury. Design: We systematically searched six databases for non-randomized and randomized controlled trials of CT in TBI patients at least 12-months post-injury reporting cognitive and/or functional outcomes. Main Measures: Efficacy was measured as standardized mean difference (Hedges’ g) of post-training change. We investigated heterogeneity across studies using subgroup analyses and meta-regressions. Results: Fourteen studies encompassing 575 patients were included. The effect of CT on overall cognition was small and statistically significant (g = 0.22, 95%CI 0.05 to 0.38; p = 0.01), with low heterogeneity (I2 = 11.71%) and no evidence of publication bias. A moderate effect size was found for overall functional outcomes (g = 0.32, 95%CI 0.08 to 0.57, p = 0.01) with low heterogeneity (I2 = 14.27%) and possible publication bias. Statistically significant effects were also found only for executive function (g = 0.20, 95%CI 0.02 to 0.39, p = 0.03) and verbal memory (g = 0.32, 95%CI 0.14 to 0.50, p < 0.01). Conclusion: Despite limited studies in this field, this meta-analysis indicates that CT is modestly effective in improving cognitive and functional outcomes in patients with post-acute TBI and should therefore play a more significant role in TBI rehabilitation.


INTRODUCTION
Traumatic brain injury (TBI) causes ongoing disability for millions worldwide (Wilson et al., 2014), with cognitive impairment and psychosocial issues presenting major barriers to positive social outcomes such as community reintegration and employment (Rice-Oxley and Turner-Stokes, 1999). Cognitive impairment in TBI frequently affects the domains of attention, memory, executive functions, processing speed, language, and visuospatial skills (Dikmen et al., 2009). Reviews (Gordon et al., 2006;Cicerone et al., 2011;Lu et al., 2012) have suggested that cognitive rehabilitation for TBI, which encompasses several therapeutic strategies and interventions, can be beneficial for improving these cognitive domains and even community functioning. These interventions may include education, goal-setting, counseling, and internal and external compensation strategies targeting specific cognitive domains.
An on-going issue within the wider field of cognitive rehabilitation is a lack of a consensus for taxonomy of cognitive interventions, including of cognitive training (CT), but here we utilize a working definition consistent across key contributors to the literature (Clare et al., 2003;Buschert et al., 2010;Gates and Valenzuela, 2010). Here, we define and assess the impact of one specific form of cognitive rehabilitation which is seen to be cost-effective, scalable, adaptive (Gates and Valenzuela, 2010): CT. We and others have operationally defined CT to include four main characteristics: (1) repeated practice, (2) on problem-orientated tasks, (3) using standardized stimuli, and (4) targeting specified cognitive domains (Gates and Valenzuela, 2010;Bahar-Fuchs et al., 2013). CT aims to restore impaired skills or harness compensatory mechanisms (Buschert et al., 2010) and can include drill and practice exercises or applied mnemonic strategies. It can be administered either in paperand-pen format, typically facilitated on a one-on-one basis by a therapist, or computer-assisted CT that can be supervised in a group setting or delivered at home at the individual level. It is therefore important to distinguish CT from the more holistic concept of cognitive rehabilitation that may include aspects of CT targeted to improve cognitive deficits, but also includes non-CT interventions aimed at improving psychological, emotional, motivational, and interpersonal functioning (Gordon et al., 2006).
Restorative treatments and compensatory strategies are generically recommended for the rehabilitation of TBI patients displaying cognitive deficits (INCOG guidelines; (Bayley et al., 2014)). Based on efficacy in other clinical populations (Wykes et al., 2011;Lampit et al., 2014;Leung et al., 2015), CT may have therapeutic potential for TBI. Yet prior reviews of cognitive interventions (Cicerone et al., 2005(Cicerone et al., , 2011Rees et al., 2007) have not specifically addressed the efficacy of CT for TBI patients. These reviews have attempted to synthesize across mixed samples with various kinds of acquired brain injury (ABI), as well as combine different types of cognitive therapies, and permitted a diversity of study designs. A recent meta-analysis (Rohling et al., 2009) highlights the potential therapeutic benefits of CT for specific brain injury deficits, but similar to the reviews, their study is of mixed etiology and also combines samples of varying time since injury, which although is inevitable, potentially introduces spontaneous recovery as a confounder. However, this could be attenuated by confining research to before or after 12 months post injury.
Accordingly, using a meta-analytic approach, this study aims to systematically evaluate whether operationally defined CT is effective in improving cognitive and functional outcomes at least one-year post-TBI, and to analyze potential moderators that may affect treatment outcomes. The study will analyze individual cognitive and functional domains, as well as overall cognition and overall functioning by pooling the individual domains together, respectively. Investigation of individual domains allows for identification of specific training effects, whilst pooling together individual domains allows for the identification of more general or overall effects that may not be apparent at the individual level for a multitude of reasons such as low sample size or poor study design. Additionally, to investigate potential moderators of training, a sub-group analysis will be conducted. Studies in this field are often small, underpowered and vary in design, thus a meta-analysis can add clarity, as it allows for amalgamation of these small studies to produce an overall analysis with greater statistical power and further reaching conclusions. Thus, as the field of cognitive rehabilitation in TBI is still in its infancy and much more research is required, a meta-analysis could prove crucial in identifying the future direction of CT, and potential design factors that may prove most effective.

MATERIALS AND METHODS
This systematic review and meta-analysis adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Supplementary Table S1), (Liberati et al., 2009) was prospectively registered with PROSPERO (CRD42014013274) and largely follows methods established in our previous meta-analyses (Lampit et al., 2014;Leung et al., 2015).

Eligibility Criteria
We included both non-randomized and randomized controlled trials (RCTs) provided they investigated the effects of a CT intervention on cognitive and/or functional outcomes in individuals (both intervention and controls groups) with postacute TBI (time since injury ≥12 months, study mean). Thus we excluded studies that included healthy or acute TBI controls. Eligible outcomes were baseline and post-training performance on measures of cognition, Instrumental Activities of Daily Living (IADL) or dysexecutive functioning, defined as holistic disruptions to frontal lobe functions such as behavior, executive functions and cognition. CT was defined as any intervention incorporating computer-assisted CT, penciland-paper-administered CT, or cognitive strategy training, practiced systematically for a minimum of 4 hours. Studies that used combined interventions (e.g., CT with standard physical rehabilitation) were eligible if CT comprised at least 50% of the total intervention duration. and DC conducted initial screening for eligibility using title and abstract, and then independently examined full-text articles for inclusion.

Study Appraisal and Risk of Bias Within Studies
A modified form of the Physiotherapy Evidence Database (PEDro) scale (Maher et al., 2003), designed for rating the quality of RCTs, was used by HH and DC to assess the methodological rigor of included studies. As blinding of participants and therapists is impractical in CT trials, these two PEDro items were not assessed, and the maximum overall score (i.e., highest study quality) became 9 (Lampit et al., 2014). Risk of bias resulted from lack of assessor blinding or adherence to intention-totreat analysis was assessed using the Cochrane's risk of bias tool (Higgins and Green, 2011). RCTs with high or unclear risk of bias for either of these categories were defined as having a high risk of bias.

Data Extraction
Cognitive and functional outcome data were extracted in the form of means and standard deviations for each group immediately pre-and post-intervention using a correlation of 0.6 between timepoints or mean group change, and entered into Comprehensive Meta-Analysis version 2 (CMA, Biostat, Englewood, NJ, USA). Coding of outcomes into cognitive domains and effect direction were performed using the Compendium of Neuropsychological Tests (Strauss et al., 2006) or by consensus.

Data Analysis
The dependent variable was standardized mean differences (SMD) (calculated as Hedges' g to correct for small sample sizes) of change from baseline to post-intervention between CT and control groups. Precision of SMD was estimated using 95% confidence intervals (CI). Analyses were conducted on individual cognitive (executive functions, verbal memory, working memory, attention, processing speed, non-verbal memory, visuospatial, language) and functional domains (IADL and dysexecutive functions). Analyses were also conducted on Overall Cognition and Overall Function, which were a result of combining or pooling the respective individual cognitive or functional domains together (Wykes et al., 2011;Lampit et al., 2014). An effect size of g < 0.30 was considered small, g ≥ 0.30 moderate, and g ≥ 0.60 large.
To avoid selective analyses of outcomes, study-level SMDs from the same cognitive domain were combined into a single effect estimate, corrected for inter-correlation across outcomes using a correlation of 0.7 (Gleser and Olkin, 2009). Pooling of outcomes across studies was conducted using random effects model. Heterogeneity across studies was quantified using the I 2 statistic, which quantifies the proportion of variance due to heterogeneity in true effects rather than random error (Higgins et al., 2003). I 2 values of 25, 50, and 75% imply low, moderate, and large heterogeneity, respectively.
To assess publication bias (small-study effect), funnel plots were visually inspected and formally tested using Egger's Test of the Intercepts if at least 10 studies were available for analysis (Egger et al., 1997;Sterne et al., 2011). If significant asymmetry was detected (p < 0.1), we estimated the magnitude of smallstudy effect using Duval and Tweedie's Trim and Fill method (Duval and Tweedie, 2000).
In order to detect design factors that may affect CT efficacy, we performed subgroup meta-analyses based on mixedeffects model. Between-subgroup heterogeneity was tested using Cochrane's Q statistic (significant at p < 0.05). Analyses were performed for overall cognitive and overall functional outcomes based on the following study characteristics: study design (randomized or non-randomized), intervention type (combined, strategy or training), control type (active or passive), total hours of training (≤20 or >20 h), session length (≤60 or >60 mins), and session frequency (<4 or ≥4 a week). Univariate metaregressions were used to detect relationships between cognitive results and PEDro score, sample size and year of publication. All analyses were conducted in CMA.

Study Selection
After removal of duplicates, 3464 articles were screened for inclusion based on published title and abstract. 421 articles were suitable for full-text screening, including one manually added study (Figure 1). After full-text screening, 15 studies were eligible for review, however one focused solely on children and adolescence (Thomas-Stonell et al., 1994) and was therefore excluded, leaving 14 studies for analysis. Age was not a screening criterion, but given the fact that TBI can manifest quite differently during adolescent brain development, it was deemed appropriate to exclude this study.

Efficacy on Overall Functional Outcomes
A pooled analysis of the seven studies reporting functional outcomes revealed a moderate and statistically significant effect size (g = 0.32, 95% CI 0.08 to 0.57, p = 0.01; Figure 6A). Heterogeneity across studies was low (I 2 = 14.27%, 95% CI 0% to 75.39%). The funnel plot revealed asymmetry, indicating more positive results in smaller studies. A trim and fill analysis revealed a smaller and statistically non-significant effect size (g = 0.23, 95% CI −0.05 to 0.51, p = 0.11 Figure 3).

Moderators of CT Efficacy
Possible moderators of training effects on overall cognitive ( Figure 7A) and functional ( Figure 7B) outcomes were investigated using sub-group analyses. For overall cognition,  we did not find significant between group differences for study design, intervention type, control type, total hours of training, session length or session frequency. However there was a strong trend toward less training being more effective on overall cognition, with studies providing 20 h or less of training (g = 0.41, 95% CI 0.14 to 0.68, p < 0.01, I 2 = 21.40%) being more effective than those that provided more than 20 hours (g = 0.06, 95% CI −0.15 to 0.28, p = 0.55; Q = 3.80, df = 1, p = 0.05). To further investigate this trend, we conducted an analysis on a post hoc basis. This correlation comparing length of training and severity of injury was found to be non-significant (r = 0.26, p = 0.44, n = 11). There were no significant between-subgroup differences with overall functional outcomes for any of these moderators. As both IADL and working memory outcomes had moderate heterogeneity, subgroup analyses were conducted, but no significant differences were found for either. For other domains, heterogeneity was close to zero, thus subgroup analyses were not warranted.
A matrix was constructed to investigate whether the content of training (the domain/s that were trained) moderated outcomes on specific cognitive domains outcomes, i.e., if there was transfer. A summary of these cognitive outcomes is presented in Figure 8, and categorized by study and cognitive domain trained. No statistical analysis was run on this data, but the matrix illustrates which cognitive domains were trained (gray color cells), and the effect sizes at a study level or pooled together at a domain level.

DISCUSSION
Cognitive-based interventions are effective in several clinical populations (Wykes et al., 2011;Lampit et al., 2014;Leung et al., 2015), and here we expand the evidence base to include postacute TBI. CT was particularly effective on overall cognition, as well as the cognitive domains of verbal memory and executive function, and jointly improved individuals' IADLs whilst reducing severity of dysexecutive signs and symptoms. TBI is extremely heterogeneous in its etiology and origins. Accordingly, patients present with a variety of cognitive deficits (Dikmen et al., 2009), with information processing speed and verbal memory most commonly affected (Skandsen et al., 2010). It is therefore promising that this study found not only general cognitive efficacy, but specific efficacy for executive function and verbal memory. Contrary to this, a previous meta-analysis (Rohling et al., 2009) found that cognitive rehabilitation was not effective in TBI patients. However, that study combined several different types of cognitive interventions and patients varied greatly in the time since injury. As mentioned, cognitive rehabilitation encompasses a variety of therapeutic approaches, and here we aim to focus on CT as operationally defined in the introduction. Moreover, timing of intervention may well be critical. TBI often progresses through stages of unconsciousness and emerging consciousness; confusion with dense anterograde amnesia that can vary from days to several weeks; and a longterm period of restoration of cognitive, neuropsychological and social functioning that can last for several years (Povlishock and Katz, 2005). Here we have clarified the literature to some extent and shown that one approach to cognitive rehabilitation, CT, is effective for certain cognitive domains in the post-acute phase.
Cognitive rehabilitation, which can include CT, is known to improve community functioning even several years after TBI (Gordon et al., 2006). Our analyses suggest that CT may itself be sufficient to retrain functional skills or facilitate compensatory mechanisms that can translate into everyday outcomes. In the TBI literature, functionality is often measured by IADL scales and assessment of dysexecutive syndrome. Given the importance of both IADLs and dysexecutive syndrome to everyday life, it is noteworthy that CT produced a moderate effect size on these outcomes when combined. Furthermore, the low heterogeneity surrounding this estimate indicates that the result is subject to little explainable variation and is thus an accurate estimate of effect size. Whilst the combination of these two outcomes may appear to be novel, previous studies have shown loose connections between the two (Pa et al., 2009;Marshall et al., 2011). Importantly, this result suggests that CT has the potential to achieve so-called "far-transfer" (Barnett and Ceci, 2002) to positively influence real world issues faced by TBI patients.
Despite these combined results, IADL or dysexecutive functioning did not produce significant improvements when considered separately. This may be due to insufficient power, as not only were there limited studies examining these outcomes, and small sample sizes, but a separate analysis of the two domains displayed larger CI. Positive effects on daily function were restricted to a pooled analysis of combined dysexecutive and IADL outcomes. Whilst this approach has some precedence (Pa et al., 2009;Marshall et al., 2011) and was planned a priori, when each type of outcome was considered individually no significant effects were observed. This therefore brings up the issue as to what can be reasonably combined in terms of outcomes measures within a meta-analysis -a topic treated in detail by   Borenstein et al. (2009). In their example, combining tests of Maths and English is justified, "If our goal is to assess the impact on performance in general, then the answer is Yes." (Borenstein et al., 2009, p. 357). Our goal was to assess the impact of CT on those areas that most impact day to day function in TBI rehabilitation, inclusive of both dysexecutive syndrome (Rao and Lyketsos, 2000) and impaired IADLs (Colantonio et al., 2004). Hence, there are promising indications that CT can help support daily function in chronic TBI patients, but clearly more research is required to parse these effects out.
Interestingly, CT had a significant effect on executive function but not dysexecutive outcomes. This may appear paradoxical given the two outcomes are intrinsically (and inversely) related (Ardila, 2013). However, this pattern of results can be explained by the nature of the data. Executive outcomes originate from neuropsychological tests that are generally objective, quantitative and continuous, and thus sensitive to change, whilst dysexecutive instruments are generally subjective, qualitative and ordinal. By nature these instruments are therefore of lower resolution and require much larger behavioral change before detection. Further research is therefore required to determine whether CT can improve not just psychometric executive function but also minimize the presentation or severity of dysexecutive symptoms in post-acute TBI.
Of the potential moderators analyzed, a strong trend was found only for training hours. Studies where subjects trained for ≤ 20 h showed improvements in overall cognition compared to studies where patients trained more. This is consistent with evidence of weaker effect sizes in studies that provided intense training schedules (Lampit et al., 2014) or long training durations (Toril et al., 2014) in healthy older adults. A possible explanation for this trend could be the heterogeneity in injury severity amongst the population, whereby those with more severe injuries required more training, with the assumption that increased severity means lower improvement. We conducted a post hoc analysis to test this theory, but we did not find a relationship between length of training and injury severity across studies. However given that this post hoc analysis was conducted on such a small sample size, and thus lacks power, we cannot completely rule out that the trend in training time is linked with injury severity. Nonetheless, it is intriguing that across different clinical cohorts there may be converging evidence for the importance of avoiding overdosing, or over-training participants. This concept is even more salient in the field of TBI, where rehabilitation is often guided by the principle that greater intensity or number of repetitions is better. Here, we conclude that CT at a circumscribed dose, at the right time in the post-acute stage, is preferable.
Other possible moderators analyzed were found to be nonsignificant, consistent with the small number of studies and minimal explainable between-study variance -a concept we have previously discussed (Leung et al., 2015). More specifically, our data suggests that an important study design factor, whether randomization occurred or not, did not impact CT efficacy in post-acute TBI. However, we cannot rule out that this could be due to a lack of power, a notion counter-weighed by similar effect sizes from the two design methods. This finding supports our decision to combine both non randomized and RCTs into a single analysis.
To further explore moderating or driving factors of cognitive outcomes, we investigated whether there was a link between training content and cognitive outcomes (Figure 8). For this population, cross-transfer, the idea that training in one domain can result in improvements in another untrained domain, appears to be unlikely. This is evident when looking at the columns for working memory, speed, language and executive functions. We can see here, as indicated by the gray cells, or lack thereof, that there was minimal training on these domains, however there was training in many other domains. The fact that there are no significant results, in addition to the obvious lack of power, suggests a lack of cross transfer, a sentiment mirrored in previous research (Edwards et al., 2002). Importantly, we cannot conclude that certain domains, such as working memory, speed, language, and executive functions are ineffective or nonresponsive in this population. Instead, this figure suggests that there is need for more trials that are training or targeting multiple different cognitive domains.
Limitations include potential selection bias that may have influenced results. Our narrow eligibility criteria and decision to include only studies published in English, resulted in wellcontrolled CT studies being excluded from this analysis, such as trials implemented before 12 months post-injury. We chose this temporal window for clinical reasons, namely to minimize the confounding effects of spontaneous recovery of function that can occur during the acute and sub-acute stages (Sohlberg and Mateer, 2001). A caveat to this criterion was a reliance on studylevel characteristic. Some studies included participants from 3 months post-injury and onward, resulting in large variations in time since injury, despite the reported study average being >12 months post injury. To further clarify the specificity of our findings to this temporal window, a patient-level meta-analysis is required. In addition, our decision to only include functional outcomes that could be categorized as IADL or dysexecutive functioning was a potential source of selection bias, but a decision we consider clinically principled since functional outcomes from three studies e.g., 'Life-3' (Dawson et al., 2013;Cantor et al., 2014;Twamley et al., 2014) were idiosyncratic and deemed incomparable.
A notable limitation of our analysis is the heterogeneity in injury severity, but this is reflective of the state of the field. Indeed, many of the included studies themselves comprised patients of varying TBI severity (from mild to severe). However, low statistical heterogeneity indicates that there was no other important source of bias between study variance besides total hours of training. Average PEDro quality scores were relatively low, but this was mainly attributable to two points being allocated for randomization procedures. We specifically tested this factor and found it was not influencing effect size estimates. Perhaps the largest limitation of our study is the relative infancy of the field. With only six of the studies included being RCTs, the field is somewhat nascent, thus our results must be viewed with some skepticism. Nonetheless there is enough power to show effectiveness of CT on overall cognitive and functional outcomes, however, clearly future research with more rigorous trial design and reporting is required.
TBI is fundamentally heterogeneous and manifests in complex and unpredictable patterns, resulting in diverse physical, behavioral, cognitive and functional outcomes. Discerning therapeutic efficacy in this population is therefore challenging. Despite this potential for background 'noise' and the limited studies in a still developing field, we found encouraging results with implications for clinical practice. Namely, significant cognitive gains were seen as a result of CT more than one-year post-injury when spontaneous neurological recovery is assumed to have stabilized. It is encouraging to see that there may be a possible link between training intensity and overall efficacy, but further studies with larger sample sizes and more heterogeneous populations are required to explore this relationship. Small samples and lack of power have meant that the effectiveness of CT in TBI has been inconclusive in individual studies -this is precisely the condition when a meta-analysis can add value and clarity to a field (Borenstein et al., 2009). This meta-analysis thereby provides evidence that CT may be modestly effective in promoting cognitive and functional gains in everyday life. Accordingly, further investigation of different approaches to CT is required along with health economic analyses of the costs and benefits of CT for post-acute TBI.