Tanja Könen1,2*
Julia Karbach1,2- 1Department of Psychology, Goethe University, Frankfurt, Germany
- 2Center for Research on Individual Development and Adaptive Education of Children at Risk (IDeA), Frankfurt, Germany
Over the last decade, the prospect of improving or maintaining cognitive functioning has provoked a steadily increasing number of cognitive training studies. Central target populations are individuals at risk for a disadvantageous development, such as older adults exhibiting cognitive decline or children with learning impairments. They rely on cognitive resources to meet the challenges of an independent life in old age or requirements at school.
To support daily cognitive functioning, training outcomes need to generalize to other cognitive abilities. Such transfer effects are, however, highly discussed. For example, recent meta-analyses on working memory training differed in the conclusion on the presence (Au et al., 2015; Karbach and Verhaeghen, 2014) or absence of transfer effects (Melby-Lervåg and Hulme, 2013). Usually training-specific design factors such as type, intensity, duration, and feedback routines are discussed as reasons for such inconsistent findings. However, even individuals participating in exactly the same training regime highly differ in their training outcomes. We argue that it is time to study the individual development during trainings to understand these differential outcomes. It is time to have a closer look at the intraindividual training data.
Within-Person Information in Training Data
The classical findings of a training study – whether a cognitive training group showed training and transfer effects compared to a control group – could be amended and sometimes even better understood by further analyzing the training sessions on the within-person level. Intraindividual training data could offer four types of information: (1) Intraindividual performance trajectories across all training sessions can demonstrate which participants show training effects and when they reach their performance maximum. (2) Intraindividual performance fluctuations – between and within training sessions – show which participants vary substantially in their performance (despite general training improvement). (3) Intraindividual couplings of performance fluctuations with other variables can reveal which internal and external factors contribute to individual performance and to what extend participants differ in the strength of these relations. (4) Further combinations of these types can be considered as well. For example, substantial performance fluctuations (type 2) can in theory be both, an indicator of adaptive (e.g., varying strategies; Siegler, 1994, 2007) or maladaptive processes (e.g., vulnerability to disturbing influences) during training. Relating fluctuations to other variables such as daily motivation and affect (type 3) or to performance trajectories (type 4, here combining 1 and 2) can contribute to exposing them as either beneficial or obstructive for the individual training success.
Rationale of the Approach
There is a strong rationale behind this approach of looking at the intraindividual and dynamic characteristics of cognitive training data. Until recently, the training phase of intervention studies constituted a black box and we gradually tested hypothesized mechanisms through the variation of training conditions. Combining this tradition with within-person analyses is a more efficient approach to understanding how cognitive training works, for whom it works, and in which contexts and situations it works by combining the benefits of two different research perspectives. The cognitive training field provides cumulating evidence for the role of individual differences in training (e.g., Lövdén et al., 2012; Karbach and Unger, 2014, for a review), where some individuals benefit more than others. Most of the time, however, one can only speculate about the underlying mechanisms that lead to these differences. Measurement intensive research, such as the field of ambulatory assessment, provides cumulating evidence that individuals differ in intraindividual cognitive processes. They differ in the strength of cognitive performance fluctuations (i.e., short-term variations) and their couplings with possible antecedents and consequences (e.g., Riediger et al., 2014; Könen et al., 2015).
Combining both perspectives is reasonable, because a central assumption behind cognitive training is that it fosters intraindividual change in cognitive performance. But how exactly do individuals come from A to B (i.e., pre to post level)? Dynamic systems theory predicts that a later state of cognition (yt+1) is a function of an earlier state (yt) with the function being an adaptive mechanism to perturbation (yt+1 = f(yt); cf. Weisstein, 1999; van Geert and Steenbeek, 2005). In our case, the perturbation would be a challenging cognitive training and individuals do respond to this situation. Ideally, they develop new cognitive resources because they experience a prolonged mismatch between their resources and the situational demands (cf. Lövdén et al., 2010). Practically, some individuals gain more than others, even if the training is adaptive and well designed (thus, neither too easy, nor too difficult). Individuals likely vary in within-person processes over time that eventually produce between-person differences in training outcomes. Consequently, it is just a logical step to look at the intraindividual level to find out what happens over the course of the training.
Implications for Specific Populations
A within-person approach to cognitive training data is all the more beneficial the more heterogeneous the trained individuals are. Good examples to illustrate this point are specific populations, such as children with learning disabilities and older adults. On the one hand, they demonstrate between group differences compared to healthy controls or young adults, and on the other hand they likely exhibit substantial within group differences. Such differences are of major concern because they can influence training outcomes or even mask effects. Therefore, both populations are particularly useful to highlight the benefits of a within-person approach.
Learning disabilities constitute an important target for cognitive training, because they have been related to substantial working memory impairments (e.g., Schuchardt et al., 2008; Fischbach et al., 2014). However, the profile of these impairments varies considerably between disabilities (e.g., reading vs. spelling disability, Brandenburg et al., 2014). Varying impairments can influence training outcomes because the initial performance level is often related to training and transfer gains (e.g., Zinke et al., 2013; Karbach et al., 2014). Consequently, individuals sharing a specific learning disability might not only function differently from healthy controls, but also from those with other learning disabilities. Further, they likely show a substantial amount of within-group variability and may, for example, vary in the etiology of the learning disability and with regard to possible treatments they previously received (cf. Shah et al., 2012). In cases with such crucial heterogeneity, within-person analyses can reveal whether and to what extent participants perform differently in the course of a cognitive training (i.e., show differential intraindividual effects). For example, they might show different cognitive performance trajectories and different antecedents and consequences of performance fluctuations (i.e., differ in the internal and external factors contributing to individual performance). A related within-person finding comes from a sample of elementary school children. Daily working memory performance was related to last night's sleep quality and this within-person coupling varied reliably between children. It was stronger for low performing children, indicating that they were more vulnerable to the influence of last night's sleep (Könen et al., 2015).
Further promising examples for the usefulness of within-person analyses come from training research with older adults. Older adults demonstrated on average a slower growth during working memory training than younger adults (Bürki et al., 2014) and their performance fluctuated less in all tasks of a broad cognitive training (working memory, episodic memory, and processing speed, Schmiedek et al., 2013). In addition, their performance fluctuations in reasoning and perceptual speed were positively associated with practice-related gains on the same tasks (Allaire and Marsiske, 2005), implying that fluctuations likely indicated an adaptive process in this case. Taken together, these examples demonstrate that the performance of older adults during cognitive trainings differs from the performance of younger adults, which is valuable additional information on top of between-person differences in training outcomes (e.g., in Schmiedek et al., 2010). They suggest a need to question what causes these differences and whether older adults' behavior can be modified through, for example, instruction and feedback (e.g., Garrett et al., 2012). Interestingly, the within-person relation between daily motivation and daily working memory training performance was considerably lower in older compared to younger adults (Brose et al., 2010), raising the question whether there are untapped motivational resources and whether the effectiveness of cognitive trainings for older adults may be improved by building more on these motivational resources. Still, much more research is needed to further confirm and elaborate these first recent findings.
Statistical Modeling
Suitable modeling approaches are, for example, multilevel modeling (e.g., Brose et al., 2012; Schmiedek et al., 2013), structural equation modeling (SEM; e.g., Brose et al., 2010; Bürki et al., 2014), dynamical systems analysis (e.g., Gasimova et al., 2014), and combinations thereof (e.g., multilevel SEM, e.g., Könen et al., 2015). We suggest using or at least starting with multilevel models because they are easy to access and to apply (e.g., closely related to standard regression, implemented in all commonly used software packages) and they are perfectly suitable to study intraindividual trajectories, fluctuations, and couplings. In these models, a certain number of measurement occasions (e.g., training sessions, Level 1, within-person level) are nested within a certain number of individuals (Level 2, between-person level). One can, for instance, predict a Level 1 variable (e.g., daily cognitive performance) with another Level 1 variable (e.g., daily motivation) and test the mean intraindividual effect (fixed effect) and examine whether individuals (Level 2) differ reliably in the strength of this relation (random effect). Hoffman and Stawski (2009) provide a detailed discussion of multilevel analyses with longitudinal data.
Practical Considerations
There is already a number of existing cognitive training studies with data suitable for all or at least a part of the proposed within-person analyses (e.g., Bürki et al., 2014). We want to encourage the field to further explore the potential of the existing data and to consider within-person processes when designing future training studies. Therefore, one should pay particular attention to the number of measurement occasions, the sample size, and the sensitivity of the measures to fluctuations.
The number of measurement occasions (K) and the sample size (N) should be reviewed together in a multilevel context. Cognitive trainings are expected to change cognitive performance on a construct level, so the frequency of training sessions (here: K) is usually high. Whether a given K is sufficient for a certain within-person analysis depends on N as well as the size and nature of the effect of interest (e.g., an intraindividual coupling). The typical 10–20 training sessions applied in cognitive trainings (cf. Melby-Lervåg and Hulme, 2013, Appendix) are sufficient for within-person analyses if N is appropriate. The first step to calculate the necessary N would be to conduct a traditional power analysis concerning the central transfer effects of the training (e.g., with G*Power; Faul et al., 2007). Then one could use the resulting N of trained participants as a lower starting point for Monte Carlo simulations on the multilevel parameters (to estimate the probability of recovering known population parameters given K and N; for example with Mplus; see Bolger et al., 2012; Bolger and Laurenceau, 2013; for a step-by-step description). One should simulate different combinations of K and N and could also consider other design factors (e.g., the number of observed indicators) to find an optimal trade off and study design (cf. von Oertzen and Brandmaier, 2013). In case an existing N is slightly lower than preferred, Bayesian estimation could be eligible (Hox et al., 2012).
Variables that might impact daily cognitive performance (e.g., current motivation, affect, and health) should be observed with every training session, if feasible. This allows for the estimation of couplings over time. For instance, Brose et al. (2012) found that working memory performance during a cognitive training in young adults was lower on days with reduced motivation, reduced control of attention, and enhanced negative affect. To allow for such analyses, one has to consider the temporal dynamics of the variables and carefully select measures that are sensitive to fluctuations. For example, an affect scale has to capture the current state and should not include items for rare affective states. The variables of interest could be assessed with short scales (see Ziegler et al., 2014) to reduce testing time and participant burden. The reliability of these scales and their sensitivity to fluctuations can be analyzed with multilevel models (e.g., Wilhelm and Schoebi, 2007). We highly recommend the handbook of Mehl and Conner (2012) for a detailed and elaborate introduction to measurement intensive research.
Summary and Outlook
Cognitive training data could offer more information than is currently used in the field. We suggest analyzing intraindividual performance trajectories, fluctuations, and couplings and to consider such within-person analyses when designing future training studies. This seems to be particularly promising for studies with heterogeneous samples. Individuals likely vary in within-person effects over time that eventually produce between-person differences in training outcomes. Thereby, a within-person approach could contribute to understanding training outcomes and to generating theories about the underlying mechanisms.
Some hypotheses could then be further tested and validated through classic variations of training conditions. Experimental variation is the only way to ensure valid causal inferences in cognitive psychology. However, it is practically impossible to test all thinkable explanations for the current heterogeneous findings in training research only in this way. It seems much more feasible to test certain mechanisms that were already identified in the intraindividual dynamics of the training data. This highlights how both perspectives complement each other and could be combined to an efficient approach to study the mechanisms that drive or hamper cognitive training success.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We thank Jan Kühnhausen and Florian Schmiedek for valuable comments on the first draft of this manuscript.
References
Allaire, J. C., and Marsiske, M. (2005). Intraindividual variability may not always indicate vulnerability in elders' cognitive performance. Psychol. Aging 20, 390–401. doi: 10.1037/0882-7974.20.3.390
Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., and Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: a meta-analysis. Psychon. Bull. Rev. 22, 366–377. doi: 10.3758/s13423-014-0699-x
Bolger, N., and Laurenceau, J.-P. (2013). Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research. New York, NY: Guilford Press.
Bolger, N., Stadler, G., and Laurenceau, J.-P. (2012). “Power analysis for intensive longitudinal measurement designs,” in Handbook of Research Methods for Studying Daily Life, eds M. R. Mehl and T. S. Conner (New York, NY: Guilford Press), 285–301.
Brandenburg, J., Klesczewski, J., Fischbach, A., Schuchardt, K., Büttner, G., and Hasselhorn, M. (2014). Working memory in children with learning disabilities in reading versus spelling: searching for overlapping and specific cognitive factors. J. Learn. Disabil. doi: 10.1177/0022219414521665. [Epub ahead of print].
Brose, A., Schmiedek, F., Lövdén, M., and Lindenberger, U. (2012). Daily variability in working memory is coupled with negative affect: the role of attention and motivation. Emotion 12, 605–617. doi: 10.1037/a0024436
Brose, A., Schmiedek, F., Lövdén, M., Molenaar, P. C. M., and Lindenberger, U. (2010). Adult age differences in covariation of motivation and working memory performance: contrasting between-person and within-person findings. Res. Hum. Dev. 7, 61–78. doi: 10.1080/15427600903578177
Bürki, C., Ludwig, C., Chicherio, C., and de Ribaupierre, A. (2014). Individual differences in cognitive plasticity: an investigation of training curves in younger and older adults. Psychol. Res. 78, 821–835. doi: 10.1007/s00426-014-0559-3
Faul, F., Erdfelder, E., Lang, A.-G., and Buchner, A. (2007). G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191. doi: 10.3758/BF03193146
Fischbach, A., Könen, T., Rietz, C. S., and Hasselhorn, M. (2014). What is not working in working memory of children with literacy disorders? Evidence from a three-year-longitudinal study. Read. Writ. 27, 267–286. doi: 10.1007/s11145-013-9444-5
Garrett, D. D., MacDonald, S. W. S., and Craik, F. I. M. (2012). Intraindividual reaction time variability is malleable: feedback- and education-related reductions in variability with age. Front. Hum. Neurosci. 6:101 doi: 10.3389/fnhum.2012.00101
Gasimova, F., Robitzsch, A., Wilhelm, O., Boker, S. M., Hu, Y., and Hülür, G. (2014). Dynamical systems analysis applied to working memory data. Front. Psychol. 5:687. doi: 10.3389/fpsyg.2014.00687
Hoffman, L., and Stawski, R. S. (2009). Persons as contexts: evaluating between-person and within-person effects in longitudinal analysis. Res. Hum. Dev. 6, 97–120. doi: 10.1080/15427600902911189
Hox, J., van de Schoot, R., and Matthijsse, S. (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Surv. Res. Methods 6, 87–93.
Karbach, J., Strobach, T., and Schubert, T. (2014). Adaptive working-memory training benefits reading, but not mathematics in middle childhood. Child Neuropsychol. doi: 10.1080/09297049.2014.899336. [Epub ahead of print].
Karbach, J., and Unger, K. (2014). Executive control training from middle childhood to adolescence. Front. Psychol. 5:390. doi: 10.3389/fpsyg.2014.00390
Karbach, J., and Verhaeghen, P. (2014). Making working memory work: a meta-analysis of executive-control and working memory training in older adults. Psychol. Sci. 25, 2027–2037. doi: 10.1177/0956797614548725
Könen, T., Dirk, J., and Schmiedek, F. (2015). Cognitive benefits of last night's sleep: daily variations in children's sleep behavior are related to working memory fluctuations. J. Child Psychol. Psychiatry 56, 171–182. doi: 10.1111/jcpp.12296
Lövdén, M., Bäckman, L., Lindenberger, U., Schäfer, S., and Schmiedek, F. (2010). A theoretical framework for the study of adult cognitive plasticity. Psychol. Bull. 4, 659–676. doi: 10.1037/a0020080
Lövdén, M., Brehmer, Y., Li, S. C., and Lindenberger, U. (2012). Training induced compensation versus magnification of individual differences in memory performance. Front. Hum. Neurosci. 6:141. doi: 10.3389/fnhum.2012.00141
Mehl, M., and Conner, T. (2012). Handbook of Research Methods for Studying Daily Life. New York, NY: Guilford Press.
Melby-Lervåg, M., and Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Dev. Psychol. 49, 270–291. doi: 10.1037/a0028228
Riediger, M., Wrzus, C., Klipker, K., Müller, V., Schmiedek, F., and Wagner, G. G. (2014). Outside of the laboratory: associations of working-memory performance with psychological and physiological arousal vary with age. Psychol. Aging 29, 103–114. doi: 10.1037/a0035766
Schmiedek, F., Lövdén, M., and Lindenberger, U. (2010). Hundred days of cognitive training enhance broad cognitive abilities in adulthood: findings from the COGITO study. Front. Aging Neurosci. 2:27. doi: 10.3389/fnagi.2010.00027
Schmiedek, F., Lövdén, M., and Lindenberger, U. (2013). Keeping it steady: older adults perform more consistently on cognitive tasks than younger adults. Psychol. Sci. 24, 1747–1754. doi: 10.1177/0956797613479611
Schuchardt, K., Maehler, C., and Hasselhorn, M. (2008). Working memory deficits in children with specific learning disorders. J. Learn. Disabil. 41, 514–523. doi: 10.1177/0022219408317856
Shah, P., Buschkuehl, M., Jaeggi, S., and Jonides, J. (2012). Cognitive training for ADHD: the importance of individual differences. J. Appl. Res. Mem. Cogn. 1, 204–205. doi: 10.1016/j.jarmac.2012.07.001
Siegler, R. S. (1994). Cognitive variability: a key to understanding cognitive development. Curr. Dir. Psychol. Sci. 3, 1–5. doi: 10.1111/1467-8721.ep10769817
Siegler, R. S. (2007). Cognitive variability. Dev. Sci. 10, 104–109. doi: 10.1111/j.1467-7687.2007.00571.x
van Geert, P., and Steenbeek, H. (2005). Explaining after by before: basic aspects of a dynamic systems approach to the study of development. Dev. Rev. 25, 408–442. doi: 10.1016/j.dr.2005.10.003
von Oertzen, T., and Brandmaier, A. M. (2013). Optimal study design with identical power: an application of power equivalence to latent growth curve model. Psychol. Aging 28, 414–428. doi: 10.1037/a0031844
Weisstein, E. W. (1999). CRC Concise Encyclopedia of Mathematics. Boca Raton, FL: Chapman & Hall/CRC.
Wilhelm, P., and Schoebi, D. (2007). Assessing mood in daily life. Structural validity, sensitivity to change, and reliability of a short scale to measure three basic dimensions of mood. Eur. J. Psychol. Assess. 23, 258–267. doi: 10.1027/1015-5759.23.4.258
Ziegler, M., Kemper, C. J., and Kruyen, P. (2014). Short scales – Five misunderstandings and ways to overcome them. J. Individ. Differ. 35, 185–189. doi: 10.1027/1614-0001/a000148
Keywords: working memory, training, intraindividual variability, micro-longitudinal, within-person, trajectories, fluctuations, learning disabilities
Citation: Könen T and Karbach J (2015) The benefits of looking at intraindividual dynamics in cognitive training data. Front. Psychol. 6:615. doi: 10.3389/fpsyg.2015.00615
Received: 29 January 2015; Accepted: 26 April 2015;
Published: 12 May 2015.
Edited by:
Barbara Carretti, University of Padova, ItalyReviewed by:
Yvonne Brehmer, Karolinska Institute, SwedenJoni Holmes, Medical Research Council Cognition and Brain Sciences Unit, UK
Copyright © 2015 Könen and Karbach. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Tanja Könen, koenen@psych.uni-frankfurt.de