DIGITAL SKILLS AND LIFE-LONG LEARNING: DIGITAL LEARNING AS A NEW INSIGHT OF ENHANCED LEARNING BY THE INNOVATIVE APPROACH JOINING TECHNOLOGY AND COGNITION

EDITED BY : Dina Di Giacomo, Pierpaolo Vittorini and Pilar Lacasa PUBLISHED IN : Frontiers in Psychology and Frontiers in ICT

#### Frontiers Copyright Statement

© Copyright 2007-2019 Frontiers Media SA. All rights reserved. All content included on this site, such as text, graphics, logos, button icons, images, video/audio clips, downloads, data compilations and software, is the property of or is licensed to Frontiers Media SA ("Frontiers") or its licensees and/or subcontractors. The copyright in the text of individual articles is the property of their respective authors, subject to a license granted to Frontiers.

The compilation of articles constituting this e-book, wherever published, as well as the compilation of all other content on this site, is the exclusive property of Frontiers. For the conditions for downloading and copying of e-books from Frontiers' website, please see the Terms for Website Use. If purchasing Frontiers e-books from other websites or sources, the conditions of the website concerned apply.

Images and graphics not forming part of user-contributed materials may not be downloaded or copied without permission.

Individual articles may be downloaded and reproduced in accordance with the principles of the CC-BY licence subject to any copyright or other notices. They may not be re-sold as an e-book.

As author or other contributor you grant a CC-BY licence to others to reproduce your articles, including any graphics and third-party materials supplied by you, in accordance with the Conditions for Website Use and subject to any copyright notices which you include in connection with your articles and materials.

All copyright, and all rights therein, are protected by national and international copyright laws.

The above represents a summary only. For the full conditions see the Conditions for Authors and the Conditions for Website Use. ISSN 1664-8714 ISBN 978-2-88945-753-3 DOI 10.3389/978-2-88945-753-3

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

## Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

# DIGITAL SKILLS AND LIFE-LONG LEARNING: DIGITAL LEARNING AS A NEW INSIGHT OF ENHANCED LEARNING BY THE INNOVATIVE APPROACH JOINING TECHNOLOGY AND COGNITION

Topic Editors: Dina Di Giacomo, University of L'Aquila, Italy Pierpaolo Vittorini, University of L'Aquila, Italy Pilar Lacasa, University of Alcalá, Alcalá de Henares, Spain

Recently, technology and aging have been key research areas in human cognition.

The Research Topic "Digital Skills and Life-long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition" investigated technology's impact on cognitive and intellective processes, highlighting how intensively technology can change and/or enhance the cognitive functioning throughout one's lifespan.

The aim of this Research Topic was to provide an outlook through multidisciplinary research and development while addressing the dynamic intersection of cognition, mind, and technology. Our scope was 1) to favor the cognitive technology debate, 2) to overcome the dichotomies of technology and psychology, 3) to emphasize the advances in knowledge and well-being.

This Research Topic comprises review studies and original articles, focused on digital skills that enhance human potential. Transversal approaches and cross-sectorial analysis were encouraged, leading to investigation areas related to cognitive and mental processing—in educational, rehabilitation, clinical settings—across aging.

Articles of high relevance to the Research Topic were submitted on the subjects of a) research in human performance and human factors, b) new research and technologies addressing the needs of a growing populace, and c) cognitive aging and cognitive rehabilitation research.

Citation: Giacomo, D. D., Vittorini, P., Lacasa, P., eds. (2019). Digital Skills and Life-long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition. Lausanne: Frontiers Media. doi: 10.3389/978-2-88945-753-3

# Table of Contents

# EDITORIAL

*04 Editorial: Digital Skills and Life-Long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition* Dina Di Giacomo, Pierpaolo Vittorini and Pilar Lacasa

# 1. ORIGINAL RESEARCH ARTICLES


David Rodriguez-Andres, Magdalena Mendez-Lopez, M.-Carmen Juan and Elena Perez-Hernandez

*37 Digital Inclusion in Older Adults: A Comparison Between Face-to-Face and Blended Digital Literacy Workshops*

Claudia I. Martínez-Alcalá, Alejandra Rosales-Lagarde, María de los Ángeles Alonso-Lavernia, José Á. Ramírez-Salvador, Brenda Jiménez-Rodríguez, Rosario M. Cepeda-Rebollar, José Sócrates López-Noguerola, María Leticia Bautista-Díaz and Raúl Azael Agis-Juárez

*54 Reading on Paper and Screen Among Senior Adults: Cognitive Map and Technophobia*

Jinghui Hou, Yijie Wu and Erin Harrell

*64 Development and Evaluation of a Cognitive Training Game for Older People: A Design-based Approach*

Ming-Hsin Lu, Weijane Lin and Hsiu-Ping Yueh

*79 Examining the Relationship Between Teachers' ICT Self-Efficacy for Educational Purposes, Collegial Collaboration, Lack of Facilitation and the Use of ICT in Teaching Practice* Ida K. R. Hatlevik and Ove E. Hatlevik

# 2. TECHNOLOGY REPORTS

*87 Unobtrusive Observation of Team Learning Attributes in Digital Learning* David C. Gibson

# 3. REVIEWS

*92 Brain Training in Children and Adolescents: Is It Scientifically Valid?* Teresa Rossignoli-Palomeque, Elena Perez-Hernandez and Javier González-Marqués

# Editorial: Digital Skills and Life-Long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition

Dina Di Giacomo<sup>1</sup> \*, Pierpaolo Vittorini <sup>1</sup> and Pilar Lacasa<sup>2</sup>

<sup>1</sup> Department of Life, Health and Environment Sciences, University of L'Aquila, L'Aquila, Italy, <sup>2</sup> Faculty of Humanities, University of Alcalá, Alcalá de Henares, Madrid, Spain

Keywords: digital, lifelong learning, digital skill, technology, ICT

#### **Editorial on the Research Topic**

#### **Digital Skills and Life-Long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition**

Digital skills represent enhanced learning abilities within which cognition and technology interact to model the knowledge processes in aging populations. Since they are digital natives, new generations are tech savvy, using improved learning capabilities. Adults and older generations do not possess native technological competence. However, they are learning digital skills to improve their lives and to age well in a digital world.

Digital and technological innovations are impacting the daily living of people influencing significantly the demographic changes. New kind of jobs, new way to manage the healthcare, education, and social contexts. In the "White Paper on the Future of Europe," the European Commission (2017) highlights that it "is likely that most children entering primary school today will end up working in new job types that do not yet exist" and that coping with this "will require a massive investment in skills and a major rethink of education and lifelong learning system."

The Research Topic aimed to elaborate the benefits of digital skills such as to overcoming the barriers of technology, investigating the human factor in digital life-long learning, advancing the cognitive technology debate, overcoming the dichotomies between technology and psychology, and emphasizing the improvements to human knowledge and wellness. The studies' focus was on investigating technology's impact on cognitive and intellectual processes by highlighting the intensity with which technology can change and/or enhance cognitive/behavioral functioning over a lifetime. Multidisciplinary research has been the favored scientific approach.

The Research Topic Issue is composed of original research, as well as a technical report and review. The original research addressed relevant digital inclusion themes and self-efficacy in ICT and teaching practices. The cognitive training game focused on the adult population to see the impact of digital technology on aging adults' cognition and behaviors.

Several researchers have provided evidence of the potential for technology to improve cognitive development and learning outcomes in children. Many studies have highlighted the efficacy of technology in an educational context to promote innovative learning and greater cognitive autonomy (Zimmerman and Tsikalas, 2005; Alvermann et al., 2006; Di Giacomo et al., 2016a,b). Using an evidence-based approach, Di Giacomo et al.'s study investigated digital learning as smart learning in children by examining the influence of digital experiences on children's

Edited and reviewed by:

Anton Nijholt, University of Twente, Netherlands

> \*Correspondence: Dina Di Giacomo dina.digiacomo@cc.univaq.it

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 15 November 2018 Accepted: 05 December 2018 Published: 19 December 2018

#### Citation:

Di Giacomo D, Vittorini P and Lacasa P (2018) Editorial: Digital Skills and Life-Long Learning: Digital Learning as a New Insight of Enhanced Learning by the Innovative Approach Joining Technology and Cognition. Front. Psychol. 9:2621. doi: 10.3389/fpsyg.2018.02621

**4**

cognitive development. Chao and Yu's study investigated Internet access and cyberbullying behavior in adolescents, and Rodriguez-Andres et al. focused on how videogames influence children's navigation skills. All of the research listed showed the impact of technology on daily living of people highlighting the changing in lifelong learning.

Older adults' abilities to use digital solutions and tools is a crucial issue, because low adherence to digital living creates a barrier in daily living, which reduces quality of life, independence, autonomy, and mental health. Moreover, such tools could effectively enhance medical care for the elderly. However, the fear of technology is more prevalent among older generations who did not grow up with computers, complicated acronyms, or digital games.

While the benefits of learning about computers and applications are abundant, on the negative side, learning can be stressful for people due to cognitive as well as psychological factors. This affect is commonly identified as "computer anxiety" (Cambre and Cook, 1985; Desai and Richards, 1998; Thatcher

# REFERENCES


and Perrewe, 2002). Martínez-Alcalá et al.'s study aimed at the digital inclusion of older adults, whereas Hou et al. dealt with technophobia as barrier to digital living in adulthood, and Lu et al.'s study developed a cognitive training game for older people. The last original research from Hatlevik and Hatlevik's was dedicated to the impact of technology on adults and their teaching practices.

Gibson's technology report designed a new framework to unobtrusively observe and analyse knowledge and skills-inaction by continuously collecting data from individuals as they interacted with digital assets either alone or on problem-solving teams.

Finally, the review by Rossignoli-Palomeque et al. debated the scientific concept of brain training in children and youth, specifically its efficacy in populations with neurological diseases.

# AUTHOR CONTRIBUTIONS

DD, PV and PL wrote the Editorial.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Di Giacomo, Vittorini and Lacasa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Digital Learning As Enhanced Learning Processing? Cognitive Evidence for New insight of Smart Learning

#### Dina Di Giacomo<sup>1</sup> \*, Jessica Ranieri<sup>1</sup> and Pilar Lacasa<sup>2</sup>

<sup>1</sup> Department of Life, Health and Sciences, University of L'Aquila, L'Aquila, Italy, <sup>2</sup> Department of Psychology, Documentation and Audiovisual Communication, University of Alcalá, Alcalá de Henares, Spain

Large use of technology improved quality of life across aging and favoring the development of digital skills. Digital skills can be considered an enhancing to human cognitive activities. New research trend is about the impact of the technology in the elaboration information processing of the children. We wanted to analyze the influence of technology in early age evaluating the impact on cognition. We investigated the performance of a sample composed of n. 191 children in school age distributed in two groups as users: high digital users and low digital users. We measured the verbal and visuoperceptual cognitive performance of children by n. 8 standardized psychological tests and ad hoc self-report questionnaire. Results have evidenced the influence of digital exposition on cognitive development: the cognitive performance is looked enhanced and better developed: high digital users performed better in naming, semantic, visual memory and logical reasoning tasks. Our finding confirms the data present in literature and suggests the strong impact of the technology using not only in the social, educational and quality of life of the people, but also it outlines the functionality and the effect of the digital exposition in early age; increased cognitive abilities of the children tailor digital skilled generation with enhanced cognitive processing toward to smart learning.

#### Edited by:

Ana Lucia Pereira, Ponta Grossa State University, Brazil

#### Reviewed by:

Gianluca Castelnuovo, Università Cattolica del Sacro Cuore, Italy Yuejin Xu, Murray State University, United States

> \*Correspondence: Dina Di Giacomo dina.digiacomo@cc.univaq.it

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 11 April 2017 Accepted: 19 July 2017 Published: 03 August 2017

#### Citation:

Di Giacomo D, Ranieri J and Lacasa P (2017) Digital Learning As Enhanced Learning Processing? Cognitive Evidence for New insight of Smart Learning. Front. Psychol. 8:1329. doi: 10.3389/fpsyg.2017.01329 Keywords: digital skills, cognitive process, childhood, technology, enhancing of cognitive processing

# INTRODUCTION

The influence of technology on people life is almost evident. The impact of technology is on the quality of life and wellness such as better living in social context, improved efficiency in working context and enhanced educational system. Lately, technology have had large and quick spread out making better: (a) the management of the working performance reducing elaboration processing data and providing fast processing of tasks, (b) the healthcare efficacy by application of the technology in the screening, diagnosis and treatment of the patients, (c) the educational proficiency enhancing the learning processing in lifespan in order to favor equality and wellness in daily living. Childhood, adulthood, and old age have taken advantage by technology through national and international scientific synergies and investments, providing innovative solutions for people's needs (Tikhomirov, 1974; Gamberini et al., 2006; Githens, 2007; Blaschke et al., 2009; Wang et al., 2010).

'Digital native' or 'Net generation' are ones of most used labels applied to the young people defining the generation of 'tech-savvy' learners (Tapscott, 1998; Prensky, 2001). Cognitive, social and emotional development processing results influenced by technology; as well as the related educational practice seems reflecting a changing in the daily living.

Interesting research topic is the influence of young children's use of technology on the learning system of young generation. Initially, researchers were rather cautious and they have been oriented to highlight the technology as impediment for mental development. In short time, the research addressed opposite view. Armstrong and Casement (2000) pointed out the technology should be avoided in order to don't keep out of the reach of children from interaction with others and so do not influence negatively the social and emotional development. Following review studies have evidenced that the concept about the technology as impediment in mental development as to be a myth to debunk (Yelland, 2011; Plowman and McPake, 2013): the awarded use of technology should be considered a valid instrument to improve the growth of children and doesn't represent an impediment for mental development.

Particular lively debate is about the technological impact on the educational system promoting new kind of learners. In fact, the radical occurred changing in education is strictly joined to the rapid technological development providing an impressive gap between the traditional teaching method and the needs of young people (Bennett and Maton, 2010). Moreover, Hsin et al. (2014) have highlighted the presence of fundamental gap between the technologically skilled and technological unskilled individuals, and the influent factors can be: economical status, cultural and social context, parents education. Kerawalla and Crook (2002) has pointed out as the home access to the technology can be strongly measure of the technological advantage: costs of devices affect the exposition to the digital tools and so having a low effect on the development of digital skill. Location of computer, rules about access and value placed on technology are three main determinant factors in the home access to the device that can mark the difference among young generation (Facer and Furlong, 2001; Campbell et al., 2006): timing, exposition time, availability of experienced digital tools can be somehow the measures of technological skill development. Besides, Furlong et al. (2000) suggested that the availability of computer doesn't mean genuine access. Parental control and much more parental supervising and supporting are getting became the focal key point in the early stage of digital skills.

The suggestion of these studies addressed that the new generations grow living inhomogeneous exposition to the technology (in terms of access to the device and digital tools use) and that draw different opportunities so then different technology experience that makes the difference the development of digital skills. Bennett and Maton (2010) sustained that the young people are pushed to the diversity of interests, motivations and needs. The simple exposition to the technology doesn't mean to be digital (or native) skilled. The complexity of young people as technological skilled has to be undertaken better and it is getting became the emerging research agenda. Specific focus has to be reserved to the impact of the technology on the intellective development in lifelong learning: as daily evident, the digital skills represent the enhancing of learning processing but those are detected by qualitative data (see review Hsin et al., 2014): it misses quantitative elaboration data about the use of technology influence starting in young age and then in lifelong going.

Verenikina et al. (2003) have evidenced the understanding of the digital exposition like new research trend in order to contribute to analyze the influence of its on cognitive and mental development in childhood. The utility and effectiveness of the technologies in childhood are largely spread out, but literature is focused rather on external variables as measurement of their influence: for instance scholastic successful and/or gaming ability. However, even though the scholastic successful could be a good measure, it doesn't explain if digital capabilities have effect on intellective improvement of children. In other hand, gaming and pedagogical supports are two relevant areas of Information Technology System and users. Measures about the influence of the exposition to technology in early age can define better the impact of its on intellective development in order to better understand and manage its application into educational system. According to Griffiths (2002), the gaming gets educational benefits not only in terms of entertainment value but also in the increasing of skills. Indeed, Author showed the efficacy of the technology in the development of skills among special need groups: e.g., brain-wave biofeedback on children with attention deficit disorders, or in general in rehabilitation in which several case reports to stimulate the motivation; enlarge the investigation from vulnerable to more largely population could favor the better use of technology based on awareness of its usefully in the enhanced growth of youth. In particular, Cecilia et al. (2015) have analyzed the influence of gaming activities on cognitive performance. The results showed the technological exposition in childhood can favor a better cognitive flexibility and enhanced learning; the autonomy in the use of the technological tools and/or applications represents a good practice to improve the learning abilities in developmental age.

Hsin et al. (2014) have analyzed a review about the influence of technology on learning processes of young children. The interaction children-technology is driven by different modality; the children characteristics are distinguished in age, experience, time and gender; the characteristics of technologies are located in mechanism design, teaching and learning approaches applied and content of technology. This complex interaction is moderated by third factor: adult influence articulated in facilitating children's engagement in technology, adapting teaching to integrate technology, perceptions of technology, adults' interaction maximizes the effect of technology. Hsin et al. (2014) proposed a conceptualizing young children's learning with technology highlighting the factors related to the learning outcomes. In this regard, interesting is the topic about the digital impact in weak leaning processing during childhood. Even if increasing slowly, few studies have been conducted so far. Cofini et al. (2012) have conducted a study about the effectiveness of digital support in poor comprehender children. Starting from innovative approach, experimental digital tool based on Adaptive Learning System (ALS), Authors provided a digital tool in order to support the cognitive weakness of children in reading tasks.

Findings showed positive impact on reading performance of poor comprehenders evidencing an added value of technology in learning processing.

In addition, Di Giacomo et al. (2015) have highlighted an increasing performance in the lower reading children applying a smarty digital tool. The results suggested the efficacy and the positive influence of technologies in the cognitive process: in the silent reading, the child may be better stimulated to learn and to comprehend the information using technology interactive. The adaptive learning technology might be considered a strong ally in mental development.

Albeit progressively, scientific community is focusing more deeply on 'how much' and 'what about' the technology can influence the intellective growth of children. Some studies have been conducted analyzing the impact of the technology in childhood evaluating the school performance (scholar successful data), others are have examined the gaming performance (Di Giacomo et al., 2016; Gomez et al., 2016; Monjelat et al., 2016). These investigations are focused on the performing data on specific digital tasks by processing the behavioral data. Being explored yet is the impact of digital stimulation on cognitive performance in mental development: particularly, evaluating the cognitive performance would mean to measure the digital impact on intellective development, and might be more suitable to argument about new insight of smart learning, no more in the view of digital and cognition distinguished.

Aim of present study was to investigate the influence of digital experience on cognitive development in childhood by evidence-based approach; our objective was to detect if the access to digital gaming in early childhood could determine a changing into mental development. We wanted to measure the effect of technology influence on childhood by assessment of cognitive performance using standardized psychological tests. Our research wants to detect preliminary data in order to understand if the digital experience in early age (or later) impacts the cognition.

# MATERIALS AND METHODS

# Subjects

The sample was composed of n. 191 native Italian speaker children (n. 82 females; n. 109 males), in range age 7–10 (mean age 8.7 ± 1.1).

The recruitment has been in a public primary Italian school and was mandatory the signed Informed Consensus by parents to process the inclusion of the children.

The inclusion criteria to take part of the sample have been: (a) range age 7–10 years olds, (b) no sign of psychological weakness and/or behavioral deficits, (c) signed Informed consent, (d) filled in of Self-report Questionnaire by parents. The exclusion criteria have been: (a) the presence of neurological and/or psychiatric signs; (b) no signed Informed consent, (c) ambiguity or refusal to fill in the Self-report Questionnaire by parents.

In **Table 1** are reported the details of demographic data of sample.

# Test

To test the cognitive performance of children we applied a Psychological measurement (psychological test composed of n. 8 standardized cognitive tests) and the Self-report Questionnaire was carried out ad hoc to detect the technology use from children.

## Psychological Measurement

The psychological measurement was conducted applying the psychological battery composed of eight standardized cognitive tests divided in two functional testing: visuospatial and verbal task. Visuospatial tasks are: Tower of London, Attentional Matrices, Corsi Blocking Tapping Test, Raven Progressive Matrices. Verbal tasks are: Fluency, Phonological Fluency, Syntactic Comprehension, Naming.


The psychological battery is composed of standardized tests and in the scoring was applied the standard foreseen procedures.

## Self-Report Questionnaire

A Self-report Questionnaire was carried out to detect information about the intensity of daily use of digital/board games of the sample. The examined variables have been: (a) the using of digital

#### TABLE 1 | Demographic data of the sample.

fpsyg-08-01329 July 31, 2017 Time: 15:4 # 4


games (PC/laptop, Ipad, Wii/Xbox, PSP, Smartphone, others); (b) the daily time in the using of digital games (half hour, 1 hour, 2–3 hour, more than 3 hours); (c) the using of board games (Monopoli, Risiko, Chess, Lady's play, Domino, Scarabeo, Goose game, Puzzle, Card's play, Other); (d) the daily time in the using of board games (half hour, 1 hour, 2–3 hour, more than 3 hours). In **Table 2** was reported the core of Self-report Questionnarie. The scoring of intensity of use of digital and/or boards games was based on four-Likert-point scale (Item 2 and Item 4 of the questionnaire). The other questions applied a qualitative scoring.

Applying questionnaire, we could classify the sample into two groups: (a) High Digital group (HD) = children that spend most high time for digital games; (b) Low Digital group (LD) = children that have spend most high time for board games.

# Ethics Statement

This study was carried out in accordance with the recommendations and approval of 'Consiglio Didattico' of the involved Schools with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki.

# Procedure

Testing was conducted into school attended by children, during school time. Participants have been tested in a quiet room reserved for testing. The test administration time was approximately 40 min. The examiners were trained Psychologists, graduate students from the Postgraduate School of Clinical Psychology of the University of L'Aquila, Italy. The examiners were blinded to the research objectives.

The procedure was composed of two detecting data steps: (1) by self-report parents and (2) children cognitive evaluation. First step was represented by the involvement of the parents to fill out the self-report questionnaire in order to detect the technology using form children in their daily time. Second step was the cognitive assessment of the children by application of psychological battery.

#### Self-Report Parents

Applying the self-report questionnaire it was possible to detect the frequency of the use of technology by children; the children were classified in High Digital users (HD) and Low Digital (LD) users.

#### TABLE 2 | Self-report questionnaire.


(2) During the day, how long time your child spends for electronical devices?


(3) Which board games your child use?



(6) If yes, when does he/she prefer to read:


Only when the teacher asks it

	- Half hour
	- 1 hour
	- 2–3 hour
	- More than 3 hours

#### Children Cognitive Evaluation

Standardized psychological tests have been applied to measure the cognitive abilities by quantitative data.

# RESULTS

The data have been analyzed by the SPSS program version 20, and the significance level have been fixed at alpha < 0.05.

In **Table 3** are reported the raw scores of the sample in the psychological battery.

Firstly, data were elaborated by age variable. The sample was distributed into four groups of age: 7 year olds, 8 year olds, 9 year olds, 10 year olds groups. One-way-ANOVA was conducted to elaborate the performance of children along the age. As expected, children improve progressively their performance in verbal and visuoperceptual abilities: the results have evidenced significative effects for the groups in observed cognitive abilities (see **Table 3**).

Then, the scores obtained in standardized tests were transformed in z score and then we combined tests measuring visuoperceptual and verbal abilities.

Then, we divided the sample in n. 2 categorical digital use groups: high digital user (HD) and low digital user (LD); the applied criteria was the frequency and intensity of technology the children by detected data by experimental questionnaire filled out from parents. The distribution, as expected and inline of literature, resulted significant different: HD group was composed of n. 141 children (74%), and LD group n. 51 (26%).

In **Table 4** are reported the cognitive performance in z score.

A multivariate analysis of variance (MANOVA) was conducted to determine the cognitive profile of sample distributed in two groups (HD and LD users) obtained in the composite score.

A MANOVA 4x2 on verbal tasks was conducted; the results have evidenced a significative multivariate effect for the groups [Pillai's Trace F(4,186) = 0.690; p = 0.02; η 2p = 0.05]; additionally, the Box's M value of 13.2 was associated with a p-value of 0.23. Univariate results showed a significative effect for some verbal tasks: naming test [ F(1,189) = 5.10; p = 0.02; η 2p = 0.02] and Category fluency test [ F(1,189) = 8.7; p = 0.004; η 2p = 0.04].

Then, a MANOVA 4x2 was applied on visuoperceptual tasks and it has showed a significative multivariate effect [Pillai's Trace F(4,186) = 2.7; p = 0.03; η2p = 0.05]; the Box's M value of 16.8 was associated with a p-value of 0.09. Univariate showed a difference only on below visuoperceptual tasks: Corsi tapping test [ F(1,189) = 4.5; p = 0.03; η 2p = 0.02] and Raven Progressive Matrices [ F(1,189) = 7.5; p = 0.007; η 2p = 0.03]. In **Figures 1** , **2** are represented the significative performance of two digital groups.

Finally, it has been examined the gender effect on two digital groups. A MANOVA 4x2x2 (4 tasks × 2 digital groups × 2 sex) on both verbal and visuoperceptual performance; it have evidenced no significative multivariate effects.

# DISCUSSION

Aim of the research was to detect the impact of the technology in childhood in order to analyze its influence on cognitive development. Frequently, it is accepted the digital skills as


#### TABLE 4 | z score of the sample's cognitive performance.


enhanced learning: scholastic successful, intellective performance and daily advantage represent consistent variables in childhood and youth. Hsin et al. (2014) have highlighted two aspects in the development of digital skills: (a) skills and competences needed to use technology and (b) perceptions of technology use. Both of them are basic for the development of digital learning. Emerging debate is the measuring of the technology effectiveness.

Indeed, our results showed that the high digital skills are related to the improvement of the cognitive development. Children with high level have better performance in cognitive ability of language (naming and semantic skills) and visual (planning of the logical reasoning and visual memory). Those results are interesting: the digital exposition by gaming in early time could be a great instrument to favor the increased cognitive development of the children in digitalization world. Children so well-stimulated in early time by tasks related to pleasure (digital games) present an advantage in the intellective competence that will be useful the learning processing and much more in the adaptation to the educational and

then working requests. Our data evidenced also a large technology use in childhood: the large part of children is digital skilled and can access to the technology in different way and easily; moreover, a small but relevant children group is digital unskilled and represents a lack of efficiency and equality to the access to the complexity of learning system.

In literature, our study is interesting because propose to investigate the efficacy and the functionally of the digital exposition in early age; more researchers have studied on the use of the technology in life of the people, in lifespan, but they were focused on quality of life. The key point of our study is the evidence that the digital skills are featured by tailored cognitive performance. Our research wanted to contribute to the new research trend of the digital impact on the cognitive development of human like technology is not only the enhancing of the learning processing but also just a good instrument for the intellective development. Technology has a strong impact on people life not only in terms of changing but much more of better leaping up the individual potentiality.

Our finding confirms the data present in literature and suggests the strong impact of the technology using not only in the social, educational and quality of life of the people, but also it outlines the functionality and the effect of the digital exposition in early age in order to increase the cognitive abilities of the children tailoring digital skilled generation with enhanced cognitive processing.

Secondary data of our study was the detection of a small (definitely) but still present group of children technologically unskilled, just because inaccessible in early time of life. That is a relevant aspect representing the early disadvantage (external variable) to develop the smart learning following traditional educational scheme.

# CONCLUSION

The technology has a strong impact on the quality of life of the human: the exposition of the children in young age to digital gaming favors advantage of learning capability not only in terms of high successful in educational path, but much more in the cognitive functionality by enhancing the verbal/visuoperceptual performance. The technology changes the life of the people improving the wellness in the social context and refining the learning processing, enhancing the cognitive performance and so favoring high adaptability in the elaboration information processing. Taking into account that, the learning processing reflects the changing becoming 'smart learning' for improved cognition.

A limit of present study is to have detected the digital skills by self-report method; beside we applied a standard measurement of cognitive performance. Could be interesting to measure digital skills by direct measurements and evaluate the correlated effect on cognitive development in childhood. Moreover, our

preliminary analyses are interesting and significant to outline our next research focus for future investigations. Implications of our study can be in future leaning context and procedures in order to support educational needs of new generations fitting the efficacy on the cognitive potentiality.

# REFERENCES


# AUTHOR CONTRIBUTIONS

The study was leaded by DG; and she wrote and analyzed data. JR has collaborated to the data collecting. PL has supervised the manuscript.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Di Giacomo, Ranieri and Lacasa. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Associations among Different Internet Access Time, Gender and Cyberbullying Behaviors in Taiwan's Adolescents

Cheng-Min Chao<sup>1</sup> and Tai-Kuei Yu<sup>2</sup> \*

<sup>1</sup> Department of Business Administration, National Taichung University of Science and Technology, Taichung, Taiwan, <sup>2</sup> Department of Business Administration, National Quemoy University, Kinmen, Taiwan

With the increasing convenience of social networking sites and their interconnectedness with human interaction, verbal and image bullying have turned digital, making cyberbullying a new form of bullying attracting the attention of researchers, social workers, and schools. This study focuses on the status quo of attitude toward cyberbullying and cyberbullying behavior, explores associations of attitude toward behavior on cyberbullying behavior in gender and different internet access times. In a cross-sectional survey, adolescents were surveyed through self-report questionnaires, 13,864 respondents were received among 150 high schools in Taiwan. Statistical analyses were performed using structural equation modeling (SEM). The study revealed that attitude toward cyberbullying has a direct effect on cyberbullying behavior; among the greatest direct impact were when students use the Internet during the time frame 10:00–14:00. Parents and schoolteachers pay special attention to students using the Internet during this time frame 10:00–14:00 and give guidance, express care, and help those being bullied to communicate and use the Internet in a correct manner. Among genders, male student attitude toward behavior has a greater effect than those of female students. Both male and female students know what cyberbullying is and have witnessed, heard of, or personally encountered cyberbullying behavior. We recommend students talk to parents or teachers or other people who care to reduce the negative effects of cyberbullying. We hopeful that the conceptualization model presented in this study serves as an activator for researches on attitude toward cyberbullying and cyberbullying behavior, and serves a guide and a call to attract more researches in this area.

Keywords: adolescent, cyberbullying behavior, attitude toward cyberbullying, internet use, partial least squares (PLS)

# INTRODUCTION

With the advancement of technology and the widespread access to the Internet, the Internet has revolutionized the way in which people connect and communicate lives (Chao et al., 2013; Pawlikowski et al., 2014; Good and Fang, 2015; Quinones and Kakabadse, 2015; Wegmann et al., 2015; Hsieh et al., 2016). According to the Taiwan Communication Survey (2015) Internet project

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

Patricia Moreno-Peral, Instituto de Investigación Biomédica de Málaga, Spain Andrej Košir, University of Ljubljana, Slovenia

> \*Correspondence: Tai-Kuei Yu yutk2000@gmail.com

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 22 March 2017 Accepted: 14 June 2017 Published: 30 June 2017

#### Citation:

Chao C-M and Yu T-K (2017) Associations among Different Internet Access Time, Gender and Cyberbullying Behaviors in Taiwan's Adolescents. Front. Psychol. 8:1104. doi: 10.3389/fpsyg.2017.01104

reported findings from a national survey of about ninety percent of Taiwanese young people between 9 and 17 years are on the internet and this statistic has been on the rise (Taiwan Communication Survey, 2015). These young people spend about an average of 3 h per day on the Internet. And 14 year old teenagers spend the longest time about an average of 3.5h online per day (Taiwan Communication Survey, 2015). In the past decade, the exponential growth of information technology and the widespread access to the Internet, adolescents have tended to spend more time in the cyber world. Internet provides new information, social-related information, and social networking opportunities but also simultaneously contains risks and a serious form of misbehavior among adolescents, such as: Internet addiction, cyberbullying, cyber pornography, health risks, Internet fraud and cyber victimization that can hurt and distort an adolescents' development (Bauman et al., 2013).

Cyberbullying and Internet addiction are a serious issue for young people in the World. The prevalence rates of cyberbullying vary based on the population studied (i.e., gender, age group), the form of cyberbullying is being measured, and the measure is used to assess cyberbullying incidents. Of note, the majority of cyberbullying studies have focused on youth between the age of 10 and 17 (Kowalski and Limber, 2007; Yılmaz, 2011; Chang et al., 2013; Cleemput et al., 2014; Özdemir, 2014; Taiwan Communication Survey, 2015). Cyberbullying among adolescents has emerged as a new form of bullying and has attracted the attention of many researchers in recent years because the ratio of adolescents using the Internet has increased quickly, especially in social networking sites, chat rooms, and instant messaging applications. Adolescents feel a sense of solidarity and identity through the aforementioned applications (Anderson et al., 2014; Dredge et al., 2014; Palladino et al., 2015). Cyberbullying behavior and traditional school bullying have some commonality (Bauman et al., 2013; Waasdorp and Bradshaw, 2015). In fact, it has been found that cyberbullying may lead to more severe negative outcomes when compared to traditional bullying. From a traditional point of view, the perpetrators of cyberbullying use online videos, pictures and words in digital form to threaten, ridicule, and insult others. Research suggests that cyberbullying may negatively impact multiple aspects of adolescents mental health includes: depression, social anxiety, suicide, low self-esteem, etc; and behavioral problems include deterioration of relationships between family members and a drop in grades and other negative outcomes (Anderson et al., 2014; Dredge et al., 2014). However, not all victims of cyberbullying have had a negative repercussions (Patchin and Hinduja, 2006; Burgess-Proctor et al., 2009); exploring the consequences of cyberbullying is also rather important.

Cyberbullying is defined as the behavior of using modern communication technology to post pictures, videos, communication device [including: mobile phones, social networking sites (SNS), instant messenger (IM), e-mail, facebook, blog, etc], or texts that ridicule, insult, threaten, or intimidate a person and cause feelings of hurt in said person (Huang and Chou, 2010, 2013; Anderson et al., 2014; Cleemput et al., 2014; Hong et al., 2014; Watts et al., 2017). In a United States sample of 3,767 middle school students in 6th-8th grades (aged 11–14), around 11% of 6th-8th grade students had cyberbullied others within the 2 months prior to completing the survey; 4% had cyberbullied someone (cyberbully); and 7% had cyberbullied others and had also been both cyberbullies and victims (Kowalski and Limber, 2007). The Taiwan cyberbullying of adolescents project reported findings from a national survey of 1,959 students (ages 9–17). Of the respondents, 5.8% said that they had been the victims of cyberbullying, whereas 8.3% said that they had engaged in bullying behavior online (Taiwan Communication Survey, 2015). Among all the ways of youth cyberbullying, social media (e.g., Facebook or Plurk) is the most common one. About 2/3 (68.7%) of the interviewees said that they had been bullied on social media, and about 1/2 (42.3%) had been bullied when playing online games. Even on IMs, chatrooms, forums, BBS (Bulletin Board System), cyberbullying is a frequently occurred incident (Taiwan Communication Survey, 2015). There is approximately 1 among 4 people who had been bullied while using the above mentioned online services (IMs: 25.6%; chatrooms, etc.: 24.8%) (Taiwan Communication Survey, 2015). Ditch the Label (2013) released its annual cyberbullying report which combines the largest bullying-related data set of over 10,000 young people with key questions surrounding cyberbullying and the use of integrated digital technology within the lives of young people. In cyberbullying and social networks issue, of all youths polled, 75% of them use facebook and 54% have experienced cyberbullying. In addition, this report also revealed that cyberbullying happens mostly on Facebook, Youtube, Twitter, and Myspace. Ditch the Label (2016) surveyed 8,850 young people aged 12–20 in partnership with schools and colleges from across the all across the UK. The results showed that people who have been bullied are almost twice as likely to bully others; boys bully twice as more as (66% of males versus 31% females); 44% of young people who have been bullied experience depression; 33% of those being bullied have suicidal thoughts. The ease of producing information online, fast and widespread dissemination, coupled with anonymous posting, and the elimination of the pressures of face-to-face confrontation have made it easier and quicker for a cyberbully to hurt others. The evolution of this new form of bullying deserves the attention of educators because this kind of bullying and oppressive behavior causes great harm to students and adolescents undergoing personality development.

Past studies have shown that the effects of cyberbullying are more severe than traditional bullying in four different ways: (Anderson et al., 2014; Dredge et al., 2014) (1) the identity of the perpetrator of cyberbullying is unknown; (2) the number of bullies is greater; (3) the rate of information sharing is faster; (4) the record of bullying lasts forever, much like insulting a person repeatedly. With cyberbullying, there is no direct way for perpetrators to know the effect of their behavior on the victim. Hence, the victim may feel terror, pressure, and other adverse effects when the victim does not know the identity of the perpetrator. When the victim is under pressure, there is a

higher probability for violent behavior (Huang and Chou, 2013; Anderson et al., 2014; Hong et al., 2014).

Existing empirical research have examined the influence of cyberbullying and victimization among adolescents. The results show that parenting behavior, good family relations (such as: parental monitoring, positive parent–adolescent relationships and communication) can decrease involvement in risky activities of cyberbullying (Law et al., 2010; Stadler et al., 2010; Özdemir, 2014). But, adolescents tend not to share negative experiences with their parents. For instance, Tokunaga (2010) found that although about 20–40% of adolescents are victims of cyberbullying, most choose not to tell their parents. Among the victims, suicidal tendency is higher than among non-victims. Also, 16% of adolescents admitted to cyberbullying behavior once or regularly. Yılmaz (2011) found that about 62% of students who had been cyber bullied did not discussed the problem with parents or teachers. In a recent study of 2,992 Taiwan high school students in 10th grade, 18.4% had been cyberbullied (cybervictim); 5.8% had cyberbullied others (cyberbully); 11.2% had both cyberbullied others and been cyberbullied (cyberbully victim) (Chang et al., 2013). According to findings published from the Taiwan Communication Survey (2015), middle and high school students are more likely to become victims of cyberbullying. Also, among 9–17 year olds in Taiwan, 5.8% have been victims of cyberbullying and 8.3% have bullied others. The most common channels were social media sites and online games. When faced with cyberbullying, 36.7% of the adolescents surveyed ignored the behavior. For example, during information class, one high school student, May, learned about attitudes toward cyberbullying and grasped a deeper understanding as the teacher frequently applies real world cases to teach students the definition of cyberbullying and how to deal with such behavior in her information course. Sometimes May witnesses classmates bad-mouthing another classmate or sending photos or videos mocking said classmate. May has realized that cyberbullying does exist in real life from her observation of online news stories on the negative effects of cyberbullying on artists and students. As May increasingly used the Internet, she saw more cases of cyberbullying and understood the contexts of cyberbullying.

Behavioral beliefs can vary from individuals' attitude toward the behavior. In other words, individuals with a more positive attitude toward a certain behavior tend to perform the behavior. Ho et al. (2017) explored factors related to cyberbullying on social media among children and adolescents in Singapore. The results showed that individuals with less favorable attitude had a lower likelihood to engage in cyberbullying behavior. Doane et al. (2014) explored cyberbullying perpetration among American college students. The results presented strong positive correlations between attitude, subjective norms, and cyberbullying perpetration. Many previous studies have found that attitude toward cyberbullying is significantly associated with cyberbullying (Perren and Gutzwiller-Helfenfinger, 2012; Doane et al., 2014; Ho et al., 2017).

About the relationship between cyberbullying and genders of adolescents. Previous research investigating the significant differences of the incidences of cyberbullying displays inconclusive findings. Huang and Chou (2010) argue that gender has long been associated with aggressive behaviors, and it may result in different types of bullying among adolescents. Some researchers found no significant differences of cyberbullying experiences between male and female youth (e.g., Ybarra and Mitchell, 2004; Similarly, Patchin and Hinduja, 2006), while others do not support this view (e.g., Kowalski and Limber, 2007). Some researchers have shown that males are more likely to be perpetrators of cyberbullying than are females (Huang and Chou, 2010; Chang et al., 2015; Lee and Shin, 2017). In conclusion, the relationship between adolescent cyberbullying and the genders seem ambiguous. Therefore, further investigation is necessary. In addition, most prior research of cyberbullying has focused on the factors and the influences it has on mental and physical health (Chang et al., 2013; Anderson et al., 2014; Dredge et al., 2014; Hong et al., 2014) and the differences between cyber and traditional bullying. With the Internet become more common, and people spend more time online, cyberbullying, especially between students, has become an important topic. However, there is a lack of examinations how different attitudes toward cyberbullying of students with different levels of Internet addiction affect their cyberbullying behaviors.

In recent years, adolescent Internet addiction has emerged as a major issue. Internet addiction is the inability of people to control their Internet use—an inability that can eventually lead to be a global major mental health, social, and academic problems (Wegmann et al., 2015; Chen and Nath, 2016; Hsieh et al., 2016; Montag et al., 2016; Musetti et al., 2016; Servidio, 2017). The prevalence of youth Internet addiction varies widely across countries, with the early studies of Internet addiction reported findings from a survey conducted in 11 European countries, taken by about 11,000 adolescents (mean age: 14.9 ± 0.89). The results showed that the overall prevalence of Internet addiction was 4.4%, higher among males than females (5.2% versus 3.8%) (Durkee et al., 2012). In Poland, about 2–9% of young people have very serious problems caused by Internet usage (Hawi et al., 2015), and the prevalence in Hong Kong at between 6.7 and 26.7% (Wang et al., 2015). According to the survey of Taiwan Network Information Center [TWNIC] (2015), during the last 6 months, 79.7% of Taiwanese people aged 12 and more who use the Internet have spent approximately 3.42 h online per daily, and 3.35 h per daily during holidays. The most common Internet activities are visiting online communities and using IMs (Taiwan Network Information Center [TWNIC], 2015). Internet addiction prevalence estimation varies across countries due to differences in diagnostic criteria as well as the psychometric tools utilized for assessment (Hawi et al., 2015; Boysan et al., 2017). Kuss et al. (2014) found that there are currently 21 existing instruments for measuring Internet addiction. Current screening scales of Internet Addiction are mainly adopted from Young's Internet addiction Scales (Young, 1998), which appeared as one of the earliest conceptualizations and is still being commonly adopted nowadays. Taiwan's popular Chinese versions of Internet addiction scales also uses the symptoms of DSM-IV-TR Diagnostic Criteria, for substance addiction as the diagnostic indicator (Ko et al., 2005; Yen et al., 2009). To conclude, Internet addiction among teenagers is a relatively serious problem. A comprehensive Internet addiction

measuring tool might effectively categorize the addiction levels. In addition, previous research seldom covers Internet addiction together with cyberbullying. It is especially important to examine the influence of different Internet addiction levels on the cyberbullying behaviors.

To summarize, cyberbullying gives birth to a new kind of public physical and mental health problem that negatively impacts the everyday life of a substantial number of adolescents. The effect of attitude toward cyberbullying on cyberbullying behavior is an important topic for discussion. Therefore, the purposes of the present research are (1) to investigate antecedent (attitude toward cyberbullying) for cyberbullying behaviors, and (2) to examine the effects of gender, most frequent Internet access time, and Internet addiction situations on cyberbullying behaviors. In order to understand the association between attitudes toward cyberbullying and cyberbullying behavior modulates by gender and different internet access. This study aims to address the following research questions: (1) Does attitudes toward cyberbullying influence the cyberbullying behaviors? (2) How does the gender, most frequent Internet access time, and Internet addiction the impacts of cyberbullying behaviors? To minimize cyberbullying, the results of this study provide references for parents, schools, and government educational units, further adding emphasis on the effects of cyberbullying and the problems raised.

# MATERIALS AND METHODS

# Hypotheses and Research Model Development

In order to understand the association between attitudes toward cyberbullying and cyberbullying behavior modulates by gender and different internet access. The purposes of the present research are (1) to investigate antecedent (attitude toward cyberbullying) for cyberbullying behaviors, and (2) to examine the effects of gender, most frequent Internet access time, and Internet addiction situations on cyberbullying behaviors.

According to the previous research (Perren and Gutzwiller-Helfenfinger, 2012; Doane et al., 2014; Ho et al., 2017) pointed out that positive correlations between attitude toward cyberbullying and cyberbullying behaviors. Consequently, this appears to support that attitude toward cyberbullying is the strongest predictor of cyberbullying behaviors. In addition, the effect of gender on cyberbullying is controversial. Some researchers have shown that males are more likely to be perpetrators of cyberbullying than are females. Other researchers, however, indicate no gender effect on cyberbullying behaviors (Ybarra and Mitchell, 2004; Similarly, Patchin and Hinduja, 2006; Roberto et al., 2014).

On a global scale, the reported prevalence estimates for adolescent Internet addiction vary between 2% (Poland) and 26.7% (Hong Kong) (Hawi et al., 2015; Wang et al., 2015). Thus, adolescent Internet addiction has emerged as a major issue. Although there exists extensive research in the field of Internet addiction, most researches are lacking in cyberbullying. It is especially important to examine the influence of different Internet addiction levels on the Internet bullying behaviors. Based on the abovementioned considerations, we advance the following hypotheses:

Hypotheses 1: Attitude toward cyberbullying has a significant effect on cyberbullying behaviors.

Hypotheses 2: The moderating effect of gender exists between the Attitude toward cyberbullying and cyberbullying behaviors.

Hypotheses 3: The moderating effect of most frequent Internet access time exists between the Attitude toward cyberbullying and cyberbullying behaviors.

Hypotheses 4: The moderating effect of Internet addiction exists between the Attitude toward cyberbullying and cyberbullying behaviors.

# Measures

This study used questionnaires as the main instruments for data collection. In the questionnaires, the instrument was designed 38 questions with a four-part questionnaire, includes: The content of the questionnaire includes four components: (1) basic information of the student and Internet usage; (2) attitude toward cyberbullying (1 = strongly disagree, 4 = strongly agree), (3) cyberbullying behavior (1 = not at all, 4 = very often), and (4) Internet addiction (1 = strongly disagree, 4 = strongly agree). The Likert four point scale was used in the "Attitude toward Cyberbullying," "Cyberbullying Behavior," and Internet addiction sections. The self-reporting inventory was used for filling out the questionnaire. The questionnaire used is shown in **Table 1**. Due to the nature of the questions in "Cyberbullying Behavior," there is a tendency for participants to avoid them, causing common method variance (CMV). We chose to write the questions from a third party perspective to evaluate stories the respondents have heard or saw about cyberbullying. We undertook a large sample to avoid CMV and to increase the reliability power of this study.

The instruments were developed after a thorough review of several previous studies on attitude toward cyberbullying, cyberbullying behavior, and Internet addiction. Our scale development followed MacKenzie et al. (2011) and the development procedures suggested by DeVellis (2003) for standard psychometric scales. The measurement items for the constructs of attitude toward cyberbullying and cyberbullying behavior were adapted from the measurement developed by previous studies (Yılmaz, 2011; Chang et al., 2013; Anderson et al., 2014; Cleemput et al., 2014; Dredge et al., 2014). To define adolescents Internet addiction scale, which was modified from the Chen Internet Addiction Scale (CIAS), Internet Addiction Test (IAT) and other measurement tool about Internet Addiction (Young, 1998; Ko et al., 2005; Kuss et al., 2014; Hawi et al., 2015; Wang et al., 2015; Chen and Nath, 2016; Hsieh et al., 2016; Boysan et al., 2017; Servidio, 2017) to measure Internet addiction.

A pilot test using the questionnaire was conducted on 1,225 senior high school students (including vocational high school students) in central Taiwan to evaluate the revised questionnaire in terms of readability, ease of understanding, and formatting issues prior to the actual test. Further, a Cronbach's alpha test

#### TABLE 1 | Questionnaire items.

fpsyg-08-01104 June 28, 2017 Time: 18:49 # 5


was performed to test the reliability and internal consistency of each of the 32 measured attributes. The alpha coefficients for all of the 32 attributes ranged from 0.62 to 0.74, which respectively exceeded the minimum value of 0.6 that is widely used as an indicator of reliability (Hair et al., 2010). Subjects who participated in the pilot test were excluded from the subsequent study.

# Participants

In the Taiwanese educational system, elementary, junior and senior high school education is compulsory, children enter the elementary school at the age of 7 or 8 years and complete this stage of their education by the age of 13 or 14 years. Then, they enter junior high school for 3 years followed by a further 3 years at senior high school (including vocational high school). The years at junior high school are termed as the 7th–9th grade, and those at senior high school (including vocational high school) are termed as the 10th–12th grade. To understand the effect of attitude toward cyberbullying on cyberbullying behavior among senior and vocational high school students in Taiwan. As of May 2014, 344 senior high schools and 155 vocational high schools (499 in total) were registered. The sample was first stratified by region (northern, central, southern, and eastern), then stratified by school type (senior high schools and vocational high schools). This cross-sectional survey was conducted in Taiwan among 17- to 19-year-old senior and vocational high school adolescents. Participants were recruited from 150 high schools in Taiwan through stratified and random cluster sampling. Of these, 103 senior high schools (selection rate, 29.9%) and 47 vocational high schools (selection rate, 30.3%) were selected. At every school, 100 students were surveyed through questionnaires.

This research adopted a quantitative survey and utilized mail or face-to-face interviews for high schools that were willing to participate in the survey. The first step, we contacted a teacher who service in academic affairs in each high school to ensure his/her cooperation. They helped explain the content of the questionnaire to the respondents, the duration of data collection took around 8 months. A total of 15,000 questionnaires were sent at the same time. All responses to the self-report instruments were collected during a regular school-day in classrooms and in the presence of the class teacher. All students enrolled in the sampled schools were the participants of this study. Participation in the study was completely voluntary. In total, 13,864 valid responses were received, and the response rate was 92.4%.

# Statistical Analysis

fpsyg-08-01104 June 28, 2017 Time: 18:49 # 6

Structural equation modeling (SEM) was used in a comprehensive, combined analysis of both the measurement and structural model. According to Hair et al. (2011), although the covariance-based structural equation modeling (CB-SEM) has dominated since its first appeared in the 1980s, the partial least square SEM (PLS-SEM) has called a great deal of attention in recent years. PLS is component-based and uses a least-squares estimation procedure. For the actual data analyses, we used the SmartPLS 2.0 M3 software developed by Ringle et al. (2005) to test the research model of both the measurement and structural model. The results of the SEM estimation, including standardized path coefficients for each hypothesized path in the model, significance based on one-tailed t-tests, and the amount of variance explained (R 2 ).

# RESULTS

# Demographic Variables

In total, 13,864 (6,747 were male, 7,117 were female) usable responses were obtained. The average age of the participants was 17.33 years (SD = 0.94 years). The first time Internet access for the sample population indicated 39.3% were K3 to K4 grades. Most frequent Internet access time for the sample population indicated 73.6% were 18:00∼22:00. Place of Internet usage for the sample population indicated 92.6% were Home. **Table 2** shows the demographic and Internet usage characteristics of the sample.

# Measurement Model Evaluation

Using linear structural relational estimations and traditional alphas, the assessment of the measurement model includes three indices: reliability coefficients (Cronbach's α), average variance extracted (AVE), and composite reliability coefficients. The results presented in **Table 3**, all values displayed a higher Cronbach's α coefficient than the 0.60 benchmark recommended by Hair et al. (2010). The constructs also exhibited a higher composite reliability than the benchmark of 0.6 (Fornell and Larcker, 1981). That all of the convergent validity were exceeded this criteria, except for behavioral of cyberbullying.

# Hypotheses Testing

In the PLS analysis, the R 2 values are used as a goodness-offit measure. The construct of attitude toward cyberbullying was significant determinants of cyberbullying behavior (β = 0.33, t = 37.33, p < 0.01). This research model explained 10.9% of the variance in cyberbullying behavior. **Table 4** shows the results of further investigation: firstly, male students have higher coefficient (β = 0.34, t = 25.84, p < 0.01) and explanatory ability (R <sup>2</sup> = 0.116) in their effects of attitude toward cyberbullying on behavior. Secondly, students surf the Internet during 10:00 to 14:00, which leads to higher coefficient (β = 0.40, t = 9.66, p < 0.01) and explanatory ability (R <sup>2</sup> = 0.161). That reveals attitude toward cyberbullying has more significant impact on cyberbullying behavior for students who access the Internet during 10:00 to 14:00. Third, among different Internet addiction situations, most students are Internet addicted, which leads to higher coefficient (β = 0.28, t = 15.75, p < 0.01) and explanatory ability (R <sup>2</sup> = 0.078). The data reveals that when students are addicted to the Internet, the influence on their cyberbullying behaviors are highly significant.

# DISCUSSION

The analysis of this study shows that attitude toward cyberbullying has an effect on cyberbullying. The explanatory power is 10.9%, the reason being when faced with cyberbullying, most are afraid to publicize it, nor do they tell their parents or ask for help. Also, students are unsure of the definition of cyberbullying and are unable to detect cyberbullying when confronted. Some students mistake cyberbullying as making fun of peers, so they do not pay attention, and suddenly realizing the severity of the situation after the effects of cyberbullying have already snowballed. For example, high school student Jason picked up his classmate, Jackson's, phone. Jason posted indecent photos of Jackson from his phone on the Internet as a joke, unaware that this was cyberbullying behavior. Jackson committed suicide as a result. We recommend teachers remind students to not like, respond, or share posts criticizing or bullying others and to not participate in suspicious search engine activities to avoid being an accomplice to cyberbullying. When a student is faced with cyberbullying or discovering a classmate being cyberbullied, teachers and parents should be notified immediately to put an end to this behavior.

The purpose of this study was to explore the factors associated with cyberbullying behaviors based on attitude toward cyberbullying. In this study, the individuals with less favorable attitude had a lower likelihood to engage in cyberbullying behavior. This is consistent with previous research, where a positive correlation was found between positive attitude toward cyberbullying and cyberbullying behaviors (Perren and Gutzwiller-Helfenfinger, 2012; Doane et al., 2014; Ho et al., 2017). Results also indicated that attitude toward cyberbullying was the key predictor of cyberbullying behavior. Besides, some previous studies (Selkie et al., 2015; Watts et al., 2017) have shown that when one has been a cyberbully or a victim of it, they tend to fall into these same categories in college. Therefore, high school teachers should try to prevent students from being cyberbullies or cybervictims. In order to achieve that, students' attitude toward cyberbullying is important as it is to the behaviors. Past scholars (Washington, 2014; Watts et al., 2017) suggested instructors must take a more active role in monitoring online interactions to prevent cyberbullying among students.

It is difficult for parents or teachers to monitor the Internet; everyone browsing SNSs or other websites is a spectator, and the number of spectators can multiply, augmenting cyberbullying. Therefore, to deter the growth of online violence, future cyberbullying prevention and relevant education should start with spectators taking action and the establishment of the concept of using the Internet for positive uses. From the school's perspective, teachers need to understand the motives for cyberbullying. Every online user, regardless of their identity or

#### TABLE 2 | Profiles of respondents (N = 13864).

fpsyg-08-01104 June 28, 2017 Time: 18:49 # 7


#### TABLE 3 | Validity and reliability.


#### TABLE 4 | Result of SEM analysis.


<sup>∗</sup>p < 0.05; ∗∗p < 0.01.

whether they are an adult or a high school student, should be educated.

Among genders, male student attitude toward behavior has a greater effect than those of female students. Both male and female students know what cyberbullying is and have witnessed, heard of, or personally encountered cyberbullying behavior. The results of the survey show that regardless of gender, when confronted with cyberbullying, students choose to keep to themselves and not ask for help. We recommend victims of cyberbullying to not keep quiet about an incident and to mitigate the negative effects

through the care of concerning adults. Cyberbullying is not a short-term phenomenon, but a vicious cycle. One feels the need to retort when being bullied, then the cyberbully strikes back, creating a cycle of bullying. Parents and teachers can inform the students of the correct attitude to deal with cyberbullying and even ask for the school's help, minimizing the damage done by cyberbullying.

In terms of most frequent Internet access time, students online during the 10:00–14:00 were most affected by attitude toward cyberbullying on cyberbullying behavior. Students utilize this time frame to check the latest status on SNSs as it is their rest time at school. Cyberbullying messages may be intermixed with replies on any given video and photo post. This group is not only at high risk for cyberbullying, they are also considerably addicted to the Internet and unable to stay off the Internet for long periods of time. Because this is during school hours, schools have a stronger capability in disciplining this group. We recommend schools and teachers to pay special attention to and try to understand the browsing habits and online interactions of this group to reduce the incidence of cyberbullying. Also, to diminish the incidence of cyberbullying, teachers need to understand the reasons for such behavior, strengthen guidance offered to students, debunk myths pertinent to the Internet and cyberbullying, emphasize the harm done to victims of cyberbullying, and educate students about the consequences of cyberbullying from a legal standpoint.

Additionally, focusing on students of different Internet addiction levels, this study examined the influence of cyberbullying attitudes on the behaviors. The results indicate that students who are addicted to the Internet showed the influence of higher cyberbullying attitudes on the behaviors and explanatory power. Social media (e.g., Facebook, Plurk) and online games are the most participated online activities for Internet addicts. These activities are also where teenager cyberbullying happens. When students are playing online games or chatting via social media, they are exposed in the cyberbullying environment. Cyberbullying incidents are no rare cases for students to witness. Therefore, it is suggested that teachers should develop correct attitude toward Internet usage, which parents should also pay attention to. When a problem occurs, it is recommended to reach related organizations or schools for help to decrease the impacts led by different Internet addiction levels.

# Limitations and Future Research

There are several limitations in this study that should be addressed in future studies. First, this study focuses on the status quo of attitude toward cyberbullying and cyberbullying behavior, explores associations of attitude toward behavior on cyberbullying behavior in gender, different internet access times, and Internet addiction situations. The results showed gender, different internet access times, and levels of Internet addiction, the influence of cyberbullying attitudes on the behaviors. In the research, it was not considered whether the different internet access times, and levels of Internet addiction as variables precede (or predict) the cyberbullying behaviors. Future research should treat Internet access time and Internet addiction as antecedent variables in order to fully examine their effects of cyberbullying behaviors. Second, this study did not specifically classify the objects into groups of Internet addiction and non-Internet addiction. Therefore, we suggest using experimental designs to examine the differences between Internet addictions and noninternet addictions, so as to provide references to senior and vocational high school teachers, parents, and school managers Third, while we used cross-sectional data in this study, for it is useful for correlation purposes, causal relations cannot be concluded from such data. Fourth, previous studies' participants have reported cyberbullying behavior according to their own personal experience. Participants are more likely to evade such questions so as to avoid judgment, causing CMV. This study uses the self-report inventory, but the items in the questions are from a third person point of view, measuring cyberbullying behavior the participant saw or heard of online. Finally, one potential limitation of this study is that we measured variables in only one timeframe using self-report questionnaires, suggesting the potential for CMV. We tested for CMV using the Harman one factor test, which showed that all constructs explained roughly an equal amount of variance, suggesting that common method bias was not an issue in this study. This study also undertakes a large data collection and sample to circumvent the problem of CMV.

# ETHICS STATEMENT

The study procedures were carried out in accordance with the APA Ethics Code. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. All subjects were informed about the research, and all provided informed consent. Given the subject matter, no ethical external approval was required under Taiwan law. All participants in the study were received the opportunity to refuse participation at any time without consequences.

# AUTHOR CONTRIBUTIONS

C-MC: data collection, concept and design, interpretation of data, writing up; T-KY: obtaining funding, statistical analysis, interpretation of data, study supervision. All authors wrote the manuscript together and approved the final manuscript.

# FUNDING

The authors thank the Ministry of Science and Technology of Taiwan for financially supporting this research under contract MOST 105-2511-S-507 -001 -MY3.

# REFERENCES

fpsyg-08-01104 June 28, 2017 Time: 18:49 # 9


and school support protective? Child. Psychiatry Hum. Dev. 41, 371–386. doi: 10.1007/s10578-010-0174-5


literacy, and psychopathological symptoms. J. Behav. Addict. 4, 155–162. doi: 10.1556/2006.4.2015.021


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Chao and Yu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Virtual Object-Location Task for Children: Gender and Videogame Experience Influence Navigation; Age Impacts Memory and Completion Time

David Rodriguez-Andres<sup>1</sup> , Magdalena Mendez-Lopez<sup>2</sup> \*, M.-Carmen Juan<sup>1</sup> and Elena Perez-Hernandez<sup>3</sup>

1 Instituto Universitario de Automática e Informática Industrial, Universitat Politècnica de València, Valencia, Spain, 2 IIS Aragón, Departamento de Psicología y Sociología, Universidad de Zaragoza, Zaragoza, Spain, <sup>3</sup> Departamento de Psicología Evolutiva y de la Educación, Universidad Autónoma de Madrid, Madrid, Spain

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

Remco Veltkamp, Utrecht University, Netherlands Kening Zhu, City University of Hong Kong, Hong Kong

> \*Correspondence: Magdalena Mendez-Lopez mmendez@unizar.es

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 28 November 2017 Accepted: 19 March 2018 Published: 04 April 2018

#### Citation:

Rodriguez-Andres D, Mendez-Lopez M, Juan M-C and Perez-Hernandez E (2018) A Virtual Object-Location Task for Children: Gender and Videogame Experience Influence Navigation; Age Impacts Memory and Completion Time. Front. Psychol. 9:451. doi: 10.3389/fpsyg.2018.00451 The use of virtual reality-based tasks for studying memory has increased considerably. Most of the studies that have looked at child population factors that influence performance on such tasks have been focused on cognitive variables. However, little attention has been paid to the impact of non-cognitive skills. In the present paper, we tested 52 typically-developing children aged 5–12 years in a virtual object-location task. The task assessed their spatial short-term memory for the location of three objects in a virtual city. The virtual task environment was presented using a 3D application consisting of a 120<sup>00</sup> stereoscopic screen and a gamepad interface. Measures of learning and displacement indicators in the virtual environment, 3D perception, satisfaction, and usability were obtained. We assessed the children's videogame experience, their visuospatial span, their ability to build blocks, and emotional and behavioral outcomes. The results indicate that learning improved with age. Significant effects on the speed of navigation were found favoring boys and those more experienced with videogames. Visuospatial skills correlated mainly with ability to recall object positions, but the correlation was weak. Longer paths were related with higher scores of withdrawal behavior, attention problems, and a lower visuospatial span. Aggressiveness and experience with the device used for interaction were related with faster navigation. However, the correlations indicated only weak associations among these variables.

Keywords: virtual environment, behavior, emotion, short-term memory, visuospatial skill, children

# INTRODUCTION

Loomis et al. (1999) reviewed the potential of immersive virtual environment (VE) technology for experimental psychology. They described its value as a tool in research on spatial cognition. They highlighted its advantages in terms of methodological issues, that are difficult to achieve in practice without this type of technological support (e.g., facilitating the control of the delivered stimuli, manipulating variables, recording measurements and allowing exposure to complex and natural-appearing environments). VEs have also become quite popular for their contributions to

neuropsychological assessment. Measures of performance (e.g., correct responses and completion time) derived from tasks using VEs have shown moderate sensitivity in detecting cognitive impairments in clinical populations, especially in the assessment of visuospatial and memory skills (see the review of Negut et al., 2016). Some VEs have been used to study children's performance, reporting differences between typically-developing children and children with developmental issues (Bioulac et al., 2012; Courbois et al., 2013; Kalyvioti and Mikropoulos, 2013; Broadbent et al., 2015; Farran et al., 2015). Therefore, virtual reality-based tasks currently play an important role in the field of child psychological assessment as an adjunct to standardized classical tests.

The study of human spatial cognition using VEs became quite popular by emulating virtual tasks based on animal mazes (e.g., Astur et al., 2004; Cánovas et al., 2008). Other virtual tasks simulated familiar places for humans (e.g., Maïano et al., 2011; Purser et al., 2012; Burles et al., 2014). The VEs can be viewed on a computer screen (e.g., Astur et al., 2004; Merrill et al., 2016) or other virtual reality platforms, such as head-mounted displays (HMDs), which can provide a full 360◦ view (Siemerkus et al., 2012). In a typical spatial task, the person controls their movements in the virtual space to memorize places, objects, or routes using a joystick (e.g., Astur et al., 2004; Siemerkus et al., 2012; Walkowiak et al., 2015) or a keyboard (e.g., Purser et al., 2012; Merrill et al., 2016).

Virtual environments used for researching spatial navigation abilities in children have been very similar to those used for adult research (e.g., Hamilton et al., 2003; León et al., 2014; Broadbent et al., 2015; Nys et al., 2015). Children have been asked to navigated the VE and then were tested on their ability to retrace routes or to memorize places or objects. The results found can be extrapolated to results obtained within real environments (e.g., Schmelter et al., 2009). Also, these virtual tasks have been used to draw conclusions about difficulties in orientation in children with developmental disorders (e.g., Hamilton et al., 2003; Courbois et al., 2013; Fornasari et al., 2013; Broadbent et al., 2015).

Most of the studies looking at factors influencing children's performance in spatial tasks have been focused on cognitive variables such as visuospatial abilities (e.g., Nys et al., 2015), memory (e.g., Purser et al., 2012; Nys et al., 2015), executive functions (e.g., Purser et al., 2012), or navigational strategies (e.g., Bullens et al., 2010; León et al., 2014). The impact of children's non-cognitive skills on spatial task performance has been less studied. Van den Brink and Janzen (2013) considered the effects of self-care skills measured using the Vineland Screener. The authors found that there was a significant relation between adaptive functioning and the performance of 2 to 3-year-olds on a VR spatial task used for the assessment of orientation skills. They suggested that independence in everyday activities presented by some of the children was critical in improving their spatial knowledge because of a greater number of opportunities for exploring their spatial surroundings. Also, exploratory behavior was related to emotional factors in a study that tested children with autism (Fornasari et al., 2013) who were less active during free exploration of a virtual town. Children suffering from anxiety disorder also performed more poorly than control participants in a virtual Morris water maze (Mueller et al., 2009). They showed thigmotaxis behavior (i.e., the adaptive tendency to avoid exploring the central zone of a novel place) at the beginning of the test and higher numbers of heading errors and unsuccessful trials. Psychometrical measures of anxiety, but not depression, were related to the number of heading errors.

To our knowledge, there are no published studies about relationships between emotional factors and spatial performance in VEs in healthy children. We suggest that affective components and adaptive behavior could influence the performance of typically-developing children in a spatial task involving exploration of a VE. Previous studies performed in adults found that thigmotaxis behavior was positively correlated with affective components (Kallai et al., 2007). These results were obtained after controlling for gender differences in the levels of fear (i.e., women scored higher than men). Also, neuroticism and psychoticism traits had a negative impact on spatial performance (Burles et al., 2014; Walkowiak et al., 2015). These studies yielded conclusions for young adults, but little is known about the relationships between these psychological variables and performance in the child population.

In the present study, we aimed to determine if the performance of typically-developing children in an emotionally neutral virtual-based spatial task is related to their behavioral and emotional outcomes. To do this, we used a basic shortterm memory test in which children were to learn the spatial locations of objects (i.e., the learning phase) and later were asked about the correct position of one of these objects (i.e., the testing phase). The VE of this virtual object-location (VOL) task consisted of a city square. To provide visual guidance, the square was surrounded by distal cues and proximal cues. The objects were associated to a place holder and located in the central area of the VE. The VE was presented using a 3D application consisting of a 120<sup>00</sup> stereoscopic screen. The children could actively explore details required for orientation by traveling across the interaction area taking a first-person perspective. We chose a gamepad as the device for interaction because it has been preferred by children due to its playability (Rodríguez-Andrés et al., 2016). We tried to reduce the potential influence of individual differences in the experience with videogames and gamepads. For this reason, the participants were trained with the gamepad and performed a tutored trial of the task.

The VOL task was similar to the task used in Rodríguez-Andrés et al. (2016) in terms of the visual and procedural aspects; however, in the present study, we aimed to obtain information about how children's exploratory behaviors were. The present task assessed not only the ability of the children to recall the place of the objects, but also their way of exploring the interaction area of the VE. The aims of the study by Rodríguez-Andrés et al. (2016) were mainly to present the task, to validate the task for the assessment of spatial short-term memory, and to examine the influence of the type of interaction used on the ability to recall the place of the objects, and the perceived usability and satisfaction of the children with the task.

We obtained objective performance measures of the participating children on the VOL task (i.e., learning and displacement indicators). We assessed their perception about the task (3D perception, satisfaction, and usability) and previous

videogame experience. We also considered participant individual differences in the performance of small-scale visuospatial tasks (building blocks and visuospatial span), emotional outcomes (i.e., anxiety, depression, and aggressiveness), and behavioral outcomes (i.e., hyperactivity, withdrawal, and attention problems), which were obtained with a psychometric rating scale. We also considered the age and gender of the participants. The research questions are: (1) Does age, gender, or previous videogame experience of the children affect their performance on the VOL task?; and (2) Are there any significant relationships among performance of the VOL task and the user's variables (i.e., videogame experience, ratings about the experience with the VOL task, visuospatial skills, emotional and behavioral outcomes)?

We hypothesized that age would affect performance in the VOL task. We studied a wide age range, as the values of the learning indicators would be lower in children younger than 6 years. We did not expect to find an effect of gender on VOL task performance because of its low level of difficulty. The task involved remembering the locations of three objects that were shown sequentially with several proximal visual cues aiding orientation. Both boys and girls might use specific orientation strategies to solve the task. We did not expect to find an effect of videogame experience on task execution because the participants were trained before being tested. Finally, a higher ability for recalling objects in the VE would be linked with higher visuospatial skills on small-scale tests. The displacements made across the interaction area of the VE during performance of the task would be related to emotional and behavioral outcomes. Specifically, higher scores on anxiety would be related to a higher tendency to explore the boundaries of the VE.

# MATERIALS AND METHODS

# Participants

The participants were 52 right-handed, typically-developing children from 5 to 12 years old (22 girls and 30 boys; Mage ± SD = 8.06 ± 1.60). They were divided into three age groups: preschool (5–6 year olds; 5 boys and 4 girls); the first cycle of primary school (7–9 year olds; 18 boys and 13 girls); and the second cycle of primary school (10–12 year olds; 7 boys and 5 girls). They were recruited from a summer school. This final sample was selected after applying the inclusion criteria to a larger sample composed of 66 children. None of the participating children had visual or hearing impairments or had had a breech birth, required neonatal resuscitation, had a body temperature higher than 40◦ in the first 5 years of life, had suffered a brain injury, had any impairment in motor performance, or were treated with a medication that could potentially impair their cognitive functioning. A questionnaire was completed by their parents concerning their development and medical history. The parents also completed the Movement ABC-2 Checklist (Henderson et al., 2012) to discard any motivational or emotional difficulty related to motor tasks. We used the Lang-Stereo-Test (Lang, 1983) to check that the children could perceive 3D properly. All parents gave written informed consent before their children's participation. The study was conducted in accordance with the European Directive 2001/20/EC and the Helsinki Declaration for biomedical research involving humans. The University Ethics Committee approved the research protocol.

# The Testing Room and Instrumentation for the VOL Task

The testing room consisted of a square room of about 20 square meters (**Figure 1**). The child was placed in the middle of the room facing one of the walls on which was mounted a 120-inch screen. We used two projectors to send two images to the screen from the back. Each of the two projectors had a linear vertical polarizer. There was a difference of 90◦ between the directions of the two polarizers. The children wore linear polarized glasses to perceive the 3D sensation. These glasses had two vertical polarizers, one for each eye, that were aligned to match the directions of the projectors' polarizers. The interface used to control the child's movements in the VE was a PlayStation gamepad. We used Unity 3D as the game engine.

To run the application, we used a PC with an i7-4770k processor, 16 GB RAM memory, and a graphic card NVIDIA GTX770. The software and hardware used to develop the VOL task were described in Rodríguez-Andrés et al. (2016).

# The VOL Task

The task consisted of a short-term memory test for object location. Briefly, the participant had to search for objects that were placed on tables distributed in certain locations of a VE. Then, he/she had to remember their locations in order to place them in their correct positions later. There was a narrator who guided the participant through all phases of the task with her voice. She told the child what to do each time (e.g., "Remember the location of the objects that you are going to see now"; "Approach it and push the button when its color changes"; "You have to put this object in its correct position"). In Section "Training Phases and VOL Trials," we briefly describe the phases of the VOL task. More details about the VOL task can be found in Rodríguez-Andrés et al. (2016).

FIGURE 1 | The testing room for the VOL task.

## Training Phases

Before starting the trials of the VOL task, each child completed two separate phases: the adaptation phase and the tutorial phase. The aim of the adaptation phase was to familiarize the child with the interaction method. The child learned how to move inside a VE using the gamepad. In this phase, the participant was transported to a VE in mountainous terrain (**Figure 2**). Then, he/she had to follow a path across the mountains to arrive at a goal at the end of the path. Some arrows and bubbles showed the child which direction to follow.

In the tutorial phase, each child completed a short tutorial about how to perform the VOL task. He/she learned what the goal of the task was and how to achieve it. This phase was like a trial of the VOL task (see section "VOL Trials") except for the fact that the child received a visual indication of the position of the object during the testing phase. The visual indication consisted of a vertical green arrow pointing to the position of the object.

#### VOL Trials

The VE of the VOL task was simulated as a city square. The square was surrounded by several buildings (**Figure 3**). The child could move within the limits of an interaction area of the city square (**Figure 3A**). The buildings were outside of the interaction area of the child and worked as visual cues to help orient the participant with spatial orientation (distal cues). Inside the square, there were eight common objects of a city (proximal cues), which also served as guidance (**Figures 3A,B**). We defined two separate areas within the interaction area: the peripheral area, and the central area (**Figure 3C**). The peripheral area included a zone that was three meters away from the tables, whereas the central area included the area where objects were placed on tables (**Figure 3C**).

There were four VOL trials in the VOL task (see **Figure 4**). Each child completed these trials consecutively. The goal of these trials was to assess the children's short-term memory for object location. Each trial was divided into two separate phases: the learning phase, and the testing phase.

Short-term memories for visuospatial items were formed in the learning phase. In this phase, the child searched three gray tables with the aim of finding three hidden objects. These objects were shown one by one. The system guided the search process using a green arrow that pointed to one of the gray tables.

The child had to walk to the table, and, when the child was close enough, the table changed color from gray to green, and the child could see the object on that table for 5 s. The child had to repeat this process two more times to discover all of the objects. It is important to note that the children had to remember the objects they saw and where the objects were placed. At the end of each learning phase, the child returned to the center of the scene, and the VE was rotated 180◦ from the original position before starting the testing phase. Therefore, idiothetic information could not be used as a reference for orientation.

The testing phase consisted of the retrieval of the short-term memories for the visuospatial items that were formed in the learning phase. In this phase, the system showed an object on the screen and the narrator asked the participant for the position of that object. The participant had to put the object in its correct position to complete the trial. The position of the tables and the objects varied in the four different trials of the VOL task as shown in **Figure 4**. We included a score screen to keep the children motivated. They received a star when they finished the tutorial phase and each of the four VOL trials, regardless of the quality of their responses.

# Videogame Experience, 3D Perception, Satisfaction, and Usability Questionnaires

We designed a questionnaire to determine the participants' videogame experience, consisting of two items:"How often do you play with videogames on a PC or smartphone?" and "How often do you play videogames with a gamepad?" The children answered the two items using a five-point Likert scale ranging from "(1) Never" to "(5) Everyday." Also, the children gave their opinion about 3D perception during the performance on the VOL task by answering the statement "At certain moments, the objects came out of the screen" using a five-point Likert scale ranging from "(1) Strongly disagree" to "(5) Strongly agree." The questionnaires were adapted to children. The items of the questionnaires were filled in using text labels that were accompanied by graphical icons (Read, 2008).

We collected information about the satisfaction and usability perceived by the children by using two questionnaires with fivepoint Likert scale items. The satisfaction questionnaire was made up of four items: "How much fun did you have?" [response scale: "(1) None" to "(5) A lot"], "I would invite my friends to play the game" [response scale: "(1) Never" to "(5) Every day"], "Would you play this game another time?" [response scale: "(1) Never" to "(5) Every day"], and "Score the game from 1 to 5" [response scale: "(1) Very bad" to "(5) Very good"]. Finally, a usability questionnaire had two items: "Was the VOL task easy to play?" [response scale: "(1) Very difficult" to "(5) Very easy"], and "I always understood what I had to do" [response scale: "(1) Strongly disagree" to "(5) Strongly agree"].

# Spatial Ability Tests

We used two classical psychometric tests to assess the children's basic visuospatial abilities. We obtained an index of their visuospatial span with the forward version of the Corsi Block

Tapping Test (CBTT), and we used the backward version of the CBTT to collect a measure of their visuospatial working memory capacity (Kessels et al., 2000). We also assessed their visuospatial and visuomotor ability with the Block Construction subtest (BC) from the Nepsy-II battery (Korkman et al., 2014).

# Emotional and Behavioral Rating Scale

We used the Parent Report form of the Behavioral Assessment Scale for Children (PRfBASC) (Reynolds and Kamphaus, 2004) to assess their emotional and behavioral outcomes. PRfBASC is one of the most widely used behavior rating scales for the assessment of behavioral problems, emotional symptoms, and adjustment patterns in children across the following domains of functioning: Internalizing, Externalizing, and Adaptive Behavior. The PRfBASC consists of 130 items (3–6 years old) or 134 items (6–12 years old) about a child's behavior at home and in the community measured on a four-point Likert scale. In this study, we considered the following subscales of the PRfBASC:

delimits the peripheral area.

Anxiety, Depression, Hyperactivity, Aggressiveness, Withdrawal, and Attention Problems.

# Procedure

The participants were tested individually in sessions of approximately 55 min, which took place on the same day and between 9:00 A.M. and 2:00 P.M. They were randomly assigned to one of the following experimental conditions: Condition I and Condition II. In Condition I, the participants were tested with the Lang-Stereo-Test and then completed the questionnaire about videogame experience. Afterward, they performed the VOL task and then completed the questionnaires about 3D perception, satisfaction, and usability. Finally, they performed the CBTT and the BC. In Condition II, the participants performed the CBTT and the BC first and were then assessed with the remaining tests and questionnaires in the same order as described in Condition I. In the recruitment phase of the study (see section "Participants"), theparents completed the PRfBASC to obtain the emotional and behavioral measures. Before the child started, the child and his/her parents met the person responsible for the procedure, who accompanied the child during the whole session. The child and the experimenter talked for about 5 min, until the child felt comfortable with the situation. Then, the parents left the room and the session began.

# Data Analysis

We considered two variables that are related to the videogame experience questionnaire: (item 1) the child's previous experience in playing videogames, and (item 2) child's previous experience using the interaction method of the VOL task. We used the direct scores of these two items to calculate these variables. Similarly, we used the direct score of the 3D perception statement. For the satisfaction and usability questionnaires, we calculated the mean of the children's direct scores for each item of these two questionnaires in order to obtain a general measure of satisfaction and usability in performing the VOL task.

We considered five variables that are related to the performance of the VOL task: VOL Task Score, Total Distance, Total Time Average Speed, and Peripheral Distance. The VOL Task score is an indicator of visuospatial memory and involved the sum of the trials of the VOL task that were correctly performed (range: 0–4). The Total Time consists of the time (in seconds) taken to complete the four trials of the VOL task. The Total Distance is the total distance (in virtual meters) traveled by the child in the four trials of the VOL task. The Average Speed is an indicator of the velocity (in virtual meters/sec) with which the child explored the VE. We calculated this variable by dividing the Total Distance traveled by the Total Time spent to perform the task. The Peripheral Distance consists of the distance traveled by the child in the peripheral zone of the interaction area in the four trials of the VOL task. For the variables: Total Distance, Total Time, Average Speed, and Peripheral Distance, we also calculated the values obtained by the sum of each phase of the VOL task separately (learning and testing).

For the measures of the visuospatial ability, we used the direct scores of the CBTT (forward and backward versions) and BC. Finally, the scores of the subscales measured with the PRfBASC-2 (Anxiety, Depression, Hyperactivity, Aggressiveness, Withdrawal, and Attention Problems) are reported as T-scores (M = 50, SD = 10).

We applied the Shapiro–Wilk test (Shapiro and Wilk, 1965) to check the normality distribution of the dataset variables. This test is especially powerful for samples of small size. The tests showed that only the Anxiety variable followed a normal distribution. We decided to perform non-parametric tests with the entire data-set which are more suitable with distributions of this kind. All analyses were done using the free Software R-Studio (Version 0.98.1079). The results were considered to be statistically significant if p < 0.05.

# RESULTS

**Table 1** shows descriptive statistics for the five variables that are related to performance in the VOL task. In the case of time, speed, path length and peripheral path length, we present descriptive statistics for both the learning and testing phases of the VOL task. **Figure 5** shows the paths made by the children in the testing phases. **Table 1** also shows the descriptive statistics for the participants' experience in playing videogames and using the interaction method, their 3D perception during the VOL task, and their perceived satisfaction and usability. **Table 2** shows the descriptive statistics for the children's visuospatial abilities assessed with CBTT and CB, and their scores on the emotional and behavioral subscales of the PRfBASC-2.

# Effects of Age, Gender, and Previous Videogame Experience on Performance in the VOL Task

The task performance variables were analyzed using the Kruskal– Wallis test with four factors: Gender, Age, Experience in Videogames, and Interaction Method Experience. **Table 3** shows the results of the statistical analyses. The Kruskal–Wallis test

TABLE 1 | Mean scores (standard deviations) for the variables of the VOL task, videogame experience, 3D perception, satisfaction, and usability questionnaires (N = 52).


revealed a significant effect of Age group on the VOL Task Score. The older children had higher scores than the younger ones [χ 2 (2) = 15.8, p < 0.01]. A post hoc test showed significant differences between Age 5–6 and 7–9 (r = 0.49, p < 0.001), and between Age 5–6 and 10–12 (r = 0.84, p < 0.001). The test also indicated that the younger children spent more time completing the task [χ 2 (2) = 13.98, p < 0.01]. **Figure 6A** shows the influence of Gender and Age on the VOL Task Score. The gray and white boxes of the same age group are placed at the same height. The boxes are closer to the maximum score in the oldest group.

The Kruskal–Wallis tests also revealed that there was a significant effect of Age on the Total time spent to complete the task. The younger children required more time than the older ones [χ 2 (2) = 13.98, p < 0.01]. There are statistically significant differences related to the Gender factor. The girls spent more time than the boys to complete the task in all age groups (**Figure 6B**). This difference was especially high in the 5 to 6 year-old group. The girls in this group spent a mean of 16 min to complete the entire task.

To check if previous experience in videogames or previous experience with the interaction method influenced performance on the VOL task, we included these variables in the analyses. The Kruskal–Wallis tests show that only the average speed in the task is influenced by previous experience with videogames [χ 2 (4) = 12.25, p < 0.01]; the children who had more experience with videogames completed the task faster than those who did not have as much experience. Previous experience with the interaction method did not influence any of the variables considered (p > 0.05).

# Usability, Satisfaction, and 3D Perception

We performed one Kruskal–Wallis test for each VOL task measure of performance, using Usability, Satisfaction, and 3D perception as dependent variables. The results of these tests are shown in **Table 4**. The tests indicated that there were no statistically significant differences in the measures of the VOL task performance in relation to these variables. These results reflect that the users' perception of the task and the system did not influence the way users performed the task.

# Relationship Between Performance on the VOL Task and the Participant's Outcomes

To determine the relations between the different performance outcomes in the VOL task and the different scores obtained depending on videogame experience, perception of the VOL task, visuospatial ability, and emotional and behavioral variables, we performed a partial Spearman correlation extracting the influence of Age (**Table 5**).

Some displacement indicators in the VE showed significant correlations. There were significant direct correlations between the VOL task score and two variables of visuospatial abilities in small-scale real space: the visuospatial span backward (CBTT backward; Spearman's r = 0.29, p = 0.04), and blocks construction (BC; Spearman's r = 0.43, p < 0.01). Furthermore, there were relations between the Total Distance (Spearman's r = −0.27, p = 0.04) and Peripheral Distance (Spearman's r = −0.31, p = 0.03) with the CBTT forward score. We also found that with for those with less experience with gamepads, the completion time for the VOL task was longer (Spearman's r = −0.29, p = 0.04).

There were significant direct correlations between the Average Speed of navigation in the VE and experience with the gamepad interaction (Spearman's r = 0.30, p = 0.03). The same type of meaningful relationship was found between Average Speed and the score on the Aggressiveness subscale (Spearman's r = 0.30, p = 0.04). Longer navigation paths were related to higher scores on the Withdrawal (Spearman's r = 0.31, p = 0.03) and Attention Problem subscales (Spearman's r = 0.32, p = 0.02). In addition, longer navigation paths in the peripheral area of the VE were related to higher scores on the Withdrawal subscale (Spearman's r = 0.31, p = 0.03). Shorter path lengths in both the whole interaction area and in the peripheral area of the VE were related to higher visuospatial span scores measured with the CBTT forward (Spearman's r = −0.31, p = 0.03).

# DISCUSSION

We studied the performance of typically-developing boys and girls in a VE that was designed to test short-term memory for the location of objects placed in specific places in a city square (i.e., the VOL task). The VE worked as an open field, which included proximal and distal cues that are common in a city. The area of interaction was divided into two areas (i.e., central and peripheral). The VE was actively explored using a gamepad.

We considered participants' age, gender, and previous videogame experience as potential variables that could influence success on the VOL task and the way of exploring the VE. We also examined relationships among the variables in performance on the VOL task and visuospatial, emotional, and behavioral outcomes.

The children's performance on the VOL task and their visuospatial skills correlated. The task also obtained high values of usability and satisfaction by the children. Hence, we considered that the task was appropriate for studying the spatial performance in a child population without disabilities.

As we hypothesized, the participants' age affected their performance on the VOL task. The task involved the retrieval of short-term memories of three visuospatial items. Also, the proximal and distal cues were important for orientation. The children could not use a strategy based on routes since their point-of-view position was rotated between the learning and testing phases. Their success was dependent on the creation and use of a mental map of the city square and/or links between the target and its surrounding cues. The lower scores of the youngest participants suggest that their visuospatial short-term memory and/or their spatial strategies were relatively weaker during this developmental period. This result is consistent with previous studies (Bullens et al., 2010; Purser et al., 2012; Nys et al., 2015; Mendez-Lopez et al., 2016; Merrill et al., 2016) and with the results found by Rodríguez-Andrés et al. (2016). They performed descriptive analyses taking into account the age of the

participants and the VOL task score. They found a trend toward a better score on the task by the older children than the younger ones.

Age and gender also affected the total time spent on the task. This time was especially longer in the youngest group studied and was related to gender differences found in navigation

TABLE 2 | Mean scores (standard deviations) for the spatial ability tests and the subscales of the Parent Report form of the BASC-2 used in the study (N = 52).


CBTT, Corsi Block Tapping Test; BC, Block Construction subtest from the Nepsy-II; PRfBASC, Parent Report form of the Behavioral Assessment Scale for Children.

TABLE 3 | The results of the Kruskal–Wallis tests for the variables related to the performance of the VOL task.


The tests that reached significance are displayed in bold type.

speed (**Figure 6C**) and amount of previous experience with videogames. Children who played videogames frequently were found to navigate with greater speed, precision, and agility. The amount of experience with technological devices increases with the age (Sayers, 2004). Also, girls play videogames less frequently (Terlecki and Newcombe, 2005). Interestingly, experience with playing videogames influenced only performance variables that reflected the way in which the participant explored the VE but did not affect the score obtained. The VOL score was a measure of the visuospatial abilities of the children based on the correlations found between this outcome and the score obtained on the paper–pencil spatial tests. This result suggests that being less skilled in videogames does not influence the visuospatial ability of the user obtained in a virtual spatial task, but it does influence the speed of exploration of the VE.

Familiarity with the interaction method used in the VOL task did not impair how the way the children explored the space; however, there was a trend toward a lower ability to locate the objects by the less experienced participants. Also, those more skilled with the gamepad were faster in the completion of the task and the navigation of the VE, but the strength of the correlation was weak. Our sample was not very familiar with the gamepad overall. The gamepad was the preferred device of interaction by children from 5 to 10 years old when compared with a device based on a natural user interface in a previous study (Rodríguez-Andrés et al., 2016). As noted by the mean score, our participants had played with a gamepad occasionally, but their frequency of use was less than once a week. Based on our informal notes, they played more frequently using touch-based interfaces in mobile devices, including tablets. We gave them training to reduce the possibility of differences in experience-based performance (Sandamas et al., 2009). The VOL task included two phases for practice with the interface (i.e., adaptation and tutorial). The first one involved practicing in a VE which was more difficult to explore than the VE of the learning trials. The VE of the adaptation phase required strong fine motor skills. These two phases gave the users training in the procedural aspects of the virtual navigation. In this way, we attempted to reduce any potential bias due to experiential factors in the interpretation of children's ability to locate objects.

The children's perception of the task experience and with overall system was very positive, especially for its usability aspects. The children gave a score close to maximum on the usability questionnaire; he means were 4 on a scale from 1 to 5. This result shows that the task was easy for them to perform. Three reasons for the high usability level include (1) the procedure aimed to facilitate the familiarity and comprehension of task phases, (2) the users were habituated to the interaction system prior to being tested, and (3) the children appreciated the innovative nature of the system. For example, the 120-inch stereoscopic screen, wearing polarized glasses, and the sense of immersion in the virtual city were novelties for most if not all of the children (Wells et al., 2010). However, we suggest that novelty alone is unlikely to influence the positive scores. If that were the case, the children would have given the maximum score on the satisfaction and 3D perception questionnaires. All of the children had stereoptic vision, but their 3D perception was not highly

TABLE 4 | Multifactorial Kruskal–Wallis tests for the Usability, Satisfaction, and 3D Perception variables.


positive considering their mean rating of the experience (3.56 out of a maximum of 5). Similarly, the children were satisfied with the task, but some aspects of the task might have increased the perceived satisfaction more than others. The task provided motivating feedback after the completion of each trial regardless of the quality of execution. This was to prevent any frustration that might have been caused by a feeling of inadequacy and to keep the children engaged throughout the task. Despite this, the children gave the task a relatively high score (3.39 out of 5).

As we mentioned above, boys were more skilled than girls in the exploring the VE. However, contrary to what we expected, boys did not outperform girls in their ability to locate the virtual objects. The similar performance between boys and girls was also found in several studies in which children were trained in a navigational short-term memory task (Juan et al., 2014; León et al., 2014; Piccardi et al., 2014; Mendez-Lopez et al., 2016). Also, Rodríguez-Andrés et al. (2016) did not find significant differences between 5 and 10-year old boys and girls in their ability to locate the objects in the task. They performed similarly regardless of the type of interaction used (i.e., natural interaction or gamepad). The level of difficulty of the VOL task was low in terms of its VE and the memory load required. The spatial layout of the VE had proximal and distal cues guiding orientation. All of these cues could be seen from any viewpoint of the interaction area by the rotation of the VE during the exploration. In addition, the task requirement was to store three spatial locations temporarily that had been sequentially presented. The results agree with those of León et al. (2014) who suggested that gender differences emerge only in spatial tasks that are more challenging.

We found significant correlations between task performance and children's visuospatial abilities in some paper and pencil spatial tests. The moderate strength of the correlation found between the VOL score and the score on the BC subtest indicates that the success in object location in our task is related to the general ability to calculate position and directionality (Korkman et al., 2014). It is also related to spatial working memory, but to a lesser extent as indicated by the weak correlation found between the VOL score and the score on the CBTT backward subtest. High scores on this subtest reveal good skill in holding in mind and manipulating a large number of visuospatial items (Kessels et al., 2000). We suggest that the mental manipulation of spatial representations is a key factor in solving our virtual task because there was no correlation with the simple ability to maintain spatial information in short-term memory. This supports the relevance of executive functions in working memory tasks involving spatial relations (Purser et al., 2012; Korthauer et al., 2017). It is also interesting to note that those participants with lower spatial spans executed longer path lengths in the VE, but the strength of the correlation was weak. The spatial span measure is affected by attentional capacity (Kessels et al., 2000), thus the result could also be interpreted as reflecting the relation

#### TABLE 5 | Partial Spearman correlations (N = 52).

fpsyg-09-00451 March 30, 2018 Time: 16:18 # 11


CBTT, Corsi Block Tapping Test; BC, Block Construction subtest from the Nepsy-II. The correlation coefficients that reached significance are displayed in bold type.

between execution of longer paths and increased attentional difficulties. Those children that are more prone to distraction might be less able to navigate efficiently to a spatial target. Thus, the level of the participants' attentional capacity could be an important factor to be considered in spatial task performance, as discussed by Farran et al. (2015).

Contrary to what we expected, we did not find correlations between anxiety measures and displacements within the peripheral area of the VE. We propose three possible explanations for this result: (1) the children scored within the normal limits in all emotional and behavioral outcomes considered, whereas previous studies were conducted using samples with clinical symptoms (Mueller et al., 2009; Burles et al., 2014), (2) our VE was emotionally neutral, and we gave children the optimal conditions to reduce anxiety regarding the testing situation (i.e., an initial familiarization with the experimenter and positive reinforcement during the task), and (3) our VE target items were all in the same space. A more complex environment with several spaces that are not accessible at first sight might be more prone to reveal significant correlations with anxious behavioral traits.

Nevertheless, withdrawal behaviors were related to an increase in the exploratory behaviors in our sample, but they did not affect spatial learning or the time spent on the task. It might be suggested that the children withdraw made an intense exploration of the VE in order to achieve a good spatial representation. Fornasari et al. (2013) found an effect of withdrawal on the exploration of a virtual city. Contrary to what we found, this behavior was related to a reduction in the exploration of the VE, but the differences between these results could be explained by the populations studied. In the case of Fornasari et al. (2013), they studied a clinical population of children with autism. Based on levels of withdrawal within the normal limits, we can speculate that withdrawal might have a negative impact on social outdoor games. This result partially supported the relevance of previous experiences in spatial behaviors proposed by other authors (e.g., Lawton and Kallai, 2002).

Finally, those children who are more prone to aggressive behaviors navigated the VE faster, but there was no significant correlation with time savings. This result is in line with studies confirming that the feeling of anger predicts faster motor behavior (e.g., Deffenbacher et al., 2003; Roidl et al., 2014).

The present research has some limitations. First, our task tests spatial short-term memory learning for three locations in a VE that works like an open field. The level of difficulty is low. It would have been interesting to compare results with those obtained in a task that was more difficult and a VE that was more complex. The second problem is related to the sample. It would have been desirable to increase the number of participants in each age group, especially in the youngest group.

The VOL task presents a VE in which participants use their navigational competencies and their spatial short-term memory for the location of objects. The key factor in an object-location task is the possibility to mentally represent a spatial configuration of interrelated objects. We used three objects because previous research has tested adults in spatial tasks with three or four objects (Zimmer et al., 2003; Iachini et al., 2009), and, from 5 years of age, a person is able to recall the location of 2–3 objects (Mendez-Lopez et al., 2016). The VOL task is attractive for children and is also challenging for adults. We considered that this is a positive aspect of this task because the VOL task provides an opportunity to increase knowledge about spatial memory and navigation and to directly compare these skills in participants of all ages. However, this aspect also puts us at a disadvantage in determining the effects of individual factors in spatial performance. If the task had incorporated more objects

that would make it more difficult to perform, there might have been gender effects in favor of the boys. As we have discussed above, the difficulty of a spatial task is a key factor in the existence of gender differences (León et al., 2014). Also, we hypothesize that an increase in the number of objects to recall would have negatively affected the scores on satisfaction and usability given by the younger children because it would be very difficult for them to perform it. In addition, their exploratory behavior would have been more prone to reveal significant correlations with anxiety or withdrawal behaviors.

# CONCLUSION

Age affected the spatial short-term memory for the location of three objects in a virtual city in children between the ages of 5 and 12 years. Three factors contributed positively to improving the accuracy of the children's performance during the navigation: age, being male, and having more experience with videogames. There were weak associations among variables which showed that the individual differences in visuospatial skills correlated mainly with the ability to recall objects' positions. Behavioral and emotional variables were not related to object location memory. However, three variables were associated with differences in the exploration of the VE, namely: levels of attention, aggressiveness, and withdrawal.

# REFERENCES


# AUTHOR CONTRIBUTIONS

DR-A, MM-L, M-CJ, and EP-H: conceived, designed, and performed the experiments, interpreted the data, and drafted the manuscript. DR-A and MM-L: analyzed the data.

# FUNDING

This work was funded mainly by the Spanish Ministry of Economy, Industry and Competitiveness (MINECO) through the CHILDMNEMOS project (TIN2012-37381-C02-01) and co-financed by the European Regional Development Fund (FEDER). Other financial support was received from the Generalitat Valenciana (Conselleria d'Educació, Investigació, Cultura i Esport) through the grant for consolidable research groups (AICO/2017/041), the Gobierno de Aragón (Departamento de Innovación, Investigación y Universidad), and the FEDER "Construyendo Europa desde Aragón".

# ACKNOWLEDGMENTS

We would like to thank the following for their contributions James F. Juola, the "Escola d'Estiu" and the ETSInf for letting us use their facilities.

development: a comparison of sparse and rich environments. J. Neurodev. Disord. 7:37. doi: 10.1186/s11689-015-9133-6



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Rodriguez-Andres, Mendez-Lopez, Juan and Perez-Hernandez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Digital Inclusion in Older Adults: A Comparison Between Face-to-Face and Blended Digital Literacy Workshops

Claudia I. Martínez-Alcalá1,2 \*, Alejandra Rosales-Lagarde1,2 , María de los Ángeles Alonso-Lavernia<sup>3</sup> , José Á. Ramírez-Salvador <sup>3</sup> , Brenda Jiménez-Rodríguez <sup>2</sup> , Rosario M. Cepeda-Rebollar <sup>2</sup> , José Sócrates López-Noguerola<sup>2</sup> , María Leticia Bautista-Díaz <sup>4</sup> and Raúl Azael Agis-Juárez <sup>2</sup>

<sup>1</sup> Cátedras CONACyT, Consejo Nacional de Ciencia y Tecnología, Mexico City, Mexico, <sup>2</sup> Área Académica de Gerontología, Instituto de Ciencias de la Salud, Universidad Autónoma del Estado de Hidalgo, Pachuca, Mexico, <sup>3</sup> Área Académica de Computación y Electrónica, Instituto de Ciencias Básicas e Ingeniería, Universidad Autónoma del Estado de Hidalgo, Pachuca, Mexico, <sup>4</sup> Área Académica de Psicología del Instituto de Ciencias de la Salud, Universidad Autónoma del Estado de Hidalgo, Pachuca, Mexico

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

Andrej Košir, University of Ljubljana, Slovenia José Carlos Núñez, Universidad de Oviedo Mieres, Spain

#### \*Correspondence:

Claudia I. Martínez-Alcalá c\_isabel\_alcala@hotmail.com

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in ICT

Received: 29 November 2017 Accepted: 26 July 2018 Published: 28 August 2018

#### Citation:

Martínez-Alcalá CI, Rosales-Lagarde A, Alonso-Lavernia MÁ, Ramírez-Salvador JÁ, Jiménez-Rodríguez B, Cepeda-Rebollar RM, López-Noguerola JS, Bautista-Díaz ML and Agis-Juárez RA (2018) Digital Inclusion in Older Adults: A Comparison Between Face-to-Face and Blended Digital Literacy Workshops. Front. ICT 5:21. doi: 10.3389/fict.2018.00021 As information and services are becoming more and more decentralized and they are often available in the cloud, an increasing number of older adults are expected to use Internet-based services—health, education, finance and others. For this reason, it seems important to plan models and/or strategies to allow the older adult population to acquire and enhance digital competencies more easily. The goal of this research is to show a blended workshop based on a Learning Management System (LMS) as a supporting tool for older adults' digital literacy. This blended workshop was based on the adoption of an instructional model and on prior experiences of the groups of elderly that participated in the face-to-face workshops. This study involved 98 adults aged 60 and above, 72 Females (68.5 ± 6.9) and 26 Males (73.3 ± 7.4). 61 older adults participated in the face-to-face workshop (FFG) on digital literacy and 37 participated in the blended workshop (BLG). Digital literacy increased at the post-evaluation after the workshops but even more for the BLG. Likewise, in the validation of the blended workshop the results were positive regarding ease of use, perceived usefulness, attitude toward using and intention to use, which showed that older adults believe that it is useful to implement this type of supporting systems for developing their digital competencies. Hence, it is possible to conclude that older adults are capable of learning and acquiring digital literacy skills as long as they are strongly motivated or they know the functional benefits related to ICT.

Keywords: digital literacy, older adults, face-to face workshop, blended workshop, learning management system (LSMS)

# INTRODUCTION

At present, Information and Communication Technologies (ICT) are becoming omnipresent in our daily lives due to the increasing tendency to use the Internet and mobile devices such as smartphones and tablets, that have allowed access to information and services anytime, anywhere, thanks to their portability (Navarro et al., 2017; Engel et al., 2018). Consequently, an increasing number of older adults are expected to use Internet-based services—health, education, finance, and others- as information and services are becoming more and more decentralized and they are often available in the cloud.

Furthermore, this population has shown considerable interest in learning how to use ICT, to stay socially connected, to access instant information, and to perform everyday tasks such as shopping, traveling, and banking. However, this population's digital skills are minimal, since they are not included in the new interaction environment that marks current technological breakthrough. There is a wide variety of technologies that have caused an increasing gap between the tools used by the young population and the ones used by the elderly population (Vroman et al., 2015; Kuerbis et al., 2017), since the latter are left behind vis-à-vis the rest of the age-groups (Choi and DiNitto, 2013; Hodge et al., 2017).

According to Internet World Stats (2017), from a total of 3.6 billion people worldwide, 48.3% of the population is digitally excluded. In the case of Mexico, with a total population of 130, 222, 815, 34.7% experience digital exclusion. In Mexico, the Instituto Nacional de Estadistica y Geografía (Instituto Nacional de Estadistica y Geografía, 2016) suggests that 78.4% of adults aged 55 or more do not know or use the Internet.

These disparities regarding Internet and ICT use are commonly referred to as a digital divide, which suggests that people with certain demographic and socioeconomic characteristics can be at a disadvantage to access and use the Internet in comparison with other groups (van Deursen and Helsper, 2015; Delello and McWhorter, 2017; Hodge et al., 2017). Delello and McWhorter (2017) mention that the population with no access to ICT or no interaction with digital products or services is called "digitally excluded." Digital exclusion implies unequal access and incapacity to use the ICT, both of them now considered essential to fully participate in society (Schejter et al., 2015).

It is a fact that constant evolution in ICT brings about the need for people to acquire ever higher levels of digital literacy to maintain their sense of inclusion. Digital literacy is a set of skills associated with the use of ICT that every individual should develop to be able to perform in a computerized society (Friemel, 2016; Van Deursen et al., 2016). Thus, it can be said that digital literacy constitutes a fundamental element in the development of any individual, as it allows its insertion in today's society in a more participatory manner. For this reason, some countries are interested in increasing older adults' digital competencies, as these grant them a variety of advantages and benefits at a personal and social level. It is essential, then, to design strategies that facilitate older adults' participation and presence in the use of Internet-based services.

The following sections constitute a review of how learning occurs in the elderly and the description of educational initiatives that have been implemented for the digital literacy of this population. Firstly, we present two types of learning (face-toface and blended) that were implemented for literacy instruction in older adults within the Institute of Health Sciences of the Autonomous University of the State of Hidalgo. Subsequently, a prior and subsequent comparison of the level of digital literacy obtained by older adults in each modality is made. Next, the validation of the Learning Management System (LMS) implemented within the blended workshop as a support tool for digital literacy of older adults is shown. Finally, the challenges that older adults and instructors face when adopting a mixed workshop for ICT instruction are highlighted.

# Education and Learning for the Elderly

One of the fundamental objectives of this study is to analyze under what learning environment elderly adults learn best, taking into account the bases of andragogy. Andragogy is considered as the discipline that allows to know the most relevant educational principles and processes based on the characteristics and needs of adults, both in their maturity and in their old age (Muchtar and Yanuarsari, 2018). Learning in the elderly presents distinctive features that must be taken into account for any approach to educational programs that include older adults.

In addition to considering the main theories of learning and the pedagogical characteristics applicable to the education of older adults, it is important to highlight other factors that make the teaching-learning process special in the elderly. Some of these factors are: Physical and mental changes, memory loss, decline in cognitive abilities and, finally, life experience. According to Wlodkowsk and Ginsberg (2017), the fundamental element that determines the learning process is motivation, and it should be the main factor in any educational program designed for this type of population.

The fact that older people are often considered dependent and lacking in initiative and determination can lead to education programs being conceived in a condescending manner, from top to bottom, without giving learners any opportunity to set their own priorities and make their own decisions.

There are studies that indicate how the present generations of older adults show the capacity to be active and the interest in increasing the possibilities of human, social and educational development (Oliver et al., 2017; Rubenson, 2018). In this sense, Rubenson (2018) demonstrated that the present generation of older adults wishes to continue education, they have a greater commitment to learning and are aware that through education they can improve their quality of life and their participation in the communities.

Gonzalez et al. (2015) point out that some fundamental aspects for the learning process in older adults are: motivation, experience, need, self-concept, learning usefulness and orientation to learn.

In addition, it is important to consider that learning is not only carried out in classrooms or other formal contexts, but also under many and varied conditions. Above all, the elderly have accumulated a countless number of hours of informal learning. So, it is not enough that older people are given access to existing services, it is also important to create educational environments that recognize and support all types of learning and all types of previous experience (Hodge et al., 2017; Tam, 2018).

Fausset et al. (2013) mention that when older adults notice personally relevant usefulness of technologies, and when, at the same time, they receive family support, they will regularly use and adopt ICT. Therefore, gaining greater understanding of the experience lived by older adults in the use of technology will facilitate the implementation of appropriate technological solutions for this population. That is why some authors suggest that ICT patterns of use in older adults should be studied (Tennant et al., 2015; Tsai et al., 2015) and that it is necessary to point out the importance of researching and proposing models and/or strategies for this population to acquire digital competencies more easily.

Therefore, the educational model toward digital literacy of the elderly should be based on 4 important points:


In relation to the above, it can be said that the educational model for digital literacy of the elderly should be developed in the framework of a personalized, cooperative, collaborative and meaningful learning, which can provide them with the basic tools as a starting point, and with a source of motivation so that they adopt ICT as part of their life. Likewise, this should be based on andragogy, taking into consideration the biological, psychological and social characteristics typical of their age.

Finally, the teacher is a fundamental element in any teachinglearning process, especially in the case of elderly learning, where the teacher plays an essential role to achieve balance among the student, the group and their needs, and to maintain or increase their motivation, which is essential for the training of older adults. One of the strategies implemented in this study was that the digital literacy workshops had a tutor with knowledge about andragogy, as well as support staff who were gerontology students.

# Digital Literacy Workshops for Older Adults

Due to an increase in recent technology, public institutions and international organizations have been obliged to develop initiatives for the inclusion of older people in the use of ICTs. **Table 1** describes some initiatives that have been implemented in the world, to help older adults to be included in today's digital society.

One of the fundamental objectives of digital literacy in older adults is that they take on a more participatory role in society, and hence improve their quality of life. In this sense, the studies mentioned above sought to implement initiatives that allowed the development of digital competencies in older adults, through the creation of face-to-face courses. However, this type of courses presents some limitations that could affect learning in the elderly; for example, working methods in face-to-face courses are not focused on the particular needs of students, and there is no availability of learning materials for consulting outside of class.

# MATERIALS AND METHODS

# Context

The Academic Area of Gerontology at the Instituto de Ciencias de la salud (ICSa) of the Universidad Autónoma del Estado de Hidalgo began to teach the Digital Literacy Workshop for adults over 60 years of age in 2014 and, from the outset, the educational strategy was based on gerontological foundations and andragogy. Six workshops have been delivered to date, with groups of between 15 and 25 older adults.

The diffusion of the workshops is done through printed (advertisements) and digital (radio) media. Older adults enroll in the workshop voluntarily. The workshops last ∼ 4 months and they are carried out in computer rooms equipped with All-One computers connected to Internet. Every older adult is provided with a printed manual with information on the modules and topics addressed during the workshops. A tutor leads every workshop and s/he indicates the topics to be developed. Moreover, the workshops are supported by professional staff (gerontologists) who provide personalized attention to every older adult.

The digital competencies of each adult are evaluated by a Test which is described below and which is systematically implemented in all the workshops (pre and post). The participants were included in the study as long as they had little or no knowledge of ICT, they were healthy, they could read, write and speak, and were over 60 years of age. Finally, it is important to mention that this study was reviewed and approved by the Investigation and Ethics committee of the ICSa, UAEH. All research participants provided their written and informed consent.

# Participants of the Workshops

This study involved 98 adults, 72 Females (68.5 ± 6.9) and 26 Males (73.3 ± 7.4). Sixty-one older adults participated in the face-to-face workshop group (FFG) on digital literacy and 37 participated in the blended workshop group (BLG). The group of older adults who interacted in the face-to-face workshop on digital literacy was composed of 46 women and 15 men, with an average age of 69.54 ± 7 years and an average schooling of 8.26 ± 3 years. The group of older adults who interacted with the blended workshop was composed of 26 women and 11 men, with an average age of 70.43 years ± 7 and an average schooling of 9.9 ± 4 years.

## Digital Literacy Workshops Face-to-Face Workshops

The objective of these workshops was to assist older adults with the development of digital literacy skills through face-to-face tutoring. The face-to-face workshop consists of 3 lessons, namely, Introduction to ICT, Computer Programs and Getting to Know the Internet, with a total of 16 topics.

Regarding learning methods, each student had a printed manual with the topics that would be studied in the workshop. TABLE 1 | Description of initiatives for digital literacy of older adults.


For the instruction of digital skills, the tutor used digital presentations and a projector as support material. At the beginning of each class, the tutor asked random questions to each student so that they could remember the concepts and topics seen in the previous classes.

### Blended Workshops Based on a Learning Management System

The objective of these workshops was to assist older adults with the development of digital literacy skills, through the presentation of multimedia learning activities and materials that enhance their knowledge. The blended workshop consists of 3 lessons, namely, Introduction to ICT, Computer Programs and Getting to Know the Internet, with a total of 16 topics. The blended workshop organization contains the following labels: Welcome, Lessons, Resources, Chat, Course Outline, Learners and Facilitators (see **Figure 1**).

Regarding learning methods, the lesson sequence was organized in initiation, development and closure activities. In the initiation activity, the learner has to identify and activate background knowledge. Afterwards, during the development activity new learning is generated. Lastly, the closure activity reinforces learning by suggesting a review of the topics. Also, this workshop had 8 evaluations that allowed older adults to review the concepts at different times (up to 10 attempts), with the aim of reinforcing the knowledge of certain terms and tasks that were seen in each workshop module.

Learners had guides, activities, multimedia learning materials (digital presentations, videos, web pages) and resources that allowed them to acquire the necessary knowledge. Some materials could be viewed within the platform and others were distributed through links or they could even be downloaded for local reproduction on the equipment. It is important to mention that in this workshop both the teacher and the student worked together to build knowledge, generate learning and develop digital skills more easily.

The platform selected for the workshop implementation was NEO LMS, since the integration of administration and management tools facilitate the instructor's work in controlling the development of the course. Furthermore, it is endowed with communication tools that significantly enable course monitoring and development. Additionally, this platform has a version that is adaptable to mobile devices, enabling users to access the platform from any mobile device, any time. Clearly, one of the advantages of a LMS is the possibility of keeping available all the resources and files as often as necessary (Queiruga-Dios et al., 2015).

#### Instructional Model

For this study, the ASSURE instructional design model was used, since it has the necessary characteristics to implement face-to face and blended instruction, and it can be adapted to design a course or a lesson systematically on a specific topic (Lopez-Betancourt and Garcia Rodriguez, 2015). In the ASSURE instructional model 6 phases are presented: Analyze learners, State objectives, Select instructional methods and materials, Utilize materials, Require learner response, and Evaluation and revision.


# Instruments

The "Senior Digital Literacy Evaluation (SDLE)" is an instrument that was designed to measure the digital literacy level and it is based on previous questionnaires (Rangel and Peñalosa, 2013; Hall et al., 2014; Cabezas et al., 2017). On the basis of this, one of the authors (CIMA) designed an adapted version of the instrument, thanks to her education as a doctor in Multimedia Engineering. The result is the SDLE test which contains 110 items divided into three sections: Use and knowledge of the computer, Use and knowledge of the Internet and Knowledge of domestic and daily life devices (see **Supplemental Material**).

The Use and Knowledge of the Computer section includes 54 items divided into four dimensions: (1) Ownership and use of technological and computer devices; (2) Knowledge of computer resources; (3) Identification of computer terms, and (4) Activities normally performed with technological devices. The Use and Knowledge of the Internet section includes 44 items divided into three dimensions: (1) Use of Internet resources; (2) Identification of terms related to the Internet, and (3) Activities normally performed with the Internet. Finally, the section of Knowledge of domestic and daily life devices includes 12 items related to the use and frequency of domestic and daily life devices.

The main variables explored were related to ICT use, access, possession, activities, number of times and level of management, along with sociodemographic variables. The score an older adult can obtain in the Evaluation in its Spanish letter is: A+ (Total competence); A (Moderate competence); +M (Medium competence); M (Medium-Low competence); +B (Low-Medium competence) or B (Low competence). These scores are only indicative and it is not possible to assert that the scale provides absolute digital literacy values, but it presents specific and significant indicators of the technological competencies of older adults (Cepeda-Rebollar, 2016).

Likewise, at the end of the blended workshop a validation test was applied based on the Technology Acceptance Model (TAM). In order to validate this workshop, different variables were considered such as perceived ease of use, perceived usefulness, attitude toward using and intended use. The scale corresponds to a five-point Likert scale. The values assigned were: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Lastly, an open time to older adults was allowed to express their experiences and perceptions about the blended workshop (see **Supplemental Material**).

# Content Validity Analysis

The validity of an instrument refers to its ability to measure and describe what it is supposed to measure and describe. In particular, content validity aims to verify the degree to which an instrument reflects a specific domain of content of what is measured, that is, the degree to which the measurement represents the measured concept. For validating the Senior Digital Literacy Evaluation (SDLE), content validity methodology was used, in order to determine whether the items or questions proposed in the instrument reflect the knowledge, abilities and skills that we want to measure, in this case the Technological Literacy Level in the elderly.

Content validity is usually evaluated through a panel or an expert judgment, which is defined as an informed opinion of people with experience in the subject, who are recognized by others as qualified experts, and who can provide information, evidence and judgment (Almanasreh et al., 2018). Content validity was carried out with the aid of a group of experts, who evaluated individually every single test item, so as to determine its relevance for the variable it is intended to measure and consequently to verify how adequate the instrument was according to their judgment.

This evaluation was carried out in the second semester of 2014, by a group of 7 experts in the area (5 female 2 male) who have a doctoral degree. These experts undertook the task of evaluating the content validity and the relevance of the instrument. Below, the experts' academic and professional profiles are briefly described:


Every expert was given a questionnaire which included directions and the conceptual definition of the construct. The options available to classify each item were: (1) essential, (2) useful but not essential, and (3) non-essential. The initial instrument was composed of 145 items represented in three sections. **Table 2** describes the constructs, conceptual definition, dimension, items, and total number of items.

Subsequently, every expert determined content validity ratio (CVR and CRV') for each of the items, by means of the equations described in Lawshe and Tristan's Model. Results showed that the majority of items are considered acceptable, given that the values exceed the minimum cut-off point of 0.58.

Based on the results obtained, items of the instrument were eliminated in those cases where their value was lower than 0.58. All those items in which the value exceeded the minimum cutoff point of 0.58 were kept. A total of 35 items, which were considered irrelevant by the panel of experts, were eliminated. In the final version, the instrument consisted of 110 items (see **Tables 3**, **4**).

The content value analysis of the items incorporated in the instrument revealed which items are considered relevant for the evaluation of technological literacy level in older adults. In general, the results of the index show that the SDLE instrument can be considered valid in its content.

# Statistical Analysis

It is known that in Mexico there is a 25% of analphabetism among older adults (Consejo Nacional de Población (CONAPO), 2010), so Pearson correlations were made between age and years of education for each group. Likewise, Spearman correlations between age and the scores of the pre SDLE; and years of education and the scores of the pre SDLE for each group were done, because negative associations between age and digitalization and positive ones between education and digitalization could be expected. Also, the comparison between



years of education and the post SDLE was made for each group to notice if changes in this relationship occurred. To verify if the FFG and BLG for the variables age and years of education were similar, independent Student t-tests were used. Later, pre and Post SDLE scores were compared using Wilcoxon t-tests for each group. At last, to find if FFG and BLG were initially different and/or after the intervention, Mann-Whitney U tests were employed. At the end, correlation analyses were carried out to verify if the cumulative number of correct answers on the eight evaluations was related to the final examination score; also, the number of attempts of the participants in each activity was associated to the cumulative number of correct answers on the eight evaluations and the final examination score.

# RESULTS

# Comparison Within and Between the Workshops

Age and education were inversely correlated but not in a significant way in any group [r(59) = −0.16, p < 0.21; r(35) = −0.26, p < 0.11], for the FFG and the BLG, respectively]. Neither between age and the pre FFG SDLE scores, nor between age and the pre BLG SDLE scores (r = −0.17, p > 0.10; r = −0.01, p > 0.10, respectively). Nevertheless, a positive correlation between years of education and the pre SDLE scores was found for the FFG and a marginal one for the BLG (r = 0.53, p < 0.01; r = 0.34, p < 0.05, respectively). Likewise, as it can be seen in **Figure 2**, a positive correlation was found between years of education and the pre and likewise for education and the post SDLE scores only for the FFG (r = 0.53, p < 0.01); for the BLG (r = 0.16, p > 0.10).

There were not significant differences in Student t-tests for age [t(96) = 0.58, p < 0.54]. Nevertheless, the level of education was almost 2 years different between groups [M = 8.2 ± 3 vs. M = 9.9 ± 4 for the FFG and the BLG, respectively; t(96) = 2.19, p < 0.03], because the FFG had a lower level of education in comparison to the BLG.

When pre-post comparisons were made within each group with Wilcoxon tests, it was evident that the workshops ameliorated the older adult digital competence in both groups (z = −6.79, p < 0.0001; z = −5.30, p < 0.0001, for the FFG and the BLG, respectively). Also, their previous level of

#### TABLE 3 | Reason for content validity of the SDLE.


(Continued)

#### TABLE 3 | Continued


(Continued)

#### TABLE 3 | Continued


The red colored values indicate that the item does not have content validity.

#### TABLE 4 | Final Version of the Instrument SDLE.


digital competence seemed to be similar. In **Figure 3**, frequency polygrams of the two groups in both conditions are presented. Both groups had low and low-medium competence at the beginning of the study and after the workshop they increased their digital competence.

The later was confirmed by statistical analysis with Mann-Whitney U tests when their pre evaluations means were considered [U(61, 37) = 1,122.5, p < 0.96]. Mann-Whitney U tests were again computed and it was evident that, after the workshop, the BLG furthermore increased its digital competence [U(61, 37) = 810.5, p < 0.01; see **Figure 4**].

The cumulative number of correct answers on the eight activities was related to the final examination score in a significant way (r = 0.51, p < 0.01; **Figure 5**).

The number of attempts of the participants in each activity was not associated to the cumulative number of correct answers on the eight activities and the final examination score (r = 0.08, p > 0.10).

## Validation of the Blended Workshop

After the workshops ended, the BLG was invited to answer a validation test that evaluated aspects such as design, usability, and technological acceptance. In the analysis of the first variable, ease of use, 13 of the older adults indicated a positive agreement stating that the interaction with the system is clear and understandable and even the menu is easy to use. For their part, five older adults disagreed that the workshop is easy to use, indicating that a lot of mental effort is required to be able to use the platform and some materials.

In the variable of perceived usefulness, favorable results were obtained from older adults, where 16 stated that it is useful and indispensable to implement this type of workshops so that the population acquires digital literacy skills. Only two adults indicated that they disagreed.

The evaluation of attitude toward using showed that 15 older adults were enthusiastic about using the platform. On the other hand, three adults were neutral and two disagreed. This indicates that the Blended Workshop based on a LMS offers an attractive environment to support older adults to acquire digital skills more easily. However, some aspects of the presentation of materials and access to them must be improved. Finally, in the intention to use variable, 15 older adults indicated a positive agreement stating that they will use the system to reinforce their knowledge during and after the workshop (see **Figure 6**).

An analysis of comments and experiences of the BLG both by the elderly participants and by the instructor when adopting an online support system for teaching ICT are presented below.

# Learning Difficulty

Despite the fact that older adults are experiencing a decline in their cognitive and perception abilities, according to these results they are able to learn new skills. Also, Navarro et al. (2017),

mention that older adults are able to learn and acquire digital skills more easily as long as there is a strong motivation or when they are aware of the functional benefits in relation to ICT.

In this study, participants had the opportunity to explore the blended workshop with the guidance of the instructor and support staff, so they could focus on interacting with the platform and acquiring new knowledge. In addition, it was found that the availability of support and the adults' own confidence influence how they acquire their digital skills.

Regarding the role of the instructor, the difficulty was to design materials and activities that would serve as a review and/or feedback for the topics seen in class, since in the face-to-face workshops adults often forgot some steps to perform certain activities, so they had to be repeated several times for them to

remember. Derived from this, it was necessary to design materials and activities that would reinforce adult learning.

groups (p < 0.01). FFG, Face-to-Face Group, BLG, Blended Learning Group.

# Usability and User Experience

Usability is related to the intrinsic characteristics of the system with regard to the user's abilities, perceptions and attitudes. When adults start using the platform, it is clear to them how useful it is.

One of the reasons for having this perception is that the platform was presented in a relaxed and friendly atmosphere without any pressure and clarifying all their doubts when interacting with it. Likewise, some participants mentioned that the blended workshop is a very good idea as long as it is used as a support system, because if it were adopted in a totally asynchronous manner, they would have difficulties using it due to lack of assistance. Some of the interaction difficulties encountered by some participants were:


the window where they had opened the blended workshop and performed the steps that were previously shown. This fact showed that for some older adults browsing between windows is something complicated.

• Access to the platform. To access the blended workshop, the user needs to enter a username and password. Adults with no or little knowledge of ICT, sometimes entered some mistaken data and after several attempts they had to change their password. In most cases, it was not because they forgot their access codes, but because they did not type it correctly.

According to the above, it can be observed that the interaction difficulties were due to the inexperience of some older adults in relation to the use of the computer and the Internet. However, the experience of these users was improved, taking into account all their observations and comments, in order to adapt a more usable platform. This entails a challenge that is possible to fulfill, as long as continuous improvements are made based on the experiences of the users.

# Strengths of the Workshop Instruction

One of the strengths of the blended workshop is that it was developed under the guidelines of the ASSURE instructional model, which allowed to know the characteristics of the population, taking into account their visual deficiencies, cognitive and emotional abilities (insecurities). As a result, a space was obtained in which there is an adequate structure and design, attractive text colors and images, quality teaching materials and clear instructions.

The blended workshop also facilitates the learning and acquisition of knowledge by considering the constant and continuous review of each of the topics, in addition to having a learning method where the sequence of the class is organized by initiation, development and closure activities.

# DISCUSSION

The present research combines digital literacy assessment, comparison between face-to-face and blended digital literacy workshops and the validation of the LMS. Previous studies have shown how older adults perceive and use ICT in their daily lives. For example, Schreurs et al. (2017) conducted a study of how older adults perceive their own digital skills, what barriers to digital literacy they face, and what social and institutional support systems they have in place to achieve greater digital literacy. The results of this study indicated that older adults do not have digital skills and that there have limited support systems, which makes it difficult for them to gain experience and comfort with ICT.

Delello and McWhorter (2017) aimed to explore whether ICTs, specifically iPads, improve the lives of older adults. Derived from the results, the authors suggest that the use of the iPad reduced the social isolation of older adults, leading to closer family ties and a greater overall connection to society. In addition, there was also a significant increase in digital skills with respect to the use of the iPad.

Vroman et al. (2015) analyzed the patterns of ICT use (experience, socio-personal characteristics and use) of older adults living in the New England region. The results showed that most participants use ICT to maintain family connections and to access health information and routine activities. Based on the above, the authors propose a community-centered model that takes into account socio-personal characteristics for future ICT training programs.

Nevertheless, at present there is insufficient documentation on face-to face or blended courses designed for digital competence training of older adults. Moreno Ramírez et al. (2009) and Abad (2014) mention that the courses that currently exist have limitations that affect the learning and understanding of older adults in relation to ICT. For example, face-to-face workshops have numerous groups; this causes distractions between the older adults (Jaggars, 2014). Also, working methods in faceto-face courses are not focused on the particular needs of students (Jaggars, 2014; Margulieux et al., 2016), and do not have availability of learning materials for consulting outside of class (Kemp and Grieve, 2014). This could affect learning in older adults because their cognitive skills decline with age (Davidson and Guthrie, 2017; Meltzer et al., 2017), and they need more stimulation for understanding complex items and concepts.

By contrast, the blended workshops stimulate the use of new ways of learning, build knowledge and cover several limitations presented in face-to-face workshops (Oncu and Cakir, 2011; Chiu and Churchill, 2016; Conole, 2016). The idea of developing an online support tool for digital literacy workshops in this study emerged just to provide a more customized instruction to older adults. Contrary to young students, older adults have clear ideas about what to learn and how. Moreover, blended workshops allow students to have a more active role, as it grants them continued accessibility to study materials. This benefits older adults because they can carry out constant review of the topics and concepts seen in class, avoiding prejudice on those who are not able to attend class periodically for personal reasons or disease. This last point resulted in an opportunity for older adults who participated in blended workshops because they could access to LMS and review the class materials they missed when they were sick or in a medical appointment.

In our study, participants in the blended learning workshops recognized the importance of continued practice and the additional experience provided through the multimedia learning materials added to the LMS. It is also interesting to note that after the end of the digital literacy workshops (both cases), the participating adults not only came forward to ask additional questions, but many also asked "when the next workshop would open."

As part of the analysis in this study, it could be verified that the level of digital competencies of adults raised in both groups, however, the scores in post-evaluations were higher in the BLG group vs. FFG group. This was because adults of BLG group had access to evaluations and information available within the LMS. These evaluations allowed older adults to review the concepts at different times (up to 10 attempts), with the aim of reinforcing the knowledge of certain terms and tasks that were seen in each workshop module. This fact reflected that in the post-evaluations the participants of the blended learning workshop identified more computer terms and carried out more tasks on the Internet than the adults of the classroom workshop. Furthermore, the older adults expressed interest and enthusiasm most of the time because they had a support tool for their learning.

In this case, it can be concluded that the adaptation of blended workshops based on appropriate methodologies, instructional models and teaching-learning models may provide inspiration to more investigations about training the older adult population in any area of knowledge and to enrich the understanding of the background to the topic.

# CONCLUSIONS

For this study it was important to understand the reasons why older adults believe that digital technologies are difficult to use and that some perceive that they are not capable of learning to use them. This understanding is necessary not only to find better ways to introduce digital technologies to currently excluded potential users, but also to improve the design of digital products so that they are easy to use and easy to learn, which can facilitate adoption by all types of users.

Within this study it was possible to implement an adequate blended workshop so that the older adult population could develop their digital skills more easily. Likewise, it is important to take into consideration the experiences and perceptions of older adults in order to adapt and to continue to design both the system and the materials included, so as to present a more usable system.

The proposal of innovative practices, such as the one presented in this study, should make it possible to open up more areas of study to provide older adults with effective intervention strategies to include them digitally, creating spaces that are even adapted to the learning styles of each older adult. One fact to consider is that digital literacy has evolved rapidly from an option into a need, due to the constant change in technology. These constant advances make it insufficient to provide digital literacy to older adults using a single technology. It is therefore important to propose strategies where adults can develop flexible skills and technological self-efficacy to maintain their digital literacy despite the changes and advances.

# ETHICS STATEMENT

This study was carried out in accordance with the recommendations of The Ethics committee of the Instituto de Ciencias de la Salud, Universidad Autónoma del Estado de Hidalgo and it is registered in the Research Directorate with the key UAEH-DI-ICSA-GE-CF-006. All subjects gave written informed consent. The protocol was approved by the Research Directorate of the Universidad Autónoma del Estado de Hidalgo.

# AUTHOR CONTRIBUTIONS

CM-A and JR-S designed the concept and development of the virtual workshop. AR-L, MA-L, MB-D and JL-N carried out data interpretation and critical review of the study. RC-R, BJ-R and RA-J did bibliographic search, data collection, and capture. CM-A drafted the manuscript and received the support of JR-S and AR-L. All authors approved the final version of the manuscript for submission.

# ACKNOWLEDGMENTS

We thank the older adults for their participation in this study. Also, the Instituto de Ciencias de la Salud (ICSa) for access

# REFERENCES


to computer rooms. Also, we thank Monzerrat Cepeda for the authorization to use her statistical data of her Bachelor's thesis Impact of digital literacy in adults over 60 years of the Centro Gerontológico Integral de Punta Azul de Pachuca de Soto, Hidalgo. Finally, we thank Reyna Lopez-Blé for the revision of the English language version. This publication was financed with resources PRODEP 2018.

# SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fict.2018. 00021/full#supplementary-material


Appl. Neuropsychol. 24, 364–375. doi: 10.1080/23279095.2016.11 85428


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Martínez-Alcalá, Rosales-Lagarde, Alonso-Lavernia, Ramírez-Salvador, Jiménez-Rodríguez, Cepeda-Rebollar, López-Noguerola, Bautista-Díaz and Agis-Juárez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Reading on Paper and Screen among Senior Adults: Cognitive Map and Technophobia

#### Jinghui Hou<sup>1</sup> \*, Yijie Wu<sup>1</sup> and Erin Harrell<sup>2</sup>

<sup>1</sup> School of Communication, Florida State University, Tallahassee, FL, United States, <sup>2</sup> Department of Psychology, Florida State University, Tallahassee, FL, United States

While the senior population has been increasingly engaged with reading on mobile technologies, research that specifically documents the impact of technologies on reading for this age group has still been lacking. The present study investigated how different reading media (screen versus paper) might result in different reading outcomes among older adults due to both cognitive and psychological factors. Using a laboratory experiment with 81participants aged 57 to 85, our results supported past research and showed the influence of cognitive map formation on readers' feelings of fatigue. We contributed empirical evidence to the contention that reading on a screen could match that of reading from paper if the presentation of the text on screen resemble that of the print. Our findings also suggested that individual levels of technophobia was an important barrier to older adults' effective use of mobile technologies for reading. In the post hoc analyses, we further showed that technophobia was correlated with technology experience, certain personality traits, and age. The present study highlights the importance of providing tailored support that helps older adults overcome psychological obstacles in using technologies.

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

José Baptista Coelho, Universidade de Lisboa, Portugal Dubravko Culibrk, University of Novi Sad, Serbia

> \*Correspondence: Jinghui Hou jhou@fsu.edu

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 13 October 2017 Accepted: 07 December 2017 Published: 19 December 2017

#### Citation:

Hou J, Wu Y and Harrell E (2017) Reading on Paper and Screen among Senior Adults: Cognitive Map and Technophobia. Front. Psychol. 8:2225. doi: 10.3389/fpsyg.2017.02225 Keywords: older adults, reading, mobile technology, cognitive map, technophobia

# INTRODUCTION

The senior population has been increasingly integrating technology into their everyday lives. One area of technology that older adults particularly embrace is electronic books. Known as the most avid book consumers, older adults also make up the highest percentage of tablet and e-reader owners according to a national poll (Harrison Interactive, 2012). Reading has proven to bring great benefits to older adults, including providing mental stimulation to the brain, slowing memory decline, reducing stress and enhancing sleep, and possibly delaying the onset of dementia and Alzheimer's disease (Kawashima et al., 2005; Uchida and Kawashima, 2008; Nouchi et al., 2012; Kulason et al., 2016). Digital mobile devices may be particularly useful for senior readers because their advantageous technology features, such as backlit displays, adjustable text sizes, portable and lightweight frame, and touchscreens with large buttons, could accommodate for many common age-related limitations (Jochems et al., 2012; Findilater et al., 2013).

However, despite the continuing growth of seniors using mobile reading devices and their great values to the elderly population, older adults typically show a strong preference for traditional printed books (Kretzschmar et al., 2013). Much research on the effectiveness and experience of different reading media has focused on the younger population, including students in primary and

secondary school (Horton and Lovitt, 1994; MacCann et al., 2002; Sahin, 2011; Mangen et al., 2013; Porion et al., 2016; Wollscheid et al., 2016), and college (Schwartz et al., 1998; Ackerman and Lauterman, 2012; Stoop et al., 2013; Vincent, 2016) as new technologies continue to infiltrate in classrooms (Noyes and Garland, 2003; Mangen et al., 2013; Margolin et al., 2013; Millar and Schrier, 2015; Mizrachi, 2015; Aharony and Bar-Ilan, 2016). While the senior population has received some scholarly attention (Velikova et al., 1999; Kretzschmar et al., 2013; Fanning and McAuley, 2014), empirical research suggest that the elderly tend to lack confidence or have a certain degree of anxiety toward digital reading media as well as emerging technologies in general (Ellis and Allaire, 1999; Umemuro, 2004; Barnard et al., 2013). Such psychological factors could prevent seniors from reaping the myriad of benefits of using emerging technologies. To help older adults successfully use reading technologies and realize their potential benefits, it is important for researchers and technology designers to understand the specific users, their unique cognitive functions, psychological needs and preferences, and limitations (Di Giacomo et al., 2013). To this end, the current study aims to investigate how different reading media (screen versus paper) might result in different reading outcomes among older adults due to both cognitive and psychological factors.

As people age, they experience a number of cognitive changes that impair their reading performance. Recent studies (Li et al., 2013; Hou et al., 2017) suggested that a key factor that influences people's reading performance lies in the extent to which the presentation of the reading text facilitates or impedes readers' ability to form a cognitive map of the text. Similar to how people remember a physical landscape, such as the road taken to a place, readers form a mental map of the physical location of a text within a page. Both scholarly evidence and anecdotal experience testify that when people try to locate a particular piece of information they have read, they are often able to recall where in the text it appeared, such as a formula on the top of a righthand page (Jabr, 2013). Thus, during reading, human brains not only gather information about a text but also process information about its context, and associates the two (Li et al., 2013; Hou et al., 2017). As such, individuals read by constructing a cognitive map of the text based on the spatial placement or presentation of the text on a page (Payne and Reader, 2006; Li et al., 2013; Hou et al., 2017).

Generally speaking, printed books have fixed layouts; such text presentation aids the formation of a physical map in readers' minds of where certain pieces of information are. In contrast, screens typically make it difficult for readers to know the location of information in the document, thereby impeding readers in constructing an effective cognitive map. For example, scrolling text on a screen prevents readers from forming a coherent mental map, as there is no point for a reader to remember that a piece of text was on the top of page, because soon it might not be as the reader scrolls down the page (Piolat et al., 1997; Lee, 2005). The weak efficiency for forming cognitive maps impairs comprehension, and increases readers' feeling of fatigue (Hou et al., 2017). Studies found that compared to young people, older individuals were less efficient in forming and using a cognitive map (Iaria et al., 2009). Difficulty with cognitive map processing might explain why older individuals particularly report difficulties with reading on screens (Iaria et al., 2009). To our knowledge, little research has investigated the influence of cognitive map formation on reading outcomes among older adults. We anticipate that a text presentation supporting the formation of a cognitive map of the text will bolster reading outcomes, while a text presentation with weak efficacy for forming a cognitive map will impair such outcomes for older adults.

Beyond the influence of cognitive modification in aging, psychological preferences of older adults with respect to technology could also play an important role. By tracking eye movements and brain activity of participants aged 60 and above as they read, a study suggested that older participants read faster and with less effort when reading from a tablet compared to a printed book. However, these participants still self-reported to gain more pleasure reading from a printed book and claimed that it was easier to read than reading on the tablet (Kretzschmar et al., 2013). These findings align with previous survey studies showing the strong subjective preference for reading on paper among older readers (Two Sides, 2015). The researchers suggested that the perception that digital devices reduce the pleasure of reading may be attributable to the positive attitude or cultural attachment associated with traditional printed books rather than a cognitive phenomenon (Kretzschmar et al., 2013). Thus, reading outcomes could potentially be affected by individuals' psychological attitude toward the reading medium.

While psychological attitude toward technology varies across age groups, the psychological disposition of technophobia (Rosen and Weil, 1995a) is a particularly common and salient phenomenon among the senior population. Technophobia can be described as anxiety and overall negative attitudinal and affective response toward technology, its operation, or societal impact (Rosen and Weil, 1990). Technophobia could lead to physical symptoms, discomfort, and inefficiency (Brosnan, 2002; Gonzales and Wu, 2016). For example, an experimental study on technophobia revealed that individuals who experienced greater computer anxiety made more errors on a computer database searching task (Brosnan, 1998). Furthermore, longitudinal field studies have also demonstrated that technology anxiety can influence learning in technology-mediated environments (Fuller et al., 2006) and task achievement using technologies (Marcoulides, 1988). Research has documented evidence that individuals who have less past experience in using technologies are more likely to experience technophobia (Rosen and Weil, 1995b; Anderson, 1996). The elderly in general have significantly less technology experience than their younger counterparts (Dyck and Smither, 1994) whom many refer to as digital natives. With relatively less technological background, older people tend to lack confidence in their capabilities to understand and use technology, and as a result often feel insecure, discouraged, and stressed when using technology (Laguna and Babcock, 1997). As such, albeit having good access to technology, older adults showed a higher level of technophobia (Hogan, 2009) and exhibited lower performance in using technologies (Laguna and Babcock, 1997) compared to younger people.

# MATERIALS AND METHODS

fpsyg-08-02225 December 17, 2017 Time: 16:35 # 3

# Hypotheses

Taken together, existing literature implies that older adults may experience difficulties and frustration with technologies due to both cognitive and psychological influences. In this study, we aim to examine how reading on a screen versus paper might result in different reading outcomes among older adults due to both cognitive map and psychological attributes. As noted above, previous research (e.g., Li et al., 2013; Hou et al., 2017) suggested that the extent to which a text presentation facilitates or hinders the construction of a cognitive map of the text influences reading outcomes. Meanwhile, individuals' psychological disposition of technophobia could also impact how they can effectively use technologies for reading. As such, we postulate that:


While early research suggested that reading in print outperforms reading onscreen (e.g., Gould and Grischkowsky, 1984; Kurniawan and Zaphiris, 2001), recent studies found little evidence that people reading on paper show significantly different reading outcomes compared to people reading via digital technologies (e.g., Hou et al., 2017; Kretzschmar et al., 2013; Margolin et al., 2013). Due to the inconsistent findings, we tested the following non-directional hypotheses in the present study.

**H3.** Reading medium (paper vs. screen) will impact reading outcomes among older adults on reading comprehension (a), time (b), fatigue (c), general discomfort (d), and enjoyment (e).

# Experiment Design and Materials

This experiment employed a 2 (Medium: Paper vs. Screen) ∗ 2 (Cognitive Map Formation: Easy vs. Difficult) betweensubject factorial design. The basic activities in the experiment involved reading texts and completing questionnaires. All experiment materials were in English. The reading material for this experiment was two texts of different types – a fictional story (i.e., a narrative text) and a scientific article (i.e., an expository text). Both were chosen from consumer publications intended for the general reading public, and were adapted by the researchers to be of similar length. The narrative text was 15 pages in length and had a total of 3,469 words. The expository text was also 15-pages in length and had a total of 3,150 words. None of the participants had read the texts prior to the experiment.

# Experiment Conditions Reading Medium

The reading texts were presented on either a touchscreen tablet (Apple's iPad) or on paper with identical font, font size, and line spacing. The tablet was 241 mm <sup>∗</sup> 186 mm <sup>∗</sup> 8.8 mm, with the display area being 193 mm <sup>∗</sup> 147 mm. The paper condition was the print version of the tablet condition. Thus, the two conditions had identical display size and page layout; the only difference was the reading medium.

## Cognitive Map Formation

The text presentation was manipulated so that cognitive map formation of the text was either easy or difficult for participants. In the easy conditions, the reading texts were presented on a tablet screen or on paper in the format of a traditional book. In the screen version, participants swiped-through to turn pages on the tablet screen and saw one page at a time; this navigational method mimicked page turning similar to reading a physical book. In the paper version, participants were given a printed copy of the identical text used in the screen condition.

In the difficult conditions, scrolling was employed. Scrolling method has been shown to impede readers in forming cognitive maps because scrolling provides less contextual information during reading (Piolat et al., 1997; Lee, 2005). With paging many readers would get the contextual information where certain passage was on a page, such as whether it was at the left or right, top or bottom. With scrolling, however, pages are constantly changing as readers scroll through them. What is at the top at one second can be on the bottom the next; this weakens the association between text and pages, thus scrolling cannot effectively indicate a reader's location in a text (Wästlund et al., 2008; Li et al., 2013). Therefore, we used scrolling as our manipulation for the difficult cognitive-mapformation conditions. In the screen version, participants read by scrolling texts upward to reveal succeeding pages. In the paper version, we employed a specifically designed paper scroll. Participants rolled the paper scroll upward as they read in a similar way as they scrolled on the tablet screen. The display size and page layout were held constant between the two reading media.

Additionally, across conditions, no additional reading activities, such as text highlighting, annotating, and bookmarking, were allowed. All participants were required to read in the portrait orientation.

# Participants and Procedure

fpsyg-08-02225 December 17, 2017 Time: 16:35 # 4

The study obtained human subjects approval from the University Institutional Review Board. We obtained written informed consent from all participants, and they were compensated \$10 for participating in the study.

### Sample

A total of 81 older adults (n = 53 female, n = 28 male) aged 50 and above were recruited from a southeastern city in the United States and its surrounding areas. The participants ranged from 57 to 85 years of age with a mean age of 69.43 (SD = 6.01). They were randomly assigned to one of the four conditions with gender approximately balanced across the conditions.

# Recruitment

Various methods were used for participant recruitment that included: social media, attendance at community meetings, interactions with agencies serving older adults (e.g., local senior services), and participant registries. A telephone screening was first conducted among potential participants to (1) describe the requirements of the study, and (2) to assess their eligibility status. Eligibility status was evaluated using the 10-item Short Portable Mental Status Questionnaire (SPMSQ; Pfeiffer, 1975). Participants making more than two errors on the SPMSQ may experience cognitive deficit that would prevent them from successfully participating in this study and bias our results, and thus were excluded. During our screening process, all participants passed the SPMSQ test.

#### Protocol

After being greeted to a quiet room on a university campus, participants were given an introduction to the experiment and finished the consent procedure. A pre-test paper-and-pencil questionnaire was administered first. Depending on their randomly assigned experimental condition, participants were asked to either read the two texts on a tablet or on paper. Prior to reading, all participants were given a demonstration by the experimenter on how to use their assigned reading medium. Participants were also instructed to read the texts at their normal pace, with the understanding that they would answer some comprehension questions afterwards. The experimenter recorded how long each participant took to finish reading, but participants were not made aware that they were being timed. After reading, participants completed the comprehension questions. The above steps were repeated with the second text. The order of the two reading texts was randomized. After reading the two texts, participants completed a post-test paper-and-pencil questionnaire. Finally, participants were debriefed and compensated. Most participants completed the experiment in approximately 50 min to 1 h.

# Measures

#### Comprehension

Participants were asked 20 comprehension questions, 10 questions for each text. One question was a sequence of events question; the rest were multiple-choice. On average, the number of correct answers ranged from 7 to 20, M = 15.730, SD = 2.579.

### Time

The length of time it took participants to read was measured. This variable reflected participants' reading speed. Participants completed the reading within 877–3087 seconds (i.e., 14.61– 51.45 min), M = 1985.37 (i.e., 33.09 min), SD = 506.918.

#### Fatigue

Participants reported their feeling of fatigue while reading the texts on a 4-point scale (1 = None, 4 = Severe) in the post-test questionnaire, M = 1.120, SD = 0.399.

#### General Discomfort

Participants also reported their feeling of general discomfort (1 = None, 4 = Severe) in the post-test questionnaire, M = 1.280, SD = 0.617.

### Enjoyment

Participants self-reported their reading experience on three items – Enjoyable, Fun, and Interesting, ranging from Not at all (1) to Very much (5). The items were averaged to create the scale score; higher scores on this scale denoted higher enjoyment of the reading experience, M = 3.551, SD = 0.881, Cronbach's alpha = 0.82.

#### Technophobia

Technophobia was measured using five items adapted from the internet phobia scale from Brosnan and Rosen (2001). The five 7-point scale (1 = strongly disagree and 7 = strongly agree) items were: (1) "I always feel anxious when using mobile devices," (2) "I go out of my way to avoid using mobile devices and technologies," (3) "It is easy for me to use mobile technologies (reversed coded)," (4) "My anxiety about mobile technologies bothers me", and (5) "I am more anxious about mobile devices than I should be." The five items were averaged to create the technophobia scale, M = 2.094, SD = 1.099, Cronbach's alpha = 0.84.

#### Experience in Mobile Technologies

Participants reported their experience with using mobile technologies on a 7-point scale. They were instructed to pick one of following choices regarding their mobile technology use: Only once or twice in total (1), Less than once a month (2), A couple times a month (3), Once a week (4), Several times a week (5), and Every day (6), or Never use (0). 54.3% (n = 44) of our sample reported using mobile technologies every day, while 21% of our sample reporting never having used mobile devices, M = 4.124, SD = 2.482.

#### Personality Characteristics (TIPI)

A brief measure of the Big-Five personality dimensions - the ten-item personality inventory (TIPI; Gosling et al., 2003) scale was used. The TIPI uses a 7-point Likert scale, and creates a score for each of the Big Five Personality traits: extraversion, openness to experiences, emotional stability, conscientiousness, and agreeableness.

# RESULTS

# Statistical Analysis

fpsyg-08-02225 December 17, 2017 Time: 16:35 # 5

Data analyses for this study were based on a series of fullfactorial two-way ANCOVA using technophobia as a covariate. Technophobia was considered as an appropriate covariate that reflects the dispositional characteristic of individuals. As such, the two-way ANCOVA model was the fitted statistical test for the present study aiming to investigate the influences of both media factors (i.e., cognitive map formation and reading medium) and individual factor (i.e., technophobia) on older adults' reading experience. Participants' age, education level, and experience in mobile technologies were included in the analyses as control variables. In the post hoc test section, Pearson's correlation tests were employed to further explore relationships between technophobia and other individual characteristics.

# Primary Results

Descriptive statistics for the study measures are presented in **Table 1**. The results of the ANCOVA analysis are shown in **Table 2**.

H1 expected that when exposed to texts that have weak efficacy for forming a cognitive map, individuals would show poorer reading outcomes, including displaying poorer comprehension performance (a), taking more time to read (b), experiencing greater feeling of fatigue (c) and general discomfort (d), and enjoying the reading less (e).

The ANCOVA analyses testing H1a and H1b failed to detect significant effects of cognitive map formation on comprehension and reading speed, respectively. However, the analyses indicated a significant effect of education on reading comprehension, F(1,73) = 6.07, p = 0.016, η <sup>2</sup> = 0.08. In testing H1b, a significant effect of education on reading speed was also found, F(1,73) = 11.384, p = 0.001, η <sup>2</sup> = 0.135. The results suggest that seniors with higher levels of education scored better on the comprehension questions and spent less time reading the texts. But H1a and H1b were not supported.

Consistent with H1c, the results of the ANCOVA revealed a significant main effect of cognitive map formation on fatigue, F(1,73) = 6.975, p = 0.010, η <sup>2</sup> = 0.087. Participants in the scrolling conditions reported stronger feelings of fatigue than those in the paging conditions. H1c was supported. There was no significant main effect for cognitive map formation on feelings of general discomfort or enjoyment of reading. H1d and H1e were not supported.

It was also hypothesized (H2a–e) that technophobia, measured as an individual characteristic in the pre-test, would significantly influence the reading outcomes. The ANCOVA analyses showed that technophobia had a significant effect as a covariate for reading speed, F(1,73) = 6.256, p = 0.015, η <sup>2</sup> = 0.079, and for general discomfort, F(1,73) = 6.394, p = 0.014, η <sup>2</sup> = 0.081. Participants who had a higher level of technophobia spent more time reading the texts and showed stronger feelings of general discomfort while reading. Technophobia also had a marginally significant effect on fatigue, F(1, 73) = 3.440, p = 0.068, η <sup>2</sup> = 0.045. That is, participants with a higher level of technophobia also reported increased feelings of fatigue compared to those with a lower level of technophobia. Thus, both H2b and H2d were supported. H2c was supported with marginal significance. H2a and H2e were not supported.

H3 postulated that reading medium (paper vs. screen) would impact reading outcomes among older adults on reading comprehension (a), time (b), fatigue (c), general discomfort (d), and enjoyment (e). The ANCOVA analyses, however, did not show significant differences between the conditions when older adults read on digital screens as opposed to paper in any of the five reading outcomes. Thus, H3a–H3e were not supported.

# Post hoc Analyses

To further understand the psychological disposition of technophobia and, in particular, to explore who (in terms of demographics and personality traits) is more likely to have a high level of technophobia, Pearson's correlational analyses were conducted to examine associations between technophobia and individual variables.

Our data showed that technophobia was negatively correlated with openness to experiences, r = −0.355, p < 0.001, agreeableness, r = −0.266, p = 0.017, and emotional stability, r = −0.233, p = 0.037. Technophobia was also negatively correlated with experience in mobile technologies, r = −0.521, p < 0.001. It also showed a marginally significant relationship with age, r = 2.12, p = 0.058 (two-tailed). Technophobia was not correlated with education level, and it did not differ between male (M = 2.30, SD = 1.25) and female (M = 1.98, SD = 1.01), t(79) = −1.275, p = 0.206. **Table 3** presents the results of the correlation tests.

# DISCUSSION

The purpose of this study was to investigate the effects of cognitive map formation and reading medium on reading performance and experience among older people in terms of comprehension, reading speed, fatigue, general discomfort, and enjoyment. It also investigated the influence of technophobia on these reading outcomes. Our results showed that the efficacy of cognitive map construction influences readers' feeling of fatigue. This finding is in line with existing literature (Hou et al., 2017) suggesting that reading a text that hinders cognitive map formation is more tiring for readers because they need to constantly adjust the processing of the text in relation to its spatial placement on the page. Our results replicated existing findings and extended it to the elderly readers. However, we failed to observe compromised comprehension performance in the conditions that was difficult for readers to form an effective cognitive map, a finding that has been documented in previous studies (e.g., Li et al., 2013; Margolin et al., 2013; Hou et al., 2017). It is possible that general participants in our sample achieved high comprehension accuracy. In the experiment, we allowed sufficient time for participants to read, and participants were made aware that they would be tested with comprehension questions after reading. This might have motivated them to read the texts carefully to prepare for the test. Future studies could consider allocating a time limit to the reading task, or measure comprehension while controlling the influence of motivation. Another possible explanation is that since comprehension was strongly influenced by readers' education level, as was shown in our analysis, and that our participants had overall high education level, most participants were able to do well on

#### TABLE 1 | Descriptive statistics of study variable by condition.

fpsyg-08-02225 December 17, 2017 Time: 16:35 # 6


TABLE 2 | ANCOVA analysis on cognitive map formation, reading medium, and technophobia.


TABLE 3 | ANCOVA analysis on cognitive map formation, reading medium, and technophobia.


Two-tailed ∗∗p < 0.001, <sup>∗</sup>p < 0.05.

the comprehension questions. In sum, we suggest it requires further investigation in future research to see how the cognitive map mechanism influences comprehension among older adults. Furthermore, in line with previous studies (Kretzschmar et al., 2013; Hou et al., 2017), our data did not show significant differences between reading on screen versus on paper in terms of reading performance and experience. Thus, we add another piece of empirical evidence to the recent postulation that reading on digital media could match that of reading from paper if the representation of the document digital reading devices resembles that of the print book (Hou et al., 2017).

Our findings highlight the importance of a psychological disposition factor – technophobia – that could threaten older adults' ability to effectively use digital technologies for reading. In the current study, older adults with higher levels of technophobia spent more time on reading, but they performed as well on reading comprehension as those with lower levels of technophobia. This finding was consistent with Darke's (1988) suggestion that anxious individuals allocate more cognitive resources to off-task efforts such as worrying about their performance, and as a result are slower to complete the actual tasks, but do not necessarily make more errors. Our finding resonated with previous research showing that the people who experience more computer-associated anxiety may exhibit lower self-efficacy to perform reading well on digital media (Brosnan, 1998), thereby showing slower reading speed.

It is worth noting that previous studies have mainly focused on the negative influence of technophobia on performance of a novel computer-based task, such as database searching (Brosnan, 1998), data entry (Mahar et al., 1997), and word-processer usage (Brosnan, 1999), which require some level of computer literacy and software operating skills. Thus, one can argue that the negative impact of technophobia on performance might stem from individuals' reluctance to learn a novel skill. Few studies, however, have looked at daily actives that are mediated by computers, yet require minimum learning of novel skills, such as reading on a touch-screen mobile device. Reading on a touch-screen mobile device is not a computer-based task in the traditional sense because the intuitive user interface of the reading devices nowadays requires little effort from the users to perform any computer operations. Therefore, the negative impact of technophobia on reading performance is likely to be a result due to users' negative attitudes directly towards computer itself rather than negative attitudes towards computer-based operations. Meanwhile, reading on a touch-screen is becoming increasingly incorporated into our everyday experience, which may have important consequences for people, especially older adults, with technophobia. In testing the effects of older adults' technophobia on reading on a touch-screen, we contribute to previous technophobia literature by analyzing a new yet common context of technology use, and by further clarifying if technophobia is a result of the negative attitudes towards the computer itself or the idea of having to perform computing operations.

Interestingly, our results also showed that older adult users with higher level of technophobia experienced more subjective feeling of discomfort and fatigue. This finding suggested that despite that older adult users performed minimum computer operations during the reading, just the thought of reading on a computer device would activate negative responses within users who possessed pre-existing negative posture toward computers. On a different research topic, an experimental study by Gonzales and Wu (2016) found a similar pattern among college students with higher level of technophobia. When excluded by someone reading on a cellphone in their presence, college students who harbor more negative attitudes toward computer technology reported worsened mood, a reduced sense of self-esteem and control over their environment. Our study results extend these findings, and contribute to the broad research dialog regarding how technophobia may deter the process of human-computer interaction.

Moreover, our post hoc analyses explored the associations between technophobia and individual characteristics. Overall our results were consistent with past research examining technophobia. For instance, existing research indicated that previous computer experience and technology ownership is a main predictor of technophobia (e.g., Maurer, 1994; Chua et al., 1999). Our data also showed that as an individual's experience in technology use increases, their level of technophobia decreases. Further, studying older adults between the ages of 60 to 97, Ellis and Allaire (1999) found that age was positively correlated with computer anxiety. Similarly, our data indicated a marginally significant relationship between age and technophobia. Furthermore, we provided some indications that technophobia could be associated with an individual's personality traits (Anthony et al., 2000), particularly one's openness to experience. As a person is more open to innovative experiences, s/he is less likely to experience technophobia. Future studies could employ other research methods, such as large-scale survey or in-depth interview, to further understand what may cause technophobia and how older adults can overcome it.

Unlike the youth population, growing up as digital natives, who are more likely to face problems such as internet addiction, the senior population might have an opposite concern associated with anti-digital-reading-attitudes and anxiety in using technologies. Thus, research-driven evidence-based technology design and interventional programs that understand this population group and accommodates for their unique cognitive functions and psychological needs could lead to development of innovative technologies that bring important benefits to the senior population, and in turn, promoting successful aging as a whole. This issue becomes increasingly important as the global aging population continues to grow and individuals' life expectancy gets longer. In sum, while we embrace a variety of positive impacts of using technology on seniors' lives (Topo, 2009), the degree to which the technology innovation is compatible with one's values, experiences, and needs (Rogers, 2003) should receive more scholarly attention.

# Limitations and Future Research

This study has limitations that warrant discussion and can inform future studies. First, self-selection bias is considered a limitation

to our study. It was likely that older people who expressed interest in participating in our study were more techno-savvy than the general aging population, because many of our participants were able to respond to the study advertisements sent via email and on social media. Second, due to the voluntary nature of subject recruitment, the composition of our sample was primarily Caucasian with a relatively high education level on average. Previous research has revealed that individuals' demographic characteristics, such as ethnicity and socioeconomic status are associated with technophobia (e.g., Rosen and Weil, 1995b). Hence, this study sample could have potentially excluded a technologically underserved population who experiences higher level of technophobia. Future studies should pay attention to the sample diversity in order to make more generalizable conclusions regarding digital reading. Lastly, our measure of fatigue and general discomfort relies on a single self-reported item. Future research could employ more comprehensive self-report measures or more objective measures targeting the older adult population to triangulate our findings.

# CONCLUSION

Our study provides implications for scholars, technology designers, and training program practitioners to promote digital reading among aging users. While existing literature and interventional programs mainly focus on improving access to digital devices and increasing digital literacy among seniors, our study further highlights the importance of providing tailored

# REFERENCES


support that helps overcome psychological obstacles in using technologies for the aging population.

# ETHICS STATEMENT

This study was carried out in accordance with the recommendations and approval of "Thomas L. Jacobson" from the Human Subjects Committee of the involved Institute with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki.

# AUTHOR CONTRIBUTIONS

This study was led by JH: she conceptualized and designed the study, ran experiments, collected data, performed the data analysis, and wrote the paper. YW filed the IRB application, recruited participants, ran experiments, collected data, and contributed to the manuscript write-up. EH recruited participants, and collaborated to data collection. All authors edited the paper and approved the final manuscript. JH takes primary responsibility for the research reported here.

# FUNDING

This work was partially supported by the United States National Science Foundation Award #1640587 (PI: Arghandeh).




**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Hou, Wu and Harrell. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Development and Evaluation of a Cognitive Training Game for Older People: A Design-based Approach

Ming-Hsin Lu<sup>1</sup> , Weijane Lin<sup>2</sup> \* and Hsiu-Ping Yueh1, 3 \*

*<sup>1</sup> Department of Bio-Industry Communication and Development, National Taiwan University, Taipei, Taiwan, <sup>2</sup> Department of Library and Information Science, National Taiwan University, Taipei, Taiwan, <sup>3</sup> Department of Psychology, National Taiwan University, Taipei, Taiwan*

In the research field of cognitive aging, games have gained attention as training interventions to remediate age-related deficits. Cognitive training games on computer, video and mobile platforms have shown ample and positive support. However, the generalized effects are not agreed upon unanimously, and the game tasks are usually simple and decontextualized due to the limitations of measurements. This study adopted a qualitative approach of design-based research (DBR) to systematically review and pragmatically examine the regime, presentation and feedback design of a cognitive training game for older adults. An overview of the literature of cognitive aging and training games was conducted to form the theoretical conjectures of the design, and an iterative cycle and process were employed to develop a mobile game for older adults who are homebound or receiving care in a nursing home. Stakeholders, i.e., elderly users and institutional administrators, were invited to participate in the design process. Using two cycles of design and evaluation, a working prototype of an iPad-based app that accounted for the needs of elderly adults in terms of form, appearance and working function was developed and tested in the actual contexts of the participants' homes and an assisted living facility. The results showed that the cognitive training game developed in this study was accepted by the participants, and a high degree of satisfaction was noted. Moreover, the elements of the interface, including its size, layout and control flow, were tested and found to be suitable for use. This study contributes to the literature by providing design suggestions for such games, including the designs of the cognitive training structure, interface, interaction, instructions and feedback, based on empirical evidence collected in natural settings. This study further suggests that the effectiveness of cognitive training in mobile games be evaluated through field and physical testing on a larger scale in the future.

Keywords: design-based research, elderly adult, cognitive training, mobile game, app

# INTRODUCTION

The number of elderly people in the world is increasing rapidly. Taiwan will have become an aged society by 2018 and a super-aged society by 2025 (National Development Council, 2016). To cope with aging populations, governments and institutions around the world have proposed various guidelines, such as successful aging (Rowe and Kahn, 1997), active aging (Foster and Walker, 2015) and healthy aging (World Health Organization, 2015). The purpose of these guidelines is generally

#### Edited by:

*Pierpaolo Vittorini, University of L'Aquila, Italy*

#### Reviewed by:

*Dilana Hazer, University of Ulm, Germany Alejandro Catala, University of Twente, Netherlands*

\*Correspondence:

*Weijane Lin vjlin@ntu.edu.tw Hsiu-Ping Yueh yueh@ntu.edu.tw*

#### Specialty section:

*This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology*

Received: *16 May 2017* Accepted: *02 October 2017* Published: *17 October 2017*

#### Citation:

*Lu M-H, Lin W and Yueh H-P (2017) Development and Evaluation of a Cognitive Training Game for Older People: A Design-based Approach. Front. Psychol. 8:1837. doi: 10.3389/fpsyg.2017.01837*

**64**

to propose how elderly people's cognition, capacity and physical function can be maintained, to encourage their social participation and to emphasize the competence and knowledge that older people possess. The idea of cognitive aging has gained attention as domain-specific, age-related cognitive changes that—at a minimum—affect attention, executive functioning, memory, language and visuospatial function, as indicated by neurochemical theory, localized theory and process theory (Dempster, 1992; Salthouse, 1996; West, 1996; Volkow et al., 1998; Bäckman et al., 2006; Clay et al., 2009; Drag and Bieliauskas, 2010; Craik and Rose, 2012; Turgeon et al., 2016). Normal aging leads to changes in inhibition and selective attention. These changes make older people more susceptible to the negative effects of divided attention, and task-switching is thus harder to perform (Kray and Lindenberger, 2000; Mayr and Kliegl, 2000). One significant finding of related studies is that the aging-related decreases in working memory affect executive functioning. Executive functioning is a high-order cognitive structure that uses and organizes large amounts of information to support goal-directed behavior (Dobbs and Rule, 1989; Wang et al., 2011). The lower speed of time perceived by the elderly also affects their executive performance (Turgeon et al., 2016). The age-related deficiency of executive function is a strong predictor of functional impairment in the elderly living in communities or assisted-living facilities (Wang et al., 2011). In addition, the age-related memory decline has a particular effect on recollection ability during information finding and retrieval (Park et al., 2002). Recall is more difficult than recognition for elderly people, and their semantic memory appears to have a deficit (Naveh-Benjamin, 2000). However, the related cues of familiarity, auditory context, and external context can help the recollection process of older people (Park et al., 1990; James and Burke, 2000; Bastin and Van der Linden, 2003). Another change is the agerelated decline in language ability performance, which is affected by inhibition capability, working memory and the recollection process. This decline makes it more difficult for elderly people to understand sentences and recall text because of the great syntactic complexity these tasks involve (Gold and Arbuckle, 1995; DeDe et al., 2004). According to previous studies, visuospatial abilities such as visuospatial attention, memory and orientation are also affected by age. Integrative visuospatial tasks such as problem solving (e.g., route-finding) and mental transformation (e.g., map reading) are sensitive to age and associated with executive function. In addition, testing visuospatial access can more efficiently detect Alzheimer's patients at the early stage of cognitive decline than can testing verbal access (De Federicis et al., 2016).

Pursuing these questions, an increasing number of studies have demonstrated that games can have a positive impact on senior citizens and can provide them with cognitive training (Flynn et al., 2007; Ackerman et al., 2010; Calvillo-G'amez et al., 2010; Anguera et al., 2015). These empirical studies have proposed certain componentsthat make cognitive training games successful, such as training content comprised of customized and adaptive tasks (Lachman, 2006), well-designed interaction via friendly interfaces (Schmiedek et al., 2010), and the accessibility of the devices (Lindenberger et al., 2008; Schmiedek et al., 2010). While these components are important references for the formation of design guidelines, their effects on cognitive training are not agreed upon unanimously (Owen et al., 2010). The rapid development of the devices and users' adjustability to the affordance of the devices introduce great variety and difficulty to systematic and empirical investigations (Van Muijden et al., 2012; Anguera et al., 2015). In addition to the employment of advanced methods and techniques to collect neural and behavioral evidence, a few, yet critical, attempts have also been made at the situated effect of the training context (Rush et al., 2006; Trefry, 2010; Binder et al., 2015). The results have suggested a more comprehensive range of research for cognitive training game studies. **Figure 1** summarizes the relationships among aging theory, age-related cognitive changes and associated cognitive training games, and their effects revealed by the related studies.

An overview of the literature has shown that designing technology products with a friendly interface for elderly people requires that designers consider the declines in older users' perception, sensation, motion and cognition (Drag and Bieliauskas, 2010; Lu and Yueh, 2015). For instance, considering older adults' declines in perception and sensation, decisions to use visual presentations such as reflective, decorative or animated images should be made cautiously (Namba et al., 1995; Green and Bavelier, 2003). Other design principles regarding auditory feedback (Charness and Jastrzembki, 2009; Fisk et al., 2009), haptic feedback (Lee and Kuo, 2001; Harada et al., 2013), and memory assistance (Ferreira and Pithan, 2005; Fisk et al., 2009; Sauve et al., 2015) have also been reported in related studies. According to this review, the present study proposes five principles that should be considered during the development of the interface and interaction in a digital game for the cognitive training of elderly people: (1) Provide clear and multisensory game instructions and suitable interaction. (2) For the game context and tasks, adopt content and themes that are familiar to the user from daily life. (3) Design several different tasks corresponding to the training of various cognitive capabilities; these tasks should be easy to complete and provide practice modes. (4) Provide feedback on the training consequences. (5) Engage users and stakeholders to evaluate the design and its effects. This study proposes that for elderly adults, cognitive training games that are casual, contain familiar life contexts and have a user-friendly interface are necessary and satisfying.

In terms of game devices, a mobile tablet-based training app has several advantages that are especially beneficial for training older adults. The most important benefit is that older adults perceive tablets as personal devices that they can carry with them easily; as a result, they can flexibly integrate training into everyday life (Binder et al., 2015). For example, mobile devices are easier to carry and operate than portable devices (i.e., laptops) when the elderly have to move between day-care nursing homes and their own homes. Moreover, the screen size of tablets fits older adults' visual abilities better than smartphones, especially during tasks that are complex or involve a high cognitive load, such as reading (Werner et al., 2012; Yueh et al., 2012). Because of these advantages of tablets, older adults have rapidly adopted mobile devices and expect mobile technology

to flexibly support their leisure activities, social interactions and learning.

Although studies on digital games have provided valuable insight into cognitive training and also suggested design principles for the content and interactions, they have not frequently used multiple sources (such as stakeholder attitudes and qualitative and quantitative sources) for analysis, have not described the design processes of games according to users' evaluations and have seldom tested games in a realistic context. Elderly adults adopt a technological innovation when they see that it is useful in their lives (Hanson, 2010). Understanding what components of cognitive training games affect elderly users' experiences is necessary because the usability of mobile applications is of great importance.

Meanwhile, to address the need to clarify how older users' cognitive functions can be enhanced by game training in a genuine context with stakeholders, this study adopted the research method framework of design-based research (DBR) to investigate the complex phenomenon. DBR focuses on the development of hypotheses and a framework and the contribution to model formulation, rather than model estimation or validation. As a result, it can obtain a different outcome, such as new theory (Simon, 1969; Sloane and Gorard, 2003). DBR typically triangulates multiple sources and types of data to connect intended and unintended outcomes of processes of enactment, and it increases the validity of findings through its typical partnerships and iteration (The Design-Based Research Collective, 2003). DBR enables researchers to grasp problems through iterative development and places emphasis on the authenticity of the context; it also enables a clearer understanding of the goals and implications of research, resulting in maximum design optimization (Joseph, 2004; Lin et al., 2014).

In the design of an optimized and user-friendly cognitive training game for elderly people, the specific research questions that must be answered are as follows: (1) What types of cognitive training do elderly people accept in a mobile game? and (2) What components are crucial to improving or reducing the usability of a cognitive training game for elderly people? The current study involved collaboration among experts from the medical, interaction design and engineering fields, and it designed a cognitive training app for seniors in home or nursing home settings. The iterative cycles and process that were employed to develop the app are described. The needs, preferences, technological experience and cognitive deficits of older people were evaluated and considered in the design of the cognitive training app so that it would offer comprehensive support.

# METHODS

In its approach to the relatively new concept of producing cognitive training games for elderly people, the present study employed DBR (Barab and Squire, 2004). The study focus was the evolution of a cognitive training game as a design artifact through iterative processes to determine both the theoretical and practical implications of the mobile game's design. First, elderly people's experience with mobile technology and their needs and interactions with the cognitive training game were investigated. Second, their perceptions of, and the cognitive load entailed by, the mobile game were subjected to a formative assessment. This formative assessment was used to determine theoretical propositions concerning aging-related cognitive training and human-computer interactions for the design of novel cognitive training supported by mobile games. The opinions of expert stakeholders were gathered and incorporated through iterative evaluations aimed at optimizing the design.

# Research Design

The cognitive training game developed in this study was defined as a casual game. The game was investigated in complex real-life settings involving multiple variables, and various aspects of the design were refined using DBR. Two design cycles were adopted and are illustrated in **Figure 2**.

In the two iterations, successive refinements of the design, perception, analysis and redesign were made. The first iteration focused on evaluating whether the game environment with the cognitive training structure proposed in **Figure 1** was accepted and also explored suggestions regarding its redesign to improve its usability.

The second iteration focused on developing and testing the form and function of the prototype to verify and refine the context and usability of the game. An acceptance evaluation and usability testing were executed at elderly participants' homes so that the data would be collected in authentic usage contexts.

# Participant Recruitment

This study employed purposeful and convenience sampling methods to recruit participants. The participants included elderly people (older than 60 years) as target users who were homebound and those who received care either during the day or around the

clock in a nursing home. Elderly people were recruited to perform heuristic evaluations of the mobile cognitive training app. Heuristic evaluation allows researchers to employ a small set of evaluators (three to five evaluators) to examine the interface and judge its compliance with recognized usability principles (Nielsen and Molich, 1990; Nielsen, 1994). Moreover, medical experts, interaction designers and engineering programmers were also recruited as stakeholders to provide professional suggestions on the designs of cognitive training content and interactions.

In the first iteration, four female target users, who had agerelated memory decline and lived in a day-care or full-day nursing home, were recruited to participate (**Table 1**). They were aged from 73 to 90 years (mean = 82.75, SD = 7.27). All had experience with using mobile technology (smartphones and tablets), and three had their own mobile devices. They were interviewed and asked to evaluate current mobile games related to the cognitive training structure (**Figure 1**). These mobile games were chosen on the basis of their cognitive training structure, as determined by stakeholders and four experts, including interaction and user interface designers and programming engineers (**Table 2**, S-2–S-5). Game design suggestions were also collected from a medically proficient manager of a day-care nursing home (**Table 2**, S-1). All professional stakeholders participating in this study had more than 3 years of experience in their fields.

In the second iteration, five older adults who lived in their own homes participated and tested the usability of the prototype (**Table 1**). They were aged from 61 to 84 years (mean = 69.60, SD = 9.45). Two had experience with using mobile technology (smartphones and tablets), and three had their own mobile devices.

Every participant in this study was asked to sign an informed consent form before participating in the research. They were fully informed about the study purpose, research process, data recording, potential risk and relevant compensation. No additional ethical approval was required in this study, in accordance with national and institutional requirements.

# Procedures

The procedures of the DBR process used in this study are explained as follows.

## First Iteration

The first iteration involved the following steps:



#### TABLE 1 | Demographics of elderly participants.

*Mobile technology experience, experiences of using smartphone or tablet; F, female; M, male.*



3. Acceptance evaluation: The current mobile games with cognitive training structures selected in step 2 were presented to the participants (U-1–U-4). The elderly users were asked to complete the game tasks, and their usage behaviors and general feedback were documented.

Design cycle I was based on a theoretical review of the aging cognition and technology, and a user evaluation; the cognitive training structure of the game was optimized accordingly. The design principles, game tasks and training goals of the cognitive training game became clearer during this stage. The results of the evaluation were used to refine the initial conjecture and thus supported the second design cycle.

#### Second Iteration

The second iteration involved the following steps:


the mobile game in genuine home and full-day nursing home contexts. Their activities were documented and investigated. They were asked to complete the NASA Task Load Index (NASA-TLX) questionnaire to evaluate the cognitive load of the game. These evaluation results were used to identify successes and failures and to plan further improvements.

During the second iteration, the focus of the formative assessment was on the game process and cognitive load of the cognitive training game. The interaction and functions of the mobile game were modified, which completed the development of the game.

# Data Collection and Analysis

## Interview of Cognitive Training Structure of the Game

To develop a cognitive training game, the theoretical bases of aging cognition and technology were reviewed and organized into design guidelines. The cognitive training structure of the game was first proposed on the basis of the presented literature review (**Figure 1**) and then suggested by a medical expert (S-1). A semistructured interview was conducted to acquire a professional stakeholder's (S-1) suggestions regarding the cognitive training structure of the game. The guidelines of the interview covered four aspects: (1) What are the everyday life and physical and mental statuses of elderly people in day-care nursing homes? (2) What cognitive training activities can residents participate in when in day-care nursing homes? (3) What experience do elderly people have of using technology, and what is their technological ability? (4) What is important in the design of a cognitive training game for elderly users?

This study adopted content analysis to analyze the interview document (Strauss and Corbin, 1990; Ericsson and Simon, 1993). First, the protocol analysis was employed to classify the content of interview transcripts into the categories of cognitive training activities, specific-domain cognitive training, older adults' attitudes toward and feedback on cognitive training activities, older adults' technology experiences and the expectations of the cognitive training game. Second, the classified contents were encoded. Third, the encoded contents were categorized and conceptualized. The process of encoding and categorizing were agreed upon by the three researchers.

## Acceptance Evaluation

An acceptance evaluation was conducted to examine whether or not the cognitive training structure of the game (**Figure 1**) was suitable for elderly people. First, the evaluation content of this structure had to be selected. The researchers and experts, i.e., designers and programmers (S-2–S-5), selected the current mobile game-dementia prevention self-diagnosis test as the evaluation content. This mobile game is relevant to various specific domains of cognition in the cognitive training structure of the game (**Figure 1**). However, this mobile game lacks visuomotor and respelling tasks, so the designers suggested designing simple interactions such as drawing and card reading activities to be included (**Table 3**). Second, the participants (U-1–U-4) were asked to complete discrimination, calculation,


visuomotor and reading tasks in the games. Third, the researchers documented the users' usage behaviors while playing the game. Finally, the participants' attitudes toward and preferences of the games were collected.

The usage behavior of game playing was recorded and analyzed according to hierarchical task analysis (HTA) of each game trial. HTA is a structural method for describing how users complete tasks through hierarchical action plans and executions (Reagan and Kidd, 2013). HTA comprises redescription and decomposition. A primary task can be transferred into a structural model with these two important elements. The former helps designers identify the primary goal and break it down into several sub-goals with various levels until they reach the expected criteria. The latter helps designers to examine every sub-goal in order, such as inputting, receiving feedback, completion time and recording errors (Shepherd, 1998).

#### Formative Assessment and Usability Testing

A formative assessment was performed to collect feedback from the representative users and thus further the context and usability considerations of the cognitive training game. The users (U-5– U-9) played the game, completed the NASA-TLX questionnaire and were then interviewed on their general opinions of the game. The NASA-TLX questionnaire includes six load indices—mental demand, physical demand, temporal demand, performance, effort and frustration—measured using a 10-point Likert-type scale (1 = strongly disagree; 10 = strongly agree) (Hart and Staveland, 1988) and analyzed with descriptive statistics.

When the users played the game, the researchers observed and documented participants' behaviors and feedback to evaluate the interaction design. The usage behavior was recorded and analyzed according to HTA of each game trial. The time spent on each game trial was also recorded.

# RESULTS

# Expert Interview and Confirmation of Theoretical Propositions of Cognitive Training Game Structure

To design appropriate game tasks for cognitive training, this study interviewed a professional manager (S-1) of a day-care nursing home. His suggestions were adopted to confirm the theoretical propositions of the cognitive training game structure. He also provided his observations of the elderly adults' attitudes toward the cognitive training technology.

**Table 4** presents the cognitive training activities and tasks performed in the day-care nursing home that were suggested by the manager. All of the activities and tasks corresponded to associated specific-domain cognitive training of the conjectural structure (**Figure 1**). The easy and free creation activities such as drawing and clipping facilitated the cognition, accomplishment, activity engagement and emotional stability of elderly people. For example, **Figure 3A** shows the works of clipping. The elderly participants clipped the products from product catalogs to show what they wanted to buy when they imagined shopping contexts. The social interaction activities with multisensory stimulation promoted their attention, memory and language abilities, such as during karaoke and foot-bathing (**Figure 3C**). Certain complex activities such as the folding of balloons and handicrafts aimed to facilitate the cognitive functions of attention, executive function,


memory and visuospatial function. However, older adults may perform worse on these tasks because such tasks require the execution of a complicated sequence of steps. On the other hand, the computer-based game with the simulation of an authentic context, such as baking a cake, was considered easier and was accepted by the elderly participants. As shown in **Table 4**, the conjectural structure was sufficient to develop a cognitive training game. The current and easy cognitive training activities were appropriate for use as training content in the game.

In terms of the expert's attitude toward the cognitive training game, he specifically suggested four principles for the design of the cognitive training game for elderly people: (1) The game should train social interaction and life function capabilities; (2) the content or tasks should be associated with users' life experience; (3) the game should provide multisensory stimulation, including lights, sounds and colors; and (4) the game tasks should give users a sense of accomplishment to motivate them to play the game regularly.

# Users' Acceptance Evaluation of Cognitive Training Games

According to the theories of aging cognition, this study proposes the cognitive training structure of a game, which includes discrimination, visuomotor, respelling and calculation tasks. These tasks are intended to train various cognitive aspects in elderly users: attention, executive function, memory, language and visuospatial function (**Figure 1**). The identification of this structure was followed by the selection of current cognitive training games (**Table 3**) to investigate users' acceptance in cycle I.

As demonstrated in **Table 5**, tasks related to mental rotation and counting were perceived as more difficult by the participants. Their failure in these two games mainly resulted from the inappropriate content and interface designs. The participants responded that they preferred larger icons and words (26 pt). They also suggested that the input function should be more intuitive. For example, in the game of time discrimination, they expected to respond by tapping the clock directly, rather than by using the number keyboard.

Although some game tasks were difficult and the interface and interaction designs were not sufficiently appropriate, every participant perceived the game as interesting. They were willing to use these types of mobile games in the future. Therefore, the cognitive training structure of the game proposed by this study was accepted by the elderly participants. However, some modification of the tasks and interaction design were required to improve the experience and performance.

# Game Design: Refinement and Development of the Cognitive Training Game

The consensus among the elderly users and the stakeholder's evaluation were used to support, validate and refine the design structure of the cognitive training game. The game comprised not only the cognitive training structure but also experiential contexts, clear and concise instructions, and multisensory and intuitive interaction with immediate feedback to improve the acceptance, experience and performance. Following this optimized game structure, a working prototype was developed. The prototype consisted of game tasks involving suitable experiential contexts and a user-friendly interface and interaction.

### Game Tasks, Contexts, and Cognitive Training

The working prototype, named Brain Win, had a functional appearance and was designed to involve four types of cognitive tasks—discrimination, visuomotor, respelling and calculation tasks—in six game contexts that were connected with the life experience of older adults. The game contexts and elements were designed according to the interview with the manager of the day-care nursing home (**Table 4** and **Figure 3**) and the preferred leisure activities of elderly people, including shopping, singing, playing chess and mahjong, taking a walk, and accompanying and playing with family and grandchildren in particular (Shi et al., 2000; Chou et al., 2004; Lee et al., 2007). The tasks corresponded to different types of domain-specific cognitive training (**Table 6**).

The following is a specific explanation of the game tasks and contexts, in the order that they are encountered in the game:



TABLE 6 | Game tasks, contexts and stimulated cognitive constructs.


FIGURE 4 | User interface prototype. (A) Home page (B) Auditory instruction setting page (C) My calendar: identify the date by turning the calendar, choosing the day and time (D) Context and task introduction page displayed before each task starts (E) Go to the market (F) Shopping in the market (G) Finding object during a phone call (H) Super singer (I) Go to the zoo (J) Feedback page of task failure (K) Game score (L) Ranking table.

concentrate on the information icons and remember the costs of their selected items to perform cumulative calculations. This task thus trained participants' cognitive functions of attention, memory and executive function (**Figure 4F**).


Once each task was completed, a feedback page indicating success and failure was displayed with applause sounds and encouragement, respectively (**Figure 4J**). When participants successfully completed tasks, they could choose to practice the particular game, check their game scores (**Figure 4K**), or check their positions in the ranking table (**Figure 4L**).

#### Interface and Interaction

The game instruction was supported by visual and auditory design elements. The task explanations were concise and displayed in a font size of at least 26 pt. To decrease the working memory load, each task instruction was displayed continuously during the task until the user made choices. The auditory instructions were read in the Mandarin and Taiwanese dialects to help users retrieve their semantic memory to ensure that they could comprehend the instructions in a language that was familiar to them (**Figure 4B**).

The buttons and icons used had a realistic or skeuomorphic appearance so that elderly users could connect them to their life experience. For instance, when users were required to buy food items, the vegetable and meat icons were all realistic pictures (**Figure 4F**). Additionally, the buttons were all raised shapes. The effective touch range of the buttons and icons extended more than 20 mm on each side to accommodate the decreased visual perception and finger sensitivity of older adults. The button and icon interaction cues were color changes and sounds. The first button in the game was designed to flash to prompt users to tap, recognize and memorize the input button (**Figure 4A**). Then, when users tapped the button, it flashed and made a feedback sound.

Unlike existing research on the interaction design of digital games, the present study investigated the design more specifically through exhaustive and iterative user evaluations. Providing a user-friendly interface can increase the willingness of elderly people to adopt new technology. A satisfying interaction experience also increases the likelihood that they will continue using the game. A formative assessment was finally conducted to confirm the usability of the game.

# Results of the Confirmation Test

The working prototype with complete functionality was presented to five users for a formative assessment in the actual settings of the users' homes. The participants (U-5–U-9) were three females and two males from 61 to 84 years old. The confirmation test focused on the usability of the game. All five participants completed the six game tasks before answering the NASA-TLX questionnaire and expressing their general attitudes toward the game.

Their performances on the six game tasks are shown in **Table 7** and **Figure 5**. The average time taken to complete the six game tasks was 462.40 seconds (SD = 205.11). Three game tasks were more difficult, namely, My calendar (mean = 102.00, SD = 71.39), Shopping in the market (mean = 91.60, SD = 26.50) and Go to the zoo (mean = 85.60, SD = 44.53). The participant older than 80 years (U-5) had to spend over twice the time taken by the younger participants to complete all the game tasks.

According to HTA, the failure manipulations and longer completion times of My calendar, Shopping in the market and Go to the zoo could be investigated and elaborated. In terms of the game contents, My calendar entailed the highest number of manipulation steps. The participants could not stop tapping the icon for turning the calendar until the date was right, and then they had to input the right day and time period. Therefore, participants had to spend more time to complete these steps. Shopping in the market required participants to calculate the accumulated amount of money of the chosen products, so this task demanded more integrative cognitive resources. Every participant presented thinking expressions and thought aloud about the calculations when tapping the product icons. Go to the


zoo asked the participants to remember two successive animal sounds, so the participants tended to forget (U-6) or needed more time to recall the second animal sound.

On the other hand, the interface and interaction designs affected the performance of the game tasks. During the assessment, the participant aged 84 years (U-5) responded that she usually forgot the task instructions and ignored the task count button in Shopping in the market (**Figure 4F**). Two participants (U-5 and U-7) were confused by the task of turning the calendar in "My calendar" (**Figure 4C**) because they tried to turn the calendar by tapping the calendar directly rather than by tapping the icons for moving to the previous or next page.

As shown in **Table 8**, the participants generally felt satisfied with their performances (mean = 8.80, SD = 0.84); more importantly, they did not find the games frustrating (mean = 1.20, SD = 0.54). The participants thought the games to be slightly physically demanding (mean = 3.64, SD = 1.82) and mentally demanding (mean = 4.00, SD = 3.74). To train the cognitive function of elderly people, games should require low levels of physical and mental demand. Generally, the participants were willing to use the cognitive training game and had positive attitudes toward the game. The participant aged 84 years suggested that the instructions of manipulation should be more salient. On the other hand, the participants who were younger than 80 years considered the game to be easy and expected the more difficult levels and virtual money or food as rewards for training.

According to the results, the contexts and tasks of the game were generally satisfying and acceptable. Interface and interaction problems were encountered by users for the My calendar and Shopping in the market tasks, so these were modified. As shown in **Figure 6**, the mode of calendar turning was changed from button tapping to swiping, which is more TABLE 8 | Cognitive load evaluation based on NASA-TLX.


intuitive, and this mode was indicated by the textual instruction and a flashing arrow. The count button in Shopping in the market was modified to flash with a yellow light to attract the user's attention after he or she had chosen three ingredients.

# DISCUSSION AND CONCLUSION

This study aimed to develop an improved design for gamebased cognitive training for seniors using mobile devices. The Design Based Research method was employed to answer the research questions, including: (1) What types of cognitive training do elderly people accept in a mobile game? and (2) What components are crucial to improving or reducing the usability of a cognitive training game for elderly people?

According to the results of expert interviews and the acceptance evaluation in the first iterative cycle, the theoretical propositions concerning aging-related cognitive training (**Figure 1**) in the design of novel cognitive training supported by mobile games was confirmed. Both medical expert and elderly participants accepted the cognitive training structure of the game. The effectiveness of cognitive training activities for elderly people

in day-care nursing homes and suggestions from the manager agreed with the reviews of aging cognition theory and interaction design for the elderly (Ijsselsteijn et al., 2007; Schmiedek et al., 2010; Trefry, 2010; Lu and Yueh, 2015). Specifically, the design of the experiential context, social interaction and positive feedback should be emphasized. Therefore, lyrics arrangement using character squares (**Figure 3D**), product catalog clipping (**Figure 3A**) and calendar board changing (**Figure 3B**) were suitable contexts for the mobile game design of this study, for they corresponded to the leisure activities of older adults, such as karaoke and shopping (Shi et al., 2000; Chou et al., 2004). They also echoed the domain-specific cognitive training and tasks in the conjectured game structure and were neither too easy nor too difficult.

The results of the evaluation echoed the theories of aging technology design. First, the content of a mobile game should be large, such as its font size, icons, buttons and tap area (Charness and Jastrzembki, 2009; Fisk et al., 2009; Yueh et al., 2013, 2014). For instance, the effective touch range of buttons should extend at least 20 mm per side in mobile games designed for elderly adults (Siek et al., 2005), and the font size should be at least 26 pt. Second, the operation should be intuitive and accompanied by multisensory cues and feedback (Lee and Kuo, 2001; Harada et al., 2013). Third, accomplishing the game's tasks should require a low amount of working memory, and auditory cues should be provided to facilitate the users' retrieval of semantic memory (Ferreira and Pithan, 2005). Fourth, the cognitive abilities of attention, executive function, memory, language and visuospatial function decline with age, so the entire design should be clear, simple, and intuitive, and suitable feedback should be provided (Fisk et al., 2009; Drag and Bieliauskas, 2010; Lu and Yueh, 2015). The results of the evaluation enabled interpretation and refinement of the theoretical propositions and identified related needs and problems before the development of a form-study prototype and further interviews in the next iteration.

In the second iterative cycle, the results in the first iterative cycle supported and supplemented the theoretical propositions concerning training contexts and the importance of a usable interface in the design of a cognitive training game for elderly users. Therefore, a working prototype that conformed to the needs of older adults in terms of form, appearance and working function was developed. Afterwards, the usability evaluation was conducted to confirm the design of both the training contexts and interface to explore what components are crucial to the game. The results showed that elderly users generally perceived that low effort was needed to complete the game and approved of their own performance. However, game tasks with many manipulation steps and complex integrative cognitive functions required more time to work through. That is, older adults' performances of attention, executive function and memory are worse when they perform complex tasks (Dobbs and Rule, 1989; Lee and Kuo, 2001; Wang et al., 2011). Furthermore, the instruction and feedback design should be more salient to guide users on how to interact with the game. A more intuitive interaction design with authentic contexts can better improve performance (Fisk et al., 2009; Drag and Bieliauskas, 2010; Lu and Yueh, 2015). Finally, the optimized interface of the My calendar and Shopping in the market tasks were proposed in line with the results of the usability evaluation.

Based on the results, this study offers design suggestions, theoretical contributions, limitations and further issues deserving future research. In terms of design suggestions, this study differed from previous studies of game-based cognitive training for elderly people (Van Muijden et al., 2012; Binder et al., 2015) in that it not only included training tasks based on theories of aging cognition but also concentrated on the game's interface and interaction design to ensure that the game would be easy, userfriendly, satisfying and relatable to the user's everyday life. The design suggestions are offered to increase the adoption and selfperceived performance of elderly people using casual cognitive training games. First, the cognitive training structure of the game must be relatable to the users' life experience if the training tasks are to be accepted. Second, the interface of and interaction within the game should correspond to familiar elements and operations in the lives of elderly people so that the interaction will be intuitive. Third, the instructions and feedback in the game must be multisensory because such input improves user performance.

This study employed a design-based approach with an aim to improve the standards of research on the use of mobile games as a suitable intervention for cognitive training of elderly people. In addition, the iterative design process was also documented in authentic contexts. Through this procedure, this study derived the following conclusions, which are crucial to the application of DBR in conducting research. First, research should involve interdisciplinary collaboration, including at least academics from the fields of interaction design and engineer science, mechanism experts and stakeholders, if the insufficiencies of conventional designer-driven methods are to be overcome. Second, theorydriven intervention and conjectures must be verified and refined for authentic contexts to solve specific research problems and improve designs.

This study also has some limitations. The cognitive training game developed in this study was user-friendly, but such games can be addictive and may have a negative effect on user health. In addition, if mobile games are implemented in nursing homes, management strategies must be devised to avoid problems regarding ownership and usage time. Finally, this study was limited by resource availability, including accessibility to institutions and users, the system stability of the prototype and the efficiency of cognitive training. It is suggested that the effectiveness of cognitive training in mobile games be evaluated

# REFERENCES


in the future through field and physical testing on a larger scale involving more elderly users, and that the research be conducted in natural settings. The next task for this research is to incorporate theoretical propositions regarding the interactions among elderly people, mobile games and cognitive training evaluation.

# AUTHOR CONTRIBUTIONS

ML, WL, and HY developed the concept and design of this study. Data was collected and analyzed by ML. All authors interpreted the results. ML drafted the manuscript under the supervision of WL and HY, and WL and HY provided critical revisions. All authors approved the final version of the manuscript for submission.

# FUNDING

This research was supported by a grant from the Ministry of Science and Technology to HY (MOST 103-2628-S-002-002- MY2) and to WL (MOST 103-2410-H-002-173-MY2).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2017 Lu, Lin and Yueh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Examining the Relationship Between Teachers' ICT Self-Efficacy for Educational Purposes, Collegial Collaboration, Lack of Facilitation and the Use of ICT in Teaching Practice

#### Ida K.R. Hatlevik<sup>1</sup> and Ove E. Hatlevik<sup>2</sup> \*

<sup>1</sup> Department of Teacher Education and School Research, University of Oslo, Oslo, Norway, <sup>2</sup> OsloMet – Oslo Metropolitan University, Oslo, Norway

Information and communication technology (ICT) is now an integrated and central element of modern life, and its rapid emergence is changing the execution and organization of work and learning. Digital technology is also important for schools, and hence for teachers' working days. However, among today's teachers, not everyone has the knowledge required to teach using digital technology. Recent research indicates that self-efficacy is important for how teachers master their practice. This paper addresses teachers' ICT self-efficacy for educational purposes, and examines the assumed antecedents of teachers' self-efficacy. Data from 1,158 teachers at 116 Norwegian schools was analyzed. The results indicate that teachers' self-efficacy for using ICT in their teaching practice is associated with their use of ICT in teaching and their general ICT self-efficacy. In addition, the results show that collegial collaboration among teachers has a positive association with the use of ICT in their teaching practice. One interpretation of these findings is that general ICT self-efficacy is necessary for developing ICT self-efficacy for educational purposes and being able to use ICT in education. However, further research is required to scrutinize the relationships between these concepts.

Keywords: teachers, ICT self-efficacy for educational purposes, ICT self-efficacy, collegial collaboration, use of ICT, lack of facilitation

# INTRODUCTION

In little more than a generation, ICT has become a ubiquitous element of modern life. As schools prepare students to live in a technology-infused society and technology-driven workplaces, we must have teachers who are well prepared to support students' learning through the use of technology. Yet, many of those teaching today came of age during a transitional time and have varying degrees of capacity and comfort with the array of technological tools at their disposal. Their capacity to enhance the learning of students with technology and to enhance their students' technological skills depends, in part, on their personal comfort with and use of technological tools in their lives outside of the classroom. Beyond that, their motivation to infuse ICT in instruction instead of more traditional forms of pedagogy with which they may be more familiar

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

Barbara Caci, Università degli Studi di Palermo, Italy Anna Pribilova, Slovak University of Technology in Bratislava, Slovakia

> \*Correspondence: Ove E. Hatlevik ove-edvard.hatlevik@oslomet.no

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 14 March 2018 Accepted: 22 May 2018 Published: 13 June 2018

#### Citation:

Hatlevik IKR and Hatlevik OE (2018) Examining the Relationship Between Teachers' ICT Self-Efficacy for Educational Purposes, Collegial Collaboration, Lack of Facilitation and the Use of ICT in Teaching Practice. Front. Psychol. 9:935. doi: 10.3389/fpsyg.2018.00935

is influenced by their belief in their capability to do so successfully. These self-efficacy beliefs regarding ICT instruction, as research in self-efficacy in other domains has demonstrated, are likely to influence the effort they invest in planning for and delivering ICT instruction, their persistence with students who struggle and their resilience in the face of the inevitable snafus and breakdowns that accompany any pedagogical innovation, and even more so an innovation involving the use of technology. Most previous studies have focused on the impact of one or two variables on either teachers' self-efficacy in teaching or on their teaching practice. Thus, there is a need to gather knowledge about how different variables interact and are associated with both teachers' ICT self-efficacy for instructional purposes and with the use of ICT in their teaching practice. In this article, we explore the associations between teachers' self-efficacy in using ICT for instructional purposes, the use of ICT in teaching practice, general ICT self-efficacy, collegial collaboration regarding the use of ICT in teaching and the lack of facilitation for using ICT in teaching by the school management. The analysis is based on Norwegian teachers' answers to questions in the International Computer and Information Literacy study (ICILS) 2013 (Fraillon et al., 2014). In the following two sections, we elaborate on the studies used to formulate the hypotheses tested in this article.

# Self-Efficacy

Bandura's (1997, p. 3) concept of "self-efficacy refers to a belief in one's capabilities to organize and execute the courses of action required to produce given attainments." For the teaching profession, "a teacher's efficacy belief is a judgment of his or her capabilities to bring about desired outcomes of student engagement and learning, even among those students who may be difficult or unmotivated" (Tschannen-Moran and Hoy, 2001, p. 783). In other words, teacher self-efficacy is about "teachers' beliefs that they are capable of carrying out good teaching in the classroom" (Christophersen et al., 2016, p. 241).

Previous research has underscored the fact that teachers' selfefficacy has an effect on their job satisfaction and professional commitment (Skaalvik and Skaalvik, 2007; Ware and Kitsantas, 2007), attrition from the teaching profession (Klassen and Chiu, 2011; Hong, 2012) and is an important predictor of students' motivation and achievements (Caprara et al., 2006; Guo et al., 2012). Thus, identifying factors that can influence teachers' self-efficacy in using ICT in their teaching practice is an important subject to investigate. Social cognitive theory points to a potential positive effect of individuals' perception of their own competence and capabilities in a specific area of interest (i.e., self-efficacy) for continual growth and a feeling of mastery in that same field and similar fields of interest. Bandura (1997) claimed that these beliefs were more powerful than one's actual abilities; thus, self-efficacy beliefs can become self-fulfilling prophesies. Bandura stated that selfefficacy in a specific area affects individuals' thought processes, levels of persistence, degrees of motivation and affective states regarding tasks within the same area, thereby influencing individuals' performances. Enhancing individuals' self-efficacy beliefs in a specific set of tasks increases their performance levels on those tasks; however, those same individuals may fail in tasks that exceed their perceived coping capabilities (Bandura, 1997). Recent research regarding self-efficacy and the use of ICT in teaching corroborates Bandura's assumptions, and underscores the notion that increased levels of computer self-efficacy can lead to higher levels of confidence in being an efficient teacher with ICT (Fanni et al., 2013). Hammond et al. (2011) examined reasons why teachers use ICT, and they discovered a relationship between lower levels of ICT self-efficacy and the less frequent use of ICT. Furthermore, recent research demonstrates a positive relationship between self-efficacy about using digital tools and the use of ICT for teaching purposes (Teo, 2014; Hatlevik, 2017). In addition there is a positive association between student teachers' use of computers and their computer self-efficacy (So et al., 2012).

According to Bandura (1997), self-efficacy is both domain and context specific (i.e., it is not a global trait). In this study, we focus on teachers' ICT self-efficacy for instructional purposes, which describes the self-confidence teachers have when it comes to using ICT in their own teaching and instruction (Krumsvik, 2014). Krumsvik (2011) distinguishes between being confident about using ICT on your own and using ICT for pedagogical purposes. Scherer and Siddiq (2015), who used the same data as we utilize in our study, also reported that computer self-efficacy in basic and advanced ICT operational and collaborative skills, and self-efficacy in using computers for instructional purposes, are highly correlated but separate constructs. One way to interpret this positive association is that teachers' general perception of their own ICT skills (general ICT self-efficacy) is a necessary, but not a sufficient, determinant for self-efficacy in using ICT for instructional purposes. This interpretation makes sense, as you need to be competent in a skill yourself in order to be able to incorporate it when instructing others. A reasonable assumption to draw from Bandura's (1997) theory of self-efficacy and the results of the various studies mentioned here is that teachers' ICT self-efficacy for instructional purposes is positively related to their general ICT self-efficacy (hypothesis 1, **Table 1**), and to their use of ICT in teaching practice (hypothesis 2, **Table 1**).

Bandura (1997) asserted that there are four major influences on self-efficacy beliefs – vicarious experiences, verbal persuasion, physiological arousal and mastery experiences. In our study, we focus on how general ICT self-efficacy and contextual factors like collegial collaboration regarding the use of ICT in teaching, and the lack of facilitation for using ICT in teaching by the school management, are associated with ICT self-efficacy for instructional purposes. One can argue that collegial collaboration in particular entails the opportunity for both vicarious experiences and verbal support and persuasion. Furthermore, a lack of facilitation by the management could be interpreted as a hindrance for developing ICT self-efficacy for instructional purposes. In the next section, we elaborate on research related to the relations between contextual factors and teachers' ICT self-efficacy for instructional purposes and the actual use of ICT in teaching.

TABLE 1 | Hypothesized relations between teachers' ICT self-efficacy for instructional purposes, use of ICT, general ICT self-efficacy, collegial collaboration and lack of facilitation.


# Contextual Factors: Collaboration and Facilitation

The results from the Teaching and Learning International Survey (TALIS) 2013 show that Norwegian teachers are requesting assistance to develop their professional digital competence (Organisation for Economic Co-operation and Development [OECD], 2014). Previous studies have identified an association between facilitating teachers' use of ICT and their professional digital literacy development (Krumsvik, 2011; Tondeur et al., 2012). Tondeur et al. (2012) emphasize that learning from peers and collegial collaboration are productive ways for preservice teachers to learn how to implement ICT in their teaching practice. Furthermore, findings from a research project including teachers from all the EU countries (Wastiau et al., 2013), indicate that teachers prefer an informal approach to learn how to use ICT. Teachers do not seem to prefer external courses when developing their digital competence (Egeberg et al., 2011). Recent research indicates that teachers want to learn about ICT together with other teachers (Bacigagulpo and Cachia, 2011) and participate in training activities related to authentic classroom settings (Balanskat et al., 2006). One way to interpret these research findings is that collegial collaboration provides informal opportunities for teachers to learn about ICT together with other teachers in order to foster ICT self-efficacy and understand how to use ICT for educational purposes. This notion is also supported by previous research, which has shown that teachers' professional self-efficacy is positively affected by interpersonal support (Tschannen-Moran and Hoy, 2007) and collegial collaboration (Goddard et al., 2007), collective work, cooperation and exchanges amongst teachers (Grangeat and Gray, 2008). Furthermore, Caspersen and Raaen (2014) identify both collegial and superiors' support as influential in terms of teachers' perceived mastery of teaching. Finally, previous research indicates that collegial collaboration is of importance when it comes to teachers' actual teaching practice and students' achievements (Goddard et al., 2007).

Thus, taken together, these findings indicate that contextual factors like collegial collaboration are positively associated with teachers' ICT self-efficacy for instructional purposes (hypothesis 3, **Table 1**) and their actual teaching practice (hypothesis 4, **Table 1**). Furthermore, we expect support and facilitation by the school management to be associated with teachers' ICT self-efficacy for instructional purposes and their actual use of ICT in teaching practice. However, the data in our study focuses on the lack of facilitation by the school management, which we expect to be negatively associated with ICT self-efficacy for instructional purposes (hypothesis 5, **Table 1**) and the use of ICT in teaching practice (hypothesis 6, **Table 1**).

# Aim of the Present Study

The aim of this paper is to investigate the relationship between teachers' self-efficacy in using ICT for instructional purposes, the use of ICT in teaching practice, general ICT self-efficacy, collegial collaboration regarding the use of ICT in teaching and the lack of facilitation for using ICT in teaching by the school management. Six hypotheses regarding the relationships between these variables were developed from the existing research literature.

Two hypotheses address to what extent general ICT selfefficacy relates to ICT self-efficacy for instructional purposes (H1, **Table 1**) and to the reported use of ICT in teaching practice (H2). Furthermore, two hypotheses address the extent to which teachers' perception of collegial collaboration relates to ICT self-efficacy for instructional purposes (H3) and to their reported use of ICT in teaching practice (H4). Finally, two hypotheses address the association between the reported lack of facilitation for ICT self-efficacy for instructional purposes (H5) and their reported use of ICT in teaching practice (H6). The hypotheses tested in this paper are presented in **Table 1**.

**Figure 1** shows the hypothesized relationships between the variables, indicating that teachers' ICT self-efficacy for instructional purposes acts as a mediating variable.

# MATERIALS AND METHODS

# Participants and Procedure

This study has a cross-sectional, correlational design, and is a secondary analysis of existing data, namely the Norwegian ICILS 2013 consisting of responses from 1158 secondary schoolteachers. The International Association for the Evaluation of Educational Achievement (IEA<sup>1</sup> ) (Fraillon et al., 2014) conducted the collection, coding and reporting of the data according to predefined quality standards. The study had a twostep design. First, 150 schools were randomly selected. Second, based on the size of the school and the number of teachers in the school, between 15 and 20 teachers were selected from each school. Respondents from 116 schools replied to the survey (79% response rate at school level). The sample consisted of 64%

<sup>1</sup>For more information about the IEA visit their website: www.iea.nl

their use of ICT in teaching practice.


∗∗p < 0.01.

female teachers. The ages of the respondents were measured in six intervals; 2% were younger than 25 years old, 9% were between 25 and 30, 30% were between 30 and 39, 27% were between 40 and 49, 19% were between 50 and 59 and 13% were 60 or older.

# Instruments

The teachers answered an online questionnaire that contained questions and statements about their ICT self-efficacy, the use of ICT in teaching and contextual factors. All the questions and statements used in the analysis are presented in **Table 2**,

along with information about descriptive statistics and univariate normality, and the factor loadings obtained from the latent variable models for the scales.

### Use of ICT at School

Seven questions asked about the extent to which teachers use ICT in their teaching (e.g., presenting information through direct class instruction and supporting collaboration among students). The corresponding response categories were: 1 = Never, 2 = Sometimes and 3 = Often.

### Collegial Collaboration When Using ICT in Teaching and Learning

Collegial collaboration when using ICT in teaching and learning was measured with two statements about whether or not they work together with other teachers and collaborate with their colleagues in developing ICT-based lessons. The corresponding response categories were: 1 = strongly disagree, 2 = disagree, 3 = agree and 4 = strongly agree.

# General ICT Self-Efficacy

Four questions related to teachers' beliefs in their capabilities to use ICT to perform certain general tasks on the computer (using a spreadsheet program, contributing to a discussion forum, collaborating with others and installing software). The corresponding response categories were: 1 = I do not think I could do this, 2 = I could work out how to do this and 3 = I know how to do this.

### ICT Self-Efficacy for Instructional Purposes

This concept was measured with three questions about how well they could carry out specific tasks using ICT related to their teaching practice (monitoring students' progress, preparing lessons that involve the use of ICT by students and assessing student learning). The corresponding response categories were: 1 = strongly disagree, 2 = disagree, 3 = agree and 4 = strongly agree.

### Lack of Facilitation

Lack of facilitation was measured through three negative statements about whether or not they experienced a lack of facilitation and support from the school management in using ICT in teaching (insufficient preparation time, provisions to develop expertise and technical support). The corresponding response categories were: 1 = strongly disagree, 2 = disagree, 3 = agree and 4 = strongly agree.

# Analytical Strategy

Prior to testing the hypothesized model, the data was analyzed with respect to its descriptive statistics (means and standard deviations) and measures of univariate normality (skewness and kurtosis). Structural equation modeling (SEM) was used to test the assumed relationship between the variables (statistical software package Mplus 7.11). Such SEM allows for testing patterns of associations between latent variables, and at the same time can incorporate a measurement model that represents observed variables as indicators of underlying factors (Kline, 2010). Furthermore, SEM provides information that can be used to discuss how well the hypothesized model fits the empirical data (Brown, 2006). The model tested in this paper is a fully latent model; that is, the model examines the relationships amongst five latent variables (see **Figure 1**).

In order to evaluate the goodness-of-fit of the model (**Figure 1**), we used the chi-square information, together with the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI) and the Root Mean Square Error of Approximation (RMSEA) (Brown, 2006; Kline, 2010). The Weighted Root Mean Square Residual (WRMR) was used (Yu, 2002), because one latent dependent variable consist of categorical data. When evaluating the information from the fit indices, we are following guidelines recommended in the literature (Hu and Bentler, 1999; Marsh et al., 2004). A good model fit can be described with levels of the CFI and TLI equal to or above 0.95 (Marsh et al., 2004), RMSEA below or equal to 0.08 (Hu and Bentler, 1999), and levels of WRMR close to or below 1.00 (Yu, 2002). There are missing values for some items, and the full-informationmaximum procedure was therefore used.

# RESULTS

# Descriptive Statistics

The values for mean, standard deviation, skewness and kurtosis are presented in **Table 2**. The levels of skewness and kurtosis were acceptable for the items used to measure the use of ICT in teaching, collegial collaboration, lack of facilitation and general ICT self-efficacy. One item (preparing lessons that involve the use of ICT by students) used to measure ICT self-efficacy for instructional purposes had higher levels of both skewness and kurtosis. All items used to measure ICT self-efficacy for instructional purposes are therefore treated as categorical data in the analyses. The responses were recoded under two categories: 1 = I do not know how to do it or 2 = I do know how to do it. The first category includes responses to both original ratings of 1 (I do not think I could do this) and 2 (I could work out how to do this). This is the appropriate way to conduct analysis when data is not normally distributed.

# Measurement Model

The computed chi-square value of the tested model is significant (p = 0.000). However, the chi-square test is sensitive to large samples, and the current sample consists of 1158 respondents. The results of the other fit indices indicate a good model fit: CFI = 0.964, TLI = 0.958, RMSEA = 0.034 (90% CI = 0.029–0.039) and WRMR = 0.977.

Item loadings can be used to examine how the items reflect the constructs. Item loadings above 0.60 are desirable, but items with lower loadings can also provide relevant information about the constructs. Most of the factor loadings of each latent variable were relatively high (range = 0.52–0.92), indicating sufficient convergent validity (see **Table 2**). Cronbach's alpha reliability coefficients for the respective latent variables were 0.79 for use of ICT in teaching practice (seven items), 0.71 for collegial support and cooperation (three items), 0.75 for general ICT self-efficacy

#### Hatlevik and Hatlevik Teachers ICT Self-Efficacy for Educational Purposes

fpsyg-09-00935 June 11, 2018 Time: 17:16 # 6


<sup>∗</sup>p < 0.05, ∗∗p < 0.01.

(four items), 0.68 for ICT self-efficacy for instructional purposes (three items) and 0.74 for lack of facilitation for using ICT in teaching by the school management (three items). Overall, it seems that most items are working quite well.

The correlation matrix (**Table 3**) shows significant correlations between all the latent variables. There are positive moderate to high correlations between collegial collaboration, general ICT self-efficacy, ICT self-efficacy for instructional purposes and ICT use. Lack of facilitation correlates negatively with all of the other latent variables.

The results of the SEM analysis presented in **Figure 2** indicate that teachers' general ICT self-efficacy has a strong positive association (β = 0.66) with teachers' ICT selfefficacy for instructional purposes, thus corroborating hypothesis 1. In addition, collegial collaboration is positively associated (β = 0.13) with ICT self-efficacy for instructional purposes, thus corroborating hypothesis 3. Furthermore, teachers' ICT self-efficacy for instructional purposes (β = 0.39) and collegial collaboration (β = 0.35) have moderate positive associations with teachers' use of ICT in teaching practice, thus corroborating hypotheses 2 and 4.

Lack of facilitation for using ICT in teaching by the school management does not have a significant direct association with either teachers' ICT self-efficacy for instructional purposes or with their use of ICT in teaching. However, there are moderate correlations between the three independent variables: teachers' general ICT self-efficacy, collegial collaboration and lack of facilitation. Lack of facilitation correlates negatively with both collegial collaboration and general ICT self-efficacy, whereas collegial collaboration and general ICT self-efficacy are positively correlated.

Explained variance is 50.1% for teachers' ICT self-efficacy for instructional purposes and 33.7% for teachers' use of ICT in teaching practice.

An alternative model was tested, in which a direct association between the variables general ICT self-efficacy and use of ICT in teaching practice was added to the original model. This specific association came out as not significant, and the rest of the associations in the model did not change.

# DISCUSSION

Previous research findings show variation when it comes to how teachers are able to use ICT efficiently in their own teaching practice (Fraillon et al., 2014; Haydn, 2014; Hatlevik, 2017). In addition, previous studies indicate that formal teaching competence alone is not a sufficient factor for effective student learning, as other individual and contextual factors are also influential (Gustafsson, 2003; Hattie, 2008). However, some individual characteristics seem to be more vital than others for good teaching practice; in particular, teachers' self-efficacy in teaching is considered a key issue (Bandura, 1997; Skaalvik and Skaalvik, 2007).

This paper addresses a model with antecedents of teachers' ICT self-efficacy for instructional purposes. A model was

developed based on recent research regarding the relationship between collegial collaboration, general ICT self-efficacy, lack of facilitation, ICT self-efficacy for instructional purposes and the use of ICT in teaching practice. This assumed model consists of six hypotheses, and our analyses indicate that four of these six hypotheses are supported by the data.

First, teachers' general ICT self-efficacy has a strong positive association with ICT self-efficacy for instructional purposes. This finding is in line with a fundamental premise within self-efficacy theory (Bandura, 1997) about the importance of distinguishing between domain-specific self-efficacy beliefs. Furthermore, the results show the importance of associating a domain-specific ICT self-efficacy with different ICT tasks or activities, e.g., ICT self-efficacy for instructional purposes. This finding seems to nuance the relationship between various types of domain- or task-specific self-efficacy, as the finding underpins the fact that the two constructs (general ICT self-efficacy and ICT self-efficacy for instructional purposes) are both distinctive concepts and highly correlated (Krumsvik, 2011; Scherer and Siddiq, 2015).

Second, teachers' ICT self-efficacy for instructional purposes has a moderate positive association with their use of ICT, which corroborates previous findings that teachers' digital competence predicts their use of ICT in their teaching practice (Krumsvik, 2011; Hatlevik, 2017). However, our analysis extends previous knowledge about the nature of the relationship between the use of ICT in teaching practice, ICT self-efficacy for instructional purposes and general ICT self-efficacy. Our results reveal how ICT self-efficacy for instructional purposes can act as a mediating variable. Thus, it is not enough to be confident in using ICT yourself (general perception of your own ICT skills); you also need to be confident about how to use it for instructional purposes. Therefore, supporting prospective and more experienced schoolteachers' development of didactical competence in using ICT for instructional purposes is crucial when it comes to implementing ICT in teaching practice.

Third, collegial collaboration has a positive association with teachers' use of ICT in their teaching practice, their general ICT self-efficacy and ICT self-efficacy for instructional purposes. Our results are in line with previous research, which underlines that teachers' self-efficacy and their teaching practice is positively affected by collegial collaboration (Goddard et al., 2007; Caspersen and Raaen, 2014). Bandura (1997) emphasizes the social aspect of self-efficacy, meaning that self-efficacy is developed and influenced by the context of the person.

Fourth, the hypotheses (H5 and H6) regarding a lack of facilitation are not supported by the data. Based on the theory of Bandura (2006), we assumed that facilitation is important for self-efficacy. One explanation could be that there is a difference between the need for collaboration and facilitation between various groups of teachers. Thus, it would be interesting to test the model on both newly qualified and more experienced teachers, and on teachers from different countries.

Overall, there are some limitations to our study. First, the data is gathered from a cross-sectional design and it does not establish which factor comes first, meaning that it is difficult to show what is the cause and what is the effect. According to social cognitive theory, there are reasons to believe that a reciprocal relationship exists between them. To uncover a reciprocal relationship and fully understand the dynamics of these mechanisms there is a need for longitudinal and qualitativeoriented studies. Second, the measure of self-efficacy is overly simplified in this study because the concept is measured using only three response categories. It is therefore difficult to have a nuanced interpretation of how teachers rate their own selfefficacy. Third, in structural modeling, all variables are run simultaneously, and the omitted variables might also have influenced the explored model.

# CONCLUDING REMARKS AND FURTHER RESEARCH

This paper addresses perspectives on teachers' ICT selfefficacy for instructional purposes. The results reveal a positive association between teachers' general ICT self-efficacy, ICT selfefficacy for instructional purposes and the use of ICT in teaching practice. One possible conclusion is that the way to develop ICT self-efficacy for instructional purposes is through a more general ICT self-efficacy. Another possible conclusion is that teachers' use of ICT in teaching practice can be facilitated by their ICT self-efficacy for instructional purposes.

It seems that collegial collaboration is important for understanding the variations in teachers' use of ICT in teaching practice. This corresponds to our assumptions that the use of ICT in teaching is a collective project, and that collaboration can provide support and make the use of ICT more legitimate.

As mentioned earlier, Norwegian teachers report that they need help to develop their professional digital competence (Organisation for Economic Co-operation and Development [OECD], 2014). One reason could be that ICT is defined as a transversal skill in the curriculum, meaning that digital skills are embedded in competence aims from different curriculum subjects. The findings from this study indicate that emphasizing teachers and student teachers' general ICT self-efficacy and ICT self-efficacy for instructional purposes could provide ways of preparing teachers to use ICT in their own teaching practice.

# ETHICS STATEMENT

This study was carried out in accordance with the demands of The Norwegian Data Protection Authority and the Norwegian Centre for Research Data. The protocol, the written information to the participants and the procedures of informed consent was approved by the Norwegian Centre for Research Data.

# AUTHOR CONTRIBUTIONS

IH and OH wrote the introduction and theoretical part of the paper. IH and OH developed the research questions and the hypothesis. IH and OH analyzed the data from an international survey. IH and OH developed the result section. IH and OH completed the discussion and the conclusion. A joint work where both have contributed equally.

# REFERENCES

fpsyg-09-00935 June 11, 2018 Time: 17:16 # 8


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Hatlevik and Hatlevik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Unobtrusive Observation of Team Learning Attributes in Digital Learning

## David C. Gibson\*

Learning Futures, Curtin University, Perth, WA, Australia

This article presents a new framework for unobtrusive observation analytics of knowledge and skills-in-action through continuous collection of data from individuals while they interact with digital assets either as individuals or on problem-solving teams. The framework includes measures of the skill and knowledge areas of collaboration, creativity, personal learning, problem solving, and global sustainability, which are observed during natural production and use of communications, intentional artifacts, and resources in a digital learning space designed for self-directed and team-based learning challenges. The article describes the digital context for data collection and shows some example data and analyses.

#### Edited by:

Gerrit C. Van Der Veer, University of Twente, Netherlands

#### Reviewed by:

Elisabetta Gola, Università degli studi di Cagliari, Italy Dina Di Giacomo, University of L'Aquila, Italy

> \*Correspondence: David C. Gibson david.c.gibson@curtin.edu.au

#### Specialty section:

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Received: 23 November 2017 Accepted: 08 May 2018 Published: 31 May 2018

#### Citation:

Gibson DC (2018) Unobtrusive Observation of Team Learning Attributes in Digital Learning. Front. Psychol. 9:834. doi: 10.3389/fpsyg.2018.00834 Keywords: unobtrusive observation, continuous assessment, digital learning, higher order thinking, learning analytics

# INTRODUCTION

Digital learning environments present new opportunities for analytics-based learning design (Shum and Ferguson, 2011; Scheffel et al., 2012; Gibson and Ifenthaler, 2017), assessments (Gibson and Webb, 2013; Griffin and Care, 2015), and learning supports such as scaffolding for selfregulation (Ifenthaler, 2012). In the case of assessments, the new opportunities for analytics can be extended to include legitimate peripheral participation (Lave and Wenger, 1991) in team learning by documenting individual behaviors, actions, and problem-solving strategies while working in the social setting of team work (Kinshuk Ifenthaler et al., 2010). Digital games and simulations designed for team performance for example, often are characterized by integrated, media-rich contexts with multiple layers of interaction with peers as well as computational resources, which provides a foundation for authentic performance of individual and team-based problem-solving processes (Clarke-Midura et al., 2012) with attendant opportunities for unobtrusive observation and documentation of strategies, tools, communications, intentional actions and artifacts (Griffin et al., 2012; Siadaty et al., 2015).

In this article, a foundation is outlined for unobtrusive observation of the knowledge and skills exhibited in an online challenge-based learning platform (Gibson et al., n.d.) during collaboration, problem-solving, personal learning, creative thinking, and working with concepts of global sustainability elicited by real-world challenges. Definitions of key terms and measures of a specific set of team learning attributes are provided and a proposed new integrated mapping is illustrated for sample interactions in an application called

'Balance of the Planet' that will serve as a concrete case example. A proposed general method for planning for unobtrusive observations involves metadata mapped to structured and unstructured digital interactions where an individual or group's intentions are explicitly prompted and play a role similar an assessment prompt. However, in team problem solving addressing complex challenges in a digital learning space, there is considerable openness in the type, degree and amount of possible learner responses documented by a highly granular data record of a learner's performance, with many attendant options for analysis.

# TEAM-BASED LEARNING ATTRIBUTES

As learners utilize resources and performance affordances of a well-designed digital learning space, they touch things, comment via text or talk, upload and download files, listen and watch, and leave other kinds of evidence of their interactions in a timebased digital record. If the interactions have been thoughtfully assembled to support a chain of evidence, then a learner's thinking patterns and actions can be observed and inferences can be drawn concerning what they know and can do, including 'knowledge of' and 'knowledge-in-action' of valued constructs (Quellmalz et al., 2012; Gibson and Jakl, 2015).

The constructs of 'collaboration, problem-solving, personal learning, creativity, and global sustainability' outlined below (**Table 1**) are drawn from multiple sources in the research literature on digital assessment and serve as part of a highlevel domain model (Mislevy et al., 2003) for the observation of team learning. The domain model helps organize potential claims about learner proficiency; the kinds of things students might say or do that would constitute evidence about these proficiencies; and the kinds of situations that elicit relevant evidence.

Second-level components in the domain model (**Table 2**) define sub-events that are partial measures for each construct. For example, collaboration is evidenced by three activities needed to construct and maintain a shared conception of a challenge: reaching a shared understanding, taking appropriate action to address the challenge, and maintaining group organization. If any

TABLE 1 | Construct definitions of team learning attributes in Balance of the Planet.

Collaboration: coordinated group activity resulting from continuous attempts to construct and maintain a shared conception of a problem (Roschelle and Teasley, 1995).

Problem solving: cognitive processing directed at achieving a goal when no solution method is obvious (Mayer and Wittrock, 1996).

Personal learning: acquisition of knowledge (e.g., new insights, capacities for thinking, acting and employing skills) that is evidenced for outside observers as well as an individual's own reflection and metacognition (Friedrichs and Gibson, 2003).

Creativity: creative solutions are novel (surprising and original), effective (useful and understandable) and whole (sensitive to the context of their creation) (Mishra et al., 2013).

Global sustainability: development that meets the needs of the present without compromising the ability of future generations to meet their own needs (Brundtland, 1987).

of these sub-components fail, then collaboration fails. Thus, the second-level components must all be observed and validated over a body of evidence over time in the digital record in order to enable a confident inference that the attribute has been identified for an individual. Because multiple observations are required over time, trajectories of the attributes are expected, that is, the attributes are expected to be seen more than once and in different contexts, and may be seen to evolve or change in some way during the time span of the multiple observations.

Provisions for individual performance over time within a group setting ensure that the observable has both individual and group characteristics, through what has been termed the 'social learning capacity' of the group (Gibson et al., in press).

To triangulate the measures of each of the key terms, secondlevel components (**Table 2**) have been identified and form the definition of metadata codes that are directly associated with digital interactions. The second-level components of Collaboration and Problem Solving are drawn from the collaborative problem solving framework of the Programme for International Student Assessment (PISA) a worldwide study by the Organisation for Economic Co-operation and Development (PISA, 2013). The components of personal learning are drawn from research on the personalization of learning (Friedrichs and Gibson, 2003; Gibson, 2003). Second-level ideas about creativity are based in the work of the Deep Play Research Group at Arizona State University (Mishra, 2012; Mishra et al., 2013). Finally, the second-level ideas associated with global sustainability are drawn

TABLE 2 | Second-level components that define the metadata for unobtrusive observations of team-based learning attributes in Balance of the Planet.


from the concept of the 'triple bottom line' in sustainability research (Elkington, 1997).

# INTEGRATED MAPPING OF ATTRIBUTES

The context for mapping the attributes to automated data collection and observations differs from mapping the constructs of a typical test or quiz in two important respects: the learner is unaware of being tested, and the learner has the freedom to act, think, and communicate without an additional cognitive load of evaluated meaning or consequences. I'll refer to these as the natural production of evidence and self-direction (including team self-direction) in exploratory learning. Outside of these two aspects, the integrated mapping of attributes to interactions proceeds as part of the development of a chain of reasoning from evidence to inferences guided by a domain model (Almond et al., 2002).

In the following section a case example of a digital learning platform designed for self-directed learning by individuals and teams is used to illustrate. The Curtin Challenge platform delivers a team-based learning opportunity called Balance of the Planet and serves as a specific case of a generic model of backward design thinking (Wiggins and McTighe, 1998; Guskey, 2014) in which a learning designer begins with the end in mind, by thinking about what a learner or learning team should be able to exhibit that they know and can do as a critical performance of their knowledge-in-action (Argyris, 1993). The designer begins with the end in mind by articulating the key criteria and performance levels for a critical performance, and creates one or more defining moments when students or a team must display and flexibly use what they know in order to accomplish some cornerstone task. A helpful metaphor is a sports team, whose coach is not on the playing field on game day, but who has prepared the team for success and then watches and encourages from the sidelines while the team faces the challenge first-hand. In professional contexts, cognitive apprenticeships (Resnick, 1984; Collins et al., 1989) of various kinds prepare people to 'act as' the professional when called upon by circumstances; for example, a doctor-to-be might diagnose a disease during hospital rounds, a teacher-to-be might teach a practice lesson, and an engineerto-be might take part as a legitimate peripheral team member during a field site inspection. In game and simulation-based learning authentic decisions similar to cognitive apprenticeships can be encountered and the feedback provided to the learner can in many cases be automated or built into the immersive experience.

Authoring a digital learning experience on the Challenge platform follows this basic outline. It starts with the end in mind by defining a package of work products and processes (deliverables for individuals or teams). The application platform then automatically generates scaffolding pages, which break-down the parts of the work package into subtasks and focuses on the feedback based on the criteria needed to achieve success on the deliverable. On each page and linked to each interaction are metadata codes (**Table 2**) and embedded hints (see for example "Extended Prompt" and "EvalVotePrompt" in **Table 3**) that are used either in near real time or post hoc to group the evidence into clusters for further processing, display and for reflection on learning.

In addition to these forms of structured data, the Challenge platform integrates the CISCO Spark application to create team spaces on each sub-task page. The Spark tool provides a flexible persistent messaging system that documents who says what, who shares files with whom at what times and in what contexts, how a conversation, document or product evolves and how decisions are reached related to each subtask. The data stream is addressable by future bots that can function as smart team members that can leverage the open APIs of the platform.

To illustrate, **Table 3** displays a sample of the fields and content for a subtask in Balance of the Planet 'Global Business Plan' contest. The subtask is the creation of a product name that will appear on the title page of a final report. The artifact is a text and the key prompts and hints for creating the name include the phrases "What is the name of your product, service or process?" and if further information is desired "The name has a big role in creating 'brand awareness' of your idea. A good brand creates a positive response in people and a desire to want or learn more about the brand." The evidence created for later assessment is tagged as an example of the group's problemsolving ability connected with representing and formulating an idea (PS2 from **Table 2**). The team members will all vote on how well they think the name meets the criteria. At the same time, the team process of talking about which name to settle on, who proposed the name, who proposed any edits, and so forth, are all saved as data and context for the PS2 evidence.

The process of turning session log files and process stream data into indicators has been recently summarized in Griffin and Care (2015) which also notes several precursor research projects and results in digital media learning, so will not be reduced further here except to say that a process of exploratory data analysis is required based on post hoc analysis of real people using an appropriately designed digital space to learn. The growing field of learning analytics focused on learning and learners (as opposed to teaching, institutional progress, curriculum, and other outcomes) is exploring and expanding the knowledge base concerning the challenges and solutions of the layered and complex analyses required nowadays for a better understanding of the impact of digitally enhanced learning spaces on how people learn.

Collecting evidence at such a fine-grained level (e.g., one small component of a title page) of a sub-task (e.g., the title page of a report) of a larger task (e.g., a report about an extended group project) is made feasible by automatic data collection and the nearly full digitization of the group's process. This approach to data collection and tagging supports assessment inferences based on the natural production of evidence during authentic teamwork and is intended to minimize disturbance of learning in the natural evolution of self-motivated team-based learning.

A limitation of any educational measurement system is that a learner's thought processes are not directly accessible and are not comprehensively represented in the externalized artifacts

of interactions (e.g., the words, images, discussion, products created, and resources used) during learning. This results in a need for learning analytics researchers to make inferences about what someone knows and can do based on limited available evidence. The framework and methods outlined here represent a plan to increase the number and quality of data captures of the natural production of artifacts during individual and team problem-solving in order to increase the data resolution (e.g., the fine-grained data details) of the processes of externalizing thought and learning processes.

# CONCLUSION

A key difference between being tested and unobtrusively observed is the extent to which the production of actions arises naturally without awareness or anxiety about being watched and evaluated. The digital platform can be designed for unobtrusive observation and capturing salient solution and construction processes of a learner's natural production of evidence via communicating, making artifacts for a known purpose (e.g., the subtask in a context), and using resources during the processes of acquiring and organizing information, creating responses and things that can be (or for which images can be) digitally uploaded and communicating ideas to others.

A second important construct of a digital challenge-based learning platform is based on the assumption that learners can make their own way toward a production or behavioral goal with minimal assistance, passing through various cognitive states along a trajectory toward a final submission when their goal has been met or their work completed. In addition, as they are making their way, having peers along on the journey raises the stakes of

TABLE 3 | Fields and content for a subtask in Balance of the Planet 'Global Business Plan' contest.


thinking and performance, raises the standards for completion, and provides valuable enhancements to individual knowledge and action.

This article has shared the details of a specific example mapping system for unobtrusive observation of higher order skills (**Table 2**) evidenced during open-ended and self-directed team-based learning on the Curtin Challenge platform. Data now being collected on over 25,000 students is developing the baseline for establishing process stream indicators of the attributes of natural actions involved in learning, problemsolving, and teamwork.

# REFERENCES


# AUTHOR CONTRIBUTIONS

The author confirms being the sole contributor of this work and approved it for publication.

# FUNDING

This research was supported by Curtin University's UNESCO Chair of Data Science in Higher Education Learning and Teaching.


**Conflict of Interest Statement:** The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Gibson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Brain Training in Children and Adolescents: Is It Scientifically Valid?

Teresa Rossignoli-Palomeque1,2 \*, Elena Perez-Hernandez <sup>3</sup> and Javier González-Marqués <sup>1</sup>

<sup>1</sup> Department of Basic Psychology II, Complutense University of Madrid, Madrid, Spain, <sup>2</sup> Department of Psychology and Education, Centro Universitario Cardenal Cisneros, Alcalá de Henares, Madrid, Spain, <sup>3</sup> Department of Development and Educational Psychology, Autonomous University of Madrid, Madrid, Spain

Background: Brain training products are becoming increasingly popular for children and adolescents. Despite the marketing aimed at their use in the general population, these products may provide more benefits for specific neurologically impaired populations. A review of Brain Training (BT) products analyzing their efficacy while considering the methodological limitations of supporting research is required for practical applications.

#### Edited by:

Dina Di Giacomo, University of L'Aquila, Italy

#### Reviewed by:

Sara Siddi, Parc Sanitari Sant Joan de Déu, Spain Artemisa Rocha Dores, Escola Superior de Saúde—Politécnico do Porto, Portugal

> \*Correspondence: Teresa Rossignoli-Palomeque teresa.rossignoli@ucm.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 07 November 2017 Accepted: 04 April 2018 Published: 04 May 2018

#### Citation:

Rossignoli-Palomeque T, Perez-Hernandez E and González-Marqués J (2018) Brain Training in Children and Adolescents: Is It Scientifically Valid? Front. Psychol. 9:565. doi: 10.3389/fpsyg.2018.00565 Method: searches were made of the PubMed database (until March 2017) for studies including: (1) empirical data on the use of brain training for children or adolescents and any effects on near transfer (NT) and/or far transfer (FT) and/or neuroplasticity, (2) use of brain training for cognitive training purposes, (3) commercially available training applications, (4) computer-based programs for children developed since the 1990s, and (5) relevant printed and peer-reviewed material.

Results: Database searches yielded a total of 16,402 references, of which 70 met the inclusion criteria for the review. We classified programs in terms of neuroplasticity, near and far transfer, and long-term effects and their applied methodology. Regarding efficacy, only 10 studies (14.2%) have been found that support neuroplasticity, and the majority of brain training platforms claimed to be based on such concepts without providing any supporting scientific data. Thirty-six studies (51.4%) have shown far transfer (7 of them are non-independent) and only 11 (15.7%) maintained far transfer at follow-up. Considering the methodology, 40 studies (68.2%) were not randomized and controlled; for those randomized, only 9 studies (12.9%) were double-blind, and only 13 studies (18.6%) included active controls in their trials.

Conclusion: Overall, few independent studies have found far transfer and long-term effects. The majority of independent results found only near transfer. There is a lack of double-blind randomized trials which include an active control group as well as a passive control to properly control for contaminant variables. Based on our results, Brain Training Programs as commercially available products are not as effective as first expected or as they promise in their advertisements.

Keywords: cognitive training, brain training, computer-based intervention, children, adolescents

**92**

# INTRODUCTION

The use of new technologies is increasingly accepted in society, not only in educational settings and the general population, but also in the clinical field. More specifically, some "brain training" (BT) platforms, BT applications and BT video gamelike products are becoming very popular. A rigorous evaluation of such applications is merited because most commercially available BT products have not been tested (Rabiner et al., 2010) despite being widely used. Traditionally, BT programs have been used mainly for relaxation therapy, as a tool to encourage self-control in children, or to restore abilities following brain damage. Furthermore, it specifically seems to have a special relevance for developmental psychopathology, being widely used, in disorders such as Attention deficit hyperactivity disorder (ADHD) (Rabipour and Raz, 2012), and in the elderly with cognitive decline (Papp et al., 2009). Despite this tradition, since Nintendo launched the BT game "How old is your brain?" in 2006, there has been increased interest in the general population in commercially available BT programs to improve, for instance, intelligence. Currently, BT is used both by the general population with typical development as well as in populations with neuropsychological impairment (Rabipour and Raz, 2012). In other words, "as we live in an increasingly technological society, the cognitive stimulation of healthy people requires more and more computerized resources" (Portellano, 2014, p. 136). Nevertheless, although BT is increasingly being marketed and aimed at the general population, it has a special use in neurologically impaired children and the elderly.

For this review, we must distinguish between different domains of BT or what researches refer to as computer-based interventions of "cognitive training" (CT). We must consider that the Eastern and European concepts differ; for instance, considering Tang and Posner (2009), we can classify attentional training (an example of CT) into two methods: the methods of the Asian tradition (for example mindfulness) and, the methods of the American or European tradition (such as BT programs). In the case of the first group, what is sought is to train a state of attention and self-regulation; while in the second, the aim is to alter specific brain processes related to cognitive tasks. In the case of American and European traditional methods, CT is based on the use of a repetition of exercises like those employed in cognitive psychology laboratories. This concept could be an extension of what we refer to as BT.

What is understood by "brain training?" BT is a program or activity which purports to improve a cognitive ability or general capacity by repeating certain cognitive tasks over a period of time. This is supposed to produce some changes in behavior, as well as at a neuroanatomical and functional level (Rabipour and Raz, 2012). Although this term is used mostly by companies rather than researchers (researchers commonly use "cognitive training"), BT refers to practicing core cognitive abilities with the goal of improving performance in other cognitive tasks (Simons et al., 2016). This model applies to computer-based programs as well as video game training or BT applications for touchscreens. Authors such as Nouchi et al. (2012) have researched the transfer effect of "video game training," an issue commonly discussed in BT research, or McNab et al. (2015) who studied human cognition while using a touchscreen BT gamelike application. For the present review, we have considered BT products supported by online or computer-based platforms, videogame-like products or applications for touchscreens.

To provide a better understanding of most BT research and BT efficacy, we refer to two concepts upon which most programs claim to be based: transfer and neuroplasticity. Neuroplasticity is supposed to alter neural connections and be reflected in the performance of cognitive skills or behavior, which is known as transfer or the transfer effect. Most researchers explain transfer effects due to neuroplasticity, but provide little data to support this idea. Here we are going to clarify both concepts.

# Transfer

Under this concept, authors such as Karbach and Unger (2014) distinguish between "near transfer" and "far transfer." In the present review, we follow this distinction. The main goal of BT or CT should be not only to produce benefits in tasks similar to those directly trained or for the same construct, namely, near transfer (NT), but rather to benefit performance in other tasks that are different to those directly trained or for another construct: far transfer (FT). FT can have an impact on the user's daily life, and is understood as the "ecological" outcome of BT interventions.

Cognitive training interventions have shown FT even in relevant skills such as general intelligence (Raz and Buhle, 2006). In this line, Tang and Posner's (2009) study with adults seems to demonstrate that CT programs which target executive control or WM can benefit a wider variety of cognitive functions. In particular, in CT aimed at attention and WM, it seems that benefits could extend to fluid intelligence (Mackey et al., 2011). Klingberg (2010) explained this transfer as a result of the confluence of the prefrontal neural networks that support WM and fluid intelligence. Westerberg and Klingberg (2007) showed that practice in WM tasks gradually improved performance in WM tasks, and that the effect of practice also caused a general improvement in performance in a non-trained task such as a reasoning task. After training, WM-related brain activity was significantly increased in the middle and inferior frontal gyrus. According to this researcher, the changes could best be described by small increases in the extent of the activated cortex rather than activating additional areas. As we have seen, it is very common to justify transfer as a consequence of neuroplasticity. Strenziok et al. (2014) demonstrated FT of three cognitive training programs with healthy elderly people: (1) Brain Fitness (BF-auditory perception), (2) Space Fortress (SF-WM), (3) The Rise of Nations (RON-strategic reasoning). They found transfer of these trainings to other untrained areas (the first two), such as problem resolution of daily life and reasoning. The authors attributed their results to neuroplasticity, in that training produced changes in the integrity of gray matter in occipitaltemporal areas (associated with improvement in problem solving of daily life), as well as in the ventral network. They hypothesized that this training produced changes in the attentional networks, leading to improvement in other processes. Some other studies in the adult population have tried to demonstrate the transfer effects of cognitive training though online platforms. Hardy et al. (2015)

in a randomized controlled trial with a considerable sample (N = 4,715 fully evaluable participants) divided into two groups: CT group (general cognitive training through 49 games of the Luminosity online platform) and active control group (they completed crosswords puzzles) showed transfer effects. After training conducted at home (15 min per day over 5 days per week for 10 weeks), the cognitive training group showed greater improvements than controls in speed of processing, short-term memory, WM, problem solving, and fluid reasoning assessments, and greater improvements in self-reported measures of cognitive functioning, particularly in concentration compared to the control group, which could be considered as an ecological benefit of training. Nevertheless, the results of Hardy et al.'s study must be considered carefully because instruments of cognitive assessments, while based on other known tests, are part of the Luminosity framework.

Studies on typically-developing children also support the idea of near and far transfer of CT. Karbach and Kray (2009) aim to dilute the effectiveness of training cognitive flexibility through shifting tasks and its transfer to another untrained area. For this purpose, they conducted a trial using children (aged 7 to 9) and elderly people. The results showed that with only four training sessions of shifting (flexibility) tasks, positive results in the two types of transfer, NT and FT, were found in the trained group in inhibition, verbal, and visual WM and reasoning. In 9-yearold, typically developed children, Jaeggi et al. (2008) suggested that the transfer of the training program (WM training over fluid intelligence) depends on the gains obtained in the training: those that improved their performance notably in the trained task (an n-back, WM task - giving a response to a given sequence in a go/no go task) obtained better scores on intelligence tests [Test of Nonverbal Intelligence (TONI) and Raven's Standard Progressive and Matrices (SPM)], which suggests that good performance in CT leads to FT. In adolescents, Zinke et al. (2012), conducted a randomized controlled study with children aged 10–14 years, comparing the effectiveness of CT (task switching based on that used by Karbach and Kray, 2009), with the addition of physical exercise. In addition to evaluating transfer in similar tasks, they observed transfer to other untrained areas (inhibition, WM, and processing speed), concluding that both groups throughout the sessions significantly reduced the cost of change (time it takes them to shift set), as well as the number of errors (NT). They also improved WM and processing speed (FT).

In children with special educational needs, another study has found FT and long-term effects in children with brain damage. Galbiati et al. (2009) conducted a controlled trial of 6–18-year-old patients with severe brain damage which produced attentional deficits. The experimental group received BT stimulation in laboratory conditions consisting of 45-min sessions, 4 times per week for 6 months using three BT programs targeting attention ("Tabletop," "Rehacom," and "Attenzione e Concentrazione"). The results demonstrated significant differences in the trained group compared to controls in sustained attention and selective attention (they maintained attention longer and produced fewer omissions). In parental reports, those who were trained showed improvement in communication, daily life skills, and social skills; and those results were maintained at follow-up (12 months after intervention). In children with a low socioeconomic level (aged 7–8 years old), a combination of commercially available cognitive games and BT video game-like products (e.g., Rush hour, Professor Brainium's Games among others) have shown benefits in reasoning and processing speed (Mackey et al., 2011). In children with ADHD, many CT studies have been conducted, some of which seem to be effective in terms of NT and others in terms of FT. Kray et al. (2012), in a randomized trial, demonstrate that a relatively short cognitive training intervention (four training sessions in task shifting) on children aged 7–12 years with ADHD (medicated with methylphenidate), improved processes of inhibition and WM (components of executive function), but not fluid intelligence. Here we see lack of FT. In contrast, a randomized controlled trial (with children aged 6– 18 years with ADHD) concluded that neurofeedback (NF), a type of CT, could be as effective as methylphenidate for treating the attentional and hyperactivity symptoms of ADHD, based on parental reports (Duric et al., 2012). According to Karbach and Unger (2014), the research on CT and ADHD seems to indicate that this training can compensate for deficits in executive functions (EF) and therefore improve school skills. Although this result has not been observed in all studies, this does not mean that the positive results are not encouraging. NF can be effective in relation to the improvement of EF, a key aspect of school performance (Illes and Sahakian, 2011).

# Neuroplasticity

Most BT programs claim to be based on brain neuroplasticity: the capacity that neurons have to modify their synaptic structures and form new neural connections (Pressler et al., 2011). There are studies that connect the practice of a certain activity to an increase in gray matter volume in the areas related to this activity. In a study in which adult participants learned to juggle, Driemeyer et al. (2008) concluded that changes in the gray matter can occur even after 1 week of training in a task; similar results were found by Scholz et al. (2009). Focusing on our area, to study neuroplasticity due to CT, researchers have focused especially on gray matter and neural activity changes. Some researchers, and especially BT developers, often relate changes in cognitive skills to neuroplasticity. Rabipour and Raz (2012) claim that due to brain plasticity, BT can alter attentional networks in the brain, and thus improve certain skills. In our view, to properly justify an association between cognitive skill improvements after training and neuroplasticity, neuroimaging techniques should be included in the trials.

In adults, studies focused on working memory (WM) training, such as Takeuchi et al. (2011) using a randomized controlled trial with young adults, demonstrated that a BT intervention, intensive adaptive training of WM using mental calculations (IATWMMC) was associated with a decrease in regional gray matter volume in the bilateral frontoparietal regions and the left superior temporal gyrus (neuroplasticity), and also with cognitive performance improving verbal letter span and complex arithmetic ability (transfer effect). Another study also found gray matter differences after undertaking WM training: in their pseudorandomized controlled trial, Caeyenberghs et al. (2016) studied a typical sample aged between 19 and 40 years, divided into two groups. The adaptive group trained WM at home using a Cogmed program (a computer-based program which aimed at WM and adjusted to user level, for 8 weeks with 45 min in each session, 40 sessions in total) vs. a non-adaptive group (training not adjusted to user level). Before and after training, cognitive assessment was applied, as well as white matter imaging techniques [diffusion tensor imaging (DTI)]. The results showed improvement in the adaptive group, not only in cognitive measures such as WM span, reasoning, and inhibition, but also changes in global integration based on white matter connectivity within a frontoparietal attention network. Another study with a similar design, related adaptive cognitive training to some changes in thickness of cortical structures (Metzler-Baddeley et al., 2016). In their pseudo-randomized study, an adult sample was divided into two groups: an active control group (who received training with no user-level adjustment) vs. adaptive training (for whom training was adapted to user-level performance); both groups trained using the Cogmed program. After training, neural changes were observed as increases in cortical thickness in some brain areas (right-lateralized executive regions) as well as reductions in others (such as the left pallidum). They related these changes in the brain to cognitive performance in near transfer assessment. These results support the idea of neuroplasticity due to a BT intervention. Apart from gray matter differences reflecting neuroplasticity due to CT, brain activity has been studied with the same purpose by means of the fMRI technique. Westerberg and Klingberg (2007) conducted a trial with three young healthy adults. Brain activity was measured on two separate days with fMRI: before practice and one day after practice of a WM task (Cogmed program). fMRI was also conducted during WM task performance. After training, WM-related brain activity was significantly increased in the middle and inferior frontal gyrus. Whereas this study provides data to support neuroplasticity, it lacks transfer evidence to other cognitive skills. With the same technique, fMRI, Clemens et al. (2013), through a randomized, controlled study of young adults and showed that some brain areas were commonly activated for alertness and focus attention training (participants trained attention through Cogniplus: four sessions of alertness or four sessions of focus attention training). Moreover, BT and assessment activated common neural areas described in the literature. These data support neuroplasticity, but there is no evidence of any transfer effect to other cognitions or behavior.

Having established a connection between neuroplasticity and BT in adults, we must question whether a similar result may also be found in children and adolescents, whose brain functioning differs due to developmental factors. In the following results section, we will mention certain studies that have proven neuroplasticity through brain activation changes in the following areas: dyslexia in which BT produces changes in language skills as well as changes in brain activation (observed by fMRI) in areas that are normally activated during performance of linguistic tasks, as well as in compensatory areas (Temple et al., 2003); cancer survivors, BT has also shown reduction in the activation of areas related to WM and attention apart from improvements in cognitive skills (Conklin et al., 2015) and increased brain activation in some areas of the prefrontal cortex (Kesler et al., 2011a); using the same technique with ADHD children-teenagers, Stevens et al. (2016) found that, apart from effects on behavior, responsiveness of WM frontoparietal circuits and executive process-specific WM brain regions were altered by training. In Turner syndrome patients, apart from cognitive improvements, it seems that after treatment (Luminosity), bilateral parietal lobe activation increased and frontal-striatal and medial temporal activation decreased in the math task (Kesler et al., 2011b). Using MEG with typically-developing children, Barnes et al. (2016) showed how WM training (Cogmed) impacts networks in the brain related to this function (inferior temporal and frontoparietal cortex). The magnitude of taskrelated patterns of brain activity was significantly associated with previous findings observed in resting-state activity (Astle et al., 2015). Studies using EEG techniques, such as Johnstone et al. (2017) with children with ADHD, showed how neurofeedback (NF), a type of CT, can produce brain activity changes, indicating normalization of atypical EEG features with reduced delta and increased alpha activity after training. In adolescents with multiple sclerosis, Hubacher et al. (2015) found that performance gains after cognitive training (attention and WM training through the BrainStim program) were accompanied by increased activity in the WM network and changes in inter-network connectivity (fMRI). Taking this into account, we must ask ourselves what types of BT engender neuroplasticity and whether neuroplasticity produces some observational effects in cognition and behavior.

# BT Current Limitations

Despite this background, other researchers highlight the lack of evidence of FT in many BT products (Cortese et al., 2015). Despite the increasing popularity of these training tools, Karbach and Unger (2014) claim that their results are neither robust nor consistent, and the transferability of training-induced performance improvements to untrained tasks seems limited. It must be considered that if learning is specific to the trained ability, as is often the case with BT programs, there is little generalization in relation to related tasks in new environments, limiting the practical impact of such learning. It may be the case that other activities, such as video games, music, and athletic training, show a more reasonable generalized effect (Green and Bavelier, 2008). What is essential for BT products is to establish clear cognitive targets that may have an impact on the user's daily live. Therefore, for many BT programs, FT is more difficult to prove than NT (Simons et al., 2016), not only in clinical populations, but also in a healthy or typically-developing populations. Supporting this concept, a large randomized controlled online study with 11,430 participants aged 18–69 years using a BT program (a BT tool designed by BBC Lab UK to improve reasoning, memory, planning, visuospatial skills, and attention) did not show any transfer effect in untrained tasks, even if they were parallel to the trained ones (Owen et al., 2010). These limitations are commonly found in research both with adults and with children. An example of these limitations may be seen in the study by Roberts et al. (2016). These authors studied the impact of WM training (Cogmed program) on WM skills and academic outcomes (reading, math, and spelling scores as primary outcomes) in children aged 6–7 years with low WM. WM training had an impact on the 4 short-term and WM outcomes, but had no impact on academic outcome (FT), which means that only NT was found and that some of these training effect did not maintain benefits over time. Another study with children with low WM scores (Ang et al., 2015) showed that training, whether updating training or Cogmed training, did not have FT on math, and NT it was not lasting in the long term. Another limitation of computer-based interventions is that in the short term they often produce improvements in the trained processes (NT), however, there are difficulties in interpreting data because of study design limitations (e.g., lack of a control group), which restricts the possible interpretations of the results, and they usually do not show improvement maintenance beyond 6 months (Rabipour and Raz, 2012). In a review of 10 randomized controlled trials with older people, the authors concluded that apart from a limited transfer effect, there is a lack of sufficiently follow-up periods to validate long-term effects and a lack of active control groups in the research designs (Papp et al., 2009). Another common limitation seems to be sustainable effects. For these reasons, an updated review of BT research in children and adolescents is required, as well as a proper classification of available programs considering their scientific background for practical reasons. The objective of this paper is to classify BT products available for children and teenagers according to research found using BT as an independent variable and analyzing its effects in terms of neuroplasticity, NT, FT, and long-term effects.

# METHOD

# Inclusion Criteria

Studies from psychological sciences and neuroscience were reviewed and then included or rejected based on their relevance. First, a study was considered relevant for our research if it was based on empirical data from the use of a BT program (as an independent variable not combined with other BT products) with children or adolescents (4–17.9 years old) and its effects on NT and FT and/or neuroplasticity. Feasibility, compliance, acceptability, or factors to better benefit BT studies were not included. Second, the use of BT had to be for cognitive training purposes (motor skill training or emotional competence training were excluded). Third, the training described in the article must be commercially available. Fourth, this paper takes into consideration computer-based programs for children developed since the 1990s. Finally, the selection was limited to include only printed and peer-reviewed material, such as articles in journals, edited books, and research reports.

# Search Terms and Databases

Searches were conducted from June 2015 until March 2017 with the filters: English, Humans, in the following electronic database: PubMed. Heading searches for the following areas were combined:

Search 1: 7648 results

(1) Cognitive training


#### Search 3: 2,589 results


Search 4: 60 results


Searches on CT products websites were also conducted to screen commercially available products as well as to screen any other published research (available in Pubmed but not found in our database searches). Following the inclusion criteria, 70 articles were included in the results.

The selection flow diagram is shown in **Figure 1**.

# Method of Analysis

Qualitative analysis was performed in this review. We established the following parameters to properly classify programs: Neuroplasticity, NT and FT, long-term effects, and study design.

In a first step, the different articles were read in order to determine whether they contained relevant information and whether they fulfilled the inclusion criteria. In a second step, for each selected article, the following information was extracted and entered into a table: study design, population and results (see **Tables 1**, **2**). The information provided by the different studies was compared in order to explore program efficacy (see **Table 3**) and gaps or the future direction of BT research was included in the discussion section.

# RESULTS

After searching for results, we selected 70 articles which met with inclusion criteria. Then, we classified different commercially available BT programs for children according to their scientific background. **Tables 1**, **2** summarizes the main research of programs selected for this article.

We have classified BT programs as follows: (1) products supported by neuroscience research: computer-based programs in which neuroimaging techniques, such as fMRI, MEG, EEG etc., have been applied to prove program impact in terms of neuroplasticity; (2) Products derived from experimental and quasi-experimental trials: computerbased programs in which psychometric tests have been applied to test program impact. Finally, to further clarify the scientific validity of the programs, we have taken into

account Mahncke and Merzenich's (2015) considerations about how to evaluate a BT program. This consideration includes questions related to program efficacy, study design, and longterm effects. Based on these criteria, **Table 3** summarizes the scientific validity of the programs mentioned in the present paper.

# Products Supported by Neuroscience Research

In this section, we included computer-based programs aimed at research which use neuroimaging techniques such as fMRI, MEG, EEG, DNA analysis etc., to prove program impact under neuroplasticity parameters.

**Table 1** shows a summary of characteristics of each research based on the aforementioned programs.

# Fast ForWord® (FFW)

This program is supported by independent research based on neuroimaging techniques for dyslexia (Temple et al., 2003). These authors have shown that people with dyslexia show dysfunction in phonological processing. FFW was applied to children with dyslexia (divided into an experimental and control group); after an average of 27.9 training days (100 min in 5 sessions per week), participants showed improvements in reading and oral language, pseudo-word decoding and comprehension, as well as changes in brain activation (observed by fMRI) in areas that are normally activated during performance of phonological tasks as well as in compensatory areas (left temporoparietal regions, left frontal inferior rotation, right hemisphere temporal and frontal regions, and the anterior cingulate gyrus). This suggests that this program alleviates dysfunctions associated with phonological processing, as well as producing compensatory activation in other areas.

TABLE 1 | Products supported by neuroscience research.


#### TABLE 2 | Products derived from experimental and quasi-experimental trials.


Year release

#### TABLE 2 | Continued

Product name



(Continued)

Year release Studies on children

Foy and Mann, 2014

Bergman-Nutley and

#### TABLE 2 | Continued

Product name



Rossignoli-Palomeque et al. BT on Children and Adolescent

(Continued)

Year release

#### TABLE 2 | Continued

Product name



#### TABLE 2 | Continued


TABLE 3 | Scientific validity of Brain training programs for children based on Mahncke and Merzenich (2015).


#### Teach-the-Brain

This program is based on independent research in neuroimaging techniques that measure brain activity though EEG (Rueda et al., 2005). It shows that 4–6-year-olds can improve EF and even intelligence quotient (IQ) after only 5 days of BT (with the aim of training the three attentional networks proposed by Posner and Petersen, 1990). They evaluate this evolution with EEG and psychologically-validated tests (Child ANT, Kaufman's brief intelligence test) and parent questionnaires, and conclude that, despite the genetic load on attention and executive functions, training produces improvements in these skills.

#### Cogmed

This program implements research based on neuroimaging techniques which measure brain activity in adults through fMRI (Westerberg and Klingberg, 2007), EEG (Liu et al., 2016), and DTI (Caeyenberghs et al., 2016). Specifically, for children, there are research studies that use neuroimaging techniques such as MEG (Astle et al., 2015; Barnes et al., 2016), fMRI (Conklin et al., 2015; Stevens et al., 2016), and DNA genotype (Söderqvist et al., 2012).

Astle et al. (2015) wanted to figure out whether WM training had an impact on brain connectivity at rest in those areas typically associated with WM and controlled attention as well as in cognitive tests. Typically developed children, aged 8–11 years, completed 20 sessions of computerized WM training at home. Before and after the training, all of the children underwent a 9-min resting state (MEG) scan and completed standardized assessments of short-term and WM. The results showed that the adaptive group (in which the training was adapted to user execution) demonstrated significant improvements in standardized scores in the untrained shortterm and WM assessments. Adaptive training enhanced resting functional connectivity: significant enhancement of connectivity was found in the bilateral frontoparietal network, superior parietal cortex, and a portion of inferior temporal cortex. Moreover, connectivity changes associated with training were greatest in those who displayed the greatest improvement in WM capacity.

Using MEG, Barnes et al. (2016), showed how this CT program impacted networks in the brain related to WM, specifically on frontoparietal and temporal brain structures. In this study on typically developed children, WM training involved at least 20 training sessions (35 min) for 4–6 weeks at home. WM task-related MEG data were collected before and after the training intervention. After the intervention, researchers identified "significantly increased cross-frequency phase amplitude coupling in children who completed training, specifically between the upper alpha rhythm (at 16 Hz), recorded in superior frontal and parietal cortex with high gamma activity (at ∼90 Hz) in inferior temporal cortex" (Barnes et al., 2016 p. 1). Thus, it seems that BT can modulate brain waves. The authors associated this altered neural network activity with cognitive skill enhancement. Furthermore, the magnitude of task-related coupling found in this study (as a pattern of brain activity) is significantly associated with previous findings observed in resting-state activity (Astle et al., 2015). In addition, the results showed that changes in frontoparietal to inferior temporal phase amplitude coupling were significantly predictive of children's improved performance in the WM task; in this case, there is evidence of a relationship between neuroplasticity and cognitive performance.

Through the fMRI technique, Stevens et al. (2016) conducted controlled trials comparing18 children with ADHD to 18 control subjects aged 12–18 years. After training (standard Cogmed protocol: 5 weeks and 25 sessions with 30–40 min per session), the trained group showed some NT and FT (less ADHD clinical symptoms reported by parents). The responsiveness of both WM frontoparietal circuits and executive process-specific WM brain regions was altered by WM training. Within the same neuroimaging technique, Conklin et al. (2015), in a randomized controlled trial on children survivors of cancer, proved that Cogmed training affects cognition and brain activity (5–9 weeks with 25 sessions of 30–40 min at home). After training, NT was found in WM and FT (attention and processing speed) as well as brain activity changes: reduction in activation of left lateral prefrontal and bilateral medial frontal areas related to WM and attention.

Finally, we found a DNA genotype study (Söderqvist et al., 2012) which examined the effects of polymorphisms in five genes involved in dopaminergic pathways after CT: WM training, Non-verbal reasoning training (NVR) or a combination, in preschoolers though a pseudorandomized controlled trial. They conducted 25 sessions of 15 min per day at home. WM training produced NT, and NVR produced gains in fluid intelligence. With regard to neuroplasticity, the authors found that polymorphisms of the DAT1 gene were associated with training effects: variation in the dopamine transporter gene (DAT1) influenced improvements in WM and fluid intelligence.

### WinABC Program

WinABC is a computer-based program developed to improved literacy skills, supported by a study which supports NT and neuroplasticity in children with dyslexia (Penolazzi et al., 2010). In their study, 11 children with dyslexia aged 9–11 years received 6 months of phonological training at home (5 times a week for 10 min per day). Besides NT, the authors found that those children who had the greatest reading speed enhancement showed the largest left posterior EEG beta power increase in phonological task execution after the training sessions. Nevertheless, as this study is an intervention study (not controlled), the result must be considered with caution.

### Luminosity

This program is based on research on neuroimaging techniques using fMRI in children with cancer or Turner syndrome (Kesler et al., 2011a,b), as well as EEG studies in adults (Schneider et al., 2013).

This program was found to be effective in training EF with children who have suffered cancer. Kesler et al. (2011a) designed a home cognitive training program (8 weeks of intervention/5 session per week/20 min per session). Not only cognitive assessment at baseline and post intervention were applied, but also fMRI measures were made. Following the cognitive intervention, participants showed a significant increase in processing speed, cognitive flexibility, verbal, and visual declarative memory scores, as well as a significant increase in pre-frontal cortex activation compared to the baseline (inferior, middle, and superior frontal gyrus activation). Nevertheless, in this study there was no correlation between cognitive scores at post-intervention and brain activation in fMRI.

Luminosity seems to be effective for children with Turner syndrome who have low math abilities. Kesler et al. (2011b) assess some mathematical skills and other involved mental processes (processing speed, attention, cognitive flexibility) as well as brain activation before and 1 week after training. The training consists of an adaptive BT program focused on number sense and general problem-solving skills (5 sessions/6 weeks/20 min per session, at home). After training, the participants significantly improved their basic math skills, including number sense and calculation, as well as processing speed, cognitive flexibility, and visual-spatial processing skills. In terms of brain activation, the participants showed significantly increased bilateral parietal lobe activation and decreased frontal-striatal and mesial temporal activation in math tasks. Nevertheless, it must be considered that a controlled randomized study in this field would contribute to contrasting or supporting this study which lacks a randomized controlled design.

### Focus Pocus

Focus Pocus is one of the BT programs based on neurofeedback (NF). NF is a process of learning in which the user is rewarded for positive brain activation modulation (Fox et al., 2005). The training consists in modulating brain waves to achieve a series of goals within a computer game. This program is based on empirical research using EEG records to demonstrate neuroplasticity due to training. Johnstone et al. (2017), in a controlled randomized study, showed how neurofeedback training (at home) can produce brain activity changes, indicating normalization of atypical EEG features with reduced delta and increased alpha activity after training in children with ADHD.

# Products Derived From Experimental and Quasi-Experimental Trials

In this section, we include computer-based programs based on research using psychometric testing to evaluate program impact. Some of them have been included in the first section such as Cogmed.

**Table 2** shows a summary of characteristics of each research project based on the different programs mentioned above.

### BrainTrain

Some randomized controlled studies have also been conducted using BrainTrain products (such as Capitain's Log) with ADHD children (Rabiner et al., 2010; Steiner et al., 2011, 2014).

A combination of CT with other techniques could also be of interest for children with ADHD symptoms (Rabiner et al., 2010). Cognitive training ("Capitain's Log") and computer intervention that facilitates the understanding of instructions, or "Computerassisted instruction," entails a decrease in ADHD symptoms in the classroom, especially for those who initially showed more symptoms of inattention, after 28 sessions of 75 min with first grade children. Steiner et al. (2011) showed the effectiveness of two neuroscientific interventions in children with ADHD; a neurofeedback program ("Play attention") and a computerized CT program ("Brain Train/Captain's Log"). After an average of 23.4 sessions in their schools, the parents reported a significantly greater improvement in symptoms associated with this disorder than in the control group. In subsequent studies, the same authors demonstrated that the effects were maintained at a 6 month follow-up (Steiner et al., 2014).

Finally, La Marca and O'Connor (2016) tried to determine whether neurofeedback training ("SmartMind Pro") is effective at improving not only attention and executive functions, but also reading comprehension and fluency in children with ADHD Inattentive Subtype. The participants followed 40 NF sessions in a school environment and three measurements of each were obtained: baseline, post-test, and 5-month follow-up. The results showed that following the intervention, improvements were observed in a continuous performance test and a shifting attention task. The results obtained from reading fluency tests revealed little change, although participants demonstrated gains in reading comprehension. In this case, it would be interesting to conduct a randomized controlled trial that included attentional measures, in order to support their findings.

#### Cogmed

A study of typically-developing 4–5-year-old children was conducted by Thorell et al. (2009). The sample was divided into three groups: a group that received training in visuospatial WM (from Cogmed), another group that received inhibition training (through a go/no-go task), and a third, passive control group. After 5 weeks of training (they attended 15-min sessions each day), the children who received WM training improved significantly in non-trained visuospatial WM tasks, as well as in attention tasks (the children who were trained in inhibition did not display significant improvements in untrained tasks). In this case, Cogmed seemed to be effective for typically-developing children aged 4–5 years in terms of NT. In another study with typically developed children of the same age, Bergman-Nutley et al. (2011) demonstrated that Cogmed was effective for training WM in this population. First graders may also receive some benefits from CT (Fälth et al., 2016). In their study, children who received WM training (Cogmed standard protocol) showed significant improvements in a word decoding test compared to the control group. The implication is that there is a WM requirement for initial readers when the decoding process is not yet automatized, and the training was effective in improving this component. In another study with typically-developing children aged 9–11 years (Söderqvist and Nutley, 2015), it seems that WM training can have some FT on math and reading. An experimental group received 25 sessions for 20 min over 5 weeks at school, while a control group continued as usual. 12 months after training, the experimental group showed greater development in reading and math compared with a matched control group (maintained at a 2-year follow-up assessment). Furthermore, the progress in both math and reading in the trained group was directly related to the amount of improvement seen in the WM tasks. These results demonstrate transfer effects of training with a long-term effect. Nevertheless, these results must be considered with caution due to the non-independent nature of the study (the researchers have any kind of connection to the company or product). In children aged 9–16, Gibson et al. (2012) found that only the active maintenance of a limited amount of information in primary memory was improved by the program, however, no other WM components were improved. Finally, Hitchcock and Westwell (2017) compared WM training in children aged 12 years (adaptive vs. non-adaptive training) and passive control group, and did not find any transfer in task-related attention, reading, mathematics, or regulation of emotional, social, and behavioral challenges. It seems that studies on typically-developing children support evidence of NT (especially in preschoolers), yet there is no independent research to support FT for this population.

An early study of WM training effects on children with ADHD (Klingberg et al., 2002) showed that WM training produces improvements in trained capacities as well as reasoning, interference control and inhibition of motor skills after 5 weeks of training. Klingberg et al. (2005) showed that after training with the standard Cogmed protocol, the trained group obtained better results compared to the active control group in verbal WM, inhibition and abstract reasoning. Transfer in both studies is not only NT but also FT. However, these initial studies are not independent and therefore must be considered with caution. Another attempt to prove the benefits of behavioral ADHD symptoms (FT) through WM training has been conducted by Beck et al. (2010). In this controlled trial, the experimental group improved in the areas of inattention, the overall number of ADHD symptoms, initiation, planning, and WM as rated by parents. Teacher ratings approached significance at posttreatment and at a 4-month follow-up in the area of initiative. Green et al. (2012), in a doubleblind randomized controlled trial, showed that WM training through standard Cogmed protocol, reduced off-task ADHD associated behavior (distractions during performance of tasks). Other studies, such as Dahlin (2013), relate WM training to school performance in math for an experimental group that received the Cogmed standard protocol. Compared to controls, the experimental group improved significantly in WM tasks and in math results. However, because the sample was not randomized, the results should be taken with caution. Egeland et al. (2013) demonstrated the effectiveness of the Cogmed program in improving processing speed in children with ADHD as well as improvements in math and reading. The experimental group's scores (after undergoing Cogmed standard training) significantly increased compared to the control group in visual and auditory WM. A later study conducted by Bigorra et al. (2016) showed that an adaptive training group, compared to the non-adaptive training group, significantly improved in WM, EF (as rated by parents and teachers), reduced impulsivity and ADHD symptoms; and those gains were maintained at a 6-month follow-up. Holmes et al. (2010) compared medication treatment for ADHD with Cogmed training. The results demonstrated that WM training produced WM and central executive gains that were maintained 6 months after treatment; nevertheless, this is a comparative study (not controlled). Despite these results using the same program on children with ADHD, van der Donk et al. (2015), did not find FT. In their study, one group received 5 weeks of cognitive training and another received a "care in class" treatment developed for the research. They valued not only cognitive outcomes and academic performance but also behavioral aspects (including after 6 months of intervention). The authors concluded that CT produced improvements at a cognitive level (in the different tests), but not in academic performance or behavior. In the same way, Chacko et al. (2014) found that WM training (Cogmed) produced benefits in WM, but not in behavior and academic achievement (FT). Similar results were obtained by Dongen-Boomsma et al. (2014) who found only NT, and in this case, it did not survive correction for multiple testing. Gibson et al. (2011) found NT after WM training in adolescents with ADHD. They conceptualize WM in two aspects: (1) retention and maintenance of information during distractions, and, (2) recovering information from the secondary memory (SM). Likewise, in a later study (Gibson et al., 2012), after modifying the exercises included in the standard version of Cogmed-RM from simple span to complex span, they did not find benefits on SM which is typically impaired in children and

adolescents with ADHD. Their results showed WM training to be effective only for the first aspect of WM. In conclusion, there is some evidence to support Cogmed intervention in ADHD to obtain NT. FT results are controversial due to a lack of consistent findings, failures to replicate, and methodological limitations.

In children with low WM capability, Bergman-Nutley and Klingberg (2014) attempted to determine whether WM training (Cogmed standard protocol) could show FT on following instructions and arithmetic. They assessed WM (five times during and after training), following instructions and arithmetic using tests developed by Pearson and Cogmed. The training group improved significantly more than the control group in all three transfer tests. Using a regression model, transfer increased linearly with the amount of training time, and correlated with the amount of improvement on the trained tasks. It must be considered that this study is non-independent. Another study with low WM children aged 9–10 years was conducted by Holmes et al. (2009). The controlled trial results showed that adaptive WM training benefitted WM and mathematical reasoning, and those gains were maintained after 6 months. Holmes and Gathercole (2014), in a randomized controlled trial with children aged 8–11 with low academic achievement, showed that after WM training (Cogmed standard protocol conducted by teachers at school), WM, math and literacy improved. No follow-up was available. Along the same lines, Dunning et al. (2013) tried to demonstrate, through their randomized controlled study, the impact of CT (6 weeks of training) on WM, general intelligence, literacy and mathematics. The sample was divided in three groups (adaptive training, non-adaptive, and passive control group). The group who received adaptive training improved significantly in WM tests, maintaining this progress in visuospatial and verbal WM after 1 year. However, they did not obtain significant results in relation to the other groups in other cognitive areas (FT). In the same way, Ang et al. (2015) showed that training, whether updating training (seven computerized games were developed for the updating training: four games were based on the running span paradigm and three games were based on the keep track paradigm) or Cogmed training, did not show FT for math, and only NT which it was not maintained in the long term, beyond six months after training. Finally, the results of a study by Roberts et al. (2016) with low WM children demonstrated benefits in NT (only visuospatial short-term memory) which were maintained at 12 months. FT was not found in reading, spelling or math. In this population, robust findings supported NT and long-term effects (but not further than 6 months), while FT and longer-term effects were not replicated.

For children with low to moderate IQ, some partial benefits of training have been shown. A study with children with intellectual disability (IQ < 70), was conducted by Soderqvist et al. (2012): the sample was pseudorandomized in two groups (adaptive training vs. non-adaptive training) of WM (Cogmed standard protocol), and non-verbal training (NVR). 20 sessions were conducted at home (80% sample) or at school (20% sample). After training, the female participants showed improvement in instruction comprehension but not in other areas (reasoning, language, behavior rated by parents etc.) After a 1-year follow-up there were no significant improvements. It seems that individual differences compromised results: only female participants without an additional diagnosis and with higher baseline performance showed greater progress. In this sense, a minimum cognitive capacity seems necessary for the training to be beneficial, and a greater training time is required to reach sustainable training effects. Similar results were found in a pseudorandomized trial with children with low IQ (Soderqvist et al., 2012). A randomized controlled study on children with Down syndrome conducted by Bennett (Bennett et al., 2013) showed that WM training (Cogmed 10–16-week period at school; three times a week for 25 min per session), produces NT and the effects were maintained at a 4-month follow-up. Partanen et al. (2015) demonstrated in a randomized controlled trial that WM training in combination with metacognitive techniques produced a significant difference in WM maintained at a 6-month followup. No transfer to arithmetic or reading and writing skills occurred in any of the two training conditions. In this population, only Dahlin (2011) has found FT; a controlled trial showed that children trained in WM Cogmed standard protocol at school increased scores in reading comprehension, and those gains were maintained at a 7-month follow-up. Some variables, such as cognitive level in lower IQ children, might influence WM training effects, but few transfer benefits in WM and reading comprehension were found.

Focusing on children with language learning disabilities, Holmes et al. (2015) compared children diagnosed with Specific Language Impairment (SLI) to children with typical language performance. There was no control group and both groups received intervention. They took part in 20 sessions of 45-min over 8 weeks in small groups at school. The results showed that both groups improved their visuospatial short-term memory. However, the SLI group improved significantly more in one of two verbal STM measures (digit span). Exploratory analyses across the sample established that low verbal IQ scores were strongly and highly-specifically associated with greater gains in verbal span-like WM tasks, and those children with higher verbal IQs made greater gains in visuospatial STM following training. In another study, children with cochlear implants received the standard Cogmed protocol (Kronenberger et al., 2011). The researchers compared scores during wait time and training. After training, children demonstrated a significant improvement in measures of verbal and nonverbal WM, sentence-repetition skills and parent-reported working memory behavior. Sentence repetition continued to show marked improvement at a 6-month follow-up. In this area, randomized controlled trials would be crucial to replicate results.

A number of studies using Cogmed have been conducted with a population at risk of learning disabilities. On the one hand, some studies have focused on low birthweight or preterm children. Grunewaldt et al. (2013) conducted a stepped-wedge randomized trial with children aged 5–6 years who were born preterm. They showed that WM training (Cogmed JM version: 10–15 min per session for 5 days per week over 5 weeks at home) benefitted WM and auditory attention, phonological awareness, facial memory, narrative memory, spatial span, and sentence repetitions. There were no effects on anxiety reduction. Later, Grunewaldt et al. (2016) also studied the effects of WM training on children with the same characteristics. An experimental group received the standard Cogmed JM protocol at home. After training, some gains or equivalent scores as the control group were found in facial memory, narrative memory and spatial span, which remained at a 7-month follow-up. No group differences in performance gain were found for attention and behavior. It seems than FT to attention and behavior was not found in this case. A study conducted by Lee et al. (2016) on children aged 4– 6 years did not find NT in preterm and normal-term children in WM after training (Cogmed JM version), and also found no FT to other domains such as attention and executive functions. Finally, a controlled trial on adolescents conducted by Løhaugen et al. (2011) showed that after training (standard Cogmed protocol) gains in WM were produced and maintained after 6 months, yet, no FT was evidenced. In this population, NT and FT in memory has been demonstrated, nevertheless there have been no findings so far for attention or behavior. On the other hand, children with a low sociocultural level (SES) are also at risk of potential learning difficulties. Foy and Mann (2014) carried out a study in an attempt to prevent learning difficulties. Through a sample of children aged 4–5 years (pre-readers) with a low sociocultural level, they assessed whether WM training had some NT in WM, as well as FT on self-regulation and pre-literacy skills. For this purpose, one group received training in WM and another group did not receive any intervention. Their conclusions are that training favors the visuospatial memory of the trained children, as well as their self-regulation or executive control (assessed in inhibition tasks), but not on the prerequisites of literacy (e.g., phonological awareness or knowledge of letters). Another study on children with a low socioeconomic level was conducted by Mezzacappa and Buckner (2010). In this pilot study with a single group design, they compared WM and behavior (symptoms of ADHD before and after training as rated by teachers). After treatment, WM and behavior improved. Further research in this area is needed to provide more robust results.

Some researchers have focused on populations with different diseases such as cancer. Hardy et al. (2013) conducted a pilot study with child and teenage survivors of cancer. Immediately after treatment, the adaptive training group displayed significant improvements (not at follow-up) in their visual WM and in parent-rated learning problems, compared with those in the active control group. Conklin et al. (2017), in a randomized controlled trial with children aged 9–14 years, showed that after intervention, the trained group improved in WM, attention and processing speed. WM and processing speed gains were maintained at a 6-month follow-up. In this area, further research is required to better clarify the efficacy of Cogmed intervention. For children and adolescents with epilepsy, Kerr and Blackwell (2015) conducted a randomized controlled trial, the results of which showed that the trained group had significant post-interventive treatment effects for visual attention span, auditory WM, and visual-verbal WM (NT). Similar results were obtained by Fuentes and Kerr (2017), nevertheless FT (in fluid reasoning) was not observed. Indeed, further research is needed in this area to replicate results and to demonstrate the existence of any FT. Finally, in terms of brain damage, Eve et al. (2016) conducted a pilot study and a long-term followup with children who had suffered from an arterial ischemic stroke. They receive the standard Cogmed WM Training at home, supervised by their parents. Measures of WM, attention, and mathematical achievement were conducted before and after intervention, and at a 1-year follow-up. The results indicated that a significant improvement in phonological-loop WM was produced, however, this improvement was not maintained after 12 months. No additional significant improvements on standardized psychometric outcome measures were seen either immediately or at the 12-month follow-up. Phillips et al. (2016) compared adaptive vs. non-adaptive training in children with brain damage. The results demonstrated a significant difference in favor of the adaptive training group in WM and reading (reading comprehension and reading accuracy); the latter was maintained at a 3-month follow-up. However, no benefits were found in math. This finding may not support WM training for these patients; thus, further randomized controlled trials with children with brain damage would help to clarify this issue.

Finally, some studies have been conducted on children and adolescents with behavioral problems. Regarding children with externalizing behavior problems, Graziano and Hart (2016) conducted a randomized trial on preschoolers. In this study, the participants completed an 8-week intervention. They were allocated to one of three programs (STP-PreK = summer treatment program for pre-kindergarteners which involved BT (Cogmed), PT = parent training (parents were trained in some parenting techniques), and STP-PreK Enhance (which involved additional social skills, self-regulation strategies). The results suggested that, although all groups improved in behavioral functioning groups at a similar magnitude, children in the STP-PreK Enhanced group experienced greater growth over time. This group and STT-PREK maintained improvements at a 6-month follow-up in academic achievement, emotional knowledge, emotion regulation, and executive functioning compared to children with PT only. In children with behavior problems aged 11–13 years, Roughan and Hadwin (2011), in a randomized controlled trial, showed that the group trained in Cogmed (standard protocol) had better post-training scores in measures of IQ, inhibition, test anxiety, teacher-reported behavior, attention and emotional symptoms, compared with a non-intervention passive group; differences in WM were also evident at a 3-month follow-up. In adolescents with high scores on anxiety questionnaires, Hadwin and Richards (2016), in a randomized controlled trial, compared WM training (Cogmed standard protocol) vs. CBT intervention (small group activities on feelings, thoughts, relaxation techniques, problem solving, and coping strategies in small groups). After treatment, the WM training group showed significant gains in WM. Both groups reported fewer anxiety symptoms, demonstrated increased inhibitory control and a reduction in attentional biases to threat post intervention, and these results were maintained after 4 months. In children with behavioral problems, the results are encouraging for better regulation of behavior though cognitive training of WM.

#### Focus Pocus

This program, mentioned in section 1, is also supported by a study using psychometric tests to improve training efficacy. Johnstone et al. (2012) showed, in children with ADHD, that the combination of CT (Focus Pocus exercises) with and without neurofeedback, and compared to a passive control group, produced significant improvements in sustained attention, inhibition, WM, as well as a decrease in behavioraltype ADHD symptoms after 25 training sessions, as rated by parents. These results were maintained at follow-up (six months after intervention). As this is a non-independent research, the results must be considered with caution.

### Play Attention

Steiner et al. (2011) demonstrated the effectiveness of two neuroscientific interventions for children with ADHD disorder: an NF training program ("Play attention") and a computerized cognitive training program ("Brain Train/Captain's Log"). After an average of 23.4 sessions in their schools, parents reported an improvement in symptoms associated with this disorder which was significantly higher than that reported for the control group. In later studies, the same authors demonstrated that the effects were maintained at 6-month follow-up (Steiner et al., 2014).

#### Braingame Brian

This online platform is designed to train EF and was endorsed by a randomized, double-blind, placebo-controlled trial on children with ADHD aged 8–12 years (Dovis et al., 2015). The experimental group received 25 sessions of 30–35 min each. After training, the trained group significantly improved in EF trained skills (NT). No FT on behavior or long-term effects were found.

# ACTIVATETM

This online platform to train attention is supported by a study which tests NT (Bikic et al., 2015). In this randomized, controlled trial with children with ADHD (aged 6–13), the results showed that the trained group (40 min per day for 6 days per week over 8 weeks at home) displayed significant improvements in the primary outcome of attention. No long-term effect was confirmed.

#### SIGUEME Application

This application designed for autistic children is supported by a controlled study to test its efficacy. The study conducted by Vélez-Coto et al. (2017) involved the training of children using this application for 25 sessions of 10–15 min each. Following training, the results showed that the children improved in the areas of attention, association and categorization, and interaction (NT). Nevertheless, it must be considered that the assessment was designed by researchers.

#### TALI Attention Training Program

This program which aims to train attention is supported by a recent study on program efficacy in children with intellectual and developmental disabilities (Kirk et al., 2017). The children were randomly assigned to a training group or to a placebo control. The trained group received 25 sessions of 20 min. Although after training no significant effects were found, scores in numeracy increased at a 3-month follow-up. It must be considered that this study only assessed FT on academic achievement.

#### Rossignoli-Palomeque et al. BT on Children and Adolescent

# DISCUSSION

The present paper highlights and summarizes the current state of BT research focused on children in recent years. It also defines different commercially available BT programs for these children by type of method or research applied to test program efficacy. This summary should be particularly useful for psychologists, educators, and parents for practical purposes. A necessary consideration is that many BT programs are commercially available for children, yet the majority have not been endorsed by empirical research results. Here we attempt to provide a better understanding of which of these programs are supported by research, including their shortcomings and suggestions for future research.

BT or CT should attempt to produce some observable brain changes. As we have found, only a few BT products that are commercially available have empirical data that support evidence of neuroplasticity. Some BT programs have shown neuroplasticity using neuroimaging techniques such as FastForWord for children with dyslexia (Temple et al., 2003), Teach-The-Brain in typically-developing children (Rueda et al., 2005), Cogmed for typically-developing children (Söderqvist et al., 2012; Astle et al., 2015; Barnes et al., 2016), cancer survivors (Conklin et al., 2015), and for children with ADHD (Stevens et al., 2016), WinABC in children with dyslexia (Penolazzi et al., 2010), Luminosity in cancer survivors (Kesler et al., 2011b), and those with Turner syndrome (Kesler et al., 2011b), and Focus Pocus in children with ADHD (Johnstone et al., 2017). These suggestive neural changes are meant to reflect some improvement in cognition or behavior. Regarding FT, the results are more encouraging in the clinical population than for typically-developing children, however, due to the limitations of many of the studies, further research is required. Despite this, most BT programs claim to be based on neuroplasticity, yet, the majority are not supported by sufficient empirical research. Furthermore, confirming the existence of a relationship between neuroplasticity and transfer would provide more robust results in terms of program efficacy, because the relation between neural changes and improvements in cognition or behavior is still largely unexplored.

One of the challenges for BT is not only to produce NT (improvement in a task or skill similar to the one that was trained), but FT (improvement in an untrained task or skill which may produce some significant difference in the user's daily life). Several studies have shown transfer of different available programs and in different populations. Brain Train (Captain's Log) have shown NT in children with ADHD (La Marca and O'Connor, 2016) as well as FT for ADHD symptoms (Rabiner et al., 2010; Steiner et al., 2011), yet, no long-term effects have been found. Cogmed is supported by the largest number of research studies on children and BT. This program has been tested on typically-developing children, yet the ones showing positive NT and FT results in these populations are nonindependent research: NT in Pre-schoolers (Thorell et al., 2009) FT in word-decoding (Bergman-Nutley et al., 2011) and math and reading for children aged 9–11 years with long-term effects at 2 years (Söderqvist and Nutley, 2015). Despite this, independent research has found inconclusive results in children aged 9–16 years related to WM (Gibson et al., 2012) and in 12-year-olds with no transfer effects and no long-term effects (Hitchcock and Westwell, 2017). In this case, we may ask ourselves why should this program be used with general population when there is a lack of consistent results. On the other hand, Cogmed seems to have some benefits in children with ADHD: Cogmed has shown NT in ADHD or children with attention difficulties, as well as low WM (Gibson et al., 2011; Dunning et al., 2013; Hovik et al., 2013; Chacko et al., 2014; Dongen-Boomsma et al., 2014; Ang et al., 2015; van der Donk et al., 2015; Roberts et al., 2016), FT over inhibition and reasoning through nonindependent research (Klingberg et al., 2002, 2005) academic performance: math (Holmes et al., 2009; Dahlin, 2013; Holmes and Gathercole, 2014), math and reading (Egeland et al., 2013) on central EF(Holmes et al., 2010), EF(Bigorra et al., 2016), ADHD symptoms (Beck et al., 2010; Bigorra et al., 2016), and reduced off-task symptoms while performing tasks. (Green et al., 2012). Nevertheless, only a few of these studies have shown long-term effects on NT (Dunning et al., 2013; Hovik et al., 2013; van der Donk et al., 2015; Roberts et al., 2016), and on FT after 4 months (Beck et al., 2010), 6 months (Holmes et al., 2009; Bigorra et al., 2016), and 7–8 months (Dahlin, 2013; Egeland et al., 2013). It seems that the majority of studies do not demonstrate longterm effects of training. NT of Cogmed has been also shown in children with special needs (Partanen et al., 2015) with effects after 4 months (Bennett et al., 2013). Despite this, the authors of these studies did not find FT. Two studies have found FT on reading or instruction comprehension (Soderqvist et al., 2012) with long-term effects after 7 months (Dahlin, 2011). In children with language disabilities or hearing problems, there are two attempts to demonstrate the efficacy of Cogmed, however, the studies have not been properly randomized and controlled. NT has been shown to occur (Holmes et al., 2015) as well as some benefits over language skills related to WM, and was maintained at a 6-month follow-up (Kronenberger et al., 2011).

With regard to children at risk of developing learning difficulties, for children born preterm, a few studies have been conducted recently, especially on preschoolers, which showed NT and FT to some language skills related to WM (Grunewaldt et al., 2013) and FT to other domains related to WM, such as facial memory and narrative memory, which were preserved after 7 months of treatment (Grunewaldt et al., 2016). In the same population, Lee et al. (2016) only found the NT effect of Cogmed and no other effects on attention or behavior, mirroring the findings of previous authors. Finally, in adolescents, NT has been demonstrated and maintained after 7 months (Løhaugen et al., 2011), yet no FT has been provided. In children with low SES, there is evidence for NT (Mezzacappa and Buckner, 2010) as well as for FT on self-regulation and pre-literacy skills (Foy and Mann, 2014), yet no long-term effects were shown. Therefore, at this stage, the results for this at-risk group are inconclusive.

Diseases which may impact cognition have also come under the scope of WM training, such as cancer, epilepsy, and brain damage. The results for cancer patients seem to be inconclusive. Using samples within a wide age range from children to adolescents, NT was found by Conklin et al. (2015) as well as FT on processing speed and attention gains maintained at a 6-month follow-up; nevertheless, with a similar sample, Hardy et al. (2013) found NT and parental reports of fewer learning problems, but the results were not maintained at a 3-month follow-up. Furthermore, a wide age range has been studied for children with epilepsy and only NT has been found (Kerr and Blackwell, 2015) with maintenance after 3 months (Fuentes and Kerr, 2017). Finally, in terms of brain damage, only a few NT effects have been demonstrated in preteens and teens, yet, these were not maintained at 1-year post-intervention (Eve et al., 2016). Adaptive training is more effective than non-adaptive (as in previous findings). In a study by Phillips et al. (2016), adaptive training was shown to produce some benefits in reading (but not math) and was maintained after 3 months. In this last study, a passive control group should be added to better interpret results.

Finally, encouraging results have been found for children with behavioral problems, especially for teenagers and in combination with other techniques. Some results have shown NT at maintenance and at a 3-month follow-up, however gains of FT on IQ, inhibition, anxiety, attention and emotional symptoms were not maintained at follow-up (Roughan and Hadwin, 2011). Treatment combinations have yielded better results and maintenance, for instance, on preschoolers; using Cogmed in combination with other techniques (social skills, self-regulation strategies) benefits WM (NT) as well as other FT (academic achievement, emotion knowledge, emotion regulation, and executive functioning) maintained at 6 months (Graziano and Hart, 2016). In this case, as Cogmed is part of a wider treatment, we cannot directly attribute improvement in dependent variables to the program. Finally, it seems that Cogmed may be as beneficial as traditional treatment for teenagers (with a focus on anxiety reduction and self-control improvement), and demonstrated maintenance at a 4-month follow-up (Hadwin and Richards, 2016). Focus Pocus, apart from its neuroplasticity results in ADHD children (Johnstone et al., 2017), has demonstrated efficacy in FT on ADHD symptoms maintained after a 6-month intervention (Johnstone et al., 2012); nevertheless, those studies are non-independent and the results need replication in independent research. In another NF intervention, Play Attention has shown some FT on ADHD symptoms (Steiner et al., 2011) and long-term effects (6 months) on children ADHD (Steiner et al., 2014). Braingame Brian has shown NT in children with ADHD (Dovis et al., 2015), but not FT or long-term effects. As this platform is quite new, future research will be needed to clarify its benefits. The same may be said about ACTIVATETM where NT have been also found in ADHD children, but with no other results (Bikic et al., 2015). Finally, we have included two touchscreen intervention products: SIGUEME has shown positive results regarding NT with autistic children (Vélez-Coto et al., 2017). In contrast, for the TALI attention training program, another touchscreen intervention, the research provided only non-significant improvement in children with intellectual and developmental disability (Kirk et al., 2017).

A number of other programs have been supported by empirical research presented at professional conferences, and we hope to find further research and publications on these programs in future major scientific reviews. For instance, Arrowsmith, one of the best-known computer-based interventions for children with specials needs, is supported by an intervention trial conducted with children with learning disabilities, showing NT after treatment (Fitzer et al., 2014; Kubas et al., 2014). In this case, despite the fact that it has been on the market for several years, there is little evidence on it efficacy. Uno brain is supported by empirical research, presented in conferences, on an adult population (Fernández-Sánchez et al., 2013a,b) and on children with ADHD (Fernández-Sánchez et al., 2014). The results of this study seem encouraging because they report NT and FT over ADHD symptoms. Nevertheless, other well-known platforms and computer-based interventions, such as Cognifit, Brain Master, Happy Neuron, Neuron UP, Fit Brains, Sincrolab Kids, Gomins application, Beebrite Edu, Identifor, and the Nexxo application still lack published empirical research conducted with child populations. Independent randomized controlled trials with proper follow-ups will aid us to clarify the efficacy of these emerging computer-based interventions for children.

In general, we have found some limitations of commercially available BT products: (1) lack of scientific validity of many programs designed to train specific brain skills; (2) only 10 studies (14.2%) have been found to demonstrate neuroplasticity yet the majority of BT platforms claim to be based on these concepts without providing any scientific data; (3) only 36 of a total of 70 (51.4%) studies have shown FT, and, only 11 of them (15.7%) maintained FT at follow-up, which may lead to question the efficacy of BT products in the long term, and, finally, (4) lack of accessibility such as high prices, which make these products accessible to developed countries, but not worldwide.

Considering the methodological designs in the total of 70 published articles included in this review, we found: (1) fewer of half of them (30 or 42.8%) were randomized-controlled; (2) only 13 (18.6%) included an active control group and only 2 (2.9%) included 3 groups (experimental group/active control group/passive control group); (3) more than half of them, (38 or 54.3%) included follow-ups; (4) a double-blind design was not common, present in only 9 studies (12.9%); and finally (5), a minority of studies were non-independent (11 or15.7%). Considering the research limitations discovered, we consider that further research is needed to scientifically validate the new BT programs available on the market, through doubleblind randomized controlled trials, which include a passive control group and active control group, in addition to proper follow-up assessments. As we have seen, the majority of studies do not include an active control group and any follow-up beyond 6 months. Furthermore, a combination of neuroimaging techniques and psychometrical tools could be a robust method to demonstrate neuroplasticity and transfer effects to everyday life. For research designs we recommend that researches review criteria proposed by the IoM report (Mahncke and Merzenich, 2015) about how to evaluate a BT program. It is necessary to consider some study limitations such as sample sizes, lack of tasks to evaluate transfer (Cortese et al., 2015), as well as the individual differences of the participants and their motivations. Thus, some authors propose different study designs to test programs including micro-trials and single-case studies (Granic et al., 2014).

Having seen the limitations of many BT programs to produce FT and long-lasting effects, together with the methodological research limitations, a combination of treatments might potentially be more profitable; i.e., using BT as part of a wider treatment. Thus, programs which involve not only BT but also other strategies, thereby offering a treatment combination, may be more beneficial for some populations, such as children with behavioral problems, and produce more sustainable effects, as suggested by Graziano and Hart (2016), or in children with special needs, as indicated by Partanen et al. (2015). These findings support the idea that a combination of methods may be more profitable to implement and maintain cognitive and behavioral improvements over time. Future research should aim to clarify whether a combination of strategy implementation and programs would have a more significant and sustainable effect.

Despite finding the benefits of BT or a treatment combination, some authors remain unconvinced by the difficulties BT programs reported here (e.g., reaching FT and long-lasting effects), and claim that other activities that form part of children's natural environment, such as video games, music, and sports, show a more reasonable and generalized effect (Green and Bavelier, 2008). These authors emphasize that these activities are natural forms of training in which several skills are practiced in parallel. If there are common activities that foster children's skills, should BT be incorporated for typically-developing children? Is it necessary to use a BT program to improve cognitive skills in typically-developing children while there are other activities in their everyday lives that seem to benefit them as well? Why should we aim to improve children's abilities beyond usual child development?

The results obtained for child populations are controversial because there is a large proportion of non-independent research. Regarding neuroplasticity, independent research has yielded positive results (Rueda et al., 2005; Astle et al., 2015; Barnes et al., 2016), and on NT (Gibson et al., 2012; Hitchcock and Westwell, 2017) and FT (Fälth et al., 2016). Non-independent research has produced better results in these populations regarding transfer or long-term effects (Temple et al., 2003; Thorell et al., 2009; Bergman-Nutley et al., 2011; Söderqvist et al., 2012; Bergman-Nutley and Klingberg, 2014; Söderqvist and Nutley, 2015). Despite the fact that BT marketing is aimed at the general population, considering the results, we believe that BT research should contribute to validate programs as treatment tools for neurologically impaired patients, such as children with ADHD, learning disabilities, and behavioral problems. Further research is required to test the efficacy of BT and to ascertain for which populations it may be suitable, and what strategies can foster the efficacy and long-term effects of CT.

# REFERENCES

Ang, S. Y., Lee, K., Cheam, F., Poon, K., and Koh, J. (2015). Updating and working memory training: immediate improvement, longterm maintenance, and generalizability to non-trained tasks. J. Appl. Res. Mem. Cogn. 4, 121–128. doi: 10.1016/j.jarmac.2015.0 3.001

# Permission to Reuse and Copyright

Permission must be obtained for use of copyrighted material from other sources (including the web). Please note that it is compulsory to follow figure instructions.

# Tables

Tables should be inserted at the end of the manuscript. Tables must be provided in an editable format e.g., Word, Excel. Tables provided as jpeg/tiff files will not be accepted. Please note that very large tables (covering several pages) cannot be included in the final PDF for reasons of space. These tables will be published as Supplementary Material on the online article page at the time of acceptance. The author will be notified during the typesetting of the final article if this is the case.

# NOMENCLATURE

# Resource Identification Initiative

To take part in the Resource Identification Initiative, please use the corresponding catalog number and RRID in your current manuscript. For more information about the project and for steps on how to search for an RRID, please click here.

# Life Science Identifiers

Life Science Identifiers (LSIDs) for ZOOBANK registered names or nomenclatural acts should be listed in the manuscript before the keywords with the following format: urn:lsid:<Authority>:<Namespace>:<ObjectID>[:<Version>] For more information on LSIDs please see Inclusion of Zoological Nomenclature section of the guidelines.

Supplementary Material should be uploaded separately on submission, if there are Supplementary Figures, please include the caption in the same file as the figure. Supplementary Material templates can be found in the Frontiers Word Templates file.

Please see the Supplementary Material section of the Author guidelines for details on the different file types accepted.

# AUTHOR CONTRIBUTIONS

TR-P, EP-H, and JG-M: Conception and design of the review; TR-P: Searches, analysis, and data classification; EP-H and JG-M: Document review; TR-P: Writing of the paper.

# ACKNOWLEDGMENTS

Our sincere thanks to James F. Juola for his valuable comments and review of the manuscript.

Astle, D. E., Barnes, J. J., Baker, K., Colclough, G. L., and Woolrich, M. W. (2015). Cognitive training enhances intrinsic brain connectivity in childhood. J. Neurosci. 35, 6277–6283. doi: 10.1523/JNEUROSCI.4517-14.2015

Barnes, J. J., Nobre, A. C., Woolrich, M. W., Baker, K., and Astle, D. E. (2016). Training working memory in childhood enhances coupling between frontoparietal control network and task-related regions. J. Neurosci. 36, 9001–9011. doi: 10.1523/JNEUROSCI.0101-16.2016


randomized con-trolled trial. Am. J. Intellect. Dev. Disabil. 122, 97–117. doi: 10.1352/1944-7558-122.2.97


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Rossignoli-Palomeque, Perez-Hernandez and González-Marqués. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.