# PSYCHOLOGY, TECHNOLOGICAL INNOVATION, AND ENTREPRENEURSHIP

EDITED BY : Jesus de la Fuente, Douglas F. Kauffman and Unai Diaz-Orueta PUBLISHED IN : Frontiers in Psychology and Frontiers in Education

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-237-4 DOI 10.3389/978-2-88963-237-4

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: researchtopics@frontiersin.org

## PSYCHOLOGY, TECHNOLOGICAL INNOVATION, AND ENTREPRENEURSHIP

Topic Editors: Jesus de la Fuente, University of Navarra, Spain Douglas F. Kauffman, Medical University of the Americas – Nevis Devens, United States Unai Diaz-Orueta, Maynooth University, Ireland

This study was supported by R&D Project, ref. EDU2011-24805 and R&D Project ref. (PGC2018-094672-B-I00), Ministry of Science and Innovation (Spain); R&D Project UAL18-SEJ-DO31-A-FEDER, University of Almería (Spain), and with Federal Funds from the European Union.

Citation: de la Fuente, J., Kauffman, D. F., Diaz-Orueta, U., eds. (2019). Psychology, Technological Innovation, and Entrepreneurship. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-237-4

# Table of Contents


Rui Hu, Li Wang, Wei Zhang and Peng Bin


Manuel García Sedeño and Jose I. Navarro

*76 Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence*

Juan-José Navarro, Catalina Mourgues-Codern, Eduardo Guzmán, Isabel R. Rodríguez-Ortiz, Ricardo Conejo, Claudia Sánchez-Gutiérrez, Jesús de la Fuente, Diana Martella and Mahia Saracostti

*95 Technological Resources to Prevent Cyberbullying During Adolescence: The Cyberprogram 2.0 Program and the Cooperative Cybereduca 2.0 Videogame*

Maite Garaigordobil and Vanesa Martínez-Valderrey

*107 Characteristics, Structure, and Effects of an On-Line Tool for Improvement in Adolescents'* Competency for Interaction With Alcohol*: The e-ALADOTM Utility*

Jesús de la Fuente, Inmaculada Cubero, Francisco Javier Peralta, Mari Carmen Sánchez, Jose Luis Salmerón and Salvatore Fadda

*118 Does the Use of Learning Management Systems With Hypermedia Mean Improved Student Learning Outcomes?*

María Consuelo Sáiz-Manzanares, Raúl Marticorena-Sánchez, José Francisco Díez-Pastor and César Ignacio García-Osorio


Moon-Ho Ringo Ho, Marilyn A. Uy, Bianca N. Y. Kang and Kim-Yin Chan

*173 I'll Follow the Minority: The Effects of Sales Level on Purchase Intention of Self-expressive Products*

Xiaoyu Wan, Tingyi Wang and Jifei Wu

# Editorial: Psychology, Technological Innovation, and Entrepreneurship

Jesús de la Fuente1,2 \*, Douglas F. Kauffman<sup>3</sup> and Unai Díaz-Orueta<sup>4</sup>

<sup>1</sup> Educational & Psychology School, University of Navarra, Pamplona, Spain, <sup>2</sup> University of Almería, Almería, Spain, <sup>3</sup> Medical University of the Americas, Devens, MA, United States, <sup>4</sup> Department of Psychology, Maynooth University, Maynooth, Ireland

Keywords: psychology, R&D value chain, technological innovation, entrepreneurship, transference

**Editorial on the Research Topic**

#### **Psychology, Technological Innovation, and Entrepreneurship**

The aim of this Research Topic is to offer an integrated view of three areas for implementing Psychology as a science and as a profession, for the benefit of both the academic and professional sphere. An initial article offers a global analysis of the R&D&I value chain (de la Fuente et al.). Complementarily, several articles then provide examples of research on the characteristics of Innovation and Entrepreneurship, whether as a review (Sánchez-García et al.), an analysis of a personal factor that is predictive of this activity (Arco-Tirado et al.), the role of psychological characteristics (Hu et al.), and even a tool for assessing this construct (Cuesta et al.).

Other articles document evidence of technological development. In primary education, evidence is presented about an app for learning mathematics (Mera et al.) and about a technological tool for assessing reading competence (Navarro et al.). In secondary education, we find evidence of online prevention of cyberbullying in adolescence (Garaigordobil and Martínez-Valderrey), as well as the characteristics and structure of an online tool for preventing alcohol intake in adolescence (de la Fuente et al.). In university education, we witness the effects of using technological tools during learning (Sáez-Manzanares et al.) and the use of another technological tool for assessing stress in university students (de la Fuente et al.). Finally, pure research has been applied to the field of intracranial stimulation for musical perception (Sánchez-Kuhn et al.).

In conclusion, other studies show examples of transfer activities, the central vision of Leadership and Entrepreneurship (Palazzeschi et al.), the effects of training for this activity in the workplace (Ho et al.) and a market study (Wan et al.).

### AUTHOR CONTRIBUTIONS

JF had the idea and coordinated the Research Topic. DK and UD-O carried out support tasks for the coordination of the Research Topic and Edition of Articles.

### FUNDING

This study was supported by R&D Project, ref. EDU2011-24805 and R&D Project ref. (PGC201 8-094672-B-I00), Ministry of Science and Innovation (Spain); R&D Project UAL18-SEJ-DO3 1-A-FEDER, University of Almería (Spain), and with Federal Funds from the European Union.

**Conflict of Interest:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 de la Fuente, Kauffman and Díaz-Orueta. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

#### Approved by:

Lawrence Jun Zhang, University of Auckland, New Zealand

> \*Correspondence: Jesús de la Fuente jdlfuente@unav.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 23 August 2019 Accepted: 17 September 2019 Published: 10 October 2019

#### Citation:

de la Fuente J, Kauffman DF and Díaz-Orueta U (2019) Editorial: Psychology, Technological Innovation, and Entrepreneurship. Front. Psychol. 10:2232. doi: 10.3389/fpsyg.2019.02232

# Adapting the Research Development and Innovation (RD & I) Value Chain in Psychology to Educational Psychology Area

Jesús de la Fuente1,2 \*, Douglas Kauffman<sup>3</sup> , Unai Díaz-Orueta<sup>4</sup> and Yashu Kauffman<sup>5</sup>

<sup>1</sup> School of Psychology, University of Almería, Almería, Spain, <sup>2</sup> Facultad de Ciencias Sociales, Universidad Autónoma de Chile, Santiago, Chile, <sup>3</sup> Independent Educational Researcher, Bedford, MA, United States, <sup>4</sup> Department of Psychology, Maynooth University, Maynooth, Ireland, <sup>5</sup> Massachusetts Institute of Technology, Cambridge, MA, United States

Educational Psychology, as an area of Psychology that specializes in formative processes, faces several important challenges in the information and knowledge society of this twenty first century. One of these challenges is to facilitate a paradigm shift from a nearly exclusive focus on social science to the scientific-technological approach of a discipline that produces innovation and meaningful transfer of science and technology. The Research, Development, and Innovation (RD & I) value chain means pursuing these three endeavors in both the academic and professional lines of Educational Psychology. It is a strategy of innovation that leads us to integrate academic or research activity (R), research-related or professional scientific-technological development of innovation (D) and transfer and entrepreneurship activity (I). Generating innovation and transfer, applicable to educational contexts, can be an important stimulus of activity for new practicing psychologists in Educational Psychology. The RD & I value chain can become an academic, research-related or professional advantage in different activities, since it pertains to the processes, products and services found in the sphere of Educational Psychology. Several examples of how the RD & I chain can help improve psychoeducational activities are presented. First, we analyze competitive improvements that the RD & I chain can offer in competitive bids. Second, we give examples of the RD & I chain in the development of new processes, products and services in Projects of Innovation and Entrepreneurship in Educational Psychology, specifically illustrating the chain in each case. In order for this conception to take shape, a new cross-functional area must be created in professional and educational organizations. Specifically, this means creating an RD & I Department, or some area that branches across the other functions. The mission of this cross-functional unit is the actual implementation of the RD & I chain in the educational organization, as well as an incentive for innovative activities: use of ICT applications, organizational improvement, improved assessment, analysis of information produced by the organization itself, cost-benefit analysis, strategic decision-making processes, and so on.

Keywords: educational psychology, RD & I value chain, RD & I projects, RD & I department, innovation and entrepreneurship projects

#### Edited by:

Michael S. Dempsey, Boston University, United States

#### Reviewed by:

Barbara McCombs, University of Denver, United States Jesús Nicasio García Sánchez, Universidad de León, Spain

> \*Correspondence: Jesús de la Fuente jfuente@ual.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 02 March 2018 Accepted: 20 June 2018 Published: 24 August 2018

#### Citation:

de la Fuente J, Kauffman D, Díaz-Orueta U and Kauffman Y (2018) Adapting the Research Development and Innovation (RD & I) Value Chain in Psychology to Educational Psychology Area. Front. Psychol. 9:1188. doi: 10.3389/fpsyg.2018.01188

**6**

## INTRODUCTION

Psychology's connection with innovation is inherent in the study and analysis of human behavior. However, a commitment to innovative activity, to knowledge transfer in the sense found in other areas of science/technology, continues to be a pressing need. There are several reasons for this endeavor. Educational Psychology is a discipline that bridges Psychology and Education, and is closely linked to Social and Educational Sciences. The Social Sciences, however, have not traditionally been oriented toward innovation and scientific-technological knowledge. This situation must change if Educational Psychology is to be present among sciences and professions with Information and Communications Technology (ICT)-based innovations, positioned in the educational sphere, just as Psychology is already positioned in other fields of knowledge.

On the one hand, a paradigm shift must be encouraged, moving from the almost exclusive Social Sciences focus to a scientific-technological approach, characteristic of experimental and health-related disciplines that produce meaningful innovation and transfer in today's information and knowledge-based society. On the other hand, new generations of psychologists must begin to engage in experiences and formative processes in the Research & Development & Innovation (RD & I) value chain. The mid- and long-term results would be: (1) Better strategic positioning of the profession and its professionals in the information and knowledge-based society of the twenty first century; (2) Creation of competitive processes, products and services, with high innovative value; (3) New professionals specialized in the RD & I value chain (Voutsinas et al., 2015).

This proposal for innovation continues to be a challenge for academia and for the profession: that the organizations and institutions that carry out the tasks of Educational Psychology would be staffed with new positions based on a new set of professional qualifications.

### EVOLUTION OF RD & I VALUE CHAIN IN THE KNOWLEDGE SOCIETY

Educational Psychology, as the area of Psychology that studies formative processes, faces several important challenges in the twenty first century. First, it must help redefine formative processes in the context of an Information and Knowledge-based Society (Punye, 2007). Second, it must encourage reflection on developing competencies of innovation and entrepreneurship in this sphere of academic and professional knowledge. Research, Development and Innovation (RD & I) is a concept that has recently appeared in the context of science, technology and society, replacing the former "Research & Development" (R&D). While "Development" as a term comes from the world of economics, the terms "Research" and "Innovation" come from epistemology and from technology, respectively, and their dynamic relationship is found when differentiating between pure and applied sciences (Cardinal, 2001; Arimura et al., 2007). Each of these terms is complex to define. Aho (2008), provocatively defines "research" as investing money in order to obtain knowledge, while "innovation" would be investing knowledge in order to obtain money, expressing quite well the feedback phenomenon that is produced in a successful RD & I strategy (Edquist, 1997; Boons and Lüdeke-Freund, 2013). When applied in politics and legislation, the concept of RD & I defines (1)research as the original, planned inquiry that seeks new knowledge and better understanding in science and technology; (2) development of technological innovation as the application of research results, or of any other type of scientific knowledge, for the manufacture of new materials or products and for designing new processes and production systems, as well as for substantial technological improvement in pre-existing materials, products, processes and systems (Bernardino and Freitas, 2017); and (3) transfer of technological innovation (entrepreneurship) as the activity that results in technological progress in obtaining new products or production processes, or substantial improvement in those that already exist. Products and processes are considered new if their characteristics or applications, from a technological point of view, differ substantially from those already exinting (Pateli and Giaglis, 2005). Therefore, this sequence of actions has been called the RD & I value chain (Sanz and Cruz, 2009).

The level of RD & I activity in a country can be calculated as the ratio between RD & I spending and Gross Domestic Product (GDP), breaking down spending into public and private spending. To the extent that they are able, all countries attempt to encourage RD & I through support policies (subsidies, deductions, soft loans, etc.), since a high level of RD & I means stronger companies, whose products and processes stand out from their competitors'. Furthermore, many such activities can potentially bring about social progress in the form of quality of life, improving the environment, health, and the ecosystem. In order to support these activities, a number of UNE standards exist: the UNE 166000 series, including UNE standard 166001 that addresses RD & I projects, UNE 166002 on requirements for the RD & I management system, and UNE 166006, relative to system requirements for technology watch. This conception is not static, but in constant evolution, and directly affects the tasks and objectives of the Universities and researchers (FECYT, 2011). See **Table 1**.

### PROFESSIONAL IMPLICATIONS OF THE DIFFERENT CONCEPTIONS OF RESEARCH, TECHNOLOGICAL DEVELOPMENT AND INNOVATION IN ACADEMICS

### Consequences of the Classical Value Chain


TABLE 1 | Moments in R&D value chain.


knowledge, preferably focused on the production of research CV and not so much on cooperative social responsibility. This implies a limited production of patents, processes, products and services of innovation and, consequently, little entrepreneurship. In the case of the professional field, interventions are carried out without being based on scientific evidence, technological developments are carried out without foundation in science or prior scientific knowledge. Therefore, it is a period of clear disconnection between the academic and professional fields. See **Figure 2**.

### Consequences of the Current Value Chain


### THE NEW RD & I VALUE CHAIN IN EDUCATIONAL PSYCHOLOGY

The RD & I value chain in the sphere of Educational Psychology means recognizing that: (1) There is an important problem to be solved, preferably defined by professional practice, in reference to a process, a product or a service (demand or need); (2) Research actions and scientific production are to be carried out (Research); (3) Research actions should give rise to new technology developments in processes, products or services (Technological Development); (4) These developments are to be transferred and implemented in real contexts, producing innovation in professional practice (Transfer Innovation or Entrepreneurship). Thus, this heuristic provides for integration of scientific or research activity (R), professional technology development activity (D), and entrepreneurship and transfer innovation (I). In real life, the RD & I value chain must be constructed in an interdependent, coordinated fashion between the academic world (R) and the professional world (D&I). However, scientific production (R) does not always translate into technology developments (D), while in the best case scenario of professional practice, new technology tools (D) and innovations (I) are being developed, but without a clear connection to research-based scientific knowledge.

Essentially, the problem lies in an ongoing disconnection between the two contexts. Even though both the academic and professional perspectives seek to address the same problems, prospects and approaches, in many cases we find that the realities addressed are different and unconnected:


In today's Global Society of Knowledge, RD & I has become an engine of the economy, generating high-skilled, competitive employment in all production and service sectors. Given this panorama of the Science-Business System, an important shift

is taking place in the academic sphere in order to promote the value chain, and innovation as an agent for strategic positioning and job creation. For example, recent proposals established a new field regarding transfer of knowledge and innovation, to be included in applications for professional advancement. Entitled "Knowledge Transfer and Innovation,"

contributions in this field are valued in the following priority order:


Similarly, in the professional sphere, there should be a shift and a new perspective on the psychologist's practice as a player in RD & I, especially in professional innovation. Professional Associations should contribute significantly toward this end, with association policies that encourage and mediate RD & I—on the one hand, closely collaborating with the University, and on the other, responding to social demands, as ascertained by their professional members in their actual practice.

In the sphere of Educational Psychology, the RD & I value chain (de la Fuente and Vera, 2010) means adopting these three links as part of both academic and professional efforts in Educational Psychology. For this reason, the chain should be considered a powerful heuristic that makes it possible to integrate academic or research activity (R), professional development activity (D) and professional innovation (I). Generating scientific-technological transfer and innovation in Educational Psychology, in different educational contexts, can mean an important boost to the activity of new psychology practitioners. In the short- and mid-term progress, these results would follow: (1) Competitive positioning of psychology as a science and a professional practice, in educational contexts; (2) Production of new processes, products and innovative services in these areas; (3) Greater value given to the educational psychologist in educational contexts; (4) Creation of high-skilled jobs in this sphere.

The Psychologist -in general- and the Educational Psychologist -in particular- cannot escape this new economic and social context. The need for coordination in order to define RD & I actions must jointly concern the academic and the professional sphere. It means creating joint structures for coordinating activation of the RD & I chain, such as work being coordinated through technology-based businesses, joint definition of broad-spectrum priority research, creation of joint consortia, and creation of foundations for social purposes.

### A SCIENTIFIC-TECHNOLOGICAL COMPONENT OF EDUCATIONAL PSYCHOLOGY, FOR PRODUCING TECHNOLOGICAL INNOVATION AND TRANSFER OF INNOVATION

Researchers and practicioners in the field of Educational Psychology typically explore relationships among developmental, learning, and teaching processes that are produced in formal, non-formal and informal educational contexts,. These relationships have been substantially altered by contemporary changes and phenomena that make up a new educational panorama:


The idea of innovation and science/technology transfer has appeared only recently in our educational and production systems. Thus, we are currently in a process of assimilating and progressively working out this personal, social and epistemological reality. In fact, the psycho-educational variables that determine this reality are yet to be defined by research: personal variables that determine an innovative and entrepreneurial attitude, educational variables that strengthen habits of innovation in science and technology, the extent to which educational and curriculum experiences encourage the choice of science and technology careers, and the level of transfer of graduates into positions requiring scientific-technological skills.

Social sciences, on the other hand, have had little orientation toward innovation and science or technology transfer. In the social realm, educational work is considered to be a services activity, not connected to the production sector. This contributes to the idea that there is little need for generating new processes, products and services. Only the arrival of ICTs has brought about a significant move toward such innovation. To encourage the necessary paradigm shift that produces meaningful innovation and transfer in today's information- and knowledge-based society, we look to progress in Educational Psychology Research (R). Besides leading to new knowledge, it can produce new technological Developments (D) that then take shape in new processes, products and services (I) with direct application (Education & Psychology I+D+i, 2008).

In addition, we would begin experiences and training in the RD & I value chain with new generations of psychologists. The new Bachelor's and Master's degree programs are an opportunity to introduce training processes that establish innovation as an essential working tool. In particular, Master's degrees should work on the innovation and entrepreneurship competency, in each of the class subjects, as an essential process for decision making in academia and professional practice, so as to create new work opportunities and new positioning as a psychologist within the labor market (de la Fuente et al., 2012). Medium- and long-term results of such a philosophy, for the science and in the profession, would include:


This innovation proposal continues to be a challenge for academia and for the profession, in the interest of creating skilled positions with a new professional slant, in organizations and institutions that work across the entire domain of Educational Psychology. Toward this end, there is a pressing need for close collaboration between academics and practitioners:

(1) New conception of the task of research and professional practice.


### The News Conception of R & D & I Projects in Educational Psychology

In the context of the most recent vision to R & D & I value chain, the actual conception of R & D Project has three components (Sterlacchini, 2008): (1) Research, (2) Technological Development, and (3) Trasfer Innovation or Entrepreneurship:


### Components and Functions of an RD & I Department in Educational Psychology Organizations and Services

Unlike other professional fields where the RD & I Department is an unquestionable reality (cf. experimental science and technology), this idea has yet to be represented and developed in the professional sphere of the Social Sciences. Today's reality is increasingly competitive. If we want to be leading societies in the production of knowledge, products and services, we must not fail to adopt an innovative spirit. We are immersed in a scientifictechnological system where we are funding innovations that will be in the market 10 years from now. It is evident, then, that decisions made in the present will give shape to our future, and will or will not make us competent. For this reason we must not take a passive posture, but we must be active and adopt the changing trends of the Knowledge Society. Development of RD & I Departments can help us rise to the challenge of this context of change.

Although Psychology itself has coined expressions that would emphasize a scientific-technological viewpoint (for example, behavioral engineering), the reality is that few psychologists consider establishing RD & I Departments in the organizations where they exercise professional influence, whether educational or other. For this concept to materialize, a new, overarching area would be created in both professional institutions and educational organizations. Whether an RD & I Department as such, or some area that cuts across the others (de la Fuente, 2010), its mission includes the actual implementation of the RD & I chain in the educational organization, and incentivizing innovation activities (de la Fuente and Zapata, 2012). Such innovation would not only relate to teaching, but to innovation in different education and psychology programs, whether pertaining to the organization, assessment, analysis of information generated by the organization itself, cost-benefit analyses, strategic decision-making processes, etc.

The RD & I Dept. with its cross-cutting nature, ought to become central to the academic and professional practice of psychology in this century. In the case of the RD & I Department, its principal objective would be research support and professional support for different sectors of education or businesses involved in this field. Such support would contribute to the realization of individual projects or collaborative projects with national or international institutions, as well as facilitate access to possible funding sources. This department would offer up-to-date information about RD & I incentives and available assistance, in addition to supporting the phase of project definition and preparing applications for assistance from each of the different public RD & I programs. Similarly, it would facilitate the search for partners, in any geographic area, that are best suited to the project (universities, SMEs, users, etc.). With these issues in mind, we define the dimensions of RD & I for Educational and School Psychology. This proposal is articuled through the following Working Topics, as previosly descrived (de la Fuente and López, 2007; de la Fuente and Zapata, 2012). See **Table 2**.

This approach would have a number of consequences: (1) Demand for new professional profiles, for educational psychologists that specialize in research, technological development and applied innovation. (2) Exploitation of resources that are presently underutilized, such as the information generated by the organization itself. (3) New technological developments, in ICT formats, for assessment and intervention (Tavassoli and Carbonara, 2014). (4) Innovation in Educational Psychology becomes a reality (Caro-Vargas, 2017).

The RD & I value chain, in the shape of an RD & I Department in educational organizations, or any organization where psychologists are involved, would generate new professional activity, improve the practice of Educational Psychology and create demand for qualified professionals to fill these posts. We as psychologists have the conceptual, methodological and applied training in order to make this idea a reality (de la Fuente and Zapata, 2012).

### Technology-Based Companies (Spin-Off) as a Tool for Entrepreneurship and Transfer of Innovation in Educational Psychology

The RD & I value chain can mean an advantage to the different activities of academics, research, and professional practice, with respect to processes, products and services that are generated in the sphere of psychology and education. Several examples are presented as to how the RD & I chain can help to improve


TABLE

2


Subareas,

justification,

competencies

and

services

of

RD

&

I

Department

in

Educatinal

Psychology.

actions. The RD & I chain is exemplified in the development of new processes, products and services, in the technologybased business itself, as a practical example of the paradigm of innovation transfer and psychological entrepreneurship (Matlay, 2008; Schaltegger and Wagner, 2008; Pyka and Prettner, 2018).

The lack of an integrated RD & I value chain in the different scientific and professional tasks from the sphere of Educational Psychology has had several practical consequencies. First, it has given rise to excessive specialization in one link of the chain, focusing on one end or the other, and losing sight of the chain itself. Second, the different tasks (research, development, innovation) are represented in isolation and with unequal value. In a classic approach, the researcher who carries out projects sees no need to move on toward later technological developments or the transfer of his/her research to new innovative processes, products or services within the professional market. Similarly, the psychology practitioner is not always sensitive to effects that innovation can produce in professional activity, considering that research and scientific-technological development are far removed from his or her immediate professional demands, and the RD & I chain has little to do with his/her reality.

The classic conceptual representation of theory vs. practice has led to gaps in the relationship between the tasks of research (R), scientific-technological development (D), and professional innovation (I). Theory cannot always be prior to and disconnected from practice, nor vice versa. We should adopt the view that all Educational Psychology work is located along some point of this chain, and that should prompt us to coordinate with its other elements (NESTA, 2008). There are different examples of how the RD & I chain can help improve the quality of actions taken in Educational Psychology. In Spain, competitive improvements in the RD & I chain can be pursued through public bidding for Research Projects (D'Ambrosio et al., 2016).

### CONCLUSIONS

Not adopting the RD & I value chain concept may have severe practical consequences in scientific and professional tasks. In the first place, the focus is placed on one or another of the three endeavors, overlooking the chain itself. Second, the tasks of research, development and innovation are represented without any connection. In the classic style, the researcher who carries out projects does not see the need to progress toward follow-on technological developments, and their transfer to the market in the form of innovation. Conversely, the professional who wishes to innovate or reinvent his or her professional activity, sees no need to begin from the research, which is perceived as distant from reality (Ertuna and Gurel, 2011).

This perceived relationship between theory and practice, now a foregone conclusion, has led us to make errors in defining

### REFERENCES

Aho, E. (2008). Cómo Construir un Silicon Valley en Europa. [How to buildng a Silicon Valley in Europe]. Available online at: http://firgoa.usc.es/drupal/node/ 40479 (Accessed Internet 01, 10, 2012).

research and professional work. Theory need not always be prior to and disconnected from practice, nor is the converse necessarily required. All of us who work in Educational Psychology ought to recognize that we are placed at some point in this chain, and that fact should prompt us to coordinate with other elements on the same chain.

The only viable solution for raising the value of Educational Psychology practice is a significant move, from both ends of RD & I chain, toward connecting the links:


This coordinated, joint work requires new actions and new structures for it to materialize on a permanent basis. Creation of cooperative agreements or scientific-technological consortia for the purpose of promoting joint RD & I between the University and Professional Associations can be a tool to help new professionals position themselves in the twenty first century Knowledge Society, adopting new professional profiles and activities in Educational Psychology (European Commission, 2006). Realization of this idea would be profitable for both research and professional practice, making a positive difference in the processes, products and services that are produced in the educational psychology sphere and that form part of the professional profile, and education for entrepreneurship in psychologists (Pittaway and Cope, 2007; Oosterbeek et al., 2010). This approach means unequivocably bridging the gap between science and profession, between researchers and professionals, in order to jointly redefine the big challenges that face the science and the profession (Walter et al., 2013). Only in this way can Psychology -and especially Educational Psychologyposition itself strategically in the present-day international context of Science and Innovation (de la Fuente and Vera, 2010; Subramanian et al., 2016), alongside other social sciences, education and healthcare (Seelos and Mair, 2005; European Commission, 2014; Ferguson, 2016). At present, steps are being taken in this direction (de la Fuente et al., 2018).

### AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Arimura, T., Hibiki, A., and Johnstone, N. (2007). "An empirical study of environmental R&D: what encourages facilities to be environmentally innovative," in Environmental Policy and Corporate Behavior, ed N. Johnstone (Cheltenham: Edward Elgar), 142–173.


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling Editor declared a shared affiliation, though no other collaboration, with one of the authors DK.

Copyright © 2018 de la Fuente, Kauffman, Díaz-Orueta and Kauffman. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Entrepreneurs' Well-Being: A Bibliometric Review

José Carlos Sánchez-García\*, Gioconda Vargas-Morúa and Brizeida Raquel Hernández-Sánchez

Universidad de Salamanca, Salamanca, Spain

The present article aims to summarize and classify existing research entrepreneurs' well-being through a bibliometric literature review. Its main objectives are: to identify the different theoretical perspectives and research strands that characterize and define literature on entrepreneurs' well-being and highlight the connections between them; as well to look for emerging trends and gaps in its literature by comparing the most recent works with those that represent the field's core. The document is based on bibliometric data: it uses citation techniques to select, analyze, and interpret citation patterns within the literature on entrepreneurs' well-being. The paper identifies six main groups, as well as several specific research flows and common themes that represent academic publications on entrepreneurs' well-being. The research strands on the topic are grouped into six different theoretical perspectives grounded in entrepreneurship related to: culture, education, innovation, sustainable development and small business; psychological well-being; social entrepreneurship and economic development; women and employment; and self-employment; life satisfaction and economic growth, and business administration. Data from the most recent publications were used to verify whether original topics and themes are reflected in contemporary debate and in which fashion. Limitations related to search engines, such as missing keywords were accounted by utilizing three different database as well as expanding keyword number. From a practical perspective, this research is expected to contribute on theory construction, management decision making, and teaching. This study describes the growing development of the literature on entrepreneurs' well-being, and the underlying structure of the different streams of research therein.

Keywords: well-being, entrepreneur, bibliometric review, social entrepreneur, self-employment, business owner, independent worker, organizational employer

### INTRODUCTION

In the last decade, the entrepreneurial activity has shown a growing development that affects governments, societies and people in general and, with the same impetus, this has been reflected in entrepreneurship research.

According to the Global Entrepreneurship Monitor (2017–2018) report, entrepreneurship levels are stable or increasing worldwide. Seventy-four percent of respondents said they had chosen to seek for opportunities as basis for their business motivations, and 43% of the world's population sees good opportunities to start a business in the next 6 months. In addition, almost 70% of the adult population across 52 economies around the world believe that entrepreneurs are well appreciated and enjoy high status within their societies.

#### Edited by:

Douglas F. Kauffman, Independent Researcher, Bedford, MA, United States

#### Reviewed by:

Laurent Sovet, Université Paris Descartes, France Matthias Philip Huehn, Universidad de Navarra, Spain

#### \*Correspondence:

José Carlos Sánchez-García jsanchez@usal.es

#### Specialty section:

This article was submitted to Organizational Psychology, a section of the journal Frontiers in Psychology

Received: 25 May 2018 Accepted: 22 August 2018 Published: 12 September 2018

#### Citation:

Sánchez-García JC, Vargas-Morúa G and Hernández-Sánchez BR (2018) Entrepreneurs' Well-Being: A Bibliometric Review. Front. Psychol. 9:1696. doi: 10.3389/fpsyg.2018.01696

**16**

Likewise, this growth is reflected in the increase of academic papers related to the field of entrepreneurship (Web of Science, 2017) in developed countries in developed countries (Bruton et al., 2008) such as the United States, England (Trueman et al., 2013), and Germany (Hetschko, 2016), with issues related to the well-being of the entrepreneur, such as the work-family conflict (Nguyen and Sawang, 2016), self-employment (Binder and Coad, 2013), stress (Cardon and Patel, 2015), firm performance (Hmieleski et al., 2013), innovation (Baron and Tang, 2011), creation of financial, social and environmental wealth (Zahra et al., 2014), as well as the possible implications depending on the type of entrepreneurship (Uy et al., 2017) and on the suitability of entrepreneurs and businessman when running their companies (Stock et al., 2016).

However, not all studies define the concept "entrepreneur," instead they presuppose an interpretation and, on many occasions, their definition is conditioned by the context of research. Initially Knight (1964) provides some characteristics of an entrepreneur and indicates that it will be the person who takes risks and occupies a position of uncertainty proposed, one who takes the initiative, who has imagination and creates new opportunities, that is, ". . . a state of mind to direct personal attention, experience and action toward a specific goal or access to achieve something" (Bird, 1988, p. 442).

Lumpkin and Dess (1996) consider that the entrepreneurial orientation refers to the processes, practices and decision-making activities, and actions that work in a dynamic generator process aimed at venture creation. Its key feature corresponds to a propensity to act autonomously, willingness to innovate and take risks, as well as a tendency to be aggressive toward competitors and proactive with regard to market opportunities.

Repeatedly, is conceived a person with entrepreneurial orientation as one that combines innovation, takes risks and is proactive (Miller and Friesen, 1983; Hansen et al., 2011; Goktan and Gupta, 2013) and "entrepreneurship" is understood as the ability of people to translate ideas into actions. It means to be creative, lead, innovate, take risks, and manage personal and professional projects to achieve specific objectives (Sánchez, 2013; Oliver et al., 2015).

Finally, Shane y Venkataraman provides a definition of "entrepreneurship," as a "scholarly examination of how, by whom, and with what effects opportunities to create future goods and services are discovered, evaluated and exploited" (2000, p. 218).

There is no doubt that entrepreneurship is a challenging effort, which entails great challenges for an entrepreneur. As we have seen in the definitions, it involves taking risks, making decisions, taking advantage of opportunities, acting in uncertain environments in a proactive and innovative way, in order to achieve objectives that are specific to each entrepreneur. This leads us to reflect whether in the midst of all this process, entrepreneurs actually manage to reach the condition of wellbeing.

It is worth mentioning that the term well-being was used even before Aristotle (IV century BC) and the issue has not gone unnoticed since then, but it is from this last decade that there has been an important increase in its study, in search of greater understanding of its link with entrepreneurship (Uy et al., 2013).

Well-being has gone from the popular perception of "being well" to being considered a measure of global interest that must be included in the analysis of human development. Such is the importance of this topic that has recently been incorporated into the official statistics of different countries (INEGI, 2013) since it is considered that the follow-up of this allows to make better governmental, business and social decisions. It has also been addressed in global forums (OCDE, 2011), in a large number of organizations (Stiglitz et al., 2009; INEGI, 2013; Survey World Values., 2017) and in programs such as the United Nations Development Program.

Many philosophers have been reflecting on happiness as the origin of well-being, which is intrinsic to human beings. However, after the Second World War, studies related to the people's well-being were associated with the treatment of the disease their causes. It is only after the emergence of positive psychology (Seligman, 1998, 2004) that well-being research, and its traditional conception associated with negative and pathological aspects of the human being, takes a turn to give way to the foundations of psychological well-being, happiness, strengths and human virtues.

Research on well-being has been approached from different perspectives, such as sociological (Veenhoven, 2008); economic (Clark and Oswald, 1996; Deci et al., 2001; Easterlin, 2001; Van Praag et al., 2003; Brown et al., 2008); and psychological (Diener et al., 1999; Park et al., 2004; Kahneman et al., 2006; Forgeard et al., 2011; Forgeard and Seligman, 2012; Csikszentmihalyi and Larson, 2014). Therefore, efforts are now focused on studies that support the construct of "well-being" as a combination of feeling good and having purpose and meaning in life, good relationships, family support; have a rewarding, attractive job, adequate income; be reasonably healthy, have important goals related to personal values; live in a democratic environment and a stable society (Diener and Seligman, 2004).

Well-being can be explained in many ways, and is generally associated with the measurement instruments used (DeNeve and Cooper, 1998). However, we will define it in hedonic and eudaimonic perspective for better comprehension.

Hedonic well-being refers to happiness in terms of achieving pleasure and avoiding pain (Deci and Ryan, 2000). The hedonic point of view focuses on subjective well-being, which is defined as the presence of positive affect and greater satisfaction with life, as well as the absence of negative affect (Diener, 1984). That being the case, we understand satisfaction with life as a cognitive, judicious process, that is, a global assessment of a person's quality of life according to their chosen criteria (Shin and Johnson, 1978). Satisfaction judgments depend on a comparison of circumstances with what is believed to be an appropriate standard, as such, it does not depend on criteria that the researcher deems important or imposed externally, but on those that people establish for themselves (Diener, 1984).

The eudaimonic approach focuses on meaning and selfrealization and defines well-being in terms of the degree to which

**Abbreviations:** SSCI, Social Science Citation Index.

a person is fully functioning (Ryan and Deci, 2001). It is defined as a fully functional and self-realized individual (Deci and Ryan, 2000). Vigor, also called vitality, is a common operationalization of eudaimonic well-being (Deci and Ryan, 2000; Ryan and Deci, 2001). The concept of subjective vitality refers to the state of feeling alive and alert, to having available energy, that is to say of positive physical functioning, for what is considered an aspect of eudaimonic well-being (Ryan and Frederick, 1997), since it is vital and energetic it is part of what it means to be in full psychological functioning.

Taking into account the previous points, we recognize that existing research in entrepreneurship provides a solid foundation for further development and, through it, can determine those aspects that have been mostly addressed to achieve entrepreneurial well-being. However, all the information related to the topic is presently not structured, and thus, not possible to understand from an overview.

To try to correct this gap in the research field, this article provides a bibliometric information platform that shows the research areas that have been studied, authors, sources and years of development, which encompass subjective well-being on entrepreneurs. For the literature search, we used the Thompson Reuters' Social Science Citation Index (SSCI), Elsevier's Scopus and ProQuest databases with the following specific objectives in mind:


We ensure that the quality of the information was excellent and could respond to the research questions: in which areas of study the entrepreneurs' well-being becomes relevant?, what has been its importance over time?, what are its major research proponents?, and what are its mayor journal sources?

In this sense, this article integrates the information found and groups it according to the different perspectives that deal with the subject of entrepreneurs' well-being.

### METHODS

### Study Design

In this article, we carry out a bibliometric review of the literature on entrepreneur's well-being, which seeks to synthesize this scientific literature. We have used strict control mechanisms in order to reduce biases to a minimum, such as the PRISMA method (Liberati et al., 2009; Urrútia and Bonfill, 2010) in the process of choosing and discarding articles. In addition, we have relied on a previous protocol of explicit criteria, uniformly applied to all articles, in order to narrow the topic and focus on the objectives set.

Only peer-reviewed articles have been used in order to guarantee the reduction of bias as much as possible and, in turn, this allow us to determine the frequency and relationship of the most co-cited authors of the topic; the frequency and relation of the sources in which it is mostly published on, and the progress in its research through time.

This bibliometric study includes the following steps: 1. Selection of articles related to entrepreneurial wellbeing 2. Application of statistical methods to extract relevant information and 3. Inclusion of a narrative synthesis about the major findings of the study.

Our structure of analysis of bibliometric networks is based on an approach for unified mapping and grouping (van Eck and Waltman, 2010; Waltman et al., 2010), which provides information on the structure of a network, on the fields of research, how they relate to topics among themselves and how the subject has developed over time. As such, through this quantitative and qualitative analysis of the evidence found, we are able to answer our research questions.

The search for relevant articles, procedures, and reviews we used the Social Science Citation Index (SSCI), Scopus and ProQuest database, without defining a certain time frame of publication. These were chosen because the science portion of SSCI covers all of the major English-language international journals (Goodman and Deis, 2005), while Scopus offers sorting and refining features so that researchers can easily find and access more than 27 million abstracts and citations stretching back to the mid-1960s (Boyle and Sherman, 2006) and ProQuest as a third and complementary source, as it offers a wide repository of business and social science research.

### Search Strategy

Citation techniques and the PRIMA method (Liberati et al., 2009; Urrútia and Bonfill, 2010) were used to select, analyze, and interpret citation patterns within the entrepreneurs' well-being literature in two stages. Once the reviewer approved the protocol, we proceeded to look for the information.

In three databases (SSCI, Scopus and ProQuest), we used the systematic procedure outlined below:

We used the search terms "well-being" and "entrepreneur<sup>∗</sup> ." For the first term, it was searched with and without a space between the two words (wellbeing and well-being) and included quotation marks so that the search engine would identify both words together, and for the second term, we included the asterisk so that the search engine would find all its possible variations.

We understand that many investigations related to the traits and actions of entrepreneurs, such as "self employment," there is "business owner," "independent worker," and "organizational employer" as well. These words were included in our search; however, for purposes of maintaining quality in our study, we only considered peer-reviewed articles and that are specifically associated with the concept "Well Being," as seen in **Table 1**.

The first step identified 399 SSCI documents, 501 Scopus documents and 81 ProQuest documents. The information was extracted without defining a specific publication period, as we sought to analyze trends in its research across time. Subsequently, we filtered any document that was not classified as a peerreviewed, and compared lists to eliminate duplicate articles. We then reviewed the abstract of each article to verify that all selected articles were, in fact, in line with the research topic. Thus, we excluded 398 duplicate articles and 210 articles whose

#### TABLE 1 | Flow diagram of the studies.


Own elaboration following the PRIMA method (Liberati et al., 2009; Urrútia and Bonfill, 2010).

information was unrelated to entrepreneurs' well-being. In total, with the VOSviewer program (van Eck and Waltman, 2010) and Excel, we analyze 373 articles. In this way, we ensured that the quality of the information was optimal and would enable us to answer our research questions.

### Flow Diagram of the Studies Retrieved for the Review

In **Table 1**, we detail in the identification, systematized article extraction, and Boolean codes that were used and the quantity of articles obtained from each of the relationships. With the purpose of being exhaustive in the search and, given that the SSCI and ProQuest databases do not have the filters we used (title, keywords and abstract), the search was extended to "subject" and "any field except full text," respectively.

In the section of choice, we indicated the number of excluded articles because they were either duplicated in the same database when performing different searches or were not peer-reviewed. Later, when joining the databases, we eliminated the duplicated articles among them, or other documents that, when making the revising their Title, keyword or abstract, were unrelated to the topic. Finally, we indicate the quantity of articles analyzed.

### Data Sources, Study Stages, and Data Extraction

We used VOSviewer software version 1.6.7 (van Eck and Waltman, 2010) to construct and visualize bibliometric maps through the technique of similarities, as well as to identify clusters and their reference networks (van Eck and Waltman, 2010; Waltman et al., 2010), EndNote and Mendeley were used as citation management software.

With VOSviewer (van Eck and Waltman, 2010) two types of bibliographic maps, one based on distance and the other can be distinguished in the graphics. "Distance-based maps are maps in which the distance between two items reflects the strength of the relation between the items. A smaller distance generally indicates a stronger relation" (van Eck and Waltman, 2010, p. 525), which facilitates the identification of groups of related items. Those based on graphics, lines are drawn between the elements to indicate their relationships. For this study, mostly distance-based maps were used.

#### Data Analysis

For data analysis, there are two strategies. First, which corresponds to the cluster analysis obtained with the VOSviewer program and the second, which corresponds to the Bibliometric analysis of the information. In this way, we were able to study our topic's within the academic field and could perceive the progress in research from 1974 to 2017.

#### **Cluster analysis**

In the first stage, in order to identify all possible research fields and associated variables we carried out a co-occurrence analysis with a minimum of two occurrences per word, for a total frequency of 196 keywords grouped into 15 clusters.

In this way, the variables were grouped by the system according to criteria of socioeconomic factors and health care policy; female entrepreneur; social entrepreneurship; development and education; self-employment and happiness; small business and management; Agriculture, culture and well-being at work; tourism and entrepreneurship policy; adaptation and social responsibility; economic growth and rural women; economic development and entrepreneurial education; capability approach and innovation; sustainable development; family business; entrepreneurialism; capitalism; citizenship and religion, and relationships. Stronger relations as seen in **Figure 1**, are represented in circles and with larger labels. For example, entrepreneurship, subjective wellbeing, self-employment, education, social capital and social entrepreneurship; women and economic growth.

Again, we carried out a co-occurrence analysis with a minimum of five occurrences per word, for a total frequency of 35 keywords with the greatest overall link strength in order to show the six most relevant clusters, as shown in **Figure 2**.

The combined mapping and grouping shown in **Figure 2** provide an overview of the structure in the field of entrepreneurship research. Each group is represented by a different color that exhibits its relative importance, proximity and relationship between them. That is, "the greater the number of neighboring elements and the smaller the distances between

these elements and the point of interest, the greater the density of the element" (van Eck and Waltman, 2010, p. 533).

As for the left-side clusters: the first and second have the same amount of items and corresponds to the red and green cluster.

The first cluster in red associates the following keywords: development, economic development, economic, poverty, social capital, social enterprise, social entrepreneurship, and values. The 17.68% occurrence of keywords under study are related to this cluster.

Social entrepreneurship is aimed at developing strategies to support social change and awareness in order to improve the well-being and conditions of people in society. Within this area, we find that the issue has been approached from various perspectives, such as that of social innovation. Dawson and Daniel (2010) indicate that social innovation generates social benefits but can also serve certain commercial, technological, organizational, or scientific purposes, so that the development of social innovation is possible in the organization, the community or society.

Likewise, when considering social innovation the entrepreneur must have a deep knowledge of social problems, and such innovations depend on the active participation of the actors involved and the availability of local endogenous resources (Bernardino and Santos, 2017). The concept of social innovation extends to the field of international entrepreneurship to affect sustainable global well-being, and is a multidimensional concept that includes the creation of financial, social, and environmental wealth (Zahra et al., 2014).

Another aspect that social entrepreneurship must consider is the possibility of contributing to improve the behavioral health problems of entrepreneurs, as well as community, economic, and social development (Ferguson, 2016). Studies have also considered how social initiatives can contribute to local growth and development (Almarri and Meewella, 2015; Bernardino and Santos, 2017), social value, and the integration of well-being (Ferreira et al., 2017).

Also, this cluster it shows a very close relationship to the main terms, because the well-being of the entrepreneur is associated with economic development and social entrepreneurship. For example, to improve the understanding of how micro-level subsistence activities might be related to higher-level phenomena to increase the well-being of individuals and communities in contexts characterized by institutional gaps (Kolk, 2014). Also, as the use of political economic strategies that advocate using more effectively the capacities of small landowners for rural development and well-being through the promotion of smallscale production systems (Pokorny et al., 2013).

In relation to social capital, we find the following: the contribution of micro-entrepreneurship has played a prominent role in the development of employment around the world; however, entrepreneurs must consider the financial, human, and social capital before starting their activity, as well as the hostility of the environment (Vial, 2011). For example, microfinance could help improve the well-being of clients and maximize business results (Newman et al., 2014).

Some research calls, therefore, for a more collective vision of management that is based on trust and social well-being, as greater well-being generates more cohesive, productive, and happy societies (Zhao and Roper, 2011). Likewise, other factors influence happiness, such as asset ownership, family ties, personal attributes, characteristics of households, human, and social capital, financial security, labor relations, and participation of the community (Mahadea and Ramroop, 2015).

The green cluster is shows the closeness and link strength in the words "entrepreneurship" and "well-being." Associated with these two keywords are culture, education, empowerment, innovation, small business and sustainable development. The 27.30% occurrence of keywords under study are related to this cluster it is the one that concentrates the greatest amount of relations with the rest of the clusters, as shown in **Figure 3**.

Additionally, the advancement of research into entrepreneurs' well-being from the perspective of culture and innovation starts very early, given the interest in improving people's well-being within society (Dawson and Daniel, 2010) and the community culture that promotes development businesses (Huggins and Thompson, 2014).

For example, subsistence entrepreneurs in developing countries carry out vital marketing activities, overcome substantial life challenges, and improve their economic capacity and that of their communities (Sridharan et al., 2014). This keywords, are associated as well with research that considers the importance of inventor-entrepreneurs, whose potential lies in their possibility of growing and societal well-being (Miner et al., 1992).

Regarding these, we found the following: first, at the absence of financial rewards, motivation for achievement does not improve the will to grow unless it is intrinsically motivated (Davidsson, 1989). Second, coping based on cognitive response facilitates well-being and business performance (Drnovsek et al., 2010; Uy et al., 2013).

Third, employers, compared to employees, show higher levels of happiness (Mahadea and Ramroop, 2015). Fourth, to achieve greater well-being and productivity, it is necessary to carry out social and leisure activities based on hobbies, since relaxation and rest are not enough to achieve real well-being (Shen et al., 2018).

The third cluster in blue associates the following keywords: agriculture, developing countries, employment, entrepreneurialism, job satisfaction, stress and women. The 13.15% occurrence of keywords under study are related to this cluster.

The variable "women" was found to have been associated with the construct entrepreneurship since the beginning of research into entrepreneurs' well-being, because it is thought that women

can be managers of a positive change in their personal, social, and economic well-being, and a positive influence on their immediate community (Sridharan et al., 2014).

Also associated with this cluster are certain financial tools, such as micro-credits, that can be effective in alleviating poverty (Ang, 2004) in order to obtain better access to markets to improve entrepreneurs' well-being in a sustainable manner, despite limitations (Kolk, 2014).

Interestingly, the report of the GEN (2017–2018) reveals that the proportion of participation of men and women in entrepreneurial activity, in the initial stage, varies considerably and is reflected in cultural differences and customs with respect to women. Generally, in economies driven by factors, such as efficiency and innovation, women will have an entrepreneurial participation motivated by necessity, and inferior to that of men.

The fourth cluster in yellow associates the following keywords: economic growth, gender, happiness, income, life satisfaction, self-employment, and subjective well-being. The 10.33% occurrence of keywords under study are related to this cluster.

From research that has focused on the search for entrepreneurial well-being in the aspect of "self-employment" and its relationship with gender, we have considered the following findings:

Individuals who are self-employed enjoy greater autonomy, have flexibility at work and report higher levels of participation in

work and job satisfaction than those employed by organizations (Parasuraman and Simmers, 2001). Despite lower incomes, self-employed workers consistently report higher job satisfaction (Binder and Coad, 2013).

However, it is also true that self-employed workers experience higher levels of work–family conflict and lower family satisfaction compared to employees of organizations (Parasuraman and Simmers, 2001). Work–family conflict has a direct negative effect on mental health, work, family, and life satisfaction (Nguyen and Sawang, 2016); additionally, high job satisfaction can make entrepreneurs neglect other important domains of life, as gratifying work excludes other pleasures (Binder and Coad, 2013). When the entrepreneur's experience in the operation helps them improve their quality of life and family well-being, they are less likely to consider leaving the business (Hsu et al., 2016).

Self-employment is also positively related to subjective well-being, despite the differences between groups of selfemployed workers. For example, self-employed workers report a higher level of life satisfaction compared to self-employed workers without employees. Immigrants experience greater life satisfaction compared to natives (Johansson Sevä et al., 2016).

We also find that being unemployed produces a much stronger decline in life satisfaction for self-employed workers than for paid employees (Hetschko, 2016). People who exchange regular employment for self-employment experience an increase in life satisfaction (up to 2 years later), while those who move from unemployment to self-employment are no more satisfied than their counterparts who go from unemployment to regular employment (Binder and Coad, 2013).

Finally, it is said that entrepreneurs, to solve their problems, can oscillate between taking active measures (active coping), or temporarily distance (i.e., avoiding confrontation).The effective use of avoidance coping improves immediate psychological well-being, since incorporating short breaks and temporary respite can be beneficial. In the long term, employers must use their ability to avoid issues, along with active coping, and learn to take advantage of both methods to deal with problems (Uy et al., 2013).

The fifth cluster in turquoise associates the following keywords: entrepreneur and psychological well-being. The 22.35% occurrence of keywords under study are related to this cluster. Although the number of words is low, it is the second cluster with the most relationships with the others.

#### TABLE 2 | Cluster 1.


(Continued)

Sánchez-García et al. Entrepreneurs' Well-Being: A Bibliometric Review

#### TABLE 2 | Continued


This cluster relates those studies where psychological aspects acquire an important role in entrepreneurship. For example, people who are willing to take short-term risks are more likely to start a business, as are those who have high psychological well-being (Zhang et al., 2015) and if they also have control of the environment, self-acceptance and autonomy, a good financial performance is foreseen (Farrington, 2017). In addition, openness to experience, autonomy and persistence in terms of goal pursuit increases the likelihood that a person will seek selfemployment, while neuroticism (Patel and Thatcher, 2014) and being a low-skilled worker (Lofstrom, 2013) reduce the likelihood of entrepreneurship.

These variables have also been used when making comparisons between entrepreneurs and non-entrepreneurs, indicating that it is possible that sometimes employers show lower returns than non-entrepreneurs do. However, this is compensated by non-pecuniary benefits, such as greater control over their work environment, greater optimism, and social capacity (Shefrin, 2011). In conclusion, entrepreneurs show greater well-being and better physical, mental and behavioral conditions (Stephan and Roesler, 2010).

Finally, the last of the clusters in purple, is located on the right side and shown in **Figure 2**, is far from the main group. This reflects the low strength of the relationship with the rest of the elements (van Eck and Waltman, 2010). The following keywords are associated with this cluster: business, management and psychology, applied. The 9.2% of the keywords relate to this cluster.

It deals with aspects related to business and its administration, thus, although well-being of entrepreneurs is present in this topic, it interacts weakly in our cluster relationship model. For example, there is mention of the metropolization process that has privileged a handful of dynamic urban centers but, on the other hand, the need for governments to activate rural areas, thus improving the well-being of the population in general (Smetkowski, 2013).

Also aspects related to different types of administration, such as that of the founders of family firms that experienced higher levels of support from other family businesses than those who owned non-family businesses, with additional support provided to the former in terms of social well-being and resilience via the family's integration into business (Powell and Eddleston, 2017).

#### **Bibliometric review**

The theme of "well-being" has been extensively discussed. For example, Mandell (1974) states in his article "The Changing Facet of the Chair of Psychiatry Departments in America: An Opinion," as "two decades ago psychiatrists were trained by a few charismatic, humanistic clinicians who did not have much to do with laboratories or fund raising. After World War II academic psychiatry was transformed, largely by federal money, into a multidimensional scientific enterprise the leaders of which needed to be scientific entrepreneurs as well as persuasive humanists" (Mandell, 1974, p. 1137) so that universal well-being was relegated to a second plane.

The next group of articles found relate to social problems, such as public transport service as a vital operation for the well-being of cities (Grava, 1980), policy statements between governments in search of the cities' well-being, and small businesses as a means of generating employment (Rothwell, 1984). We also found a study on the inequality of the black population. It concludes that "despite renewed governmental encouragement and some real growth, black-owned businesses still represent a trivial fraction of total businesses; and the black share of business firms in a city seems to have a substantial impact on the relative well-being of blacks in that city" (Villemez and Beggs, 1984, p. 137).

On recent studies, entrepreneur and well-being acquires a different connotation. As seen in **Figure 4**, from 2009 onwards the number of articles published in high-impact journals increases exponentially, revealing the importance that researchers have conferred on the subject.

As observed, the "well-being" construct is associated with many fields of knowledge, as is "entrepreneur <sup>∗</sup> ." In **Figure 5**, we have made a comparison of the progress in the research from 1984 to 2017, associated with the extracted clusters, in order to visualize the knowledge growth in those areas that has been of greater focus during this period.

For example, in recent years, the green line that relates entrepreneur well-being with culture, education and innovation is the one that shows the most growth, followed by the turquoise line, which, as we studied, is associated with the psychological aspects that allow the entrepreneur to achieve well-being.

Subsequently, the red and blue lines are presented. The first is related to social entrepreneurship and maintains a constant growth, and the second with the problem of women, which shows a decrease in the last year.

Finally, we find the lines in yellow and purple. The first, related to self-employment, shows a decrease in the last year and the purple, whose link with entrepreneurial well-being is less strength, shows a more conservative growth.

#### **Relationships between journals**

To identify which journals have the most cited articles associated with the issue of the entrepreneurs' well-being, we analyzed the data with a pre-specified minimum number of citations per source of 20. Then the VOSviewer program selected 58 sources, whose relationships are shown in **Figure 6**.

#### Clearly, four clusters are distinguished: from left to right, the first and smallest related to journals that publish on sustainable or tourism studies. Followed by the green, that frames journals related to the economic field; network, whose relationships are stronger and mediate the cluster, related to issues of business administration and the fourth cluster in blue, related to psychology issues.

The first 10 journals with the highest citation number and with the strongest link, in descending order, are: Journal of Business Venturing, Entrepreneurship Theory and Practice, Academy of Management Review, Journal of Applied Psychology, Journal of Vocational Behavior, Small Business Economics, Journal of Personality and Social Psychology, Academy of Management Journal, Journal of Small Business Management and Journal of Management. All of them classified between the first and second quartiles in 2016.

### **Relationship between authors**

To identify which authors that have the most cited articles associated with the topic of entrepreneur' well-being, we analyzed the data with a pre-specified minimum number of citations per source of 20. Then VOSviewer program selected 29 authors, whose relationships are shown in **Figure 7**.

Mapping and clustering denotes the number of links, in this case, co-citation links. This can be interpreted in terms of attractive and repulsive forces between nodes. "The higher the association strength of two nodes, the stronger the attractive

#### TABLE 3 | Cluster 2.


force between the nodes. Since the strength of the repulsive force between two nodes does not depend on the association strength of the nodes, the overall effect of the two forces is that nodes with a high association strength are pulled toward each other while nodes with a low association strength are pushed away from each other" (van Eck and Waltman, 2010, p. 631).

On the other hand, in the "full counting" option, the highly cited references are considered more representative, but not in the "fractional counting" option in which each reference cited in a publication has the same influence, or is considered equally representative in the publication. So, to carry out this analysis, the "fractioned counting" was considered, so each reference cited in a publication has the same representativeness, determining in the entrepreneurial well-being field, which are the authors with the most influence.

**Figure 7** shows 6 clusters, whose largest group comprises of 17 authors, the following 5 clusters on average has a size of 4.80 with a standard deviation of 1.94. A summary of the contents of the five groups is provided in **Tables 2**–**7**.

It is important to mention that all authors considered in the following tables have been cited in issues related to entrepreneur's well-being; therefore, both terms and derivations have not been included in the tables.

### RESULTS

### Synthesized Findings

The articles were identified using the terms "well-being" and "entrepreneur<sup>∗</sup> " and their derivations. We eliminated all studies that were irrelevant or repeated; whose keywords were not within the abstract, keywords, or title; or were not the correct document type (e.g., book chapters) (see **Table 1**). The study from the 373 articles was carried out using two analysis strategies: first the clusters are analyzed, the conformation of the clusters is presented in all areas where well-being is considered within entrepreneurship activities, and its topics reported. In the second, a bibliometric study was carried out, in which indicators are described: where is it published, years in which has been published, growth, and author's citation.

From the general analysis, topics identified in relation to the terms "well-being" and "entrepreneur<sup>∗</sup> " were found to be very diverse, and related to aspects of health, psychology, economy, society, and culture, at both the micro (individual) and macro (associated with communities or countries) level.

In the cluster analysis, the six cluster and the relationships between them are described. Mainly related with: culture, education, innovation, sustainable development and small business; psychological well-being; social entrepreneurship and economic development; women and employment; and selfemployment; life satisfaction and economic growth, and business administration.

Additionally, as shown in **Figure 4**, research on entrepreneurs' well-being has grown considerably, especially in the last decade, and the issue has been an increasing focus of governments, communities, and societies in general.

Finally, it is important to note that entrepreneurs' wellbeing has been discussed in rigorous and high-quality scientific journals, guaranteeing article and finding's reliability, also that there is a high number of authors, who are frequently cited, and


#### TABLE 5 | Cluster 4.


#### TABLE 6 | Cluster 5.


whose academic experience in the area of entrepreneurship is well-recognized. An index of collaboration per article of 1.89 was obtained (704 authors participated in the 373 articles studied).

### Risk of Bias

Our research collected information on entrepreneurs' well-being from SSCI, Scopus and ProQuest. These three databases include articles based on quality and academic relevance. Therefore, an assessment of the method used in each study to determine the integrity of the research is not required, since any risk was eliminated by using these particular databases. On the other hand, once we began to extract the information, we followed a specific protocol and established objectives, setting aside any personal criteria that may have hindered the investigation. Thus, in the first part, article relationships were analyzed according to bibliographic maps based on distance, and in the second part, other aspects were analyzed such as: citation, sources, and years of publication.

Finally, any disagreement between the authors of this study that may have led to bias was resolved through discussion and with the participation of the third author when necessary.

### DISCUSSION AND CONCLUSIONS

### Summary of Main Findings

According to GEM (2017–2018) the levels of entrepreneurship have a growing trend and also, a considerable percentage, has planned to start some activity in the near future. However,

TABLE 7 | Cluster 6.


we have argued that well-being functions as an engine in the life of the entrepreneur, because in order for companies to maintain themselves over time it is necessary for entrepreneurs to recognize the benefits of the activity. These benefits can be classified according to the aspirations and motivations of the entrepreneur, however it all comes down to the idea implicit to the human being of wanting to obtain maximum well-being. Therefore, this bibliometric review summarizes and structures in a general way (Perianes-Rodriguez et al., 2016), a large number of articles, in order to better understand the link between the entrepreneur and his well-being.

We made sure that the quality of the information was optimal and we could answer the research questions: In which areas of study does the well-being of the entrepreneur acquire relevance? What has been its importance over time? Who are the authors? mostly cited? and What are the sources that are mostly consulted?

On the basis of a formal review of 373 articles selected and published in research journals, an analysis of the literature that shows the underlying structure of its different research streams was carried out. Our study identified:


These previous points allow to fundament the importance of the entrepreneurial well-being topic in entrepreneurial initiatives, and the increasing amount of research justifies this need for a bibliometric analysis, providing a knowledge structure that had been, until now, confusing.

Our findings show that the entrepreneur's well-being is an important construct that has been used in a common body of theoretical statements and applied to different contexts. Several approaches characterize this literature, such as economic sciences, psychology, and sociology playing an important role. Hundreds of published articles recognize the role of the entrepreneur and value their subjective well-being.

From a practical perspective, our research contributes the theory construction and teaching. First, academics can now position their work in the field, as well as identify possible new issues and gaps that can be used to formulate new research questions. Some possible contributions to the existing literature have been detected and could be considered useful in the new effort to integrate the well-being into entrepreneurship. For example, to study whether re-education in entrepreneurs could improve their attitude, and with it, their well-being. A transformation in the way of thinking could modify entrepreneurial behavior.

In addition, this research could be useful for those who work in entrepreneurship and who have no experience in the theoretical fields reviewed above (culture and innovation, self-employment, women, capital and social entrepreneurship, economic development and management), as well as the comprehension in the amplitude of the constructs "well being and entrepreneurial." As indicated above, the definition of "well being," as well as that of "entrepreneur," is adapted to the objectives and criteria of the researcher. In this way, different definitions of the terms were found in the literature and, therefore, the measurements and conclusions were adjusted to the criteria used.

Second, entrepreneurs can have an overview of the area in which they operate and benefit from some of the conclusions reached by researchers, for example, how to face firm problems (Uy et al., 2013). In addition, on rethinking strategies in order to achieve a greater advantage in the market or improve positioning by analyzing well-being contributors, whether hedonic (Diener et al., 1999), eudaimonic (Ryan and Frederick, 1997; Ryan and Deci, 2001), or inherent to an entrepreneurial activity (Rotemberg-Shir, 2015).

Aspirations could affect an entrepreneur's well-being. Studies have shown that extrinsic objectives are negatively related to well-being (Ilardi et al., 1993; Sheldon et al., 2004), however, economic aspects move many entrepreneurs, obtaining wealth or fame, which, aspirations, and motivations that may have an entrepreneur could lead to the detriment of their well-being.

Third, teachers can use this research to present a general structure and development on entrepreneur's well-being, by identifying some of the most relevant aspects mentioned. Fourth, government ministers may find this research beneficial when they visualize well-being in entrepreneurship as an opportunity to improve the country's economy or community development (Samli, 2008; Harriss-White, 2010).

Finally, as expected by Shane and Venkataraman (2000), it is now possible to consolidate a set of systematic information on entrepreneurship that, not only allows to observe the field's progress, but also provides information about entrepreneur's well-being, in order that growth is assured through the different entrepreneur's stages.

Given that well-being is as much desired by entrepreneurs as it is by people in general, and considering that there are many studies whose ultimate goal is to achieve this status, possibly the greatest contribution of this research at a practical level is to motivate continuing education toward good entrepreneurial practices, which, when these become routines, will enable the growth of both the entrepreneur and the company.

### Limitations

As with most studies, this research is subject to several limitations, which are in turn associated with risks of bias. The main limitation is the likelihood that not all existing articles were included in the study, due to multiple causes, such as engine limitations and lack of included keywords in articles, even though the number of keywords and their derivations used was extensive to prevent this effect. This can improve the scope

#### of the sample, but it can also add irrelevant articles and make the sample more challenging and less practical. Additionally, to account for this effect, entrepreneurs' well-being was analyzed without restrictions in the knowledge areas and without period restriction. The databases were also expanded to include more information.

### AUTHOR CONTRIBUTIONS

All three authors conceptualized and designed the review. GV-M structured and extracted some of the information and drafted the document. BH extracted additional information and added important details. JS-G reviewed the information and adjusted the protocol for data extraction. GV-M made the requested corrections and JS-G provided the final approval. All three authors have final approval of the published article and agree to be responsible for all aspects of the work to ensure that questions related to the accuracy or integrity of any part of the work are properly investigated and resolved.

### FUNDING

This research was partially supported by a grant from Salamanca University (KAZF), and the Counseling of Education of the Junta de Castilla y León (ref. SA042G18), Spain.

### ACKNOWLEDGMENTS

We would like to thank the UNED-CR for their support, the Chair of Entrepreneurs of the University of Salamanca for supporting the research and monitoring all activities, the Doctorate Program in Economics and Business Administration of the University of Salamanca and all its professors for the teaching imparted, and researchers at the University of Beira Interior for their help and hospitality during development of the research.

### REFERENCES


systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J. Clin. Epidemiol. 62, e1–e34. doi: 10.1016/j.jclinepi.2009.06.006


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Sánchez-García, Vargas-Morúa and Hernández-Sánchez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Grit as Predictor of Entrepreneurship and Self-Employment in Spain

Jose L. Arco-Tirado<sup>1</sup> \*, Ana Bojica<sup>2</sup> , Francisco Fernández-Martín<sup>1</sup> and Rick H. Hoyle<sup>3</sup>

<sup>1</sup> Department of Developmental and Educational Psychology, University of Granada, Granada, Spain, <sup>2</sup> Department of Business Administration, University of Granada, Granada, Spain, <sup>3</sup> Department of Psychology and Neuroscience, Duke University, Durham, NC, United States

Extending the growing literature on the role of grit in different life domains, this research explores the relationship between grit and involvement in entrepreneurship. The research highlights the role of personal income and satisfaction with one's current financial situation as moderators of the relationship between grit and entrepreneurial behavior. Using a large representative sample of Spanish young adults and controlling for a number of potential confounding variables, we find that grit is modestly negatively related to the probability of involvement in entrepreneurship. As predicted, however, this relationship is qualified by both income and satisfaction with current financial situation, though in opposite directions and more weakly for satisfaction with financial status. Gritty individuals with higher levels of income are more prone to become entrepreneurs than gritty individuals with lower levels of income. Gritty individuals with lower levels of satisfaction with their financial situation are more likely to set up a business or become self-employed.

#### Edited by:

Jesus de la Fuente, University of Navarra, Spain

#### Reviewed by:

Giuseppe Mannino, Libera Università Maria SS. Assunta, Italy Evangelia Karagiannopoulou, University of Ioannina, Greece

> \*Correspondence: Jose L. Arco-Tirado jlarco@ugr.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 22 May 2018 Accepted: 07 February 2019 Published: 27 February 2019

#### Citation:

Arco-Tirado JL, Bojica A, Fernández-Martín F and Hoyle RH (2019) Grit as Predictor of Entrepreneurship and Self-Employment in Spain. Front. Psychol. 10:389. doi: 10.3389/fpsyg.2019.00389 Keywords: big five, specific traits, grit, entrepreneurship, financial resources, self-employment

## INTRODUCTION

The decision to engage in self-employment/entrepreneurship is influenced by a number of individual characteristics, with a great deal of research focusing on how personality traits can affect the likelihood that people will choose to become self-employed (Chlosta et al., 2012). Increasing attention has been given to the potential influence of grit on individual efforts during episodes of self-employment (Hmieleski and Carr, 2007), owing to the demonstrated prospective association of grit with consequential outcomes that involve persistence in the pursuit of important goals in the educational, personal, and professional domains (Duckworth et al., 2007; Duckworth and Quinn, 2009). Indeed, grit has been shown to predict higher retention among college students (Duckworth et al., 2007); higher educational attainment among adults (Duckworth and Quinn, 2009); greater engagement in the workplace (Eskreis-Winkler et al., 2014); higher labor stability and higher professional efficacy (Duckworth et al., 2007, 2009; Eskreis-Winkler et al., 2014); teacher effectiveness (Robertson-Kraft and Duckworth, 2014); the ability to accomplish strenuous military tasks (Eskreis-Winkler et al., 2014); and individuals' success operating in contexts characterized by high levels of adversity and setbacks from unexpected events (Westphal et al., 2008). Selfemployment brings many of the challenges evident in these endeavors.

The small number of studies about the role of grit in entrepreneurship, though informative, have taken a strictly psychological approach to modeling entrepreneurship, controlling mainly for cognitive factors, traits, and socio-demographic characteristics but ignoring how other structural factors such as socioeconomic background and status might moderate the gritentrepreneurship relationship. A consideration of such factors is particularly important in the

**35**

context of entrepreneurship, as the resources entrepreneurs count on to launch a new business, including public policies and programs (Tosun et al., 2016), play a crucial role in the success of the entrepreneurial endeavor and further business development (Baker and Nelson, 2005; Rawhouser et al., 2017). Moreover, some preliminary evidence indicates that structural factors such as socioeconomic status might play an indirect role in the development of grit. For example, lower socioeconomic status children spend less time in extracurricular activities that are related to the development of grit and academic achievement (Park, 2010). In this sense, recent studies have emphasized the need for research on grit to account for the role that structural factors such as socioeconomic origins and status (i.e., the material, social, and cultural resources available to the individual) play in creating the context for grit to support behaviors that help with achievement of challenging long-term goals (Kundu, 2017; Kwon, 2017).

In this research context, our study revolves around the following three key questions: What is the role of grit in predicting attempted entrepreneurship/self-employment? How do available financial resources affect the probability of a gritty individual starting a business? And finally, how does satisfaction with one's financial situation affects the relationship between grit and the probability of creating a business/becoming selfemployed? Below, we develop the arguments supporting our research questions that lead us to our hypotheses.

## Individual Differences and Entrepreneurship

The individual difference perspective involves different areas of research such as career perspective, personality differences, health and well-being, cognition and behavior, and entrepreneurial leadership (Gorgievski and Stephan, 2016; Mannino and Faraci, 2017). It encompasses two dominant research approaches to the study of entrepreneurship: the competency approach and the personality approach.

The competency approach focuses on an integrated combination of knowledge, skills, and attitudes that can be changed, learned, and attained through experience, training, or coaching (e.g., ability to plan ahead, orientation toward learning, ability to identify and seize opportunities for success, taking risk, perseverance, decisiveness, independence, ability to persuade, self-knowledge, self-confidence (Kyndt and Baert, 2015). Drawing on work from this perspective to understand entrepreneurs' activities and actions at an intra-individual level, i.e., focusing on the so-called micro-foundations of entrepreneurship, psychological scientists have an opportunity to make significant contributions to our understanding of entrepreneurial behavior (Frese, 2009).

The personality approach to the study of entrepreneurship may take two primary forms (Muñiz et al., 2014). A significant number of studies have focused on identifying those general personality factors such as the Big Five that have a clear relationship with entrepreneurship. Early reviews of relevant work raise question as to the relationship between Big Five traits (e.g., conscientiousness, openness) and entrepreneurial success (e.g., Gartner, 1985). It appears that the personality approach in this form conceptualizes traits too broadly for them to predict the specific behaviors characteristic of entrepreneurial activity (Suárez-Álvarez et al., 2014). Furthermore, work based on this form of the personality approach has produced contradictory results (Sánchez et al., 2011).

More recent studies (e.g., Zhao and Seibert, 2006; Rauch and Frese, 2007; Zhao et al., 2010) have found that an alternative form of the personality approach focused on narrow personality traits (e.g., facets) offers better prediction of entrepreneurial behavior than broad traits (Mooradian et al., 2016). These studies highlight the value of focusing more on specific personality characteristics such as innovativeness, proactive personality, and self-efficacy, which have a significant and positive relationship with entrepreneurial success, or autonomy and internal locus of control, which predict both firm creation and entrepreneurial success (e.g., Rauch and Frese, 2007; Suárez-Álvarez et al., 2014). Indeed, research syntheses indicate that allowing for matching of relevant traits to the task characteristics of entrepreneurs yields stronger results. For example, self-efficacy, proactive personality, and achievement motivation correlate more highly with business creation and success than do all other factors, including the Big Five traits and human and social capital (Frese and Gielnik, 2014).

Other studies of enterprising individuals from the individual differences perspective have taken a cognitive approach (e.g., Sánchez et al., 2011). Based on general mental ability and creativity research, these studies have sought to explain how people process information, that is, how people comprehend and make associations between information in the context of entrepreneurship. More recently, this corpus of research has put specific emphasis on cognitive capacities such as entrepreneurial alertness and abilities to detect opportunities or make quick decisions under conditions of uncertainty and limited time (Shane, 2004; Frese and Gielnik, 2014; Mooradian et al., 2016; Wolfe and Patel, 2016).

### Grit as a Narrow Trait

In addition to the nine narrow traits identified in the entrepreneurship literature (i.e., achievement motivation, risk taking, innovativeness, autonomy, self-efficacy, stress tolerance, internal locus of control, external locus of control, and optimism) (Suárez-Álvarez et al., 2014), recent studies have considered the potential of grit to strengthen the capacity of current models to predict entrepreneurial behavior. Grit, defined as "perseverance and passion for long-term goals," is associated with an individual's ability to put forth sustained effort to achieve challenging goals, particularly in the face of trials and adversity (Duckworth et al., 2007; Borghans et al., 2008; Duckworth and Yeager, 2015). In the context of broad models of personality, grit aligns most closely with the conscientiousness domain (Roberts et al., 2014; Rimfeld et al., 2016; Arco-Tirado et al., 2018) as reflected in its conceptual relatedness to conscientiousness facets such as orderliness, dependability, self-control, and industriousness (Duckworth and Eskreis-Winkler, 2013). Yet, despite its conceptual similarity to facets of conscientiousness, none of the conscientiousness facets fully captures the combination of passion and perseverance that characterizes grit. In particular, the sustained interest in

important long-term goals, a core feature of grit, is not evident in conscientiousness or its facets. Thus, although the conceptual overlap between the broad conscientiousness domain and the relatively narrow grit trait is substantial, conceptually speaking, grit includes a specific and unique focus on the pursuit of longterm, higher-order goals (Duckworth and Quinn, 2009). Whether or not grit overlaps with or is subordinate to conscientiousness, it has been demonstrated to offer important explanatory power in variables related to long term tenacity and passion for goals across time—a distinctive capacity that is likely to explain success in the entrepreneurial context (Mooradian et al., 2016).

### HYPOTHESES

### Grit as Fuel for Entrepreneurship/Self-Employment

Recently, the influence of grit has also been explored with respect to entrepreneurship, understood as self-employment (Wolfe and Patel, 2016) or the creation of a new business (Mooradian et al., 2016; Mueller et al., 2017). In a sample of the general population from developing countries, Wolfe and Patel (2016) found that grit is related to self-employment, but also that grit is the most strongly related to self-employment for risktakers, women, and younger adults. The findings are robust, holding after controlling for several individual characteristics and alternative explanations. Specifically, these authors suggest that grit is a more important factor in determining self-employment for individuals with higher risk-taking propensities than it is for individuals with lower risk-taking propensities because of the higher levels of failure and therefore frustration they are likely to face; for females more than males due to the substantially more hindrances that women have to face in the path to becoming selfemployed; and, finally, for younger more than older individuals because of their difficulties accessing the necessary resources in the entrepreneurial process.

The hypothesized underlying mechanism for the relationship between grit and the decision to engage in entrepreneurship is, according to Wolfe and Patel (2016), the increased selfconfidence gritty individuals develop. These authors base this argument on previous results of Kolvereid and Isaksen (2006) and Verheul et al. (2012), who found that the self-perceived ability to succeed depends on people's confidence in their ability to overcome adversities. This confidence could result from the role of grit as fuel for people to persevere in their efforts toward developing the skills, knowledge, and competencies necessary to judge whether they have the ability to succeed if they engage in self-employment (Wolfe and Patel, 2016). So, these arguments lead us to formulate hypothesis 1: Grit is associated with a higher probability of creating a business/becoming self-employed during young adulthood.

### Personal Financial Resources

Because not all individuals with such personality traits and characteristics will start a new business, it is useful to ask what makes some people, but not others, with such specific personality traits go on to engage in entrepreneurship (Thompson, 2009). Thus, although there has been substantial research into the influence of personal characteristics (i.e., perceived ability) on engagement in self-employment (Kolvereid and Isaksen, 2006), no previous studies have focused on the role of other personal factors (e.g., personal monthly net income level, satisfaction with personal financial situation) as moderators of the association between grit and entrepreneurship/self-employment. Previous research indicates that grit contributes to the motivation to engage in entrepreneurial behavior, and that this relationship is moderated by different socio-demographic variables such as gender and age (Wolfe and Patel, 2016). These findings suggest that those who face more barriers in the entrepreneurial process and have more resource restrictions (i.e., women and young individuals) are the ones who benefit most from grit (Meriac et al., 2015). However, prior research in the entrepreneurship field on resilient behaviors such as bricolage (i.e., making-do with resources at hand; Baker and Nelson, 2005) has found that even though these resilient behaviors can help people face adversity, they can be displayed—and in fact are displayed most strongly—in less adverse conditions (Desa and Basu, 2013; Bojica et al., 2018). For example, Baker and Nelson (2005) emphasized that, whereas making do with resources at hand helps entrepreneurs acting in impoverished circumstances to create the resources they need and respond to different challenges in the entrepreneurial process, this resilient behavior is likely to inhibit the growth of their organization by dispersing the effort to multiple bricolage subprojects. Conversely, when bricolage is applied in organizations with high levels of resource endowments as a mean to stimulate the creative use of resources, it leads to organizational growth (Bojica et al., 2018). These findings suggest that traits like grit, which help individuals make progress in spite of adversity, are likely be even more beneficial when displayed in favorable conditions.

In this vein, the resources entrepreneurs count on to launch a new business, including public policies and programs (Tosun et al., 2016), play a crucial role in the success of the entrepreneurial endeavor and further business development (Baker and Nelson, 2005; Rawhouser et al., 2017). In a sample of nascent American entrepreneurs, Gartner et al. (2012) found that entrepreneurs' personal contributions represent 57.34% of all financing used. Therefore, an entrepreneur's current financial resources are likely to be an important factor that motivates or enables entrepreneurial behavior. Moreover, some preliminary evidence indicates that structural factors such as socioeconomic status might play an indirect role in the development of grit. For example, lower socioeconomic status children spend less time in extracurricular activities that are related to the development of grit and academic achievement (Park, 2010). In this sense, recent studies have emphasized the need for research on grit to account for the role that structural factors such as socioeconomic origins, status (i.e., the material, social, and cultural resources available to the individual) play in creating the context for grit to support behaviors that help with achievement of challenging long-term goals (Kundu, 2017; Kwon, 2017). Specifically, we propose that people who can count on higher levels of personal income will have more resources for supporting themselves in the entrepreneurial process, and this will allow them to concentrate

their effort on the challenges intrinsically associated with the entrepreneurial process, thereby enhancing the relationship between grit and the probability of creating a business/becoming self-employed. Consistent with this proposal, prior research on self-employment and income distribution shows that the population of self-employed individuals concentrates in the lower and upper tails of the income distribution (Hamilton, 2000). So, these arguments lead us to formulate hypothesis 2: The relationship between grit and the probability of creating a business/becoming self-employed will be more strongly positive for young adults the greater their personal financial resources.

### Satisfaction With Own Financial Situation

Grit is a predictor of life satisfaction (Singh and Jha, 2008). In the present research we explore the role of satisfaction, and particularly satisfaction with one's financial situation, as a factor that might stimulate grit in service of entrepreneurial outcomes. We propose that gritty individuals who are less satisfied with their financial situation will be more prone to be involved in entrepreneurship, mainly because of the following reasons.

First, people who are less satisfied with their economic situation may perceive a higher level of adversity and this may make grit even more important for persisting in the entrepreneurial endeavor. In general, previous studies offer significant evidence of the value of grit for overcoming adversity particularly in relation to involving in self-employment (Wolfe and Patel, 2016).

Second, studies from the disadvantage theory of entrepreneurship (Light, 1979) indicate that some people who engage in entrepreneurship are misfits cast-off from the wage work. Specifically, these studies show that workers having a lower wage and, therefore, those who are more likely to be dissatisfied with their financial situation, are also more likely to enter selfemployment or be self-employed at a some point in time (Evans and Leighton, 1989; Acs and Audretsch, 1990). In the same vein, other studies find that most of the individuals starting a business are unemployed (e.g., Evans and Leighton, 1989; Carrasco, 1999; Millan-Tapia, 2013). The present study excludes from the analysis those individuals who started a business out of necessity, because the drivers, circumstances, and mechanisms at play in the decision to become self-employed/start a business are likely to be different than in the case of those who are pushed versus pulled into entrepreneurship. We propose that, within the group of individuals that are not excluded from the traditional labor market, those who are not satisfied with their economic situation will be the ones more motivated to explore the path of self-employment/creating a business, as dissatisfaction with one's economic situation might lead the individual to explore new avenues for generating income.

Moreover, previous research shows that occupational expectations are less consistent in time particularly when people's socioeconomic background is lower (Rindfuss et al., 1999). Therefore, we would expect people who are less satisfied with the outcomes of their occupational choices to be more prone to change and explore other alternatives. This situation can lead gritty individuals to dedicate more time and effort to explore alternative options to generate income, strengthening the relationship between grit and entrepreneurship.

Taken together, these arguments lead us to formulate the following hypothesis 3: The relationship between grit and the probability of creating a business/becoming self-employed will be more strongly positive for young adulthoods the less satisfied they are with their own financial situation.

### MATERIALS AND METHODS

The data used in this study were collected as part of a largescale research project on the cultural pathways to economic selfsufficiency and entrepreneurship and the role of family values in 11 European countries (CUPESSE). Data were collected through a survey administered to young adults in each of the 11 countries. We used the subsample of young adults from Spain; by using data from a single country, we avoided the need to consider cultural differences in the initial tests of our hypotheses.

### Participants

The sample included 1004 participants between 18 and 35 years old, with an average age of 27.44 years old (SD = 4.91), and equivalent numbers of women (n = 510, 50.8%) and men (n = 494, 49.2%). A proportional stratified random sampling technique was used with regions (i.e., Nomenclature of Units for Territorial Statistics-2), employment status, age and sex working as strata. Nearly three-quarters of the sample (71.5%) had relatively high levels of education (i.e., upper tier upper secondary education or higher) and a bit more than a half (i.e., 55.08%) was employed or self-employed.

### Materials

The CUPESSE survey focuses on economic self-sufficiency, employability, entrepreneurship, and the family transmission of traits and attitudes that affect such outcomes. The survey was fielded in Spain in 2016 after translation into Spanish by the University of Granada CUPESSE partner following recommendations in the Cross-Cultural Survey Guidelines, adhering closely to the TRAPD (Translation, Review, Adjudication, Pretesting, and Documentation) team translation model (Survey Research Center, 2016; see Arco-Tirado et al., 2018 for more information).

### Measures

### Dependent Variable

Entrepreneurship was measured by a single item asking respondents, "Have you ever started your own business/become self-employed?" It was coded as a dichotomous variable with the values 1 = Yes and 0 = No.

### Control Variables

A number of variables were used as regression control variables, as informed by previous empirical work (Brandstäter, 2011; Frese and Gielnik, 2014; Gorgievski and Stephan, 2016; Wolfe and Patel, 2016). Sex was coded dichotomously, with 1 indicating male and 2 indicating female. Age (was measured in years.

Respondents indicated their Level of Education ("What is the highest level of education you have successfully completed?") by choosing one of seven response options: 1 = ES-ISCED I, less than lower secondary, 2 = ES-ISCED II, lower secondary, 3 = ES-ISCED IIIb, lower tier upper secondary, 4 = ES-ISCED IIIa, upper tier upper secondary, 5 = ES-ISCED IV, advanced vocational, sub-degree, 6 = ES-ISCED V1, lower tertiary education, and 7 = ES-ISCED V2, higher tertiary education < = MA level. Values were re-coded as basic education, secondary education, or superior education; basic education was used as the reference group in regression models. Previous work experience ("Have you ever had a paid job for 1 year or more?") was coded dichotomously, with 1 indicating Yes and 2 indicating No. Respondents rated their risk-taking ("On a scale from 0 to 10 would you say that in general you are a person who tends to avoid taking risks or are you fully prepared to take risks?") on a Likert-type scale with values ranging from 0 = tend to avoid risks to 10 = I am fully prepared to take risks. Self-efficacy ("I am confident that I can deal efficiently with unexpected events"), personal initiative ("Usually I do more than I am asked to do"), and locus of control ("My life is determined by my own actions") were rated on a Likert-type scale with the following values: 1 = Strongly disagree, 2 = Somewhat disagree, 3 = Somewhat agree, and 4 = Strongly agree. Born in the country ("Were you born in [country]?") variable was measured based on the ISO 3166-1 numeric classification. Financial situation when 14 ("Financial situation: My family was able to pay its bills") was indicated on a scale with from the following response options: 1 = Never, 2 = Sometimes, 3 = Most of the time, and 4 = Always.

#### Independent Variables

Grit was measured using the Spanish version (Arco-Tirado et al., 2018) of the original short Grit Scale (Grit-S) developed by Duckworth and Quinn (2009) using a Likert-type scale with the following values 1 = Strongly disagree, 2 = Somewhat disagree, 3 = Somewhat agree, and 4 = Strongly agree. For analysis purpose item values were reversed for items 1, 2, 3, 5, and 6. Internal consistency estimate was acceptable: α = 0.75 for overall grit. Personal monthly total net income ("If you add up the income from all sources, which number describes your personal monthly total net income?") was measured with a Likert-type scale with values ranging from 1 = Lowest to 10 = Highest. Satisfaction with financial situation ("Thinking about your own financial situation, how satisfied are you right now?") was assessed using a Likert-type scale with the following values 1 = Very dissatisfied, 2 = Rather dissatisfied, 3 = Rather Satisfied, and 4 = Very satisfied.

### Procedure

Once the translation of the survey was completed, the implementation process comprised two steps. First, the polling firm in collaboration with the research team generated a probability sample of young adults and surveyed them with questions from the "Youth Questionnaire." Second, all respondents were asked for the contact of one or both parents, and who were surveyed using questions from the "Parental Questionnaire." The fieldwork was undertaken by a polling company operating in Spain (i.e., NetQuest, MDK). The aim was to reach a net minimum of 1.000 surveys from young adults. Regarding the young adults' data, the survey company was asked to provide a probability sample of young adults between 18 and 35 years old, representative for employment status (e.g., employed; selfemployed; unemployed; in education/training), NUTS-2 region, age categories, education, and migration background/minority group membership in Spain. The survey technique followed was the Computer Assisted Telephone Interviewing (CATI). NetQuest panel is recruited "by invitation only" using hundreds of websites with validated databases. This multisource approach results in a broad range of socio-demographic profiles. Thus, prospective respondents were sent an online invitation and given 14 days to respond. The invitation to panel members provided information on the objectives of the research, the voluntary nature of their participation, and the confidentiality of their responses. Panel members who elected to participate were provided a respondent-specific link to access the Spanish version of the youth questionnaire. The sampling frame used by the polling firm consisted of 113,739 young adults; the response rate was.205%. Questionnaire pre-tests were conducted between December 2015 and February 2016. Insights from pre-tests produced led to minor changes in the questionnaire. Responses for analysis were collected from February to June of 2016.

### Statistical Analyses

We tested our hypotheses using binomial logistic regression analysis, which estimates and tests the probability of an event occurring. We used maximum likelihood estimation, which is a robust method for dealing with data that are not normally distributed (Steenkamp and van Trijp, 1991).

As it can be seen in **Table 2**, we ran the analyses in three steps. In the first step we introduced as predictors only the control variables. In the second and third steps we added the effect of the independent variables and the interaction effects, respectively. For each variable, **Table 2** shows the beta coefficient, its level of significance, and the odds ratios (within brackets). For each model, it also presents the values of the Wald chi-square, Log likelihood, Cox and Snell R 2 , Nagelkerke R 2 , and the Hosmer– Lemeshow test, which are used to assess model goodness of fit.

## RESULTS

The results of the correlation and descriptive analysis of all the variables included in the study are presented in **Table 1**. The modest values of the correlation coefficients indicate that multicollinearity should not be a problem in estimating our models (Tabachnick and Fidell, 2001). Because the table includes covariates and moderators, a subset of the correlations are not expected to be significant. Nonetheless, more than half of the coefficients are statistically significant. Their magnitude generally is small, owing largely to the fact that many of the variables are multiply determined and thus only modestly related to any one of the multiple determinants.

**Table 2** presents the results of the binomial logistic regression analysis. Model 1 presents the effects of the control variables alone. The number of observations available for analysis is


926 because of missing values for some of the variables used in the logistic regression and because 37 the cases of entrepreneurship/self-employment by necessity (as opposed to choice) were excluded from the analysis. The Wald chi-square is statistically significant and, together with the lack of significance of the Hosmer–Lemeshov test, indicates that overall the model fits the data. Among the control variables, only previous work experience has a significant association with the probability of creating a firm or becoming self-employed. Young people who have previous work experience are more likely to create a business. The odds of creating a business is 0.535 lower for people who had no significant previous work experience (1 year or more as employed). This finding is in line with previous entrepreneurship studies, according to which young people with more work experience are more prepared to create a business (Shane, 2004).

Model 2 add variables involved in the focal interaction effects. Goodness of fit for the model is comparable to that of Model 1. Both grit and degree of satisfaction with financial situation are associated with starting a business. Contrary to our prediction, grit is significantly and negatively related to becoming an entrepreneur. Satisfaction with financial situation is also negatively related to becoming an entrepreneur. Personal monthly total net income is not associated with the starting a business.

Models 3a and 3b include the interaction effects between grit and level of income and grit and satisfaction with own economic situation, respectively. Like in the previous models, the statistics show good fit between model and data and the explanatory power of the models increases, as shown by the pseudo R 2 represented by Cox and Snell R 2 and Nagelkerke R 2 , which improve over the corresponding values for Model 2. Whereas the beta coefficient for the interaction term between grit and level of income is positive and significant, the coefficient of the interaction term between grit and satisfaction with own financial situation is negative and marginally significant. As expected, individuals who score higher on grit are more prone to become entrepreneurs when they can count on higher levels of financial resources than when they have lower levels of economic resources (see **Figure 1**). And, as predicted, a lower degree of satisfaction with own financial situation will fuel the entrepreneurial initiative of gritty individuals (see **Figure 2**). These results offer support for our second hypothesis and marginal support for our third hypothesis.

### DISCUSSION

This study explored the role of grit as a function of objective and self-perceived financial situation in determining whether young adults start a new business/become self-employed. Although the effect was weak, when controlling for a host of potential confounding variables, we observed a negative simple relation between grit and entrepreneurship, an outcome that is inconsistent with our hypothesis. Additionally, and as expected, we found that this relationship is moderated by objective and subjective features of young adults' financial situation.

fpsyg-10-00389 February 25, 2019 Time: 16:3 # 6

#### TABLE 2 | Results of the binomial logistic regression analysis.

fpsyg-10-00389 February 25, 2019 Time: 16:3 # 7


Coefficients associated with hypotheses are Grit, Grit∗PMI, and Grit∗SFS. †p < 0.10; <sup>∗</sup>p < 0.05; ∗∗p < 0.01; ∗∗∗p < 0.001.

Specifically, we found significant support for the hypothesis that young adults with higher levels of income and grit are more prone to take the step into entrepreneurship/self-employment than individuals with lower levels of income and high levels of grit. We also found marginal support for the hypothesis that a low satisfaction with one's financial situation and high levels of grit make individuals more prone to engage in entrepreneurship than when they are satisfied with their economic situation and high in grit. We suggest several plausible explanations for these findings.

Regarding the relationship between grit and entrepreneurship, the baseline argument of previous research and our hypothesis was that grit leads young adults to display more self-confidence and perseverance (Hmieleski and Carr, 2007; Wolfe and Patel, 2016), which are positively associated with entrepreneurship. Therefore, gritty individuals would be more prone to engage in entrepreneurship. However, these capabilities and behaviors are not exclusive to entrepreneurship and can be displayed in other occupational alternatives.

The nature of our sample–young adults (below 35 years old) excluding those that started a business out of necessity—suggests a relevant explanation of the results obtained. On one hand, individuals' motivations and aspirations during this life stage might prioritize other occupational choices like employment

or continue education. In fact, previous research indicates that grit is associated with the achievement of higher levels of education (Duckworth and Quinn, 2009), engagement with the workplace, and higher labor stability (Duckworth et al., 2007; Eskreis-Winkler et al., 2014). Therefore, gritty individuals might just stay longer in education and search for more stable employment opportunities.

On the other hand, the structure of economic incentives does not appear to make entrepreneurship/self-employment a better option than employment. Previous studies both in the US (e.g., Carrington et al., 1996; Hamilton, 2000) and European countries (Millan-Tapia, 2013) comparing earnings from employment and self-employment indicate that the former provides higher levels of income than the latter. If to the low economic incentives we add the high levels of uncertainty entailed by self-employment, the latter may not be a priori a desirable career choice (Moskowitz and Vissing-Jorgensen, 2002), particularly in countries with high levels of uncertainty avoidance such as Spain. In this sense Kolvereid (1996) emphasized that security is the most important reason for individuals' preferences between organizational employment and self-employment. Our finding with respect to satisfaction with one's economic situation seems to support the idea that entrepreneurship is taken as a career choice when the level of income obtained from other career alternatives is not considered satisfactory.

Even though young adults might not be aware of the probability of earning less by being self-employed, or they might be overly optimistic regarding the potential revenue of their business idea, they have less human, financial, and social capital than older adults. Lack of adequate resources constitutes an important obstacle to the establishment of a new business and persistence in the entrepreneurial endeavor (e.g., Bates, 1995; Gimeno et al., 1997; Davidsson and Honig, 2003). Available data indicate that indeed the entrepreneurship rate among young people is lower than among older adults (Red GEM España, 2017). From this perspective, even if young gritty individuals would see entrepreneurship as an attractive career opportunity, they might not feel prepared to take the step because they cannot count on adequate resources to be successful. Consistent with this line of reasoning, Clark (2016) found in an in-depth study with successful and gritty individuals, that although grit was perceived as a necessary condition to succeed, it was not considered sufficient. This is consistent with the underlying mechanisms suggested by grit research of perseverance and longterm commitment to a goal. Gritty individuals might delay the decision to step into entrepreneurship until they perceive they have built the resources required to be successful. Our finding regarding the role of income level in the relationship between grit and entrepreneurship indicates that, indeed, when gritty individuals count on higher levels of resources they are more likely to create a business/become self-employed.

Wolfe and Patel (2016) found a positive interaction effect for grit and age of the entrepreneur, suggesting that grit constitutes a way of counteracting the resource limitations that young adults have in terms of human, social, and financial capital when involving in entrepreneurship. Wolfe and Patel's results are not contradictory to our findings, as they compare the effect of grit in younger versus older adults, whereas our analyses focused strictly on young adults. Moreover, their sample was drawn from the general population from developing countries, where the resources for entrepreneurship are scarcer and the levels of necessary entrepreneurship are higher than in developed countries. In these adverse situations grit can be a more critical resource in the quest to make a living. Our sample, young adults from a developed country, excludes from the analysis entrepreneurship out of necessity as it considers that in such cases, the decision to engage in the entrepreneurial endeavor is determined by the lack of opportunities, not by a career choice, in which case the mechanisms at play should be different. Our study extends and refines therefore the results of Wolfe and Patel (2016) with a closer look at what happens within the group of young adults from a developed country, showing that gritty individuals will be more prone to set up a business/self-employ themselves when they can count on higher than current level of income and when they are dissatisfied with their current financial situation.

### Limitations

These results should be interpreted with caution, given the inherent limitations of our data set. First of all, our study uses cross-sectional data that do not allow us to trace the evolution in time of standing on the variables analyzed. Second, we used current level of income and satisfaction with one's financial situation as proxies for the objective and subjective financial situation of the individual when he/she became involved in entrepreneurship. To dispel this situational specificity threat, we took several measures. We controlled in our models for individual's economic situation at the age of 14 as an additional measure of individuals' economic resources. Correlation analysis shows a positive and significant correlation between this variable and both objective and subjective current financial situation. This suggests that current measures of objective and subjective financial situation reflect to a certain extent the financial situation of the individual along his/her life course. Additionally, descriptive analyses of data from our sample show that 81.5% of the individuals who engaged in some form of entrepreneurship did so in the last 5 years before data collection. In such a short period of time it is unlikely that their level of income dramatically changed. Based on these data, we can conclude that the proxies we have used reflect with a certain degree of validity young adults' general financial situation at the time they became involved in entrepreneurship.

Last, we would like to emphasize that other resource variables such as individuals' social capital might be relevant for analyzing the translation of grit in entrepreneurial behavior. Future studies could explore thus the interaction between grit and social capital variables and its impact on individuals' probability of becoming involved in entrepreneurship to get a deeper insight into the role that structural factors play in creating the context for grit to support behaviors that help with achievement of challenging long-term goals such as setting up a business.

### CONCLUSION

Overall, our results emphasize the importance of continuing to explore the association between grit and entrepreneurship and the factors that might moderate the association. Particularly, this study opens a new line of inquiry that seeks to gain a deeper comprehension of how young adult's material and subjective circumstances influence the display of grit and its outcomes. The findings have implications for both grit and entrepreneurship research.

### REFERENCES


Regarding the former, grit has been studied mainly as a trait that can help people overcome adversity. Our results show that understanding grit's role in the development of challenging and complex endeavors like entrepreneurship requires taking into account the resource endowments of the individual. People who are better equipped to capitalize on grit are the ones who have more resources (so face less adversity). This suggests that although grit can be an important and helpful life trait, there are other socio-economic factors that should be accounted for when study effects of grit and other individual differences.

Regarding the implications for entrepreneurship research, the study of contributions of individual differences to the prediction of entrepreneurship has ignored how the material conditions and individuals' perceptions of these conditions may condition the predictive relationship. This study opens an interesting avenue for further research in this area.

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the ethics committees of NetQuest and UPF with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the ethics committees of NetQuest and UPF.

### AUTHOR CONTRIBUTIONS

JA-T and FF-M were responsible for the data gathering process as a part of a larger research project. JA-T and AB were responsible for doing the bibliographical search and setting up the hypotheses, doing the statistical analyses and reporting the results. RH supervised the data analysis process as well as the reporting process of the results particularly the proofreading of the paper in English.

### FUNDING

This work is an outcome of the EU-funded collaborative research project "Cultural Pathways to Economic Self-Sufficiency and Entrepreneurship" (CUPESSE; Grant Agreement No. 613257; www.cupesse.eu). RH was funded by grant P30DA023026 from the National Institute on Drug Abuse during the writing of this manuscript. AB was also funded by grant ECO2016-80677-R.

Bates, T. (1995). Self-employment entry across industry groups. J. Bus. Ventur. 10, 143–156. doi: 10.1016/0883-9026(94)00018-P




J. Appl. Psychol. 91, 259–271. doi: 10.1037/0021-9010.91. 2.259

Zhao, H., Seibert, S. E., and Lumpkin, G. T. (2010). The relationship of personality to entrepreneurial intentions and performance: a meta-analytic review. J. Manag. 36, 381–404. doi: 10.1177/0149206309335187

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Arco-Tirado, Bojica, Fernández-Martín and Hoyle. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Creativity, Proactive Personality, and Entrepreneurial Intention: The Role of Entrepreneurial Alertness

Rui Hu1,2, Li Wang<sup>1</sup> , Wei Zhang<sup>3</sup> and Peng Bin<sup>1</sup> \*

<sup>1</sup> College of Public Administration, Huazhong Agricultural University, Wuhan, China, <sup>2</sup> Institute of International and Comparative Education, Beijing Normal University, Beijing, China, <sup>3</sup> School of Education Science, Jiangsu Normal University, Xuzhou, China

This study examines the extent to which entrepreneurial alertness mediates the effects of students' proactive personalities and creativity on entrepreneurial intention. Drawing on a field survey of 735 Chinese undergraduates at 26 universities, this study provides evidence for the argument that entrepreneurial alertness has a fully mediation effect on the relationship between creativity, a proactive personality, and entrepreneurial intention. The findings shed light on the mechanisms that underpin entrepreneurial alertness and contribute to the literature on key elements of the entrepreneurial process.

Keywords: creativity, proactive personality, entrepreneurial alertness, entrepreneurial intention, mediation effect

#### Edited by:

Jesus de la Fuente, University of Almería, Spain

#### Reviewed by:

Juan Antonio Moriano, Universidad Nacional de Educación a Distancia (UNED), Spain Sharinaz Hassan, Curtin University, Australia

> \*Correspondence: Peng Bin BINPENGCO@163.com

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 29 November 2017 Accepted: 23 May 2018 Published: 14 June 2018

#### Citation:

Hu R, Wang L, Zhang W and Bin P (2018) Creativity, Proactive Personality, and Entrepreneurial Intention: The Role of Entrepreneurial Alertness. Front. Psychol. 9:951. doi: 10.3389/fpsyg.2018.00951

## INTRODUCTION

Given rising unemployment rates and the richer opportunities offered by rapid globalization and marketization, more and more college students are opting to start their own businesses. A large body of empirical studies has tried to ascertain the extent to which entrepreneurship contributes to economic growth by creating new jobs and encouraging innovation. Given that researchers have highlighted the importance of intentions as antecedents of behavior in recent years (Shook et al., 2003), investigating the factors that influence individual intention to pursue entrepreneurship could accelerate the development of the entrepreneurial process. Such research could shape individual entrepreneurial intention, providing crucial support for both theory and practice. Although the relationship between different factors and entrepreneurial intention has been widely studied, the decision processes that facilitate individual entrepreneurial behavior remain an open issue (Markman et al., 2002; Zampetakis, 2008).

Scholars have debated the vital role played by entrepreneurial alertness in the entrepreneurial process. "The value of examining entrepreneurial alertness is that it concerns the individual's awareness, assessment, and orientation toward uncertainties and changes in the external environment and context—beyond the within-person, internal issue of identity" (Uy et al., 2015). The influences of proactive personality and creativity on entrepreneurial desirable and intention have also been widely studied (Crant, 1996; Zampetakis, 2008). Hansen et al. (2011) have argued that creativity and proactive personality are the main factors influencing entrepreneurial intention because they are closely linked with identifying opportunities; they serve an important function in encouraging new enterprises. However, to the best of our knowledge, the relationships between entrepreneurial alertness, creativity, proactive personality, and entrepreneurial intentions are still under-explored; few studies have tested the indirect effect of creativity or proactive personality on entrepreneurial intention outside Western culture. It is therefore important to connect entrepreneurial alertness and proactive personality with entrepreneurial intention to provide new

theoretical and practical insights. To tackle this timely issue, we have used a university student sample from the Chinese mainland to examine whether, and the extent to which, creativity and proactive personality are associated with entrepreneurial alertness and intention.

The paper is structured as follows. Section "Literature Review and Hypothesis Development" briefly reviews previous studies that touch on the relationship between proactive personality, creativity, and entrepreneurial intention. Section "Aims and Hypotheses" introduces our hypotheses. The methodology and data are presented in section "Materials and Methods." Section "Results" reveals the relationships among the targeted variables. The paper concludes with a discussion of the contributions and limitations of this study (see section "Discussion and Conclusion").

### LITERATURE REVIEW AND HYPOTHESIS DEVELOPMENT

### Entrepreneurial Alertness

One of the most significant topics in the field of entrepreneurship is the awareness of entrepreneurial opportunities, previously addressed by Campos (2016). Israel Kirzner's theory of entrepreneurial alertness has added value to the research on opportunity identification. Kirzner (1973) was the first to define the role played by entrepreneurial alertness in helping entrepreneurs become more aware of new opportunities and use limited clues in different ways. In further research, Kirzner (1979) has defined entrepreneurial alertness as an individual's ability to perceive new opportunities that have hitherto been overlooked by others; more specifically, alertness can be described as: "a motivated propensity of man to formulate an image of the future" (Kirzner, 1985). In accordance with Kirzner's research, Baron and Ensley (2006) have argued that alert individuals "identify new solutions to market and customer needs in existing information, and to imagine new products and services that do not currently exist" (Baron and Ensley, 2006). A central study by McMullen and Shepherd (2006) has confirmed that, "To act on the possibility that one has identified an opportunity worth pursuing" is the heart of being an entrepreneur (McMullen and Shepherd, 2006). Gaglio and Katz (2001) have asserted that a high level of entrepreneurial alertness leads to acute sensitivity to one's surroundings, a mental framework that helps entrepreneurs adjust to the current situation. Based on previous research, Tang et al. (2012) have produced a 13-item model, which theoretically and empirically divides entrepreneurial alertness into three distinct factors: (a) "scanning and search," (b) "association and connection," and (c) "evaluation and judgment." Tang's instrument is well-validated and has strong reliability, providing researchers with a valuable tool for probing the entire entrepreneurial opportunity-development process, including antecedents and outcomes.

In the Chinese context, Li et al. (2015) and Hu and Ye (2017) have studied Chinese university students, concluding that entrepreneurial alertness directly and significantly predicts opportunity recognition. The goal of alertness research is to discover not only the antecedents of alertness, but also its outcomes (Kirzner, 2009). The present study aims to reveal whether, and the extent to which, entrepreneurial alertness mediates the effects of creativity and a proactive personality on entrepreneurial intention.

### Creativity, Proactive Personality, and Entrepreneurial Alertness

Creativity significantly influences an individual's entrepreneurial alertness. Creativity, which refers to the development of novel and useful ideas, is closely related to innovation and generally studied at the individual level (Amabile, 1996; Zampetakis, 2008). Furthermore, the interaction between an individual and his or her environment will yield creativity (Hunter et al., 2007). Factors such as an individual's intrinsic motivation, personality, knowledge, cognitive style, and social background can play important roles in the cultivation of creativity (Woodman and Schoenfeldt, 1990; Sternberg and Lubart, 1995; Amabile, 1996).

Creativity has become a central theme in entrepreneurial process research. The relationship between creativity and enterprise has been confirmed by researchers: novel ideas are produced when access to information is clear and free (Biraglia and Kadile, 2017); such new ideas exemplify the nature of creativity and lead to new entrepreneurial ventures (McMullan and Kenworthy, 2015). Creativity is a key element at the start of the entrepreneurial process, since it contributes to the design of new products and services (Heinonen et al., 2011; Gielnik et al., 2012). Feldman and Bolino (2000) have argued that individuals with higher perceived creativity are more likely to build up their own businesses; this confirms the findings of Sternberg and Lubart (1999), who have noted that entrepreneurship is, to some extent, the result of creativity.

Given the importance of identifying entrepreneurial opportunities in the field of entrepreneurship, experts maintain that alertness about opportunities is important, not only for innovation behavior (Obschonka et al., 2017), but also for successful entrepreneurial behavior (Baron, 2006). Kirzner (2009) believes that entrepreneurial alertness involves creative behavior. This, to some extent, confirms the conclusion drawn by Ardichvili et al. (2003) that a conceptual link exists between creativity and entrepreneurial alertness. Empirical studies have confirmed that creative activities involving the use of new communication and Internet technologies may also predict entrepreneurial alertness. Campos (2016) and Obschonka et al. (2017) have empirically confirmed that creativity has a significant relationship with entrepreneurial alertness, particularly in the dimensions of scanning and searching. Campos (2016) has used a moderated mediation model to identify a positive correlation between creativity and entrepreneurial alertness. Alertness requires a creative act, which may influence the further development and improvement of entrepreneurial opportunities (Baron, 2004; Kirzner, 2009). Thus, creativity is associated with entrepreneurial alertness.

The proactive personality is also associated with entrepreneurial alertness. "Proactivity refers to active attempts made by the individual to effect changes in his or her

environment" (Zampetakis, 2008). According to Bateman and Crant (1993), people with proactive personalities tend to take the initiative to influence and even to significantly change the environment. In other words, having a proactive personality can help an individual release situational pressures, identify opportunities for advantage, make proactive moves, and thereby influence the environment to create meaningful changes (Bateman and Crant, 1993). Alongside previous research on environmental adaptability (Crant, 1995, 2000) and proactivity, additional empirical studies have confirmed that proactive individuals are more likely to achieve success at work and more positively attuned to the need for dominance, achievement, self-confidence, and conscientiousness (Claes et al., 2005). Previous studies have identified a positive relationship between proactive personality and entrepreneurship (e.g., Becherer and Maurer, 1999; Kickul and Zaper, 2000). For instance, Becherer and Maurer (1999) have related proactivity to starting rather than buying or inheriting a business, as well as to the number of businesses started.

Proactive personality is seen as the crucial antecedent of entrepreneurial alertness to opportunities because opportunity identification is an important aspect of individual initiative (Ardichvili et al., 2003; Tang et al., 2012). Using a survey that researched a diverse group of undergraduate students from Singapore, Uy et al. (2015) have shown that entrepreneurial alertness partially mediates the relationship between a proactive personality and a boundless career mindset, and confirming that personality traits, such as proactive personality, are connected to entrepreneurial alertness. Obschonka et al. (2017), using twowave longitudinal data from high schools in Helsinki, Finland, has confirmed that entrepreneurial alertness is predicted by different underlying competencies; furthermore, both creativity and proactivity improve the link between personality and entrepreneurial alertness.

### Entrepreneurial Alertness and Entrepreneurial Intention

Entrepreneurial alertness is closely associated with individual entrepreneurial intention. Entrepreneurial intention is "a selfacknowledged conviction by a person that they intend to set up a new business venture and consciously plan to do so at some point in the future" (Thompson, 2009). It constitutes a more or less concrete plan to prepare for, and then ultimately start, an entrepreneurial career of one's own in the future (Obschonka et al., 2017). Entrepreneurial intention plays a crucial role in shaping an individual's entrepreneurial behaviors (Markman et al., 2002), "the stronger a person's intention to engage in a specific behavior, the more likely it is that the actual behavior will be performed" (Ajzen, 1991).

The connection between entrepreneurial alertness and entrepreneurial intention has been empirically confirmed by McMullen and Shepherd (2006), who have argued that entrepreneurial alertness improves the judgment and opportunity identification of individuals, helping to form entrepreneurial intention and future business behavior. In a Chinese context, Li et al. (2015) have confirmed that entrepreneurial alertness is an essential strength for entrepreneurs because it directly predicts opportunity recognition. Guided by social cognitive theory, Hu and Ye (2017), using a sample of 364 Chinese students majoring in sports, have confirmed that both entrepreneurial alertness and entrepreneurial self-efficacy are key cognitive predictors of entrepreneurial intention.

Having reviewed the existing literature, we therefore raise our hypotheses in section "Aims and Hypotheses."

### AIMS AND HYPOTHESES

As discussed in the Sections "Introduction" and "Literature Review and Hypothesis Development," the relationships between entrepreneurial alertness, creativity, proactive personality, and entrepreneurial intentions remain under-explored. Therefore, this study had the objective to fill the gap in the literature in two ways. First, resorting to a nationally representative survey of entrepreneurship of Chinese college students, this study aims to investigate the relationships among creativity, proactive personality, entrepreneurial alertness and entrepreneurial intention in the Asian culture. Second, the current study mainly focuses on how the entrepreneurial alertness mediates the effects of students' proactive personalities and creativity on entrepreneurial intention. This mediation effect of entrepreneurial alertness had not yet been investigated in the context of this theory. Accordingly, this study raises the following hypotheses:


To examine the four hypotheses, the current study extends from the previous wisdom and designs the model with the mediator. The operationalization of the mediating model in current study is described in **Figure 1**.

### MATERIALS AND METHODS

### Participants

The sample was drawn from a nationally representative survey. A total of 857 undergraduate students from 26 universities volunteered to participate in this survey. Of the initial dataset, 122 respondents were eliminated due to incomplete or questionable response patterns (e.g., selecting "5" as a response across an entire

section of the survey, which included reverse-worded items); this study ultimately produced 735 valid samples. To keep the sample unbiased, the survey selected respondents studying a wide range of fields: a fair distribution across 19 areas of specialization guaranteed a representative sample of students, enabling us to analyze the entrepreneurial intentions of college students in studying fields.

The participants were majoring in Agricultural Science (6%), Animal Science (7%), Aquaculture (7%), Biological Sciences (7%), Business Administration (5%), Chemistry (5%), Computing Science (4%), Economics (5%), Environmental Science (7%), Food Science (5%), Horticulture (7%), Law (5%), Marketing Management (4%), Mechanics (5%), Public Administration (4%), Sociology (5%), and Tea Study (5%). The participants were aged 18–22 with a mean of 20.02 years (SD = 3.11). Of the total sample, 59.6% (n = 438) were males and 40.4% (n = 297) were females, with mean ages of 20.77 years (SD = 5.75) and 19.76 years (SD = 1.60), respectively. The distribution by academic year was as follows: 48.2% were 1st year college students (n = 354); 33.3% were 2nd year college students (n = 245); 6.8% were 3rd year college students (n = 50); 7.6% were 4th year college students (n = 56); 3.5% were 1st year graduate students (n = 26); and 0.5% were 2nd year graduate students (n = 4).

### Instruments

All of the independent and dependent variables were evaluated using self-report measures based on multi-item scales. Responses across all dimensions and items were made using five-point Likert-type scales, ranging from 1 (strongly disagree) to 5 (strongly agree). All of the items were translated into Mandarin by a native speaker; they were re-translated into English by other bilingual individuals to guarantee that the scales had similar quality and connotations. The instruments and some items in the constructs are outlined below.

### Proactive Personality

The proactive personality scale used in this study was the sixitem scale adopted by Bateman and Crant (1993). The original scales were those of Claes et al. (2005). The proactive personality scale included items such as, "Regardless of the odds, if I believe in something, I will make it happen." The Cronbach's alpha was 0.845. All of the items accounted for 78.08% of the variance.

### Creativity

Creativity was measured using an eight-item scale developed by Zhou and George (2001). The items referred to the production of useful and creative ideas (Zampetakis, 2008). Examples included the following: "I develop adequate plans and schedules for the implementation of new ideas" and "I suggest new ways to increase the quality of project assignments." The Cronbach's alpha for all eight items was 0.919. All of the items accounted for 64.02% of the variance.

### Entrepreneurial Alertness

Entrepreneurial alertness was tested using Tang et al.'s (2012) scale, which categorizes 13 items into three factors: scanning and search, association and connection, and evaluation and judgment. Examples included the following: "I have frequent interactions with others to acquire new information" (scanning and search), "I often see connections between previously unconnected domains of information" (association and connection), and "When facing multiple opportunities, I am able to select the good ones" (evaluation and judgment). The alpha coefficient of the general alertness factor was 0.879. All of the items accounted for 64.15% of the variance with each of the three factors accounting for 45.89, 11.14, and 7.11%, respectively. The mean for entrepreneurial alertness was 3.48 (SD = 0.6).

### Entrepreneurial Intention

We assessed entrepreneurial intention using scales developed by Krueger et al. (2000). Although Krueger et al.'s (2000) scale included nine factors, this study dropped three factors and used a six-item scale to suit the Chinese context. For example, "I intend to start my own business in the near future." The alpha coefficient of the entrepreneurial intention scale was 0.915. All of the items explained 70.39% of the variance. The mean of entrepreneurial intentions was 2.74 (SD = 0.78).

### Control Variables

We controlled for the students' gender, age, and entrepreneurial role models (whether a parent or relative was an entrepreneur). According to Díaz-García and Jiménez-Moreno (2010), male students' express higher entrepreneurial intention than female students. Demographic factors appear to significantly affect entrepreneurial intention (Hockerts, 2017).

### Procedure

fpsyg-09-00951 June 12, 2018 Time: 17:17 # 5

To reduce the sample selection bias, our survey used various classrooms and collected the questionnaires randomly. Informed consent was obtained from all participants. Participants were asked to complete a questionnaire that included 5 sections and 27 questions, covering entrepreneurial intention, entrepreneurial alertness, creativity, and proactive personality, as well as demographic features. The participants were asked whether they agreed or disagreed with each question. This study did not involve any potential risk for the participants. They were also informed that participation was voluntary and anonymous; the data were protected by applicable legislation. Four members of the research team traveled to the various universities to conduct this survey.

### Data Analysis

The current study analyzed the relationships between each variable, using Pearson's correlation coefficient and SPSS version 22. We tested the theoretical model in **Figure 1** using structural equation modeling (SEM) via AMOS 17.0 and evaluated model fit using the Chi-square statistic, the Chisquare-to-degrees-of-freedom ratio, the goodness-of-fit index (GFI), the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the root-mean square error of approximation (RMSEA). Researchers have suggested that levels of 0.90 or higher for GFI, CFI, and TLI (the closer to 1, the better the index) and levels of 0.08 or lower for RMSEA (the closer to 0, the better the RMSEA) indicate that a model fits the data appropriately (Lance et al., 2007; Kim et al., 2009).

We then tested the research hypotheses by examining whether each structural path was statistically significant. In addition, we tested the mediation effects of entrepreneurial alertness on the relationship between creativity and proactive personality and entrepreneurial intention using a Chi-square difference test (Anderson and Gerbing, 1988; Hui et al., 1999). Specifically, we compared the mediated model (as proposed in **Figure 1**) to a full model that included direct effects. We then tested whether the χ <sup>2</sup> difference between the two models was significant or not. A non-significant χ <sup>2</sup> difference indicates a full mediation effect (Hui et al., 1999; Kim et al., 2009). For the mediation test, the following three conditions had to be met: (1) predictive and mediating variables have significant effects on the outcome variable; (2) predictive variables have significant effects on the mediating variables; (3) the effect of predictive variables on the outcome variable tends to weaken when mediating variables are added, while the effect of the mediating variables on the outcome variable remain significant (Wen et al., 2016). Finally, we used bootstrapping procedures (re-sampled 1000 times, with the percentile method used to create 95% confidence intervals) to further confirm the mediation effects.

## RESULTS

### Descriptive Statistics

A confirmatory factor analysis (CFA) was conducted to assess the discriminant validity of the measures. Our hypothesized four-factor structure was confirmed by the CFA results. The four-factor model had significantly lower Chi-square statistics [χ 2 (113) = 622.185, p < 0.01] than a one-factor model with all items loaded on a single construct [χ 2 (122) = 2926.02, p < 0.01]. This is confirmed by the fact that other indices of the four-factor model (RMSEA = 0.078; GFI = 0.902; CFI = 0.922; TLI = 0.906) fit better than those of the one-factor model (RMSEA = 0.177; GFI = 0.596; CFI = 0.57; TLI = 0.521). As expected, each item loaded on its hypothesized factor with large and significant loadings; each construct extracted a variance that was larger than the highest variance it shared with any other construct, thus providing support for discriminant validity.

**Table 1** presents the descriptive statistics, reliability estimates, and correlations for all measures. It can be clearly observed that the reliabilities of all variables are in the reasonable range (0.79–0.88). A strong correlation can be observed both between entrepreneurial intention and its antecedents and between these antecedents. The main statistics in **Table 1** deliver the following messages: (1) both creativity and proactive personality are significantly related to entrepreneurial alertness and entrepreneurial intention; (2) the control variables (gender and business-owning parent) are both significantly related to entrepreneurial intention. These findings suggest that students who are more creative, proactive, and alert are more likely to start their own businesses.

### Assessment of the Structural Model

To test the hypotheses, we carried out a structural equation analysis of the relationships between creativity, proactive personality, entrepreneurial alertness, and entrepreneurial intention. In particular, we examined whether creativity and a proactive personality were both positively related to entrepreneurial alertness (Hypotheses 1 and 2), whether entrepreneurial alertness was significantly related to entrepreneurial intention (Hypothesis 3), and whether or not the effect of entrepreneurial intention was mediated by entrepreneurial alertness (Hypotheses 4a and 4b), as shown in **Figure 1**. Model 1 represents the predicted mediating model. We drew paths from creativity and proactive personality to entrepreneurial alertness and from entrepreneurial alertness to entrepreneurial intention. As shown in **Table 2**, Model 1 represented a good fit to the data (RMESA = 0.065, GFI = 0.91, CFI = 0.922, TLI = 0.91).

We adopted a second model to further test the mediation effect. As shown in **Table 2**, Model 2 includes two additional direct paths: from creativity and proactive personality to entrepreneurial intentions. Consistent with Model 1, the control variables (gender, age, and parent owns a business) are included in Model 2. As shown in **Table 2**, the difference in Chi-square between Model 1 and Model 2 was not significant (1χ <sup>2</sup> = 1.201, 1df = 2). The other indices were almost unaffected by including



N = 735. Internal reliabilities are in parenthesis.

fpsyg-09-00951 June 12, 2018 Time: 17:17 # 6

<sup>a</sup>Gender is coded-1: male 2: female.

<sup>b</sup>Parent owning a business is coded-1: yes 2: no.

∗∗p < 0.01.

TABLE 2 | Comparison of structural equation models (Models 1 and 2).


N = 735, χ 2 : Chi-square statistic.

Model 1: Creativity + proactive personality→ entrepreneurial alertness→ entrepreneurial intention (including gender, age, and parent owning a business as control variables).

Model 2: Creativity + proactive personality <sup>R</sup> entrepreneurial alertness <sup>R</sup> entrepreneurial intention (additional line: creativity + proactive personality <sup>R</sup> entrepreneurial intention; including gender, age and parent owning a business as control variables).

∗∗∗p < 0.001.

the two additional paths in the model (RMESA = 0.065, GFI = 0.91, CFI = 0.922, TLI = 0.91). The insignificant 1χ 2 /1df also suggests a complete mediation effect of entrepreneurial alertness. According to Kelloway (1998), the parsimonious model should be adopted if similar results are obtained from two competing models; for this reason, Model 1 has been adopted in current study.

To better illustrate the mediation effect of entrepreneurial alertness, a bootstrapping regression has been carried out to further demonstrate Model 1. **Table 3** shows the results of the bootstrapping regression, demonstrating that the value of the mediation effect of entrepreneurial alertness is 0.232, between creativity and entrepreneurial intention, in the confidence interval (0.164, 0.307) and 0.253, between proactive personality and entrepreneurial intention, in the confidence interval (0.187, 0.345). This suggests that entrepreneurial alertness plays a significant intermediary role between independent variables (creativity and proactive personality) and entrepreneurial intention.

We predicted that creativity would be positively associated with entrepreneurial alertness. The significant parameter estimate was consistent with this, as shown in **Figure 2** (β = 0.46, p < 0.001). **Figure 2** also shows that proactive personality was positively and significantly related to entrepreneurial alertness (β = 0.50, p < 0.001). In addition, the parameter estimate displayed in **Figure 2** indicates that entrepreneurial alertness was positively and significantly related to entrepreneurial intention (β = 0.50, p < 0.001).

To further test Hypotheses 4a and 4b, Model 3 removed the mediator (entrepreneurial alertness) from the initial model. The results of this comparison are shown in **Table 4**.

In **Table 4**, Model 3 shows that creativity has direct and significant effects on entrepreneurial intention (β = 0.28, p < 0.001), meanwhile, proactive personality has direct and significant effects on entrepreneurial intention (β = 0.20, p < 0.001). Interestingly, these effects become insignificant when the mediator is taken into the model. The values of β in Model 2 suggest that creativity (with β = 0.01) and proactive personality (with β = −0.12) are irrelevant with the entrepreneurial intention if entrepreneurial alertness performs as mediator.

### DISCUSSION AND CONCLUSION

The current research empirically supported Hypothesis 1, Hypothesis 2, and Hypothesis 3. The estimated path coefficients of our structural equation model confirm the positive effects of creativity and proactive personality on entrepreneurial alertness, and thus support the Hypotheses 1 and 2. Moreover, Hypothesis 3 is also proved by the positive and significant coefficient of entrepreneurial alertness on entrepreneurial intention (see **Figure 2**). A further comparison between models with and without mediator suggests that, the effects of the independent variables (creativity and proactive personality) on the dependent variable (entrepreneurial intention) have been mediated completely by the mediator (entrepreneurial alertness). Hence, Hypotheses 4a and 4b are further confirmed.

Although rich, the literature on entrepreneurial intentions has neglected two important items: (1) the paths from creativity and proactive personality to entrepreneurial alertness and (2) the TABLE 3 | Bootstrapping regression results for creativity and proactive personality mediated by entrepreneurial alertness.


Dependent variable: entrepreneurial intention (EI).

N = 735, bootstrap sample size = 1000.

fpsyg-09-00951 June 12, 2018 Time: 17:17 # 7

TABLE 4 | Comparison of structural equation models (Models 2 and 3).


N = 735.

Model 2: as in Table 2.

Model 3: creativity + proactive personality→ entrepreneurial intention (without mediator).

†Marginal significant.

∗∗∗p < 0.001.

mediation effect of entrepreneurial alertness on entrepreneurial intention. Drawing on a nationally representative survey, this study has filled this gap in the literature and provided evidence of the combined effects of creativity and proactive personality on entrepreneurial alertness and entrepreneurial intention. This study also suggests that creativity, proactive personality, and entrepreneurial alertness are three key factors in the entrepreneurship process.

Most fundamentally, we have found that the proactive personalities and creativity of students are positively related to their entrepreneurial alertness, which in turn influences entrepreneurial intention. As predicted, our study demonstrated that the effects of creativity and proactive personality on entrepreneurial intention were mediated through entrepreneurial alertness. The mediation model showed that the students who scored higher in creativity and proactive personality tended to have stronger entrepreneurial intention. This can be

ascribed to the contribution made by perceived entrepreneurial alertness during the formation of entrepreneurial intention. This study not only reconfirms the common view that the alertness to opportunities is a cognitive characteristic, but also confirms that creativity and a proactive personality manifest themselves in personal entrepreneurial intention through mediating processes and mechanisms. In addition, as Zampetakis (2008) has noted, "to date, however, researchers have not examined the central theoretical role creativity might play in explaining the relationship between proactive behavior and entrepreneurial intentions." The current study argues that important role of creativity must not be neglected when we study the effects of cognitive variables on entrepreneurial intentions. This study demonstrates the influence of creativity and proactive personality on entrepreneurial intentions. It extends the current entrepreneurial alertness literature by addressing their mediating effect on

the relationship between personal traits and entrepreneurial intention.

Another important contribution of current study lies in improving the understanding of the importance of entrepreneurial alertness and its related mechanisms. Most alertness studies have focused on identifying the antecedents, rather than the outcomes, of alertness. Since little research has been carried out on the consequences of alertness, the value of present study may relate to the fact that it confirms that entrepreneurial alertness is positively and significantly related to entrepreneurial intention. In other words, it is valid to state that there is a direct path from entrepreneurial alertness to entrepreneurial intention (McMullen and Shepherd, 2006). The present findings show that entrepreneurial alertness is a vital factor to consider when exploring the way in which personal traits influence entrepreneurial intentions. It contributes to the field of entrepreneurship research by showing that cognitive variables play a crucial role in developing theories related to the entrepreneurial process (Hermann et al., 2007).

This study is particularly significant because it traces the link between personal traits and intention in the context of a non-Western culture. Given that most studies of entrepreneurial alertness and entrepreneurial intention have been carried out in Western countries, in particular, the United States, our findings are noteworthy in empirically confirming that entrepreneurial alertness is associated with intention in a non-United States culture. These results also show that the college students with higher entrepreneurial alertness tend to exhibit stronger entrepreneurial intentions in mainland China, supporting previous theoretical conclusions in a range of different contexts. In effect, it shows that McMullen and Shepherd's (2006) results can be replicated in an international context.

The present study has some practical implications, as it can help policy makers, university administrators, and teachers to design and implement relevant interventions to enhance students' entrepreneurial intentions. In particular, this study points to key antecedents of entrepreneurial alertness. As it has been argued that alertness can be developed and cultivated (Baron, 2004); this reinforces the idea that entrepreneurship education should focus not only on the technical aspects of entrepreneurship, such as business planning, but also on personal traits (Heinonen and Poikkijoki, 2006; Zampetakis, 2008). As creativity and proactive personality are crucial components of the entrepreneurial process, it is also essential to cultivate creativity

### REFERENCES


and positive personality traits. In China, entrepreneurship education must move beyond traditional examination-oriented teaching styles to focus on promoting positive personal traits, such as a proactive personality and creativity. Given the potential value of creativity and a proactive personality, university training programs could be designed to facilitate cognitive processes involving entrepreneurial alertness (Solesvik et al., 2013), to enhance university students' entrepreneurial orientation, particularly in relation to building their own businesses (Zampetakis, 2008). The policy interventions need thus to be targeted so as to encourage the college students to start their own business.

### Limitations

Before closing this section, some potential limitations shall be addressed of the analysis. Although a large-scale survey was carried out to guarantee the representativeness of the sample, field surveys of this type are highly dependent on self-reported questionnaires. This can lead to common method variance; as a result, the co-variance between the explanatory variables and the explained variable could not be eliminated. Although the internal consistency of our raters was high, the ratings are subjective. In future, researchers could consider longitudinal studies to further confirm which individuals become successful entrepreneurs and what kinds of cognitive feature significantly contribute to their entrepreneurial behavior. Other important variables, such as risk preference and entrepreneurial passion, as well as control variables such as prior entrepreneurial education and experience could be highlighted in studies of entrepreneurial intention.

### AUTHOR CONTRIBUTIONS

RH designed the work. RH and WZ designed the field survey. LW assisted in the data collection. RH, WZ, and PB analyzed the data and experiment result. LW assisted in the data analysis. RH and PB wrote the manuscript.

### FUNDING

This study was supported by National Natural Science Foundation of China (71603094), China Postdoctoral Science Foundation (2017M610788), Technological Innovation Foundation of Hubei (2018ADC087), and Educational Research Foundation of Hubei (2016171).


experienced entrepreneurs. Manag. Sci. 52, 1331–1344. doi: 10.1287/mnsc.1060. 0538


Zhou, J., and George, J. M. (2001). When job dissatisfaction leads to creativity: encouraging the expression of voice. Acad. Manag. J. 44, 682–696. doi: 10.2307/ 3069410

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Hu, Wang, Zhang and Bin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Assessment of Eight Entrepreneurial Personality Dimensions: Validity Evidence of the BEPE Battery

Marcelino Cuesta<sup>1</sup> \*, Javier Suárez-Álvarez<sup>2</sup> , Luis M. Lozano<sup>3</sup> , Eduardo García-Cueto<sup>1</sup> and José Muñiz<sup>1</sup>

<sup>1</sup> CIBERSAM, Department of Psychology, University of Oviedo, Oviedo, Spain, <sup>2</sup> Directorate for Education and Skills, Organisation for Economic Co-operation and Development, Paris, France, <sup>3</sup> Mind, Brain and Behavior Research Center, Department of Methodology of Behavioral Sciences, University of Granada, Granada, Spain

#### Edited by:

Douglas F. Kauffman, Medical University of the Americas, United States

#### Reviewed by:

Jesús Nicasio García Sánchez, Universidad de León, Spain Yuejin Xu, Murray State University, United States

> \*Correspondence: Marcelino Cuesta mcuesta@uniovi.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 30 March 2018 Accepted: 09 November 2018 Published: 29 November 2018

#### Citation:

Cuesta M, Suárez-Álvarez J, Lozano LM, García-Cueto E and Muñiz J (2018) Assessment of Eight Entrepreneurial Personality Dimensions: Validity Evidence of the BEPE Battery. Front. Psychol. 9:2352. doi: 10.3389/fpsyg.2018.02352 Background: The study of entrepreneurial activity has undergone intense development in recent decades. Traditionally this topic has been addressed from three approaches: economic, sociological and psychological. In the study of enterprising personality, two fundamental perspectives stand out: the use of general personality traits, like the Big Five, and the use of more specific traits related to entrepreneurial spirit, such as selfefficacy, autonomy, innovation, optimism, and others. The objective of this study is to provide validity evidence for a new instrument for measuring eight specific dimensions of entrepreneurial personality (BEPE).

Methods: The sample was composed of 1,170 adults from the general population (59.9% women). The average age was 42.34 years with a standard deviation of 12.96. Of the sample, 13% were self-employed. Internal factorial structure and reliability of BEPE were examined. The relationships with other variables and the discriminative capacity of the BEPE between different groups of workers were analyzed.

Results: First order exploratory factor analyses show the essential unidimensionality of each of the eight proposed sub-scales, with factorial weights ranging between 0.341 and 0.825. In the Confirmatory Factor Analysis, the best fit was achieved with a Bifactor model. With regards to reliability, the eight BEPE sub-scales gave high alpha coefficient values, between 0.81 and 0.89, as did the total battery (0.97). BEPE sub-scales show a high canonical correlation with the Big Five personality factors (0.796) and with the sub-scales of the Measure of Entrepreneurial Talents and Abilities questionnaire (0.779).

Conclusion: The BEPE questionnaire for the evaluation of the eight fundamental specific dimensions of the entrepreneurial personality presents adequate psychometric properties. Its relationships with other measures of personality traits are in line with what is expected. Therefore, the BEPE is a new measurement instrument that can be used with confidence both in the applied field and in research.

Keywords: personality, entrepreneurs, assessment, validity evidence, Big Five

## INTRODUCTION

fpsyg-09-02352 April 8, 2019 Time: 19:53 # 2

Entrepreneurial activity is considered to be a crucial element in the development of a market economy (OECD/The European Commission, 2013), which is why it is regularly monitored by large international organizations in various countries (OECD, 2017; Global Entrepreneurship Monitor [GEM], 2018). Its importance has led to a substantial increase in research in recent years (Rauch and Frese, 2007a; Sánchez, 2011; Liñán and Fayolle, 2015; Suárez-Álvarez and Pedrosa, 2016; Chandra, 2018). A multidisciplinary approach has been predominant, with three main branches, economic, sociological, and psychological, the latter including cognitive, emotional, attitudinal and personality aspects. There are two main lines of investigation in research into personality characteristics of entrepreneurs, and those two lines provide the basis for this study. On the one hand, there are those who consider the Big Five type personality traits to be the appropriate paradigm for studying entrepreneurs' personality characteristics (Zhao et al., 2010; Brandstatter, 2011), and on the other, those who propose using more specific traits linked to entrepreneurial spirit (Rauch and Frese, 2007a,b; Almeida et al., 2014; Suárez-Álvarez et al., 2014). Those who hold the former position invoke the predictive capacity of classic Big Five personality models, while those in favor of the latter advocate the use of specific traits to account for entrepreneurial personality. Empirical results suggest that specific traits would have added predictive capability compared to more general models (Leutner et al., 2014).

There is a great tradition of evaluating general personality traits and a profusion of measuring instruments for that purpose, whereas the evaluation of specific entrepreneurial personality traits is relatively recent, as are the instruments which have emerged. Notable instruments include the Entrepreneurial Aptitude Test (Favretto et al., 2003), the Skills Confidence Inventory (Betz et al., 2005), General Enterprising Tendency (Caird, 2006), the Entrepreneurial Intention Questionnaire (Liñán and Chen, 2006), the Entrepreneurial Guidance Questionnaire (Sánchez, 2010), the Measure of Entrepreneurial Talents and Abilities, META (Ahmetoglu et al., 2011), and the High Entrepreneurship. Leadership and Professionalism, HELP (Di Fabio et al., 2016). In an International context, the META questionnaire has probably been the most widely accepted (Ahmetoglu et al., 2011; Almeida et al., 2014; Leutner et al., 2014). A detailed analysis of these instruments' characteristics can be found in Suárez-Álvarez and Pedrosa (2016). Most of these measuring instruments have focused on a specific trait in an entrepreneur's personality, so there are no comprehensive, exhaustive, systematic analyses of entrepreneurial personality because the instruments each focus on a single important dimension. It is also surprising that their development and analysis has not made use of the advantages of Item Response Theory models (van der Linden, 2016).

To attempt to overcome these limitations, the BEPE battery was developed. Initially, the development was made for young people as this is a key period in the emergence of entrepreneurial initiatives (Damon et al., 2015; Obschanka et al., 2017). This evaluates eight dimensions of entrepreneurial personality that were identified following an exhaustive literature review. The BEPE battery for young people and adolescents exhibits good psychometric properties, both in its classic version (Muñiz et al., 2014; Suárez-Álvarez et al., 2014), and its computerized adaptive version (Pedrosa et al., 2016). The main objective of this study is to adapt the BEPE battery to an adult population and to provide evidence of validity to support its use in research and applied situations. Evidence of validity will be gathered about internal structure, the relationship to other variables, and the discriminative capacity between different groups of selfemployed and employees.

### MATERIALS AND METHODS

### Participants

The initial sample was made up of 1,324 volunteers from the general adult population, found by a non-random snowball procedure. The participants were contacted via email and responded to various measurement instruments on the internet. The procedure was as follows: the members of the research team contacted personally with known people who fulfilled the desired characteristics (over 18 years of age and working). These people were asked for their e-mail as well as the e-mail of other possible participants. We contacted them individually (via e-mail) asking them to collaborate in the research and to provide new e-mails from people who could participate in the research. The procedure was kept active for 3 months. The initial sample was reduced to 1,170 participants, 59.9% women, after eliminating those who exhibited insufficiently rigorous behavior when responding to the questionnaires as measured by a control of attention scale. The mean age was 42.34 years, with a standard deviation of 12.96. Of the participants who were evaluated, 13% were self-employed.

### Instruments

### Battery for the Assessment of the Enterprising Personality (BEPE)

This is a questionnaire which evaluates the eight specific personality dimensions identified in the literature as the most promising when characterizing entrepreneurial personality: Selfefficacy, Autonomy, Innovativeness, Internal locus of control, Achievement motivation, Optimism, Stress tolerance, and Risktaking (Baum et al., 2007; Rauch and Frese, 2007a,b; Muñiz et al., 2014; Suárez-Álvarez et al., 2014). The items making up the battery follow a Likert-type format with five answer categories (1 totally disagree, 5 totally agree), in line with established psychometric literature which indicates that between four and six answer categories produce better psychometric indicators (Lozano et al., 2008).

Each of the scales is briefly described below; for more detail on their definition and the process of construction see Suárez-Álvarez et al. (2014). Self-efficacy refers to a person's conviction that they can organize and carry out actions effectively, and their persistence when they encounter obstacles to reaching their goals (Costa et al., 2013). Autonomy refers to the motivation for entrepreneurial creation as an attempt to achieve a certain individual freedom (Van Gelderen and Jansen, 2006).

Innovativeness is about the will and interest in finding new ways to do things (Rauch and Frese, 2007b). Achievement motivation can be defined as the desire to achieve standards of excellence (Rauch and Frese, 2007b; Suárez-Álvarez et al., 2013). Internal locus of control is about the causal attribution of consequences of one's own behavior (Rauch and Frese, 2007b; Chell, 2008; Suárez-Álvarez et al., 2013). Optimism, is defined as the beliefs a person has about good things happening more than bad things in their life (Shepperd et al., 2002). Stress tolerance may be defined as the resistance to perceiving environmental stimuli as stressors thanks to the appropriate use of coping strategies (Lazarus and Folkman, 1984). Risk-taking is people's tendency and will to assume certain levels of risk or change to achieve an objective which offers more benefits than negative consequences (Moore and Gullone, 1996).

The original battery was designed for the evaluation of young people (Muñiz et al., 2014), and demonstrated adequate psychometric properties. The adaptation of this original version for the general adult population was carried out as follows. The 87 items in the original BEPE were reformulated to make the language suitable for an adult population, and new items were constructed aimed directly at the general adult population. Following a thorough literature review on the topic, work began with a bank of 161 items. This initial bank of items was evaluated by 15 experts in psychological assessment, who were asked to rate each item on a scale of 1 to 10 in terms of their suitability for a general population. Items scoring less than 8 were rejected or reformulated. Following this first filter, the items were given to 142 psychologists to evaluate the suitability of each item in terms of measuring the BEPE dimension it was supposed to address. Items scoring less than 9, on a scale of 1–10, were revised. Finally, a quantitative pilot study was performed with a sample of 132 participants. Discrimination indices were calculated (itemtest correlation) for the items, and an exploratory factor analysis was performed for each of the eight BEPE sub-scales. After eliminating items which did not meet psychometric quality criteria (discrimination index >0.20 and factorial loadings >0.3), each sub-scale finally consisted of 15 items (Muñiz et al., 2005).

### Measure of Entrepreneurial Tendencies and Abilities (META), (Ahmetoglu et al., 2011)

Measure of entrepreneurial tendencies and abilities (META) is a self-report scale with 44 items, which measures personality traits relevant to business success. It has four dimensions defined as follows (Ahmetoglu and Chamorro-Premuzic, 2013): Proactivity, the tendency to be proactive about projects and get things done (relates to energy, confidence and self-determination); Creativity, the ability to generate innovative business ideas (relates to non-conformity, originality and preference for novel experiences); Opportunism, the tendency to identify new business opportunities (relates to being alert, informed, and detecting future trends); and Vision, the ability to see the bigger picture, the motivation to bring change and create progress (relates to values and having a higher sense of purpose). The items are measured on a five-point Likert-type scale which ranges from "completely disagree" to "completely agree." The four scales exhibit appropriate values of internal consistency (Ahmetoglu and Chamorro-Premuzic, 2013): Proactivity (0.84), Creativity (0.83), Opportunism (0.86), Vision (0.76). In our sample the alpha values were: Proactivity: 0.70, Creativity: 0.81, Opportunism: 0.86, and Vision: 0.76.

### NEO Five Factor Inventory (NEO-FFI), (Costa and McCrae, 1985)

The Spanish version from Cordero et al. (2008) was used. This questionnaire evaluates the Big Five personality factors: Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness. Each scale is made up of 12 Likert-type items with five response categories ranging from total disagreement, to total agreement. It exhibits good psychometric properties, with sub-scale reliability coefficients above 0.80 (Cordero et al., 2008): Neuroticism (0.90), Extraversion (0.84), Openness (0.82), Agreeableness (0.83), and Conscientiousness (0.88). The alpha reliability coefficients calculated in our sample were: Neuroticism: α = 0.88 and ω = 0.90, Extraversion: α = 0.83 and ω = 0.86, Openness: α = 0.81 and ω = 0.83, Agreeableness: α = 0.73 and ω = 0.76, and Conscientiousness: α = 0.81 and ω = 0.85.

### Control of Attention Scale

This is a scale comprising 10 Likert-type items with 5 response categories. Its objective is to detect those participants who respond to evaluation instruments randomly or carelessly. The questions are of the type "In this question you must select the option completely agree." Participants responding incorrectly to two or more items were eliminated. By this criterion, 154 participants (11.6%) were removed from the study.

### Procedure

The measuring instruments were administered via the internet in the same order for all participants. An application developed "ad hoc" by the research team was used for administration of the instrument. It was not considered appropriate to carry out a time control since it is a typical execution scale. However, a scale was included to control the quality of the response (see instruments section). The average response time, estimated in the test phase, to the three tests (BEPE, META, and NEOFFI) was 40 min. Participants were contacted by email and completed the aforementioned instruments anonymously in a single session.

### Data Analyses

Following the model of similar works (Chen and Lin, 2018; Sinval et al., 2018) the data analysis was carried out in different phases. Following the Standards for Educational and Psychological Testing (American Educational Research Association [AERA] et al., 2014) different procedures were used in search of validity evidences. First, the characteristics of the eight scales of the BEPE were analyzed separately using the total of the participants (1179) and the 120 items selected in the pilot study. Discrimination indices were calculated (item-test correlation), and Exploratory Factor Analyses were performed with polychoric correlation matrices. ULS (Unweighted Least Squares) was used as an

extraction method, and the number of factors to retain was determined by parallel analysis (Horn, 1965), the percentage of variance explained, and the model fit indices based on study of residuals (GFI and RMSR), as they are the most suitable and independent of the method of estimation (Ferrando and Lorenzo-Seva, 2017). Model fit is considered adequate when GFI is greater than 0.09 and RMSR is less than 0.08 (Kline, 2011). The 10 items of each scale with the highest factorial load were selected.

In the second phase, the total sample was randomly divided into two subsamples. In the first subsample, of 390 participants, EFA was performed. In the second, of 780 participants, CFA models were adjusted. In the first subsample EFA of each scale was performed with the same characteristics as in the first phase and an exploratory Bifactor analysis was also carried out. Over the sample of 780 participants, three CFA models were adjusted: a model with 8 first-order factors, a model with 8 first-order factors and a second order factor and a Bifactor model with a general factor and 8 group factors WLSMV was used as estimator and Chi-square/df, RMSEA, CFI, and TLI were used as adjustment indices. It is considered that there is a good fit when Chi-square/df < 5, CFI and TLI > 0.95 and RMSEA < 0.08 (Hu and Bentler, 1999; Jackson et al., 2009). The change in CFI (1CFI) was also calculated. A 1CFI larger than 0.01, between nested models, indicates a meaningful change in model fit (Wang and Wang, 2012).

To look for validity evidence Pearson correlations between the BEPE sub-scales and the META and NEO-FFI sub-scales were calculated, along with the canonical correlation between BEPE and META sub-scales, and between BEPE and NEO-FFI sub-scales in order to understand the overall relationship between the two blocks of variables. In addition, in order to estimate the common variance between the blocks of variables, the redundancy coefficient was calculated. To analyze convergent validity evidence, the average variance extracted (AVE) was estimated as described in Fornell and Larcker (1981). Values of AVE ≥ 0.5 were considered adequate (Hair et al., 2009).

Evidence of discriminant validity understood as the items representing a dimension are not strongly correlated with other dimensions was assessed by comparing the AVE of the scales with the squared correlation of the scales (Fornell and Larcker, 1981; Marôco, 2014). For two factors x and y, if AVEx and AVEy ≥ r 2 xy (Fornell–Larcker criterion) there is evidence of discriminant validity.

The capacity of BEPE to differentiate between groups was examined via Multivariate Analysis of Variance (MANOVA), linear regression and binary logistic regression.

Scale reliability was calculated via the alpha coefficient (Cronbach, 1951) and ω de McDonald (McDonald, 1999).

Data were analyzed using SPSS24 (IBM Corp, 2016) for subsamples selection, correlations, MANOVA and regressions; EFA analyses were performed with FACTOR10.5.03 (Lorenzo-Seva and Ferrando, 2013) and Mplus8 (Muthén and Muthén, 2017) was used for CFA.

### RESULTS

### Factor Related Validity Evidence

The analysis of the test items was performed separately for each of the eight defined scales. Using the full sample item discrimination indices were calculated (item-test correlation), all of which were above 0.20, ranging from 0.287 to 0.705.

The Exploratory Factorial Analysis gave statistically significant values of Bartlett's Sphericity Index (p < 0.01) and Kaiser– Meyer–Olkin (KMO) indices above 0.85 in all cases. Factor loadings were between 0.341 and 0.825. In each of the eight scales GFI values were above 0.95 except for Internal locus of control (0.936). RMSR was below 0.08 with the exception of Locus of control (0.122) and Stress tolerance (0.094), and in all cases the percentage of variance explained by the first factor was over 30%. From these results, each scale may be considered essentially unidimensional (Kline, 2011). In order to reduce the number of items and achieve more homogeneous scales, the 10 items of each scale with the highest factor loadings were retained.

Using SPSS, the total sample was randomly divided into two subsamples (1/3 and 2/3). In the first subsample, of 390 participants, EFA was performed, separately for every reduced scale (10 items). The Exploratory Factorial Analysis gave statistically significant values of Bartlett's Sphericity Index (p < 0.01) and KMO indices above 0.87 in all cases. Factor loadings were between 0.478 and 0.879. In each of the eight scales, GFI values were above 0.98, RMSR was below 0.08, and in all cases, the percentage of variance explained by the first factor was over 45%. An exploratory Bifactor model (Reise, 2012; Cheng and Zang, 2018; Reise et al., 2018) that showed an adequate fit to the data was also tested (GFI = 0.991; RMSR = 0.0283).

In the subsample of 780 participants, three models of confirmatory factor analysis were adjusted: a model with 8 firstorder factors, a model with 8 first-order factors and a second order factor and a Bifactor model with a general factor and 8 group factors. As seen in **Table 1**, the Bifactor model is the one that presents a better fit to the data, confirming what was found in the exploratory analysis. In **Table 2** the factorial structure of the Bifactor model is shown.


#### TABLE 2 | Factor loadings of Bifactor Model.

fpsyg-09-02352 April 8, 2019 Time: 19:53 # 5


SE, self-efficacy; AU, autonomy; IN, innovativeness; IL, internal locus of control; AM, achievement motivation; OP, optimism; ST, stress tolerance; RT, risk-taking.

### Convergent Validity Evidence

Average variance extracted, calculated over the full sample, was satisfactory for some dimensions but slightly low in others: Self-efficacy = 0.54, Autonomy = 0.40, Innovativeness = 0.53, Internal locus of control = 0.47, Achievement motivation = 0.50, Optimism = 0.55, Stress tolerance = 0.40, and Risk-taking = 0.48.

### Discriminant Validity Evidence

The discriminant validity was assessed by comparing the AVE of the factors with the squared correlation of the factors (Marôco, 2014). Discriminant validity evidence is obtained when the AVE for two factors is larger than the squared Pearson correlation between the two factors. As shown in **Table 3**, discriminant validity is reached in all cases except Self-efficacy vs. Achievement motivation, Self-efficacy vs. Risk-taking and Innovativeness vs. Risk-taking.

### Reliability: Internal Consistency

The values of reliability coefficients were adequate for all scales and for the Total score. (**Table 4**).

### Relationship With Other Variables

**Table 5** shows the correlations between the BEPE and META scales, which ranged between 0.174 (Internal locus of control and Creativity) and 0.693 (Innovation and Creativity). The canonical correlation between the eight BEPE scales and the four META scales was 0.779, and the redundancy coefficient was 0.311, which indicates 31.1% common variance. The correlation between the total scores of the BEPE and the META is 0.692, indicating high convergence between the two measuring instruments.

**Table 6** shows the relationships between the eight BEPE scales and general personality traits as measured via NEO-FFI. The highest values are seen between Stress tolerance and


SE, self-efficacy; AU, autonomy; IN, innovativeness; IL, internal locus of control; AM, achievement motivation; OP, optimism; ST, stress tolerance; RT, risk-taking. <sup>∗</sup>Meets the criteria set by Marôco (2014).


Neuroticism (−0.738), and between Achievement motivation and Responsibility (0.618). The canonical correlation between both sets of variables is 0.796, with a redundancy coefficient of 0.287, which means that there is 28.7% common variance between the two tests.

### Differences Between Groups

Two MANOVA analyses were performed in order to examine the discriminative capacity of the test between participants grouped according to their work status. Firstly, two groups were created, people who worked for others (employees; n = 1018), and people who worked for themselves (self-employed; n = 152). Statistically significant overall differences were found (F8,<sup>1161</sup> = 4.371, p < 0.001, d = 0.35). The self-employed participants had higher scores in all BEPE scales, although that difference was only statistically significant in Autonomy (See **Table 7**).

To examine the discriminative capacity of the BEPE in more depth, the sample was divided into three groups: employees, employees who plan to become self-employed in the next few months (potential entrepreneurs), and self-employed. This produced three groups of 931, 87, and 152 participants, respectively. The overall differences were statistically significant (F16,<sup>2322</sup> = 3.676, p < 0.001, d = 0.32). In the Autonomy scale, the self-employed scored significantly higher than the employees group, in Innovation and Risk-taking, it was the potential entrepreneurs who were differentiated from the employees.

On the other hand, taking the total score in the META as a criterion, we selected 25% of subjects with lower scores and 25% of subjects with higher scores. Using a stepwise binary logistic regression, we inquired about which BEPE scales were capable of predicting belonging to these extreme groups. Three scales were selected: Innovativeness, Achievement motivation, Risk-taking (Nagelkerke's R <sup>2</sup> = 0.815 and percentage of correctly classified cases = 92.2%).

Finally, given that age can be a variable related to the entrepreneurial spirit, we have explored the relationship of the BEPE scales with age within each group of subjects (employees, potential entrepreneurs, self-employed) using stepwise regressions with age as a variable dependent and the BEPE scales as independent variables (See **Table 8**).

The predictive capacity of the two types of personality traits (general vs. specific), represented by the Big-Five model and the BEPE scales, was put to the test via binary logistic regression. In both cases the predictive capacity was low, although it was improved when the NEO-FFI block of variables was added to the BEPE scales (Nagelkerke's R <sup>2</sup> went from 0.055 to 0.068, an increase of 1.3%). If we use the employees group and the potential entrepreneurs group as criteria, the predictive capacity is slightly higher and follows the same pattern (Nagelkerke's R 2 goes from 0.060 to 0.070, an increase of 1%).

### DISCUSSION

Recent decades have seen a growing interest in the study of entrepreneurship from economic, social, and psychological perspectives. One of the main focuses of attention from a psychological perspective, in addition to cognitive factors, has been the study of entrepreneurs' specific personality

#### TABLE 5 | Pearson correlations between BEPE and META sub-scales.


TABLE 6 | Pearson correlations between BEPE and NEO-FFI sub-scales.


TABLE 7 | Differences between self-employed and employed workers in BEPE sub-scales.


d: Cohen's d.

TABLE 8 | Stepwise linear regressions by groups.


characteristics as opposed to the big personality traits described by models like the Big-Five. Despite this interest, there are not many measuring instruments designed to systematically and comprehensively evaluate these specific personality characteristics. This research addressed the adaptation of one of these tests to an adult population and the study of its psychometric properties. The test is the Battery for the Assessment of the Enterprising Personality (BEPE), originally created to evaluate adolescents. The primary objective of this research was to gather evidence that the structure of the BEPE test

(Muñiz et al., 2014; Suárez-Álvarez et al., 2014) for adolescents was conserved in the adult version. The results of the CFA show that the best fit model is the Bifactor with a general factor (entrepreneurship) and eight facets. The reliability coefficients are very high, both for the eight scales and overall. The internal convergent validity evidence is not completely satisfactory, but the discriminant validity was good. So, it seems reasonable to defend a Bifactor structure such as the CFA shows.

Another important part of this research was the search for evidence of convergent validity with external variables via the study of the relationship between the BEPE and the META, the most internationally well known scale for evaluating entrepreneurial personality from the point of view of specific traits The results show a good level of convergence whether we look at the correlational analysis or the discrimination between extreme groups. The correlation between the overall BEPE and META can be termed as good according to the criteria of the European Federation of Psychologists' Associations Test Review Model (Evers et al., 2013). In terms of the relationship between the specific-trait perspective and the general-personality-trait perspective, the results of the canonical correlation analysis and the correlations between the scales indicate a moderate relationship between the two approaches.

The discriminative capacity of the BEPE scale between different groups, established ad hoc according to participants' work status, was limited, although the trend was in the expected direction. These results about discriminating between groups must be taken with precaution, and we must wait for data from larger and representative groups of entrepreneurs. As Henrekson and Sanandaji (2014) indicated, the criterion for differentiating entrepreneurs from non-entrepreneurs is itself problematic. In this study we used being self-employed or not as the criterion for simplicity's sake, while being aware that being self-employed is not synonymous with entrepreneurial spirit (Hurst and Pugsley, 2011). The results seem be in line with that idea, indicating that discriminative capacity between groups is higher when the "potential entrepreneurs" are included with the self-employed. The predictive capacity of specific traits, tested with the regression model, though low, was in line with other results indicating that specific traits would add predictive capacity to the general trait model (Leutner et al., 2014). When interpreting the predictive capacity of personality dimensions it should be remembered that personality factors are only a small part of the multiple individual, social, cultural and contextual factors which can potentially influence entrepreneurship, as is well indicated by those general models which aim to explain entrepreneurial spirit (Rauch and Frese, 2007a; Suárez-Álvarez and Pedrosa, 2016).

Regarding the age variable, the relationship found within the group of "potential entrepreneurs" is interesting and invites to investigate the role of age, together with other variables, as modulators of the "entrepreneurial spirit" (Bohlmann et al., 2017).

In conclusion, the version of BEPE for the adult population replicates and improves on the psychometric properties of the original version for young people, and exhibits very good evidence of convergent validity. It is, therefore, a measuring instrument that may be used in research and applied contexts. When interpreting the results, and when using the instrument itself, certain limitations of this research should be borne in mind. It was not possible to have clearly defined groups of entrepreneurs and non-entrepreneurs to assess the instrument's discriminative capacity. It is precisely this limitation that provides the outline for future research to find validity evidence, which is already underway. Cross-cultural studies are also needed to evaluate entrepreneurial personality characteristics in different socio-cultural contexts (Byrne and van der Vijver, 2017). In order to carry out evaluations in applied contexts, a shorter form of the BEPE is needed, as is a computerized adaptation (Pedrosa et al., 2016; Nieto et al., 2017). The use of implicit measures to avoid the possible self-reports biases is another promising research line (Martínez-Loredo et al., 2018). Research into the entrepreneurial personality has only just begun, and the results are very promising, although there is a long road to follow. We believe that beginning with evaluation instruments is a good strategy, as they are the foundation that gives us to precise diagnoses, which in turn, will lead to effective interventions, the ultimate aim of all research in psychology.

### ETHICS STATEMENT

The study was not explicitly reviewed by an Ethics Committee, given that this is not required by our University of Oviedo, nor by the national guidelines established in the Code of Ethics of the Spanish Psychological Association. There are several reasons why an explicit approval by an Ethics Committee was not necessary: the participants evaluated were adults, the evaluation was voluntarily accepted, that is, an implicit informed consent is assumed, and the data is treated anonymously and confidentially. In addition, all the recommendations established in ISO-10667 Standard for the evaluation of people were strictly followed.

The whole evaluation process and the use of the measuring instruments were carried out always following the Deontological Code of the Spanish Psychological Association (2010), as well as the International Test Commission Guidelines for Test Use (2013).

Deontological Code of the Spanish Psychological Association (2010). Madrid: Consejo General de Colegios Oficiales de Psicólogos (www.cop.es).

International Test Commission Guidelines for Test Use (2013). International Test Commission: www.intestcom.org.

### AUTHOR CONTRIBUTIONS

JM, JS-Á, and EG-C contributed conception and design of the study. LL organized the database. MC performed the statistical analysis and wrote the first draft of the manuscript.

All authors contributed to manuscript revision, read and approved the submitted version. The views expressed in the paper represent the views of the individual authors and do not represent an official position of the Organisation for Economic Co-operation and Development.

### REFERENCES


### FUNDING

This research was funded by the Spanish Ministry of Economy and Competitiveness. References: PSI2014-56114-P and PSI2017-85724-P.



A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. J. Hughes (Hoboken, NJ: Wiley-Blackwell).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Cuesta, Suárez-Álvarez, Lozano, García-Cueto and Muñiz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Coming Together: R&D and Children's Entertainment Company in Designing APPs for Learning Early Math

Carlos Mera, Gonzalo Ruiz, Manuel Aguilar, Estíbaliz Aragón, Cándida Delgado, Inmaculada Menacho, Esperanza Marchena, Manuel García Sedeño and Jose I. Navarro\*

Department of Psychology, University of Cádiz, Cádiz, Spain

#### Edited by:

Jesus de la Fuente, University of Navarra, Spain

#### Reviewed by:

Maria Pietronilla Penna, Università degli Studi di Cagliari, Italy Ana Moreno-Núñez, Universidad Autónoma de Madrid, Spain

> \*Correspondence: Jose I. Navarro jose.navarro@uca.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 20 July 2018 Accepted: 20 December 2018 Published: 24 January 2019

#### Citation:

Mera C, Ruiz G, Aguilar M, Aragón E, Delgado C, Menacho I, Marchena E, García Sedeño M and Navarro JI (2019) Coming Together: R&D and Children's Entertainment Company in Designing APPs for Learning Early Math. Front. Psychol. 9:2751. doi: 10.3389/fpsyg.2018.02751 Information and Communication Technologies (ICT) have an increasing influence on the way we interact, learn, and live. The increase in teaching and learning methodologies that are mediated by ICT in the field of education and in the domestic settings encourages the design of new effective technological tools, supported by scientific research and development to improve student learning. The challenge psychology is facing in the education field is to promote those technologies and make them available to the education community. Technologies also would produce attractive items for users and realistic commercial issues for businesses. This also allows an effective transfer for scientific work, providing visibility to Research and Development. In this context, the main aim of the article is to describe the process to get an agreement between Babyradio (a children's entertainment company: https://babyradio.es/) and our research team, starting a collaborative work between two groups of people (Babyradio's technical designer and Psychologist-Engineers software designer), in order to create several educative applications (APPs) in the field of early mathematics cognition. The institutional framework of the relationship of the R&D project and a children's entertainment company is described. The article also focuses on experience in Psychology, Technological Innovation, and Entrepreneurship. In considering the efficiency of the agreement, we present different APPs designed for tablets and smartphone devices, adapted to the different operating systems (IOS, Android, Windows). APPs are designed to instill the cognitive fundamentals associated with early math learning for students aged 4 to 7 years. The study developed after this babyradio-university enterprise agreement contributes to the development of mathematics skills in children, aged 4–7 years, so that they can successfully meet the mathematics school requirements; it also contributes to encouraging a more positive attitude toward mathematics. This study also suggests how the education system and software and educational content developers' companies would manage verified instructional APPs, with a more realistic commercial perspective.

Keywords: APPs, early math, research transfer, spin-off, gamification

### INTRODUCTION

fpsyg-09-02751 January 24, 2019 Time: 12:1 # 2

Mathematical learning is an essential tool for school success and also has an important impact on personal adaptation to everyday life. Teaching mathematics in schools has received different criticisms from experts. They believe the method should be substantially modified in order to reduce the current failure in this matter (Martínez Montero, 2017). Significant efforts have been made in the last decade to change this tendency, although results have been very dissimilar. Methods such as "open calculation based on numbers" (ABN), have allowed some countries to substantially modify the teaching-learning process of mathematics starting with the pre-primary school levels (Aragón et al., 2017). But it is necessary to carry out studies that confirm advantages of this new type of methodology (Cerda et al., 2018).

Information and Communication Technologies (ICT) have an increasing influence on the way of interaction, learning and life. ICT are being used in different contexts to teach mathematics at different school levels (Kushwaha and Singhal, 2017). These new technologies are widely used in everyday life by children. The educational system must know how to take advantage of ICT and neutralize the disadvantages.

One of the weaknesses of the use of ICT in mathematics education is related to content design. Although there are many companies dedicated to making math educational materials, a line of research and development is necessary to guarantee the reliability and validity of the contents designed. In this area, there is still a very wide margin of improvement that must be accomplished.

The business awareness for research in the field of mathematics education is not new, but has existed for a few years. However, a larger effort is necessary for experimental research results to significantly influence users in the educational field. This relationship has been most evident in educational software development. Early mathematical assessment (Van Luit et al., 2015) and math software, are disseminated in the educational literature (Aragón et al., 2013). In these cases, the business interest has been closely linked to the agencies and research group's prestige. Companies have been able to take advantage of the final product if it has shown efficiency in the applied field. This is the case of ENT, a mathematical evaluation test that has been commercialized by several European companies in the Netherlands (Van Luit and Van de Rijt, 2009), Spain (Van Luit et al., 2015), or Italy (Benvenuto et al., 2018).

But the work that remains to be done in the field of research, development and innovation (R, D&I) of educational psychology is how to involve companies in an initial investment before the development of the research products. In this sense, this paper tries to offer a model of relationship between a company in the children's entertainment sector and educational research. That is, how to link a company – whose commercial interests need to be realistically understood – with the potential of developing a research project in the educational field. The research project can show corporate profitability expectations in the mid-term. In this sense, a connection has been found between the Babyradio Company<sup>1</sup> and our research group's goals through the design of ICT prototypes.

Babyradio is a radio station company whose contents are broadcast via FM and internet that is focused on entertainment for children from 0 to 6 years old. Its programming reaches 130 countries and more than 350,000 listeners. Among these contents, Babyradio has different children's characters that act as catalysts for children's attention through music, entertainment, story, and so forth. The company has highly qualified graphic designers, an R + D + I department. Babyradio's management team had prior experience in collaboration with research and development projects (Varela, 2014).

The connection of interests between Babyradio and our research group allowed us to develop a set of computer applications for mobile devices (APPs), taking advantage of the synergies between Babyradio's graphic design team (image and sound) and our prior experience in development of mathematical education software (Navarro et al., 2012).

Consequently, the main goals of this work are threefold: (1) to present the institutional framework of the relationship between the R&D project and Babyradio, a children's entertainment company; (2) to share the collaboration scheme that may have applied attention for other R&D projects; and (3) to show some of the educational prototypes generated by this research-business collaboration and its applied experimental development.

### OBJECTIVE #1: R&D PROJECT AND BABYRADIO, A CHILDREN'S ENTERTAINMENT COMPANY AGREEMENT

In a society with a globalized economy, the innovation capacity of research carried out by universities must contribute to their economic and social progress (Mu-Hsuan and Dar-Zen, 2017). This development can be done in different ways: investing in human capital or also strengthening research lines that generate added social value. Universities have as a main focus the generation of innovative and diverse knowledge. To achieve this, one of the possible ways is to establish mutual collaboration relationships with companies. This type of collaboration should be attractive both for the purposes of the company, as for those of the university, its students and its professors.

Although these relationships have to be formally established from the institutional point of view, it must be considered that the innovation parameters and the importance in the generation of socially sustainable knowledge are very significant values to establish these cooperation channels. Likewise, if we want this cooperation between R&D projects and companies to be possible, we must consider realistic timing and companies' objectives with which strategic agreements are sought.

These agreements should not only be done in the production industry's field, but also in socio-economic sectors related to services, such as education. In recent years, there has been more interest because education is also a field where entrepreneurship

<sup>1</sup>www.babyradio.es

can be promoted, and which can facilitate the making of spinoffs allowing future students to improve their employability. Certainly, the education sector has not been the main focus of the research interest, but a progressively emerging areas of opportunity. An example of this is found in the recent work developed by Fuente et al. (2008), generating a technology-based service company with the aim of influencing the educational field. The spin-off created offers from different professional services based on the results of research, development and innovation projects<sup>2</sup> .

Collaboration between corporations and research has been conceptualized as a complex process. There are different constructs involved such as cooperation, the formation of work teams or the coordination between both groups of interest (Rajalo and Vadi, 2017). We must bear in mind that innovation in the field of research is recognized as both a result and a process. In any case, when it is carried out within the university-company collaboration, it requires taking into account the organizational aspect of the cooperation, as well as its implementation in the field of scientific research. This company's R&D relationship identifies at least three types of dealings (Rajalo and Vadi, 2017). In the first place, it needs a multidisciplinary perspective capable of focusing the interests of research and the company from different configurations. Likewise, both institutions that collaborate will be immersed in a very rewarding constant process of bilateral learning. And the investment made in both fields should also affect the two sectors involved: the company and the research. This type of relationship must also be evaluated to verify the degree of efficiency and if both institutions achieve the proposed objectives.

Within this general philosophy, we have developed a mutual collaboration agreement between Babyradio Company and the R&D project carried out by the Department of Psychology, University of Cadiz-Spain (UCA). The main objective of this collaboration was to give productive a output to the research results developed since the R&D project, related to mathematical learning for children in early childhood and primary education. The bottom line for this agreement was that, in applied research in the field of educational psychology, the most important goal is that outcomes obtained can be used as broadly as possible within the educational community. It was an important goal that the research generates some socio-educational impact, allowing the investment return that the R&D system has made in the projects.

### OBJECTIVE #2: TO SHARE BABYRADIO-R&D TEAM COLLABORATION WITH OTHER GROUPS

The relationship between Babyradio and UCA's research team existed from the beginning of the R&D project drafting. It was an extension of a relationship already established in previous projects. The difference in this case was that Babyradio not only had a role in the dissemination of results, but the company collaborated in the support of graphic design during the initial phase of the development of APP prototypes. In this sense, we incorporated into the computer design image and sound resources provided by the design team made by the company's staff. The flow of information between Babyradio's staff and research team involved in software design was continuous, thus guaranteeing the generation of synergies between the two groups.

The children's entertainment industry is highly developed by large multinational companies, and there is a company concentration process so that the culture and educational outcomes are also under the effects of globalization (Noam, 2016). Small companies in this area need to make a considerable effort to stay in this difficult market, dominated by large service platforms. One of the ways to remain in this business is seeking agreements with institutions that provide added value to the contents and services offered to potential consumers. This extra value can generate innovative outcomes, contrasted in educational settings. This is how the mutual interest of collaboration between educational research and certain companies in this business sector should arise.

The Babyradio-R&D team collaboration chart summarizes the structure (**Figure 1**). However, considering the differences in timing, resources and hierarchy of decision making between the business and research areas, we surmised that there were three requirements that had to be guaranteed for the success of the R&D-company relationship: (1) establish an official collaboration within the framework of university-company institutional relationships; (2) unequivocally decide the specific human resources that both Babyradio and the R&D team would establish for the flow of technical information; and (3) unambiguously specify the exchange of technical resources (graphics, formats, multimedia resources), as well as convenient deadlines for both parties.

In this sense, the first level of the agreement (**Figure 1**: early math APPs design project) was reached between the staff in charge of the R&D team and Babyradio's executive management. The second level would be a more technical collaboration between those responsible for the software design of the R&D team and the link designer provided by Babyradio. Finally, the third level was the prototypes' implementation in educational settings (**Figure 1**: early math applied experimental in classrooms) and dissemination on Babyradio's website, with the commercial conditions that the company understands are more advantageous.

However, this collaboration agreement's approach would not be possible without contrasted results and with an appropriate timing for the two entities. In this sense, the outcomes generated by the R&D project have consisted of several APPs prototypes, of which we will describe only four, for practical reasons. These APPs deal with content about early mathematics, in a field applied education, which we describe briefly below.

<sup>2</sup>Website of a spin-off developed by a University of Almería (Spain) project: http://www.education-psychology.com/ebt/new/espanol/index.php

### OBJECTIVE #3: APPS PROTOTYPES DESCRIPTION AND IMPLEMENTATION IN THE SCHOOL SETTING

Currently, children are immersed in an environment in which mobile devices have become the most widespread resources, in terms of communication and leisure (Livingstone et al., 2018). The design of the APPs arises from the existing need in educational settings to provide new methods of training. These should be playful and adapted to young children's everyday life. That was the target population for this project. Previous work has highlighted the advantages of the use of technological tools in teaching in general (Herodotou, 2017), showing explicitly positive effects in mathematics (Baker et al., 2018).

## Prototypes Description

### Prototype Description: Comparison of Magnitudes Task: Compare Amounts With Mon the Dragon

The main target of this APP was the training in comparison of magnitudes, one of the specific domain skills predicting high achievement in early mathematical learning (Aragón et al., 2016; Cerda et al., 2017).

This APP presents a game which consists in identifying and indicating which amount is higher or lower between two response options, as indicated in the statement (**Figure 2**: "Compare amounts with Mon the Dragon"). These two alternatives can be displayed in the same format (symbolic or non-symbolic), or be presented in a mixed style. After the initial screen, a menu with three levels of difficulty appears. These levels have implicit sublevels that grade the game complexity. Each exercise offers 10 attempts in which the player must point out where there is more or less. To overcome the level, participant must correctly answer 80 percent of the attempts.

### Prototype Description: Subitizing Task: "Quick Counting With Mon the Dragon"

The goal of this task is the subitizing training (sudden counting). The additional goal is also improving the numerical sense. According to the design, the knowledge of the numerical line should also be enhanced. The player must count a number of elements in a short period of time (4 s), with different levels of difficulty. Two screens appear on each task: (1) Stimulus-screen with verbal and written instruction (**Figure 2**: "Quick counting with Mon the Dragon"); and (2) Answer-screen with a number line for pointing the response of the sudden count. The APP has 10 items per level. To overcome this, it is necessary to get an 80 percent correct rate. After the response, feedback is given. If there is no response within 10 s, the item is registered as incorrect.

### Prototype Description: Numerical Facts Task: Calculation With Mon the Dragon

Numerical facts correspond to simple calculations stored in long-term memory (addition, subtraction, and multiplication or division). These are useful features in understanding and developing arithmetic concepts, facilitating the problems solving.

During this APP game, the child must solve different calculations presented (addition or subtraction). On other tasks, the unknown is not the result, but some element that makes up the operation (addendums or subtracts), or the sign of the numerical facts. Here, the child must identify if it is a sum or a subtraction (**Figure 2**: "Calculation with Mon the Dragon").

Each item or operation that must be solved appears for 4 s. After that time, the response screen is displayed, which will remain until the child points with his finger a numberor chooses a sign (**Figure 2**), depending on the question and level of difficulty.

### Prototype Description: Estimation on the Number Line Task: Find the Hidden Number

The purpose of this APP is to extend the knowledge of numbers and their position in a straight line. The estimation task is considered as a specific domain cognitive precursor relevant for improving mathematical skills such as counting, arithmetic operations or the understanding of mathematical concepts. The domain of the number line allows the child to answer questions regarding the magnitude without referring to specific objects; this supports the cardinality's modification rule of a set, depending on the addition or subtraction, and allows them to know the relative position of a number when the task cannot be directly solved (Gervasoni, 2005; Siegler and Booth, 2005).

Two types of estimation coexist in this APP (**Figure 2**: "Find the hidden number"). In one case, a number appears, then the child must place it in the appropriate position in a straight line (number-position); or the straight line has a distinctive mark in a specific place. Then the child must decide to which number that place belongs (position-number). A video tutorial has also been created because of the difficulty in understanding the task. The video tutorial is available from the initial screen. Each level presents 10 activities. Users have to score at least 80 percent to get to the next level. The correction of the activity is through an approximate error procedure, developed according to the following algorithm: {(X–Y)/Z} × 100. Where X, number requested; Y, number that the child indicates on the number line; Z, size of the line. The margin of success is considered as a 7.5 percent deviation from the correct answer.

#### Technical Design of Prototypes

fpsyg-09-02751 January 24, 2019 Time: 12:1 # 6

The development of the APPs interactive section on mathematical content is based on the current Web technology: HTML5, CSS, and JavaScript. The standardization of web browsers in recent years has made JavaScript one of the most important programming languages. JavaScript is an interpreted language. So, each time a program or script runs, it must be translated into recognizable codes by the microprocessor. These programs are less efficient than those developed with compilers, in which the code once translated or compiled is stored in one or more files and executed without any conversion. The great advantage of JavaScript is that its interpreter is incorporated into the Web browsers and specifically translated by the platform's operating system on which it is running. In this way, a single development that meets the language standards can run in any browser of any platform and operating system that follows these standards.

Based on this standardization, hundreds of work frameworks based on JavaScript have been developed. They generally are dedicated to very specific activities within the software programming. The CreateJS framework<sup>3</sup> is modeled on the Adobe Flash author software with which we have developed several educational software in the past. Starting from this knowledge, we are granted many advantages, facilitating and accelerating our developments, mainly in the management of images, animations and sounds, which make up the main core of the APPs we developed. However, CreateJS does not provide appropriate solutions to the treatment of form-sheets – essential for the data's control and storage. To solve this, we also used the jQuery Mobile framework<sup>4</sup> . This software contains jQuery framework<sup>5</sup> .

Another consequence of JavaScript standardization, along with HTML5 and CSS3, is the appearance of frameworks for the conversion of Web applications (created with these languages) into fully functional APPs for mobile devices of the most widespread platforms. Apache Cordova is a free software solution (open source) for the recognition of these conversion tasks to APPs. In our developments, we use Adobe Phonegap, a distribution by Apache Cordova. One of the built-in elements is Adobe Phonegap Build (see **Figure 3**), an online application that directly allows APPs packaging in the cloud, facilitating the conversion process. This application is 100 percent available for Adobe Creative Cloud subscribers. With Phonegap Build, in a simple way, we managed to expand the distribution of our applications to a wide collection of devices, such as smartphones and tablets, based on a single development and almost without changes.

In order to avoid repeatedly entering the data into the formsheets, we used the local storage resources provided by HTML5

<sup>3</sup>https://createjs.com/

**71**

<sup>4</sup>http://jquerymobile.com/ <sup>5</sup>http://jquery.com/

(local storage), known as Web storage. When working under conditions in which an Internet connection is not available, the data generated during the execution of the APPs are locally stored, using SQL Web data. In this way, all data stored in the device can be synchronized with a remote database. In the Web format, this decision limits the use of some browsers when we process data because not all of them support Web SQL. Chrome, Opera and Safari support it perfectly, but not Mozilla Firefox or Microsoft browsers.

The remote data storage was made in relational databases with the MyQSLi interface (MySQL Improvement extension), using Ajax and PHP as languages for connection and data management. The jQuery framework facilitates asynchronous transfers, through Ajax, between the local and the remote database.

#### **Media resources**

The graphic pieces of the application have been designed in Adobe Illustrator. Therefore, they were originally of a vectorial format. The complexity of some of the designs, in terms of elements and color gradients, such as the background or pet (Mon the Dragon), forced us to convert them to a bitmaps format, specifically to PNG files to keep transparencies. With Adobe Photoshop, we adjusted these bitmaps and calculated sizes and proportions so that everything could take its place according to the initial design.

Easier elements, such as buttons, icons, dice, and so forth were converted using Draw2script – a plugin from Adobe Illustrator – into a collection of graphic commands used by CreateJS (EaselJS Tiny Api). This allowed us to draw all components in real time on the screen. The simplest ones were drawn using the basic CreateJS commands.

The integration of all these features was done through CreateJS, so that everything could take its place and be observed by the user as a single design. Also, with CreateJS, we synchronized all sounds. We used Audacity and Adobe Audition software. All the sounds were stored in mp3 files.

### APPs Prototypes Implementation in School Settings

To implement the project, cognitive variables of general and specific features involved in early mathematical learning were selected. Then all activities were planned in a game context, emphasizing the process of gamification, in order to obtain positive results based on their advantages (Hanus and Fox, 2015; Papadakis et al., 2018). Subsequently, Babyradio focused on the APPs' design improvement, adding voices, graphics and animations that were processed by researchers. The APPs' prototypes implementation allowed us to validate, with a significant sample, APPs' usefulness in training those mathematical skills established by the designers.

#### Experimental Procedure

A quasi-experimental design with pre- and post-intervention measures was carried out. A quasi-experimental design was used. This distribution was based on the participant's mathematical achievement and some ecological issues, such as the groupclassroom and the school characteristics.

#### Participants

For APP validation, a sample of 112 students of the last year of early childhood education (5 years old) was used. Participants were aged between 58 and 79 months (M = 63.45, SD = 3.46). Considering participants' gender, 60 were girl between 58 and 80 months (M = 63.37, SD = 4.42); and 52 were boys aged between 58 and 71 months (M = 63.54, SD = 3.55). The children with special educational needs capable of completing the evaluation batteries and the intervention program were included. They were assessed by the pre-tests and by teachers' and the school technical team's expert judgment. In total, of the 118 students that were part of the initial group, six students were diagnosed as special needs students. This study was carried out in accordance with the recommendations of the Bioethics Committee of University of Cádiz. All subjects gave written informed consent in accordance with the Declaration of Helsinki and Singapore Statement.

#### Evaluation Instruments

Different evaluation tests were used for participants' assessment of cognitive and numerical knowledge, in order to validate the APPs (**Table 1**). The different tests allowed us to know both the domain of numerical knowledge, as well as the general and specific cognitive variables related to early mathematical learning<sup>6</sup> .

#### Procedure

First, a pre-test evaluation was carried out in two sessions for all participants. In one of the sessions, students

TABLE 1 | Evaluation tests used in the study for APP prototypes validation of improvement of early mathematical competence.


<sup>6</sup> It was not the goal of this study to show specific data analysis about the APPs' effectiveness in the field of early math learning. For that reason, just a brief summary of the evaluation instruments is presented.

individually completed specific tasks related to their early math skills. The second session was focused on assessment of cognitive parameters related to mathematical learning. Sessions were randomly run on different days. Within the sessions, the tests were randomly applied to control any effect of fatigue or facilitation in the variables measurement.

Once the total number of students was evaluated, those children belonging to the group that should receive the intervention training were selected. This selection was carried out based on the hypotheses that were intended to highlight the effectiveness of the use derived from APPs for low- and high-math-performance children. Consequently, the student's score was considered and compared with peers from their same group by selecting the three students (above the 75th percentile) with the best math skills, and the seven students who scored the lowest (below the 50th percentile) in the specific test of mathematical evaluation. This distribution was made according to the ecological characteristics of the groups (classroom grade, teachers, etc.,...).

Once the experimental and control groups were selected, the intervention was started. A total of 30 sessions were planned: three (3) sessions per week, of the duration of 30–35 min per session, in groups of 10 participants per session. All sessions were conducted with the supervision and guidance of specially trained professionals. The students worked with mobile devices (tablets and smartphones randomly). They used headphones to facilitate concentration on tasks. Once the intervention phase was finished, all students were again evaluated on the same cognitive and mathematic variables during two post-test sessions in order to judge the effectiveness of the intervention through APPs.

#### Results

Considering that the main goal of this article was to describe the institutional framework of the relationship of the R&D project and a children's entertainment company, a short result report is presented. This exploratory study presents information on the descriptive statistics of the evaluated variables (**Table 2**).

Both the students with low performances (experimental group 1) in mathematical competence and risk of dyscalculia, and the students with high mathematical performances (experimental group 2), increased the scores in each of the mathematical subtests evaluated. The experimental group 1, of low mathematical performance, was compared to the control group in all the mathematical records. The differences obtained with respect to the control group were statistically significant for experimental group 1 (t<sup>61</sup> = −14,093; p < 0.001) and for experimental group 2 (t<sup>49</sup> = −19,606; p < 0.001). The results could be explained by the effectiveness of the APP training designed to work on the specific predictors of mathematical learning. However a total comparison analysis is necessary.

### LIMITATIONS AND CONCLUSION

The main objectives of this study were to describe the institutional framework of the relationship of the R&D project and a children's entertainment company (Babyradio); to share the collaboration scheme between the working groups of both levels; and to show some of the applied educational outcomes generated by this collaboration. It was not the goal of this study to show specific data analysis about the APPs' effectiveness in the field of early math learning. The study is based on the collaboration agreement between a children's entertainment company and a university's research group. Although the experience was highly positive and productive, the conclusions must be considered, taking into account the context where it has been developed. Therefore, a generalization for cooperation agreements with any type of company R&D must be made with caution. The dynamics established between the work teams of both parties should be conveniently handled, with the least possible interference. The interest and the exchange of qualified technical information must prevail over any other information (economic, organizational, etc.) that may contaminate the objectives of the relationship. Although the empirical study still requires a multilayered data analysis, there are some practical recommendations that can be suggested. Mathematical learning


is critical for an adequate school and social adaptation. The strategies used are numerous but they should be aimed at intensifying the learner's cognitive resources. Those teachinglearning approaches that help improve the general and specific cognitive precursors of mathematics should be implemented efficiently with young children. The APP designed in this study should be one of the useful strategies to reduce the risk of mathematical difficulties in preschooler. Given the limitations of our study, a more extensive analysis of other entrepreneurship experience in the field of educational psychology would be required, with a longitudinal methodology. When strategies of entrepreneurship education are discussed, some attention should be paid to stakeholders' affinity and embeddedness within university-industry cooperation framework. Future research should spread our awareness of the role of innovative education in academic spin-off, employability and make-bring-returns of knowledge. This methodology would allow us to verify the

### REFERENCES


advantages of the need for a multidisciplinary perspective to focus on the research interests and of the company; and to, therefore, ascertain whether profitable bilateral learning has occurred and if the investment in human capital has been productive for both parties.

### AUTHOR CONTRIBUTIONS

CM, GR, MA, EA, CD, IM, EM, MGS, and JN designed, evaluated, and wrote the manuscript.

### FUNDING

This work has been partially funded by R&D project number PSI2015-63856-P of MINECO/FEDER, Spain.



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Mera, Ruiz, Aguilar, Aragón, Delgado, Menacho, Marchena, García Sedeño and Navarro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence

Juan-José Navarro1,2,3 \*, Catalina Mourgues-Codern<sup>4</sup> , Eduardo Guzmán<sup>5</sup> , Isabel R. Rodríguez-Ortiz<sup>3</sup> , Ricardo Conejo<sup>5</sup> , Claudia Sánchez-Gutiérrez<sup>6</sup> , Jesús de la Fuente7,8, Diana Martella<sup>8</sup> and Mahia Saracostti<sup>9</sup>

<sup>1</sup> Department of Psychology and Sociology, Universidad de Zaragoza, Zaragoza, Spain, <sup>2</sup> Facultad de Educación, Universidad Autónoma de Chile, Santiago, Chile, <sup>3</sup> Departamento de Psicología Evolutiva y de la Educación, Universidad de Sevilla, Seville, Spain, <sup>4</sup> Laboratoire Adaptations-Travail-Individu, Université Paris Descartes, Paris, France, <sup>5</sup> Departamento de Lenguajes y Ciencias de la Computación, Universidad de Málaga, Málaga, Spain, <sup>6</sup> Spanish and Portuguese Department, University of California, Davis, Davis, CA, United States, <sup>7</sup> Departamento de Psicología Evolutiva y de la Educación, University of Almería, Almería, Spain, <sup>8</sup> Facultad de Psicología, Universidad Autónoma de Chile, Santiago, Chile, <sup>9</sup> Centro de Investigación sobre Procesos Socioeducativos, Familias y Comunidades, Universidad de La Frontera, Temuco, Chile

#### Edited by:

Weijun Wang, University at Buffalo, United States

### Reviewed by:

Susan M. Brookhart, Duquesne University, United States Ana Miranda, Universitat de València, Spain

> \*Correspondence: Juan-José Navarro jjnavarro@us.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 07 June 2018 Accepted: 30 July 2018 Published: 28 August 2018

#### Citation:

Navarro J-J, Mourgues-Codern C, Guzmán E, Rodríguez-Ortiz IR, Conejo R, Sánchez-Gutiérrez C, de la Fuente J, Martella D and Saracostti M (2018) Integrating Curriculum-Based Dynamic Assessment in Computerized Adaptive Testing: Development and Predictive Validity of the EDPL-BAI Battery on Reading Competence. Front. Psychol. 9:1492. doi: 10.3389/fpsyg.2018.01492 In recent decades there have been significant changes in the conceptualization of reading as well as in the perception of how this activity should be assessed. Interest in the analysis of reading processes has led to the emergence of new explanatory models based primarily on the contributions of cognitive psychology. In parallel, there have been notable advances in measurement procedures, especially in models based on Item Response Theory (IRT), as well as in the capacity and performance of specific software programs that allow data to be managed and analyzed. These changes have contributed significantly to the rise of testing procedures such as computerized adaptive tests (CATs), whose fundamental characteristic is that the sequence of items presented in the tests is adapted to the level of competence that the subject manifests. Likewise, the incorporation of elements of dynamic assessment (DA) as the prompts are gradually offered allows for obtaining information about the type and degree of support required to optimize the subject's performance. In this sense, the confluence of contributions from DA and CATs offers a new possibility for approaching the assessment of learning processes. In this article, we present a longitudinal research developed in two phases, through which a computerized dynamic adaptive assessment battery of reading processes (EDPL-BAI) was configured. The research frame involved 1,831 students (46% girls) from 13 public schools in three regions of Chile. The purpose of this study was to analyze the differential contribution on reading competence of dynamic scores obtained in a subsample composed of 324 (47% girls) students from third to sixth grade after the implementation of a set of adaptive dynamic tests of morpho-syntactic processes. The results achieved in the structural equation modeling indicate a good global fit. Individual relationships show a significant contribution of calibrated score that

reflects estimated knowledge level on reading competence, as well as dynamic scores based on the assigned value of graduated prompts required by the students. These results showed significant predictive values on reading competence and incremental validity in relation to predictions made by static criterion tests.

Keywords: dynamic assessment, computerized adaptive testing, item response theory, learning potential, graduated prompts, reading processes, incremental validity

### INTRODUCTION

In educational contexts, assessing students' cognitive skills and reading processes is central to making informed decisions about the support they require to reach their full potential. In this context, Dynamic Assessment (DA) has emerged as an alternative to traditional or "static" assessment methods and is better adapted to the detection of learning difficulties and special educational needs (Jitendra and Kameenui, 1993; Swanson and Lussier, 2001; Rezaee and Ghanbarpour, 2016). DA refers to a set of procedures that embeds intervention within the assessment process through feedback, guidance on the use of specific metacognitive processes, and mediation. It can also be achieved through a progressive sequence of explicit and targeted prompts. In the latter case, the degree of learning achieved by students when receiving such support is used as an indicator of learning potential.

In recent decades significant changes have also emerged in the conceptualization of specific domains of learning, such as reading and arithmetic. These new conceptualizations come hand in hand with new perceptions of how these activities should be evaluated. Specifically, in the context of reading development, new findings in cognitive psychology have contributed to the emergence of new explanatory models (Hacker et al., 2009; Thiede et al., 2009). This has happened in parallel with significant advances in measurement procedures, especially in relation to models based on Item Response Theory (IRT), and with a significant increase in the capacity and performance of specialized data analysis and data management software. These changes have contributed to the rise of computer-based adaptive assessment procedures, known as Computerized Adaptive Testing (CAT) (Embretson and Reise, 2000). In essence, their main feature is that the sequence of items presented in the test adapts to the estimated level of the student's competency. However, the development of adaptive dynamic tests based on a mediation process has shown a lower degree of progress in the field of CAT, due in part to technical difficulties, but also to the complicated theoretical decisions that need to be taken regarding the type and intensity of the mediation provided by the system or the evaluator. Incorporating elements of DA, such as the prompts that are gradually offered to the students when they fail to correctly respond to an item, could increase the advantages of CAT. It is precisely this combination of DA and CAT that would situate these models in the realm of intervention-oriented evaluation. Indeed, when data are provided by the system on the resolution process followed by the student during a test, it is expected to be easier to infer patterns of successful intervention that directly address the specific issues observed during testing. Thus, instead of simply assessing the current state of students' competencies, it also opens avenues for improvement based on the specific types of aids that better work for each student.

In this context, the current study aims to analyze the differential contribution on reading competence of the dynamic scores obtained from the implementation of a set of adaptive dynamic tests of morpho-syntactic processes integrated into the computerized adaptive DA battery EDPL-BAI. First, some key elements related to CATs and DA of reading competence is introduced. Then the design, structure, and content of the EDPL-BAI battery are presented.

### Computerized Adaptive Testing

Computerized adaptive testing proposes the progressive adaptation of the evaluated contents to the subject's estimated abilities at each level of difficulty. Thus, the items presented to each student depend at all times on the student's demonstrated ability during the execution of the task. The fundamental idea of CAT is to get as close as possible to the behavior that a human evaluator would demonstrate when trying to obtain information about a student's task resolution abilities during a test (Wainer, 1990). The evaluator is expected to adapt questions to the answers that the student gives. If a question is too difficult for the student, he or she will most likely give a wrong answer, and therefore the evaluator will subsequently ask a question that is somewhat easier. The intention is to obtain the most accurate information possible about the student's knowledge (van der Linden and Glas, 2000). CAT items are generally displayed one at a time, and decisions about the presentation of each item, the termination of the test and the evaluation process, in general, are made dynamically based on student responses. In essence, a CAT is thus an iterative algorithm that starts with an initial estimate of a student's knowledge, which is usually represented by a probability distribution. Subsequently, all items are examined to determine which would be best suited to estimate the subject's level of knowledge most accurately; the best fit is then selected, and the student responds. Based on the student's answer, a new estimate of his or her knowledge is made, and a new item is selected that best fits the expected student's knowledge level. This process continues until the termination criterion is reached or the set time is up, if there is a time limit. The student's knowledge level is calculated as the mean or mode of the distribution calculated by this iterative process. **Figure 1** presents a typical CAT sequence.

In the configuration of a CAT, a set of parameters that determine the characteristics of an evaluation session need to be taken into consideration. Among other relevant information, the following set of information should be specified: (a) the item bank (IB; i.e., the set of items from which the final selection

will be drawn); (b) the criterion for initially estimating the level of a feature, such as the initial knowledge of the student; (c) the criterion for dynamically selecting items for the test (i.e., how to decide which item is going to be shown to the student at each trial); (d) the criterion for completion of the test (e.g., once the assessment of student knowledge has sufficient statistical accuracy, after a certain time, after completing all test items); and (e) the evaluation criteria (i.e., how the score is calculated). This is generally done by applying IRT methods, although there are also other options. For example, some heuristics could be used, such as the percentage of successful items or penalty points for each error. In short, most CATs' item selection strategies are based on an estimate of the assessed latent trait based on each response from the student. To carry out such estimates it is necessary to calibrate all the items in the IB. Item calibration is a complex process by which an item characteristic curve (ICC) is inferred, such as the one shown in **Figure 2**. This curve represents the probability that an item will be correctly answered and is usually described by logistic functions of one (1PL), two (2PL), or three parameters (3PL) based on the following formula:

$$P(u\_i = 1 | \theta) = c\_i + (1 - C\_I) \frac{1}{1 + e^{1.7a\_i(\theta - b\_i)}}$$

Where P(u<sup>i</sup> = 1|θ) is the probability of correctly responding to item i given the student's knowledge level θ. P is usually measured on a scale from −3.0 to 3.0. The three parameters that characterize this curve depend on the item and include the following: (1) Discrimination (ai) is a value that is proportional to the slope of the curve; the higher the value of this parameter, the better the item is at discerning between lower and upper levels of knowledge. (2) Difficulty (bi) corresponds to the knowledge level for which the probability of answering correctly is the same as that of answering incorrectly (regardless of random responses). (3) Finally, Guess or chance (ci) accurately measures the probability of correctly answering the question without the actual knowledge necessary to do so—that is, the probability of guessing at random. When ICC is determined by the model 1PL,

the only parameter used is difficulty; discrimination; and guessing, in this case, both present a value of zero. In the 2PL model, besides difficulty, discrimination must also be estimated.

In order to perform the calibration process, a considerable volume of data about the items needs to be gathered from students who have completed them all. The calibration procedure is also an iterative algorithm that finds the parameters of the ICC (difficulty, discrimination, and/or guessing, depending on the model selected) that better explain the response matrix obtained by the sample of students who have participated in the calibration process. Once items are calibrated, the use of a CAT requires (1) an initialization procedure of the student knowledge distribution, depending on whether or not prior information about the subjects is available; (2) an algorithm to dynamically select items during the administration of the test, and (3) a completion criterion, following which the final estimate of the trait being evaluated can be computed.

After an item i has been presented to the student, and he or she has provided an answer, the student's distribution of estimated knowledge, P(θ| u1... , ui), is updated according to the following formula:

$$P(\theta|u\_1, \ldots, u\_{i+1}) = P(\theta|u\_1, \ldots, u\_i) \ast P(u\_i = 1|\theta)^{u\_i} \ast$$

$$[1 - P(u\_i = 1|\theta)]^{1 - u\_i}$$

Where P(θ|u1, ..., ui+1) is the distribution of student knowledge that has been updated after responding to item i. If the student's answer is correct (u<sup>i</sup> = 1), the distribution is multiplied by the ICC; otherwise, it is multiplied by the opposite curve.

### Dynamic Assessment of Reading Processes

In the school context, DA results from the interaction between the evaluated student and a more proficient subject, usually an educator or psychologist, through a mediation process. This interactive process bridges the gap between the current competencies of the student and the demands of the task. As such, it creates opportunities for the joint construction of knowledge through interaction, which is especially relevant for psychoeducational interventions and the evaluation of learning processes (Newman et al., 1989; Vygotsky, 1995; Wells, 1999; Jensen, 2000; Sternberg and Grigorenko, 2002; Elliott, 2003). These mediated interactions facilitate the determination of the type and degree of support required by the students to successfully complete the tasks on which they are assessed and to acquire the established competencies. Incrementally, DA allows for establishing dynamic measures of the cognitive processes involved in task resolution, including those that are still under development (Sternberg and Grigorenko, 2002). Thus, data obtained through DA may contain relevant information that adds to that which can be obtained through more conventional static assessment methods. DA also offers more in-depth insights into the mechanisms of action that facilitate learning and task resolution in each specific student, which is necessary not only to support the students individually but also to offer better explanations of the improvement observed during an intervention. To date, various approaches to DA have been used to consider it as an assessment of (a) learning potential (Budoff, 1967), (b) test conditions (testing the limits) (Carlson and Wiedl, 2000), (c) mediated learning (Feuerstein et al., 1981), or (d) learning and assisted transfer (Campione et al., 1985). The latter approach, characterized by the use of graduated prompts during the evaluation process, has been one of the most widely used in developing computerized DAs. This is because the type and frequency of each prompt can be standardized and used as an indicator of the level of support that students need to learn a skill. For example, in relation to the type of support, one parameter may be the level of explicitness of support, making it possible to construct gradients of aid that are gradually more explicit and targeted, depending on the observed needs of each evaluated student.

Curriculum-based DA models include contextualized tasks that are closely related to educational content (Delclos et al., 1992; Ruijssenaars et al., 1993; Jensen, 2000; Guterman, 2002; Elliott, 2003; Kalyuga and Sweller, 2005; Swanson and Howard, 2005; Haywood and Lidz, 2007; Thurman and McGrath, 2008; Fuchs et al., 2011; Lidz, 2014). The adoption of such DA models could facilitate the incorporation of achievements reached during the assessment process into classwork (Jensen, 2000). In this sense, the aids that allowed the students to respond to higherdifficulty items during the assessment process could guide the type of intervention and support that the student needs. Thus, similar aids could be used in the classroom or in one-onone intervention settings. Such contextualization endeavors, including collaboration with classroom teachers, should provide greater ecological validity to the process and results of DA.

Lately, DA models have been used in the specific field of learning difficulties related to reading with the purpose of obtaining profiles of learning potential and to establish the predictive value of dynamic tests on student achievement (Jeltova et al., 2007; Gustafson et al., 2014). In this regard, Caffrey et al.'s (2008) literature review showed that the predictive value of DA was higher than that of traditional evaluation methods when the level of achievement and support required to achieve that level were taken into consideration. In the last 2 years, more than 100 published studies have reported using DA for predicting reading difficulties in preschool students (Catts et al., 2015; Gellert and Elbro, 2015; King et al., 2015; Petersen et al., 2016), and in the early school years (Clemens et al., 2015; Fani and Rashtchi, 2015; Naeini, 2015; Wolter and Pike, 2015; Stevenson et al., 2016). For example, King et al. (2015) dynamically evaluated the production of sentence structures by 4- and 5-year-old children through graphic symbols on an augmentative and alternative communication (AAC) device. Incrementally, the predictive validity of the DA on a subsequent experimental task was evaluated. The four participants had normal receptive linguistic ability but presented limitations in speech production. Graduated prompts were used throughout the assessment procedure. The measures included the amount of support required to produce sentences using symbols, as well as the changes observed during the development of the sessions (modifiability). The authors showed that participants needed variable amounts of support to produce the target structures. Likewise, modifiability was more evident in some participants than others. With regard to the predictive validity of the assessment method, the results partially supported the predictive value of the dynamic test relative to the experimental work carried out later. The researchers concluded that DA yielded valuable information on the process followed by the participants to sequence simple messages based on rules through the AAC device.

Another study that focused on morpho-syntactic processes in the context of DA was conducted by Hasson et al. (2012). Twentyfour children between 8 and 10 years old participated. All of them had specific language impairments (SLIs). A DA method was designed that aimed to use the information obtained from the testing sessions to plan interventions that would specifically address the needs of children with SLI. The researchers argued that little is known about how individuals with SLI deal with the

completion of language tasks, and that the use of static assessment has contributed to limiting our understanding of how children within this group address different types of linguistic skills. The DA procedure was applied four times for each participant, at intervals of 4 months between sessions. The predictive validity of the developed dynamic test was higher than that of the standardized test. The results of this study offered relevant insights into children's abilities to use specific strategies, to take advantage of the guidelines provided during the testing sessions and to transfer learning from one item to the next. The authors concluded that the information obtained would be useful for speech therapists who plan specific interventions for children with SLI.

Dynamic assessment has been increasingly used in various fields, and this has been particularly true for some specific areas of knowledge, such as second language learning (Kozulin and Garb, 2002; Poehner, 2007, 2008; Ableeva, 2010; Lantolf and Poehner, 2011; Poehner et al., 2015). For example, Poehner et al. (2015) developed a computerized dynamic test that assessed reading and listening comprehension in a second language. The authors argued that mediation was essential in diagnosing the level of development reached by the learners. Each test item was accompanied by a set of prompts that were graduated from lower to higher explicitness. Thus, the final result of the evaluation included information not only about the questions that students correctly answered without assistance, but also about the amount of support needed during the resolution of each question. Poehner et al. (2015) used a heuristic to calculate the difference between the score without aids and the score that included data about the required aid. These scores, which were generated by the system, made it possible to obtain a finegrained diagnostic of the developmental stage of the learners in their second language, while also providing information that is relevant to the implementation of a focused pedagogical intervention.

### Integrating Dynamic Assessment in Computerized Adaptive Testing

Adaptive testing provides the opportunity to gain valuable insight into the challenges experienced by students with the items that compose a test depending on their degree of difficulty. It is also a valuable resource for gathering data about how each student approaches their response to each item as well as his or her skill level at the end of the test. However, analysis of these data cannot provide information on the type and degree of support that the subject requires to successfully answer a particular item. This information can only be obtained by integrating DA items, such as graduated prompts, feedback, or metacognitive guides, in the more general format of adaptive testing. This type of testing, which integrates DA and CAT techniques, makes it possible to obtain information not only about the response of a subject based on the difficulty of the test items, but also about the type and degree of support required to optimize performance and successfully solve the evaluated tasks. Thus, combining these assessment techniques could offer unique insights into the task resolution strategies adopted by students, which in turn could provide evidence about these students' potential new knowledge and skills when receiving interventions that directly address their task-solving strategies. Given that CATs already offer the possibility of adapting the sequence of items in evaluation based on the skill level of the student, they provide an ideal setting for the implementation of DA.

In recent years, some CATs have included graduated prompts systems in their testing procedures as an assessment strategy in populations with learning difficulties, specifically those related to the development of reading skills (Stone and Davey, 2011; Petscher et al., 2016). Almost all of these assessment procedures have been created and marketed in the United States. Some notable examples are STAR-EL (Renaissance Learning; Shapiro, 2012) and the Children's Progress Academic Assessment (CPAA, Northwest Evaluation Association; Bechard et al., 2010). STAR-EL comprises a set of computerized adaptive assessments in the areas of reading, math, and communication and includes a sophisticated system that provides specific support according to each student's performance. The program also makes it possible to obtain reports of student performance and to compare a test with previous iterations of the test completed by the same student, tests of children under similar conditions, or standardized guidelines. McBride et al. (2010) showed that STAR-EL is technically suitable for schools and extremely convenient in terms of cost-benefit. CPAA is another computerized adaptive system that can be used three to six times during the year to monitor children's progress. This system incorporates graduated prompts, which are taken into account in the calculation of the final student performance score. The program creates performance reports and offers specific guidance on the interpretation of the results for teaching.

### The Development of EDPL-BAI Battery

EDPL-BAI is a computerized assessment device that allows for the assessment of reading processes through a battery of tests that are delivered in a dynamic/adaptive format. It focuses on elementary school students, especially those with specific needs for support and learning difficulties in reading. It aims is to contribute to the psychoeducational evaluation of reading skills in the Spanish language. The EDPL-BAI includes various tests that tap into specific processes that are involved in reading and present different levels of difficulty, as well as a system of graduated prompts associated with each item of the test. Its essential feature is the dynamic adaptation of these elements to the level of competence progressively demonstrated by the student. The system adopts a quantitative and qualitative approach that encompasses both mediation and assessment processes. EDPL-BAI can record the sequence of actions followed by the student as well as the execution times for each one of these actions. In this regard, EDPL-BAI offers an individualized assessment while still establishing parameters based on IRT methods, which makes the results comparable with each other. The EDPL-BAI battery is composed of 38 adaptive tests grouped into six blocks of processes that are involved in reading: (a) underlying psychological processes and executive functions, (b) processes involved in grapheme– phoneme association, (c) lexico-morphological processes, (d) morpho-syntactic processes, (e) processes involved in the global

comprehension of texts, and (f) personal-social adjustment processes.

EDPL-BAI is completed on a computer and is supported by the automatic evaluation web platform Siette<sup>1</sup> , which allows the development and administration of the tests as well as collection and processing of data. This platform was used during the development of the IB, the calibration process, and the configuration of the graduated prompts for each item. The system allows users to combine, design, and manage tests from the viewpoint of classical test theory and IRT while also fostering the adaptive presentation of items, as recommended by CAT theory. The Siette system (Conejo et al., 2016) was developed by the Applications in Artificial Intelligence Research Group at the University of Malaga (Andalusia-Spain).

The development of the EDPL-BAI battery followed five steps that are described below:


reaction to the answers given by the students during the resolution of the tests. In general, up to four graduated prompt levels are offered. However, in some tests, a greater number of aids are offered, depending on the particular characteristics of the task. The general sequence of proposed prompts is as follows: (a) First support level (L1): General prompts. A general prompt is first proposed (e.g., "Read the instruction again; pay attention to the instruction; remember, chose the option that better answer the question"); (b) Second and third support level (L2 and L3): Metacognitive prompts. If the first level of assistance did not help, two levels of metacognitive prompts are then offered (e.g., "Read the sentence slowly and try to imagine it in your mind. Remember that some sentences are not plausible." or "Read the sentence carefully and think about its meaning; then read carefully the question you must answer; when you have answered, review your answer carefully"); (c) Fourth support level (L4): Specific prompts. Finally, if none of the previous aids are helpful enough, specific support related to the construct under evaluation is proposed (e.g., "Pay attention to the question about who has done something. Look carefully at the choices and select the correct one. . . You must always look for who does something in the sentence"). Such differentiation is important because it makes it possible to clarify not only the amount or degree of aid required but also the type of aid that has been most effective for each student when responding to each item. At this point, it should be noted that the present paper only focused on the contribution of dynamic scores regarding the degree of support required. In any case, the inclusion of these graduated prompts during the assessment process allows for taking the subject's response to mediation into account in the assessment process. **Figure 3** shows the specific sequence followed in each test once the graduated prompts were incorporated.

In line with the work of Guthke and Beckmann (2000), the proposed evaluation system envisages that students "move" through the various activities proposed without receiving aid at first. When a student fails to resolve an item, he or she is offered support related to the type of mistake that is expected for that type of item. Subsequently, the system provides items based on preestablished selection criteria. These criteria can be set based on the difficulty of the items. In such cases, when an error occurs, the system provides an item of equivalent complexity to that which caused the initial error, and if this second item is successfully resolved, the system presents other items with progressively higher levels of complexity. Alternatively, the selection criterion may be determined by the precision with which each item reports on the construct being evaluated.

(4) Configuration of the computerized adaptive tests. The criteria adopted in the configuration of the EDPL-BAI into an adaptive test were as follows: (1) Initial trait level estimate: Based on the scores obtained during the static testing phase for each one of the tests and each of the

<sup>1</sup>www.siette.org

evaluated course levels (third, fourth, fifth, and sixth grade), an estimate of the level of difficulty for each item was computed. (2) Completion criteria: The completion criteria were established based on the most accurate information possible about the estimated current level of knowledge of the student. Indeed, the execution of each test had to be translated into a score of the estimated knowledge level of the student. For that, a scale from 1 to 7 – usually used in Chilean schools– was used. Based on this score, the next item that the student must complete was established. In principle, all tests terminate when one of the five values of the probability distribution of the estimated knowledge level reaches 80%, which is when the presented items explained 80% of the variance for estimation of knowledge level. (3) The estimated level of knowledge: A discrete distribution of student knowledge across five knowledge levels was established using the values obtained from item calibration. These estimated knowledge levels (θ), together with the values of the scale 1–7 to which they are associated are the following: very low (θ = 1.60), low (θ = 2.80), medium (θ = 4.00), high (θ = 5.20), and very high (θ = 6.40). This categorization of knowledge level also makes it possible to determine the level of difficulty of each item in the context of each one of the knowledge levels. For example, an item may be challenging for students at the low and very low levels, but increasingly easy for students with a higher knowledge level.

In addition to the estimated score corresponding to the current level of student knowledge in each of the tests performed, which is estimated as a result of items calibrated based on the IRT, dynamic scores can be obtained through two heuristics. In one of them (the integrated dynamic score) the successes obtained without aid, the value is given to the aids that were effective, and the execution time is taken into account. The second establishes the dynamic score based on the inverse of the value of the required aids to successfully solve the items performed.

(5) Configuration of the structure of the battery of tests. The configuration of the internal structure of the set of tests that constitute the EDPL-BAI was determined by the consideration that various processes involved in reading require different skills and thus entail a different set of support needs and level of difficulty. In this regard, the general design of the internal structure of the EDPL-BAI was adjusted to the network of relationships established for the EDPL dynamic assessment device (Navarro et al., 2014). As mentioned previously, the original items were adjusted, expanded, and redefined during the first phase of the project to better address the specific issues that arise in the context of computerized adaptive assessment

systems. Thus, the specific items that were shown to the student were then dynamically chosen within each block of reading processes based on the results of the calibration process.

### The Present Study: Objectives and Hypothesis

This article is part of more comprehensive research. The aim of the present study is to analyze the differential contribution on reading competence of the dynamic scores obtained from the implementation of a set of adaptive dynamic tests of morphosyntactic processes integrated into the EDPL-BAI battery. Considering this objective, a structural equation model was implemented to check the relationship between the potentially predictive variables and criterion-reference tests. A model was built to test two hypotheses. The first one maintains that the dynamic scores would significantly relate to reading competence measured by the implementation of a standardized reading comprehension task and the teacher's performance assessment (Hypothesis 1). Secondly, we expected to find a significant contribution of dynamic scores to the explained variance of both reading competence and teacher's assessment. In this sense, we expect a dynamic score to signify an incremental explicative factor of reading competence in relation to the prediction based on the static tasks of non-verbal intelligence and comprehension (Hypothesis 2).

### MATERIALS AND METHODS

### Participants

The research frame of this study involved 1831 students (46% girls) from 13 public schools in three regions of Chile (Metropolitan Region, Libertador Bernardo O'Higgins, and Araucanía). Non-probabilistic sampling was used. Initial contact was made with city councils of the three regions and through these, the public schools were accessed. Participation of the schools, teachers, and students was voluntary. The subsample selected for the present study initially consisted of 378 students belonging to six public urban and rural schools of the three regions. The students had completed the adaptive dynamic test of morpho-syntactic processes during Phase 2 of the project. From this sample, 54 students were removed based on their outlier performance on the tasks. The remaining 324 students (46% female) were in third (26), fourth (73), fifth (118), and sixth (107) grades and aged between 8 and 12 years old (M = 10.27, SD = 1.22).

### Instruments

### Tests of the EDPL-BAI Battery (Morpho-Syntactic Processes)

#### **Morpho-syntactic awareness test (MS) (Navarro and Rodríguez, 2014)**

This test consists of 60 items. Each item is composed of a sentence that lacks a word or pseudoword (**Figure 4**). The student is presented with a sentence and asked to complete it using one of the words or pseudowords below [e.g., Hoy vamos a (tabamos, tabaré, tabo, tabar) nuestro coche]. Among the words that complete the sentences are the same number of nouns (derived morphology), verbs (inflectional morphology), and flexed pseudowords. All the words used are high frequency (frequency > 10), according to the Spanish Computerized Lexicon, LEXESP (Sebastián et al., 2000). Cronbach's alpha for internal consistency for Morpho-syntactic Awareness Test (MS) (N = 261) = 0.99.

### **Syntactic awareness test**

To sort disordered sentences (OF) (Navarro and Rodríguez, 2014). This test consists of 36 items. The items are composed of sentences that have been previously disordered. The student is presented with these sentences and asked to order them (moving the cards with the mouse on his or her computer) according to Spanish grammar rules (**Figure 5**). Half of the sentences are semantically plausible once ordered; that is, they express ideas or familiar situations (bebe gata La agua. [drinks cat The water.]), and the other half are implausible (La novelas. lee mesa [The novels. reads table]). The sentences also differ in length (short/long) and syntactic complexity (active/passive, simple/compound, coordinated/subordinate). All the words used are high frequency (frequency > 10), according to the Spanish Computerized Lexicon, LEXESP (Sebastián et al., 2000). The test consists of 24 simple sentences and 12 compound sentences. Among the simple ones, four are short (< five words), all of which are active, and 20 are long (> six words), 16 of which are active and four passive/reflexive. Among the compound sentences (all of them long), four are coordinated and eight are subordinate (relative and conditional). The sentences are shown disordered, but each word is accompanied by the punctuation mark with which it appears in the original sentence, as in the example in **Figure 5**. Likewise, the first word of the original sentence maintains the initial capital letter when the phrase is disordered, as we can see in the previous example. Cronbach's alpha for internal consistency for Syntactic Awareness Test (OF) (N = 323) = 0.88.

**Syntactic awareness test (CS) (Navarro and Rodríguez, 2014)** This test consists of 44 items. It is based on a test designed by Miller (2010) with the same purpose. The CS test evaluates syntactic awareness based on the student's understanding of sentences and answers to questions that contain different syntactic keys. In each item the student is presented with a sentence and then a question referring to one of the characters or elements that have appeared in the sentence (**Figure 6**). The student must choose the correct answer among the three options. All the sentences are formed by words with a frequency greater than 10 according to the LEXESP (Sebastián et al., 2000). Of the total number of sentences, 28 are simple and 16 are compound. Of the simple sentences, 20 are active/transitive and eight are passive. Half of the questions are directed to the person who performs the action (subject), and the other half to the person who receives the action (object). On the other hand, in the compound sentences, there are eight coordinates and eight subordinates. Half of the sentences are plausible

(they correspond to possible events), and the other half are implausible (they correspond to impossible facts or facts that contradict our previous knowledge). The correct option is counterbalanced. In the semantically implausible sentences, it is not possible to access the meaning (and thereby solve the task adequately) without performing syntactic processing. Cronbach's alpha for internal consistency for Syntactic Awareness Test (CS) (N = 257) = 0.76.

The sequence of test items in the EDPL-BAI battery was presented adaptively and dynamically. The selection of the initial item was based on the student's grade average performance. Later, during the administration, the selection of the items was based

on the answers and the student's performance. If a student made a mistake, aid was offered. As mentioned before, four levels of graduated prompts were included for each item. The graduation ranged from more general to more specific aids. To obtain a weighted value of the aids, a value of 4 was given to the most general aids and 1 to the most specific ones. The items that were solved without aid received 5 points.

### Criteria Measures

#### **Reading comprehension tests CLPT (Medina et al., 2010)**

The tests evaluate different dimensions involved in reading comprehension and the writing of texts. The test has specific forms for grades third to sixth with 16 items each. In this study, only reading comprehension was assessed. The CLPT was administered twice (CLPT\_2\_Pre and CLPT\_Post, respectively) to explore the predictive and incremental validity of dynamic scores. The administration was conducted in the classrooms. The CLPT tests have been validated in Chile and are widely used in the school settings. The CLPT tests were designed based on an updated review of scientific literature of reading comprehension. The authors report acceptable values of corrected item-total correlation indexes. Also, test– retest reliability is considered acceptable. Cronbach's alpha for internal consistency was analyzed with data of Phase 1 in the present study. The results for each grade were the following: alpha third (N = 371) = 0.71; alpha fourth (N = 390) = 0.78; alpha fifth (N = 351) = 0.57; alpha sixth (N = 363) = 0.51.

#### **Pretest and posttest EDPL-BAI**

Two tests were created to be administered as a pretest and posttest of the EDPL-BAI battery. Each of these tests consisted of 80 items, which were selected based on the items that are part of the battery tests. The items selected and extracted for the pre- and posttest were equivalent in difficulty and discrimination indexes. The pretest and posttest each included 16 items from the pseudoword reading test, 16 items from the word reading test, 12 items from the morphological awareness test, 12 items from the Morpho-syntactic Awareness test, 10 items from the Syntactic Awareness test (CS), 6 items from the Syntactic Awareness test (OF), and 8 items from the text comprehension test. The percentage of correct responses was used as a measure of performance in these tests (Por\_Pre and Por\_Post2, respectively). Cronbach's alpha for internal consistency for Pretest EDPL-BAI (N = 820) = 0.88, and for Posttest (N = 736) = 0.99.

#### **Test of Raven's progressive matrices (Raven, 1995)**

This test is used to evaluate analogical reasoning skills with the aim of obtaining information regarding students' cognitive performance. Using this test in the present study provides a measure of non-verbal intelligence to control its specific contribution on reading competence. The general scale, applied from the fourth to sixth grade, consists of 60 items, and the colored scale, which applies to younger students and students who present special educational needs, consists of 36 items. The z scores were computed independently for each grade and used for the analysis.

### **Teachers' assessment of reading performance (Valor\_Prof)**

Once the application was completed, the teacher had to use a qualitative scale to assess seven specific evaluation criteria formulated in relation to the reading processes contemplated in the tasks, as well as in the criteria proposed in the Curricular Bases for Language. The qualitative grading applied to each of the criteria of the template was as follows: (1) low-very low level, (2) medium-low level, (3) average level, (4) medium-high level, and (5) high-very high level. The evaluation criteria were (a) he/she reads and understands different types of school texts appropriate to his/her grade, highlighting the topic and main ideas; (b) he/she makes diagrams or summaries in a clear and orderly manner, capturing the overall meaning of the text; (c) he/she extracts data and information from graphs and tables, using it in the resolution of problems/activities appropriate to his/her course; (d) he/she integrates explicit and implicit information and makes inferences based on elements of the text and also on previous knowledge; (e) he/she raises doubts or asks questions when reading, realizes when he/she does not understand something, and rereads when he/she has not understood the text; (f) he/she expresses his/her opinion, comments on the text already read, makes judgments, or proposes solutions to problems raised in the read texts; and (g) he/she perceives him/herself to be effective and competent when faced with reading activities and shows a positive attitude toward reading. This measure was used as an external criterion of the teacher's assessment of the performance observed during the administration period (Resing, 2000; Caffrey et al., 2008). The average score for the seven criteria was used for the analysis.

### Procedure

After the items calibration was addressed during Phase 1, the administration of criteria tests and of the EDPL-BAI was conducted during Phase 2. The implementation of the criteria tests as well as the EDPL-BAI battery tests during this phase of the study was carried out in educational centers by research assistants, who received training related to the theoretical/methodological bases of the proposal. These were postgraduate or final-year students of pedagogy or psychology. The administration of tests was collectively carried out (in class groups), in the usual educational context of the students. Each student received a total of 8 sessions: one session of 75 min for the CLPT pre-test, one session of 45 min for the administration of the Raven test, one session of 45–60 min for the EDPL-BAI pretest, four sessions of 75 min each for the administration of the EDPL-BAI battery, and one session of 45–60 min for the EDPL-BAI posttest. After 4–5 months, each student received two tests: the CLPT posttest, and the EDPL-BAI posttest. Likewise, a total of 12 teachers collaborated in the completion of the rating scales on reading performance.

Regarding ethical considerations, this study was carried out following the recommendations of the Declaration of Helsinki. The protocol was approved by the Scientific Ethics Committee of the Universidad Autónoma de Chile, Santiago, Chile. All subjects gave written informed consent in accordance with the Declaration of Helsinki. Before data collection, consent was obtained from the students' families and the students, informing them of the conditions of confidentiality and administration of the tests.

### Design and Data Analysis

fpsyg-09-01492 August 25, 2018 Time: 10:59 # 11

A correlational research design based on causal models was proposed. It aimed to determine the incremental validity of dynamic scores on students' reading competence in relation to predictions based on static measures of comprehension and non-verbal intelligence. Dynamic scores were obtained from the implementation of the tests of the EDPL-BAI battery. On the one hand, the student's knowledge level was estimated from the previous items calibration process. In this study, the calibrated score was calculated as the mean of the distribution obtained by the iterative process which represented in **Figure 1**. On the other hand, as mentioned before, two heuristics were used: the integrated dynamic score (IDS), and the dynamic score based on the inverse of the value of the required aids to successfully solve the items performed (DS\_Inv). In this study, only this second heuristic is reported. The formula for computing this dynamic score is as follows:

$$DS\_{I\nu\nu} = \begin{pmatrix} TCR \ ^\*10 \end{pmatrix} - \begin{pmatrix} VRA \ \end{pmatrix}$$

where TCR is the total correct responses in the test, with and without aids; 10 is the sum of the values assigned to the aids in each item (Aid L1 = 4 + Aid L2 = 3 + Aid L3 = 2 + Aid L4 = 1); and VRA is the value assigned to the required aids during the test execution.

The descriptive statistics were obtained to characterize the sample performance in all the measures. The outliers were explored using box plots with a labeling rule of 2, 2 (Hoaglin et al., 1986) and visual exploration of technical errors in the data recording. The correlation matrix was examined to explore the relationship between the measures as well as the concurrent and predictive validity. An initial structural equation model was built to test the hypothesized relationship between the measures as well as to test the incremental validity of the EDPL-BAI battery over the criteria test. The model included four potential predictive variables: (a) DA factor made of the dynamic scores from MS, OF, and CS tests; (b) the z scores from Raven test; (c) the EDPL-BAI pretest, and (d) the CLPT pretest. All these variables are related to each other. Then the predicted variables were: (a) CLPT posttest; (b) EDPL-BAI posttest and; (c) Teachers' assessment of reading performance. All the predicted variables were assumed to be correlated. Using this model as a template, two different models were explored. In the Model 1 the DA factor was made of the calibrated scores, and the Model 2 explored the dynamic scores based on the inverse of the value of the required aids. For the two models the non-significant paths were deleted sequentially.

Regarding the assumption of normality, the analysis showed an absence of normality, with values of asymmetry and kurtosis that exceeded the established limit of ±1.96 (p < 0.05). Thus, considering the lack of multivariate normality of some variables in the model the asymptotically distribution-free method was used to estimate the parameters (Browne, 1984). The indices used to assess goodness-of-fit of each model were: the χ 2 (p = 0.05 or greater indicating an appropriate fit); a ratio of χ 2 /df (χ 2 /df < 3, appropriate fit); Comparative Fit Index (CFI) (CFI ≥ 0.95 appropriate fit); Root Mean Square Error of Approximation (RMSEA) (RMSEA ≤ 0.06 indicating a good fit of the model) (Byrne, 2016). The statistical analysis was conducted using the programs SPSS-22 and AMOS-25 (Inc, Chicago, IL, United States). Statistical significance was set at p < 0.05.

## RESULTS

### Descriptive Results

**Table 1** shows the descriptive statistics of the values and scores obtained in the study. The differences in the sample size was due, on the one hand, to the cases in which the students could not complete all the tests administered during the study, and on the other, to the fact that the characteristic of adaptability in relation to the tests that each student took based on their answers, caused that not all students go through the exact same tests.

The mean values of the two dynamic scores computed under different methods were consistent to show that the OF task was more difficult for the students, while the performance on MS and CS were similar.

### Correlation Results Between the Values and Scores Obtained

**Table 2** shows the correlation coefficients obtained and their level of significance. It is possible to observe significant levels of correlation between the pretest and posttest scores of the CLPT criterion test. These levels were expected and confirmed the validity of this test for the purposes we have used it for. Regarding the concurrent validity, moderate correlation coefficient were observed between the dynamic scores and the CLPT pretest. Likewise, dynamic scores significantly correlated with the static criterion measures, the CLPT posttest, the teacher's assessment of reading performance, and the EDPL-BAI posttest. This occurs for the calibrated score (Nivel), which reflects the estimated student's knowledge level, and also for the dynamic scores based on the inverse of the value of the required aids (DS\_Inv).

### Theoretical Structure of the Causal Model

According to the objectives and hypotheses formulated, it is necessary to determine the differential contribution of the scores obtained on the dynamic tests of EDPL-BAI battery on reading competence, as measured by both the static posttests and the quantified teacher's assessment on reading performance. For this, a causal theoretical model is proposed for contrasting the data. This model (**Figure 7**) was developed to check the extent to which DA, the latent variable measured by the three dynamic tests, can explain the reading competence, taking into joint consideration the magnitude of the contributions produced by the static measures of reading comprehension and non-verbal intelligence. In this sense, the theoretical model presents, on the one hand, the rest of the predictor variables, all of which are observed variables (CLPT\_2\_pre, Z\_Raven2, and Por\_Pre), and on the other hand, the variables used as a criterion, which


SD, standard deviation; SE, standard error of the mean; CI, confidence interval of the mean; Por\_pre and Por\_post2, percentage of correct answers in the pretest and posttest of EDPL-BAI; CLPT\_pre and CLPT\_post, pretest and posttest scores in the CLPT test; CS, dynamic Syntactic Awareness test; MS, dynamic Morpho-syntactic Awareness test; OF, dynamic Syntactic Awareness test (order sentences); DS\_Inv, dynamic score obtained from the inverse of the value of the required aids; Z\_Raven, punctuation typified in the Raven test; Valor\_Prof, average of the teacher's assessment (1–5) on reading performance.

TABLE 2 | Correlation coefficients (Pearson) between the different values and scores obtained on the tests administered.


Por\_pre and Por\_post2, percentage of correct answers in the pretest and posttest of EDPL-BAI; CLPT\_pre and CLPT\_post, pretest and posttest scores in the CLPT test; CS, dynamic Syntactic Awareness test; MS, dynamic Morpho-syntactic Awareness test; OF, dynamic Syntactic Awareness test (order sentences); DS\_Inv, dynamic score obtained from the inverse of the value of the required aids; Z\_Raven, punctuation typified in the Raven test; Valor\_Prof, average of the teacher's assessment (1–5) on reading performance. <sup>∗</sup>p < 0.05; ∗∗p < 0.01; ∗∗∗p < 0.001.

are all observed variables as well (CLPT\_post, Value\_Prof, and Por\_Post2). The time elapsed between the two moments was 5 months. Among the predictor variables, we included the different dynamic scores mentioned previously in successive analyses.

### Structure and Standardized Solution to the Model 1

#### Contrasting the Analysis Model With Previous Assumptions

As a previous step to the analysis made with the structural equation model, several tests were carried out to verify the multivariate normality assumptions and model identification. First, with respect to the assumption of normality, the asymmetry and kurtosis values of the observed variables were analyzed. The analysis showed that five of the nine values of asymmetry (all negative) indicated an absence of normality in the distribution. These values (critical ratio) were below −1.96 (p < 0.05). With respect to kurtosis, four values (all positive) also indicated an absence of normality. In these cases, the values exceeded 1.96. Taking these data into account, together with the results of the Kolmogorov–Smirnov univariate normality test, in which eight of the nine variables did not fulfill the normality assumption (p < 0.001), it was decided to apply an asymptotically distribution-free estimation method (Browne, 1984). In relation to the model identification, the results of the analyses show that it is an identified model. In this sense, the order condition was verified (18 degrees of freedom) as well as the condition range (assuming that the covariance matrix is positively defined, the determinant of the covariance

matrix departs substantially from the value 0). We also found a lack of variance/covariance negative error, excessively high standard errors, and correlations between estimated coefficients above 0.80.

### Checking the Model Fit

First, we present the evaluation of the model adjustment, taking into consideration the score that reflects each student's estimated knowledge level—that is, the score offered by the system, which is calculated based on the items previously calibrated. Global and comparative adjustment indexes have been taken into account. Likewise, we also proceeded to introduce parsimonious adjustment measures, which provide information about the simplicity of the model. It should be noted that the application of non-parametric techniques for the estimation of parameters could lead to lower efficiency in the specification of the model, which could affect the scope of the proposed model. In relation to the adjustment of the global model, **Table 3** shows several indicators. First, the chi-square contrast shows an adequate global adjustment value (p-value = 0.161). For its part, the root mean square error of approximation indicator shows an acceptable value (RMSEA = 0.032). With respect to the CFI and TLI indexes, they show optimal values close to 1. Similarly, the adjusted parsimony measures show values that are within the acceptable range: PRATIO = 0.500 and PCFI = 0.492. Finally, we can see that the model explains 56% of the variance in reading comprehension, measured with the posttest of the static test CLPT\_post, 46% of the variance in reading performance measured by the teacher's assessment, and 68% of the variance in reading performance, measured by the EDPL-BAI posttest.

### Analysis of Individual Relationships

An individual analysis of the regression coefficients for each of the routes proposed in the model was carried out (**Figure 8**). In this sense, the standardized solution shows significant relationships between the variables at a significance level of α = 0.001. Also, both the covariance and the correlations between the exogenous (independent) variables are significant (p < 0.001). Regarding the observed exogenous variables, the highest contribution (0.54) is produced by the CLPT pretest (CLPT\_2\_pre) on the score obtained on the posttest of the same test, while the contribution of the non-verbal intelligence test is 0.23. On the other hand, the adaptive/dynamic administration of the morpho-syntactic tests (DA) shows a significant and incremental contribution on the three criteria variables (0.17 on the CLPT posttest, 0.45 on reading performance as evaluated by the teachers, and 0.40 on the EDPL-BAI battery post-test).

### Structure and Standardized Solution to the Model 2

As mentioned before, the analyses to evaluate the model adjustment were replicated by introducing the dynamic scores

#### TABLE 3 | Indicators of fit for Model 1.

fpsyg-09-01492 August 25, 2018 Time: 10:59 # 14


CFI, Comparative Fit Index; TLI, Tucker-Lewis coefficient; RMSEA, Root Mean Square Error of Approximation; PRATIO, Parsimony Ratio; PCFI, parsimony fit to the CFI.

that had been calculated as a result of the implementation of the tests. In all the analyses performed, the previous assumptions of multivariate normality and model identification were contrasted. Depending on the interest of the contributions analysis of the scores derived from the calculation of heuristics, we collect the standardized solution for the dynamic scores based on the inverse of the value of graduated aids required by the student during the task resolution. In this sense, concerning the assumption of normality, the analysis showed an absence of normality, with values of asymmetry and kurtosis that exceeded the established limit of ±1.96 (p < 0.05). Therefore, as in the previous case, we decided to apply an asymptotically distribution-free estimation method. With respect to the identification of the model, the results of the analyses show that it also was an identified model.

**Table 4** shows the indicators obtained in relation to the evaluation of the model's global adjustment. The chi-square contrast shows an acceptable value (p-value = 0.073), as does the RMSEA indicator (0.039). With respect to the other indicators evaluated, the analysis related to the CFI index shows a value close to 1, which indicates a good level of adjustment, and the TLI index also shows an acceptable value. Likewise, the adjusted parsimony measures show values that are within the acceptable range: PRATIO = 0.528 and PCFI = 0.513. Finally, we can see that the model explains 53% of the variance in the posttest of CLPT, 49% of the variance in reading performance measured with the teacher's assessment, and 66% of the variance in the EDPL-BAI posttest.

#### Analysis of Individual Relationships

In this case, we also carried out an individual analysis of the regression coefficients for each of the routes proposed in the model (**Figure 9**). The model's standardized solution shows significant relationships between the variables at a significance level of α = 0.001, except for two variables, which show a significance level of α = 0.01. Also, both the covariance and the correlations between the exogenous (independent) variables are significant (p < 0.001). Regarding the observed exogenous

#### TABLE 4 | Indicators of fit for Model 2.

fpsyg-09-01492 August 25, 2018 Time: 10:59 # 15


CFI, Comparative Fit Index; TLI, Tucker-Lewis coefficient; RMSEA, Root Mean Square Error of Approximation; PRATIO, Parsimony Ratio; PCFI, parsimony fit to the CFI.

variables, the highest contribution is still the one produced by the CLPT\_2\_pre (0.54) score on the CLPT posttest. The adaptive/dynamic application of the morpho-syntactic tests (DA) shows a significant and incremental contribution on the reading performance as evaluated by the teachers (0.51), on the CLPT posttest (0.15), and on the EDPL-BAI posttest (0.22).

### DISCUSSION

One of the main objectives with regard to the development and implementation of psychoeducational evaluation models is that they offer valuable information oriented to the intervention. The computerized adaptive DA seeks to offer incremental information to that provided by conventional tests (Sternberg and Grigorenko, 2002). This incremental information is specified in providing data on the task resolution process, including the difficulties encountered by the student during the task, as well as data related to the analysis of the established mediation process. In this sense, the study of the mechanisms of action that can explain the improvements that can be observed during the implementation of DA procedures, and through which a student would optimize his or her learning, can be significant for inferring subsequent intervention guidelines (Grigorenko, 2009). The present study aimed to analyze the contribution on reading competence of the three adaptive dynamic tests that make up the block of morpho-syntactic processes of the EDPL-BAI battery. In this sense, two specific hypotheses were formulated. The first hypothesis argued that the dynamic scores obtained from the EDPL-BAI would be significantly related to the measures of subsequent performance in reading. The second one held that dynamic scores would contribute significantly to explain the variability in reading competence and would constitute an additional explanatory factor of reading performance

over the static tests of non-verbal intelligence and reading comprehension.

With respect to the first hypothesis, we must emphasize that the dynamic scores establish a significant relationship with the standardized reading comprehension test and the qualitative teacher's assessment. In this sense, the teacher's assessment allows the introduction of evaluative elements of a procedural nature that can hardly be evaluated with static performance measures. Likewise, the teacher's assessment has proven to be a good predictor of academic performance (Navarro and Mora, 2013). Also, the evaluation of the reading performance by teachers could increase the ecological validity of the results, since the assessment corresponds to the level of demand that is considered essential by the professionals in charge of the teaching-learning process.

Regarding the second hypothesis, the results obtained show that the dynamic scores obtained from the application of the EDPL-BAI battery further explain the variability in reading competence as measured with the CLPT test, the EDPL-BAI posttest, and the teacher's assessment of reading performance. In this sense, the analysis of the regression coefficients of the model's standardized solution indicates that the dynamic application of the tests maintains a significant and incremental contribution on the three measures of reading competence once the rest of the predictor variables are controlled for. This was observed for both the estimated student's knowledge level and the dynamic score obtained from the inverse of the value of the required aids.

One of the central questions that must be answered by an intervention-oriented DA approach is to know what the incremental proportion of variance explained by the dynamic scores represents. The incremental validity means that a part of the variance of the criterion measures can be explained as a result of the information that is derived from the application of the dynamic tests. In this sense, an analysis of the elements that can explain the changes could offer valuable information about the functioning of the subject. In particular, in the context of DA, this analysis of change is aimed at establishing what the subject is capable of performing when offered guidelines and graduated prompts—that is, informing us of his or her learning potential (King et al., 2015; Poehner et al., 2015). Therefore, the incremental proportion of variance explained by the dynamic scores would represent a quantification of learning potential. This change potential is reflected in the development of competencies that facilitate learning and benefit from the mediation offered—in this case, in the form of the graduated prompts. It is important to note that these aspects are not evaluated in the conventional tests. Likewise, in accordance with the idea to establish those mechanisms of action that could optimize the learning process, the information obtained through the dynamic application of the tests is qualitatively different from what we could obtain with the application of a standard comprehension test. This additional information would be mainly about the difficulties that the student manifests during the task resolution process, as well as about the mediation guidelines that are effective during the test administration process.

In line with the results obtained by King et al. (2015), and Hasson et al. (2012), the implementation of DA tests would have provided valuable information regarding the process followed by the students during the task resolution. This information would contain, in our case, data on the aids that were most effective in successfully resolving the different items, which might be useful regarding understanding the difficulties and the ways of intervening to resolve them. In this sense, the results of the model that introduces the dynamic scores based on the inverse of the value of the required aids show that those students who needed more aids, and especially aids specifically related to the difficulties of the task, obtained worse results in the criteria measures. The next step in an intervention-oriented evaluation procedure is obviously to qualitatively analyze the specific aids required that were effective in successfully resolving the items, with the aim of guiding the educational intervention and contributing to the improvement of the subject's functioning.

Like Poehner et al. (2015) and Kozulin and Garb (2002), we used heuristics to obtain differential information from the score that included data on the required aids. In our case, the dynamic scores that comprised the graduated prompts are not yet automatically generated by the system, but we expected that obtaining them would offer us information on the amount and type of aid that would be effective. Although we have presented only the standardized solution of the model that included the dynamic scores based on the inverse of the value assigned to the aids required to successfully solve the items carried out, the rest of the calculated scores (between them the heuristics used by the authors cited above) have shown, with some differences, significant contributions on the measures of reader performance. These scores also made it possible to obtain incemental information regarding the assessment of the difficulties in the reading processes analyzed, while providing relevant information oriented to the intervention.

### Limitations of the Study and Future Analysis and Development

The present study offers information about the implementation of three adaptive dynamic tests of the EDPL-BAI battery. However, the battery has 38 tests that integrate six blocks of processes involved in reading. The research that supports the present study is still under development, so the data shown represent only a part of the sample. Likewise, we must mention that the longitudinal nature of the study, developed over the course of 2 years, required the continuous collaboration of the educational centers. In this sense, despite this collaboration, it was not possible to obtain all measures for all participants. Another consideration that we should point out is that, although the research assistants who participated in the group implementation of the EDPL-BAI battery received specific training, the logistical and technical difficulties encountered during the implementation in the schools sometimes made data collection difficult. Also, we must point out some problems related to connectivity or access to the online evaluation platform, the availability of computers in computer labs in schools, and the availability of network support in rural schools. Finally, the system of graduated prompts used in

Phase 2 could not be fully implemented, since they only appeared in audio format and not in both, audio and writing format.

Regarding future analysis, the differential validity of dynamic scores on reading performance for different subgroups of students must be analyzed. These analyses could establish differences about the conditions of greater efficiency and effectiveness in the use of the EDPL-BAI battery, as well as in relation to the information that can be obtained (Navarro and Mora, 2013). Likewise, future analyses must also differentiate the results by grade or by different age groups.

Among the main advantages that derive from the use of computerized adaptive assessment tools, there is the possibility of continuously improving the different elements that make up the device, e.g., adjustments to the test interface, the item bank, the calibration process, or the system of graduated prompts. In this sense, the complete development of the system of graduated prompts, together with the joint calibration of the items and the aids, will result in greater possibilities for obtaining optimal results about predictive and incremental validity, and especially about the analysis of the effectiveness of the aids. Likewise, in relation to the users, the use of CAT based on IRT models that also incorporate a system of graduated prompts offers a series of additional advantages: it allows for recording and analyzing the sequence of actions performed by the student, the execution time, the successes and errors based on the attempts made, the difficulty levels of each item, and the aids required to successfully solve the items. The possibility of having all these data and of analyzing them in an integrated way could offer valuable information oriented to the intervention and improvement of the evaluated processes. Specifically, the incorporation of graduated

### REFERENCES


prompts into computerized adaptive assessment models would offers control and activity regulation tools and allow observation and assessment of the degree of incorporation of these tools by the student during the task resolution.

### AUTHOR CONTRIBUTIONS

J-JN and CM-C conceived and designed the study. J-JN, CM-C, and IR-O performed tables and figures. CM-C and J-JN analyzed the data. EG, IR-O, CM-C, RC, CS-G, JdlF, J-JN, DM, and MS contributed materials and analysis tools. J-JN, CM-C, EG, CS-G, and IR-O wrote the paper. IR-O, CM-C, CS-G, JdlF, EG, RC, DM, and MS paper revision.

### FUNDING

The results included in this paper are part of the FONDEF project ID15I10067 "Design and initial validation of a Computerized Adaptive Battery for the assessment of processes involved in reading and comprehension of texts," funded by the National Commission of Scientific and Technological Research, CONICYT (Chile).

### ACKNOWLEDGMENTS

We especially appreciate the collaboration of all the educational institutions that have participated in this research.


through grade 1. J. Learn. Disabil. 50, 227–237. doi: 10.1177/0022219415 609185


Self-Regulated Learning, eds D. J. Hacker, J. Dunlosky, and A. C. Graesser (Mahwah, NJ: Lawrence Erlbaum), 85–106.


Wolter, J. A., and Pike, K. (2015). Dynamic assessment of morphological awareness and third-grade literacy success. Lang. Speech Hear. Ser. 46, 112–126. doi: 10.1044/2015\_LSHSS-14-0037

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Navarro, Mourgues-Codern, Guzmán, Rodríguez-Ortiz, Conejo, Sánchez-Gutiérrez, de la Fuente, Martella and Saracostti. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Technological Resources to Prevent Cyberbullying During Adolescence: The Cyberprogram 2.0 Program and the Cooperative Cybereduca 2.0 Videogame

#### Maite Garaigordobil\* and Vanesa Martínez-Valderrey

Faculty of Psychology, University of the Basque Country (UPV/EHU), Bilbao, Spain

#### Edited by:

Jesus de la Fuente, University of Almería, Spain

### Reviewed by:

Georgios D. Floros, Aristotle University of Thessaloniki, Greece Bernadette Ann Murphy, University of Ontario Institute of Technology, Canada

> \*Correspondence: Maite Garaigordobil

maite.garaigordobil@ehu.eus

Specialty section:

This article was submitted to Cognitive Science, a section of the journal Frontiers in Psychology

Received: 21 January 2018 Accepted: 27 April 2018 Published: 16 May 2018

#### Citation:

Garaigordobil M and Martínez-Valderrey V (2018) Technological Resources to Prevent Cyberbullying During Adolescence: The Cyberprogram 2.0 Program and the Cooperative Cybereduca 2.0 Videogame. Front. Psychol. 9:745. doi: 10.3389/fpsyg.2018.00745 Bullying and cyberbullying have serious consequences for all those involved, especially the victims, and its prevalence is high throughout all the years of schooling, which emphasizes the importance of prevention. This article describes an intervention proposal, made up of a program (Cyberprogram 2.0 Garaigordobil and Martínez-Valderrey, 2014a) and a videogame (Cooperative Cybereduca 2.0 Garaigordobil and Martínez-Valderrey, 2016b) which aims to prevent and reduce cyberbullying during adolescence and which has been validated experimentally. The proposal has four objectives: (1) To know what bullying and cyberbullying are, to reflect on the people involved in these situations; (2) to become aware of the harm caused by such behaviors and the severe consequences for all involved; (3) to learn guidelines to prevent and deal with these situations: know what to do when one suffers this kind of violence or when observing that someone else is suffering it; and (4) to foster the development of social and emotional factors that inhibit violent behavior (e.g., communication, ethical-moral values, empathy, cooperation…). The proposal is structured around 25 activities to fulfill these goals and it ends with the videogame. The activities are carried out in the classroom, and the online video is the last activity, which represents the end of the intervention program. The videogame (www.cybereduca.com) is a trivial pursuit game with questions and answers related to bullying/cyberbullying. This cybernetic trivial pursuit is organized around a fantasy story, a comic that guides the game. The videogame contains 120 questions about 5 topics: cyberphenomena, computer technology and safety, cybersexuality, consequences of bullying/cyberbullying, and coping with bullying/cyberbullying. To evaluate the effectiveness of the intervention, a quasi-experimental design, with repeated pretest-posttest measures and control groups, was used. During the pretest and posttest stages, 8 assessment instruments were administered. The experimental group randomly received the intervention proposal, which consisted of one weekly 1-h session during the entire school year. The results obtained with the analyses of variance of the data collected before and after the intervention in the experimental and control groups showed that the proposal significantly promoted the following aspects in the experimental group: (1) a decrease in face-to-face bullying and cyberbullying behaviors, in different types of school

**95**

violence, premeditated and impulsive aggressiveness, and in the use of aggressive conflict-resolution strategies; and (2) an increase of positive social behaviors, self-esteem, cooperative conflict-resolution strategies, and the capacity for empathy. The results provide empirical evidence for the proposal. The importance of implementing programs to prevent bullying in all its forms, from the beginning of schooling and throughout formal education, is discussed.

Keywords: bullying, cyberbullying, intervention, adolescence, videogames

### INTRODUCTION: CONCEPTUALIZATION, PREVALENCE AND PREVENTION OF BULLYING/CYBERBULLYING

### Bullying and Cyberbullying: Conceptualization

Bullying is a specific form of school violence, where one or more attackers intentionally cause pain, harass, and repeatedly subject another classmate. When we talk about face-to-face or presential bullying, we refer to: (1) the existence of a defenseless victim, harassed by one or more assailants; (2) who carry out different types of aggressive face-to-face behavior toward the victim, aggressive physical behaviors aimed at the victim's body or property (hitting, pushing, breaking, hiding, or stealing the victim's objects. . . ), aggressive verbal behaviors (giving nicknames, insulting, saying unpleasant things about the victim. . . ), behaviors of social exclusion (not letting the victim participate, excluding, telling lies, or spreading false rumors about the victim so she will be rejected by others. . . ), aggressive psychological behaviors (aimed at undermining the victim's selfesteem, creating insecurity and fear: threatening, blackmailing, laughing at him, humiliating him. . . ; although it should be borne in mind that all types or forms of bullying have a psychological component); (3) it is maintained physical and mental violence; (4) the aggressors intend to harm, they are purposely cruel to make the victim suffer; (5) there is usually an inequality of power between the victim and the aggressors (physical, verbal, or psychological inequality); (6) these aggressive behaviors are repeated frequently, there is a dominion-submission relationship between the aggressor or aggressors and the victim that is maintained over time; and (7) in addition, the aggression not only produces pain when it occurs, as the victim feels sustained pain and anguish due to expectations of future attacks, aggressions, or humiliation that he or she anticipates suffering (Garaigordobil, 2017).

In recent years, other forms of bullying have emerged, such as cyberbullying, which consists of using information and communication technology (ICT)—mainly Internet and mobile phones—to practice peer harassment. Cyberbullying is bullying in digital format. What behaviors do cyberaggressors perform? A review of the behaviors identified by numerous authors (Aftab, 2010; Kowalski et al., 2010; Tokunaga, 2010; Garaigordobil, 2013, 2015) identifies the following: (1) sending insulting, threatening, disparaging, or intimidating messages through mobiles or e-mail (ugly, fat, everybody hates you, you should die, be careful; we're going to beat you up. . . ); (2) making anonymous phone calls to frighten the victim; and/or making threatening, intimidating, insulting, or disparaging calls. . . ; (3) manipulating photographs to ridicule or create a false image of the victim, which the aggressors distribute by mobile phone or internet; (4) excluding, isolating the victim from social networks (Facebook, WhatsApp, Instagram. . . ); (5) stealing the victim's password, and impersonating her identity (for example, sending aggressive messages to the victim's contacts to anger them; violating the victim's privacy, changing her password to prevent her access to her email account. . . ); (6) provoking the victim in chats, online games, virtual communities. . . to achieve her violent reaction, which they then denounce to the Service Manager so he will impede her access to that service; (7) creating a false profile of the victim and, for example, making explicit offers of sexual contacts, giving the victim's mobile phone as the contact. . . ; (8) signing up on some websites with the victim's email address so he will continuously receive emails and SPAM. . . ; (9) disseminating lies about the victim to harm her (false rumors, slander. . . ); (10) disseminating secret or embarrassing information about the victim, for example, concerning his orientation to his sexual identity; (11) denigrating or badmouthing the victim on a website, a personal blog, a social network. . . ; (12) making surveys to disparage the victim, for example, choosing her as the ugliest, the least intelligent, the fattest. . . and giving her the points or votes, which go to her email; (13) beating up or placing the victim in a humiliating situation, recording it on the mobile, and broadcasting the video via mobile or uploading it to YouTube (happy slapping). . . (Garaigordobil, 2017).

Therefore, in a situation of bullying and cyberbullying, we can differentiate three roles: a victim, one or more aggressors, and various observers whose silence and passivity largely facilitate the perpetuation of the situation over time. Observers frequently do not speak out due to lack of empathy with the victim and, other times, out of fear that the aggressor will turn against them.

### Bullying and Cyberbullying: Prevalence

What percentage of students is suffering from bullying and cyberbullying? To answer this question, we carried out a review, using a variety of databases (PsycInfo, Psicodoc, Scopus, Dialnet, CSIC, Latindex, PsycArticles, Eric, Google Scholar. . . ), which identified 309 epidemiological studies that have examined the prevalence of bullying and cyberbullying at the national and international level, since the first study carried out by Olweus (1973).

The review of the results of studies on bullying, both national (e.g., among others, Irakas-Sistema Ebaluatu Eta Ikertzeko Erakundea-Instituto Vasco de Evaluación e Investigación Educativa (ISEI-IVEI), 2009, 2012; Cerezo and Méndez, 2013; Garcia-Continente et al., 2013; Fernández-Montalvo et al., 2015; Navarro et al., 2015a,b; García-Fernández et al., 2016) and international (e.g., among others, Olweus, 2013; Ybarra et al., 2014; Bogolyubova et al., 2015; Hase et al., 2015; Malhi et al., 2015; McClanahan et al., 2015; Sumter et al., 2015; Pabian and Vandebosch, 2016; Safaria, 2016; Shin et al., 2016) shows an average percentage of frequent (severe) victimization that ranges approximately between 2 and 16%, but the percentage of students who suffer violent face-to-face behavior, albeit occasionally, exceeds 80% in some studies. In relation to the percentage of aggressors, studies show a range of severe aggressors between 2 and 12%, although in some studies, the percentage of occasional aggressors reaches 45%.

The review of studies of prevalence of cyberbullying, both national (e.g., among others, Irakas-Sistema Ebaluatu Eta Ikertzeko Erakundea-Instituto Vasco de Evaluación e Investigación Educativa (ISEI-IVEI), 2009, 2012; Buelga et al., 2010; Calvete et al., 2010; Estévez et al., 2010; Cerezo and Méndez, 2013; Gámez-Guadix et al., 2013; Garaigordobil, 2015; Navarro et al., 2015a,b) and international (e.g., among others, Olweus, 2013; Stewart et al., 2014; Ybarra et al., 2014; Hase et al., 2015; Sumter et al., 2015; Tsitsika et al., 2015; Wu et al., 2015; Pabian and Vandebosch, 2016; Safaria, 2016; Shin et al., 2016) reveals a mean percentage of serious or severe (very frequent) cybervictimization ranging approximately between 1 and 10%, but the percentage of students who suffer cyberbullying behavior, even if it is occasional, exceeds 60% in some studies.

As seen in the review, cyberbullying is an increasing phenomenon, as in every study carried out, higher percentages of cybervictimized students appear. This increase is occurring in part because children have access to new technologies (Internet, mobile phones. . . ) at increasingly earlier ages; because their activities in cyberspace are increasingly more relevant as socialization and entertainment spaces; because, as they do not occur in a face-to-face situation, the aggressors have a lower perception of the harm to the victim; sometimes, they even consider their behavior like a game, as if they were interpreting a role of fiction; also, the perception of anonymity sometimes increases the feeling of impunity; and also due to the characteristics of the Internet, which facilitate the grouping of cyberaggressors and the production/dissemination of audiovisual materials. In relation to the percentage of cyberaggressors, some studies show a range of severe cyberaggressors between 1 and 8% although, in other studies, the percentage of occasional cyberaggressors reaches 70%.

The prevalence percentages of bullying and cyberbullying are not homogeneous but instead, they vary in the different studies. It is difficult to provide a specific figure that reflects their level of prevalence in children, adolescents, and young people. Regardless of whether there is more or less presence of the problem of abuse and its different forms in different countries, the research data are not easily comparable for several reasons. Different studies vary in terms of age (5–22 years in bullying, 7–25 years in cyberbullying), the assessment technique or tool employed, the type of behavior studied (especially in cyberbullying), or the time interval considered (some ask to what extent bullying was suffered in the past year, others in recent months, others do not establish any time limit). All this only allows us to offer a range of percentages of serious victimization (severe victims/cybervictims) and a much higher percentage of occasional victimization. However, the results of prevalence studies highlight that: (1) bullying/cyberbullying is a phenomenon that occurs in all countries and in all social classes; and (2) the problem is worthy of consideration, which allows us to emphasize the need for assessment, prevention, and intervention.

### Bullying and Cyberbullying: Prevention Programs

Over the past decade, higher educational and social awareness toward the phenomenon of bullying has promoted an increase in preventive and palliative measures when it occurs. Although some of these prevention and intervention programs have been experimentally validated, a broad set of them still requires experimental validation processes that prove their consistency as suitable tools for the inhibition and decrease of bullying/cyberbullying.

In relation to the antibullying programs, we carried out a review. The results underscore the scarcity of interventions aimed at children of early ages. In addition, it is important to point out that the results of the antibullying programs are inconsistent. The effects vary greatly between programs, some have very weak effects, some do not even have any positive effects.

In this sense, some meta-analyses disagree with their effectiveness. For example, Ferguson et al. (2007) concluded that antibullying programs generate few effects in young participants, whereas the meta-analysis of Ttofi and Farrington (2011) showed that antibullying programs applied in school are effective. The meta-analysis of Ttofi and Farrington noted that, in general, these antibullying programs in school are effective to reduce bullying (on average by 20–23%) and victimization (by 17–20%). The improvement is increased as the participants in the intervention grow older. Programs that involve many hours of intervention for a long time interval are more effective, and also those that include meetings with parents (parent involvement), strong educational discipline (the use of disciplinary measures with the aggressors), cooperative learning, the use of anti-bullying videos, and supervision of recess. In the same direction, Menesini and Salmivalli (2017) considered that the programs in which the whole school is involved to prevent bullying are often successful. For these authors, student awareness about the important role of the group in the elimination of bullying is crucial.

The high prevalence of bullying and cyberbullying, with their serious consequences, emphasizes the need to develop resources aimed at preventing and reducing these behaviors of face-to-face, and cybernetic (digital bullying). Within this contextualization and justification, we propose this work, which consisted of the design and evaluation of the effects of an intervention made up of two tools (Cyberprogram 2.0 and Cooperative Cybereduca 2.0) which are intended to educate youngsters about the adequate use of the ICT, as well as to prevent and reduce bullying and cyberbullying.

### THE INTERVENTION PROPOSAL TO PREVENT AND REDUCE CYBERBULLYING DURING ADOLESCENCE

The intervention proposal is made up of a program (Cyberprogram 2.0; Garaigordobil and Martínez-Valderrey, 2014a) and a videogame (Cooperative Cybereduca 2.0; Garaigordobil and Martínez-Valderrey, 2016b), which are designed to prevent and reduce cyberbullying during adolescence.

### Objectives of the Intervention Proposal

The set of activities that make up the intervention to prevent and reduce cyberbullying revolves around 4 large general objectives that include several specific objectives:


The intervention proposal promotes cognitive restructuring of the roles involved in bullying/cyberbullying, while the modification of cognitions promotes behavioral changes. Victims learn to defend themselves and observers learn to intervene in favor of the victims.

### Features, Modules, and Activities of the Proposal

The intervention program has been designed for use with adolescent groups, but it can also be implemented with youth of older age groups. This experience can be carried out by the teacher-tutor of the group or by the school psychologist.

The program's 25 activities are distributed in 3 intervention modules or axes about bullying and cyberbullying: (1) Conceptualization and identification of roles (the activities focus mainly on knowing the characteristics and behaviors of the two phenomena, identifying the various roles involved in these situations); (2) Consequences, rights, and responsibilities (the activities promote awareness of the severe consequences of bullying and cyberbullying for victims and aggressors, and the observers' responsibility for the continuation of the situation); (3) Coping strategies (the activities teach guidelines to prevent this type of violence and provide strategies to cope with it, both in the role of victim and of observer). The program concludes with the videogame Cooperative Cybereduca 2.0 (see **Table 1**).

The activities of the program provide the students with strategies to prevent and deal with bullying in all its forms. The focus of the intervention is: (1) to realize the serious harm produced in the victims in a bullying situation, the adverse effects that it can have in their lives; (2) to be aware of the

TABLE 1 | Modules and activities of the intervention proposal.


negative implications, also for the aggressors and observers; and (3) to become aware of the important role that observers play in the maintenance of this situation, as their passivity allows these situations to be perpetuated over time. The intervention sessions mobilize the development of students' social and emotional variables (communication, cooperation, empathy, conflict resolution. . . ), which has the effect of inhibiting violent behavior.

The Cyberprogram 2.0 activities were created ad hoc, and the technical specifications of the activities of the program include six informational fields: objectives, description of activity, discussion or debate, materials, time, and group structure. The manual of the program is made up of the technical cards of the activities and the methodology to implement it and assess its impact. The Cyberprogram 2.0 manual also includes a CD that contains all the necessary materials for the development of activities that can be provided by this software, for example, cards to analyze the activities, complementary materials. . . . As an example, two activities of Cyberprogram 2.0 (Garaigordobil and Martínez-Valderrey, 2014a) are presented. The first one aims to enhance awareness of the victim's suffering, foster empathy with the victim, and identify coping strategies. The second activity aims to identify actions to be carried out by the school to prevent, reduce, and cope with bullying/cyberbullying.

The activity "Let's talk about Patty" aims to: (1) analyze the feelings of the victims, the aggressors, and the observers, enhancing empathy for the victims; (2) reflect on the consequences of bullying/cyberbullying for everyone involved; (3) identify positive strategies to cope with bullying situations; and (4) identify bullying situations within the group (faceto-face and/or electronic). The children watch a video in which Lindsay, a girl, reads a composition about another classmate in the assembly hall. http://www.youtube.com/watch? v=bdQBurXQOeQ. The composition refers to the classmate's defects in an insulting and humiliating way. Moreover, the narration is read out loud and in front of a large audience of classmates and teachers who listen in silence. As Lindsay reads the narration, we see Patty's face, distorted by shock, as she hears, one by one, all kinds of critical statements about her. After viewing the video, a large group debate is carried out. The adult will launch questions, for example: What roles are identified in the story? How does the victim feel? How does the aggressor feel? How do the observers feel? and so on. The participants will respond in the whole group to the issues proposed by means of the brainstorming technique, and the adult will summarize and write the main contributions on the blackboard. Secondly, the group is divided into three teams, and each one will receive a card with the word "happy," "tragic," or "neutral." Each team should write down and represent the end of Patty's story, depending on the word they were assigned. The drama will show the situation and the assigned end. At the end of the dramatizations, there is a debate about the different outcomes. The reflection focuses on analyzing the consequences for all those involved: What consequences derive from this type of behavior for all those involved? Why do we say things in virtual environments that we would not dare to say face-to-face? Do we know of any similar case? How can we avoid having offensive attitudes toward others? What should we do in a situation like this, if we are suffering these behaviors by others?.

The activity "Secrets from cyber-rooftops" has the following objectives: (1) to identify behaviors that are part of the cyberbullying phenomenon. (2) to raise awareness of the serious consequences of cyberbullying for everyone involved; (3) to reflect on the value of trust in cyber-relations (intimacy and privacy); (4) to analyze the risks of sharing information in cyberspace; (5) to develop a critical capacity for the healthy use of ICT; (6) to promote empathy for cybervictims; and (7) to promote communication, cooperation, and emotional expression. In this activity, firstly, the group members watch a video (http://www.youtube.com/watch?v=97ZBIhvCCEg) in which a teenager spreads intimate information about two classmates, referring to their sexual orientation. The video deals with a situation in which there is a theft of images. The affective, loving relationship between two girls is photographed and broadcast in the whole institute. Subsequently, the students analyze the victims' feelings and the consequences of that behavior for all those involved (because disclosing/revealing/disseminating secrets is an offense that is typified in the penal code). Later, teams of 4–5 students are formed and each team should consider strategies for the victim and/or the observers to cope with the situation, analyze them, and select the strategy they consider the most positive and effective for the victim and/or the observers to deal with the situation. By turns, each team represents the solution, showing the coping strategy they considered the most effective. After finishing the representations, a debate takes place in which the teams report other strategies that were considered positive for victims and observers to deal with that situation.

### Implementation Procedure of the Intervention Proposal

To implement this proposal with a group, a weekly intervention session of approximately 1 h was carried out during a school year. The session is directed by the school psychologist and/or the group's tutor. It takes place in a large, diaphanous classroom, without any obstacles (tables, chairs), which provides a computer with connection to internet, a blackboard, and mats or cushions to sit comfortably on the floor. The group goes to the classroom where the intervention takes place, the participants sit on the floor in a circle, and the session begins. Firstly, the adult briefly explains the objectives of the activity, and usually after watching a short video, instructs the group on how to carry out the activity. Secondly, the group members, distributed in teams, carry out the action cooperatively. Finally, a phase of debate or discussion is conducted about the activity carried out, and the teams' reflections and/or conclusions are shared, scenes are dramatized, and/or the students discuss the products of their activity (e.g., behaviors that they think serve to prevent cyberbullying. . . ). In general, all the intervention sessions follow this structure.

The main techniques of group dynamics used in this proposal include: (1) brainstorming, which is used in many activities that require constructive responses to problems; (2) drama or roleplaying of situations in which, for example, constructive ways of coping with bullying/cyberbullying by victims and observers are presented; and (3) the study of real cases of students who suffered bullying/cyberbullying, and this experience caused severe psychological harm and, in some cases, even led them to commit suicide. In the last session, the videogame "Cooperative Cybereduca 2.0" is played.

### Cooperative Cybereduca 2.0. A Videogame to Prevent and Reduce Bullying and Cyberbullying

To complement and reinforce the effects of Cyberprogram 2.0, a videogame has been built, "Cooperative Cybereduca 2.0. A videogame to prevent and reduce bullying and cyberbullying" (Garaigordobil and Martínez-Valderrey, 2016b), which can be accessed at the following website: http://www.cybereduca.com

The videogame is played online and is free of charge. It is intended to play on computers, with an adult who guides the development of the game and who fosters reflection, although it can also be played independently by the adolescents to whom it is addressed as well as individually. It is multilingual, it can be played in Spanish, Basque, and English and it can be implemented in a variety of contexts: in schools playing with the entire classroom under the teacher's guidance, in groups of leisure time, in the family context, with parents and children playing. . . The videogame is frequently played in the classroom, as the last activity of Cyberprogram 2.0, as its conclusion and to recall and review all the points reflected on and experienced in the course of its implementation.

This videogame is a Trivial Pursuit consisting of questions and answers that revolve around the topic of bullying and cyberbullying, face-to-face bullying and bullying through ICT (Internet, mobile phone. . . ). This cybernetic trivial pursuit is organized around a history of fantasy, a comic that guides the game and has three features: it is a cooperative, constructive, and non-sexist game. Cybereduca 2.0 is a cooperative game. Cooperative games promote communication, increase prosocial behavior, enhance group cohesion, and improve one's selfconcept and of that of others. Cybereduca 2.0 is literally a constructive game. The characters are construction guilds whose goal is to use their cooperative skills to rebuild fantasy worlds. The guilds of Cybereduca 2.0 do not wield weapons, but instead tools, and they do not fight any battles, but instead, they cooperate with each other to achieve their common goals and thus restore peace and coexistence. In addition, the correct answers to the questions of each team are integrated into a global score for all the players. Cybereduca 2.0 is a non-sexist game that presents the same number of male and female characters, equitably distributed throughout the game. The collective guilds are represented by a girl-and-boy team. Both team members have similar physical and psychological profiles, not differential according to sex, and they use the same kind of clothing without any variation. The unitary guilds are represented either by an androgynous character or by a non-stereotyped feminine one.

The game starts with an earthquake in the city of Zanthia, where the characters are found. The earthquake causes the opening of a vortex, and all the characters/guilds fall into the vortex into cyberspace. The characters/guilds represent the five roles involved in a bullying/cyberbullying situation: (1) Aggressors (skull stonemasons' guild: tough, insensitive guys, who play practical jokes on others. . . ); (2) Victims (solitary painters' guild: sensitive, the target of the skull stonemason's jokes, solitary. . . ); (3) Defending observers of the victims (justice engineers' guild: they tend to get along with everyone, but they do not tolerate injustice, if someone is behaving badly, they tell them to their face. . . ); (4) Observers who support the aggressors (giggling carpenters' guild: sociable and fun, they like to be part of the group, they do not tend to play jokes on the victims, but if others do so, they laugh); and (5) Passive observers who do not intervene (impassive plumbers' guild: they go their own way, they don't argue with anyone, but remain aloof from problems).

The guilds (players/teams) must perform challenging actions (open doors, freeing missing characters) and rebuild worlds (Loot Bay, Gadget Villa, Dragon Nest, Flying High, and City of Zanthia). For this purpose, they must cooperate by answering questions because, in essence, Cybereduca is a game of trivial pursuit. The complete set contains 120 questions that revolve around 5 topics: cyberphenomena, computer technology and security, cybersexuality, consequences of bullying/cyberbullying, and coping with bullying/cyberbullying.

Topic 1. Cyberphenomena: In this topic, the contents of the questions help to identify and define bullying, cyberbullying, and other cyberphenomena related to the use of the mobile, internet, ICT, such as grooming, nomophobia, cyberbaiting, flaming, griefing, trolling, outing, cyberstalking, phishing. . . For example: The new phenomenon consisting of a compulsion that affects mobile Internet users, and characterized by having an irrational fear of going out without one's smartphone, is called: (A) Phobiaphone. (B) Nomophobia. (C) Monophobia.

Topic 2. Computer technology and Security: The questions in this topic, on the one hand, help to clarify computer concepts (firewall, cookie, blogger, antivirus, router, spam, anti-popup programs . . . ), provide data on protection rules and the safe use of ICT (mobile, internet. . . ), and, on the other hand, they identify risky behaviors and teach ethical standards of behavior in social networks and cyberspace. For example: If we suspect that someone is accessing our e-mail account or impersonating our identity, we should: (A) Delete the account and create a new one. (B) Change the password. (C) Report it to the police.

Topic 3. Cybersexuality: The questions associated with this topic, on the one hand, help to identify, prevent, and deal with sexting and, on the other hand, they foster reflecting on various sexual behaviors that are performed using ICT and that have very negative consequences, such as sextortion, grooming, sexual abuse, etc. For example: Sending sexually explicit images among people of the same age isn't a problem: (A) No, because they are people of the same age;

(B) It depends on whether there's an affective relationship between those people; (C) It's a problem indeed, because the image becomes undeletable and can be disclosed to other people.

Topic 4. Consequences of bullying/cyberbullying: In this topic, the contents of the questions allow the identification of the consequences of face-to-face and electronic bullying for the victims, the aggressors, and the observers (emotional, social, and intellectual effects. . . ), enhancing the development of empathy for the victims. For example: What is one of the effects of cyberbullying on observers, as bystanders in cyberbullying situations? (A) None, because they have nothing to do with it. (B) None, provided that they don't get involved. (C) They may become unsupportive people, insensitive toward other people's feelings.

Topic 5. Coping with bullying/cyberbullying: In this topic, problematic situations are presented, showing appropriate behaviors to deal with situations of bullying, cyberbullying and other events associated with the use of ICT, from the perspective of the victims, the observers, and the aggressors. For example: Laura and Marta were best friends "forever" and they shared everything, even their e-mail and social network passwords. One day, they quarreled and stopped being friends. Marta used Laura's password to access her social network account, and she sent offensive messages to all of her contacts, pretending to be Laura. What advice would you give to Laura? (A) Don't tell anybody anything and send an apology to all your contacts. (B) Remove all your contacts. (C) Report the issue on your social network, make Marta realize the consequences of her behavior, and change your password regularly.

In addition to the cards of these five areas, in which there is always a correct answer, the game includes some complementary cards that require performing representations and cooperating. Dramatic actions encourage the players to represent positive and negative emotions (sadness, happiness, anger, fear. . . ), and to perform prosocial actions in which they must cooperate with other team members (for example, giving a hug, cheering, singing a song. . . ).

In the dynamics of this game, the teams answer questions that allow them to open doors, free the videogame characters, and rebuild worlds to return to the city of Zanthia, always with the cooperation of all the players. The game is won when they all achieve the goals after answering questions and rebuilding the worlds through cooperative construction actions. This allows them to return to the city of Zanthia in which from that moment, a positive social climate of harmony and camaraderie among the guilds will reign.

### EMPIRICAL ASSESSMENT STUDY OF THE INTERVENTION PROPOSAL: METHODOLOGY AND RESULTS

This intervention proposal is based on evidence, that is, it has been validated experimentally. The evaluation of the effects of Cyberprogram 2.0 was conducted by applying Cyberprogram 2.0 and the game Cybereduca 2.0 in a similar, non-digital format because the Cybereduca game was transformed into a videogame later on (2016). The results of the intervention have been recently published, so below, we present a synthesis of the empirical study carried out regarding its methodology and the results obtained.

### Participants

This study was carried out with a sample of 176 adolescents, aged between 13 and 15 years, who studied Compulsory Secondary Education (grades 9 and 10). Out of the total sample, 93 (52.8%) were assigned to the experimental condition and 83 (47.2%) to the control condition. Of the sample, 25% were 13 years old, 48.9% were 14, and 26.1% were 15. The study was carried out in three schools of Gipuzkoa (northern Spain) of diverse socioeconomic-cultural level. Of the students, 44.3% attended public schools and 55.7% a private center. A random sampling technique was used to select the sample, taking into account the list of schools in Gipuzkoa and the type of center (public-private).

### Procedure

The study used a quasi-experimental, repeated pretest-posttest measures design with a control group. To carry out the research, first, the schools were randomly selected, and we described the project to the principals and requested their collaboration. We provided informed consent forms for the parents and participants of the schools whose principals agreed to participate. After the consent forms had been signed, the research team applied the pretest assessment in the schools (see **Table 2**). In each school, some classrooms were randomly assigned to the experimental condition and others to the control group. The intervention was implemented with the experimental group during the school year, while the control students carried out the usual activities of the school tutoring program. Subsequently, the posttest assessment was done, using the same tools as at pretest. The study respected the ethical values required in research with humans, and received the favorable report of the Ethics Committee of the University of the Basque Country (CEISH/112/2012).

### Instruments

In order to measure the dependent variables, before and after implementing the intervention, the experimental and control students both filled in eight assessment instruments (see **Table 2**).

### Data Analysis

After verifying the basic assumptions, to assess the program's effect on the dependent variables, firstly, we carried out descriptive analyses (means and standard deviations), and univariate and multivariate analyses of variance (ANOVA, MANOVA) with the pretest scores obtained on the eight assessment instruments, by the experimental and control participants. Secondly, we carried out descriptive analyses and analyses of covariance of the pretest-posttest differences (ANCOVA, MANCOVA) using the pretest differences between the two conditions as covariate, thereby determining the intervention's impact. In addition, we calculated the

#### TABLE 2 | Pretest-posttest evaluation instruments.


effect size (Cohen's d: small <0.50, moderate 0.50–0.79, large ≥0.80) of each variable, at pretest, and in the pretest-posttest differences. Complementary, we calculate the effect size (Eta square) for the groups of variables (Cohen's values: from 0.01 to 0.04 are judged to be small, 0.04 to 0.14 moderate, and greater than 0.14 large). The statistical analyses were performed with the SPSS 21.0 program.


Cohen's effect size. Experimental n =93, Control n =

## RESULTS

The results obtained have shown that the proposal significantly promoted the following changes (see **Tables 3**, **4**) (Garaigordobil and Martínez-Valderrey, 2014b, 2015a,b, 2016a):


significantly more in the experimental group compared with the control group.


### DISCUSSION

The results provide evidence of the effectiveness of the program, validate the intervention proposal, and, as a whole, allow us to emphasize the importance of implementing programs to prevent cyberbullying and other negative cyberphenomena (nomophobia, sexting, sextortion, grooming. . . ). The study provides a tool of effective psycho-educational intervention to prevent and reduce cyberbullying during adolescence. The use of self-reports, with the inherent bias of social desirability, is a limitation of the study.

These results are coherent with other studies showing the efficacy of antibullying interventions to decrease aggressiveness (McMahon et al., 2000; Fonagy et al., 2005; Twemlow et al., 2005) and violent peer behaviors of victimization and perpetration, either face-to-face (Olweus, 2004; Fekkes et al., 2006; Gollwitzer et al., 2006; Milton and O'Moore, 2008; Kärnä et al., 2011; Palladino et al., 2012; Williford et al., 2012) or electronic (Doane, 2011; Lee et al., 2013; Williford et al., 2013; Chaux et al., 2016). In addition, the results point in the same direction as other studies showing the efficacy of antibullying programs to increase prosocial behaviors (Grossman et al., 1997; Gini, 2004), social competence (Leadbetter et al., 2003), and self-concept—selfesteem (DeRosier and Marcus, 2005; Rawana et al., 2011).

Among the factors that can explain the positive results of the intervention are the characteristics of the activities, as they


TABLE 4 | Results of the Pretest MANOVAs, of Pretest-Posttest MANCOVAs, and Effect Size (η²) in all grouped variables, between experimental and control groups.

3, Wilks' Lambda; η², Eta squared effect size.

promote: (1) empathy toward the victim (understanding the victim's feelings and the severe harm involved in bullying/cyberbullying for its victims); (2) analysis of the consequences for all those involved in bullying/cyberbullying; and (3) mobilization of the observers to defend the victim and denounce what they are observing. The activities included in this proposal stimulate affective-emotional and social development processes (e.g., the ability to dialogue rationally, two-way communication, assertiveness, prosocial behavior, empathy, constructive conflict resolution. . . ), which play an important role in reducing violent behavior. These activities create a positive climate in the group, which promotes coexistence, increases prosociality, and reduces violence.

### REFERENCES


## AUTHOR CONTRIBUTIONS

MG: has designed the research and she has supervised the application of the instruments and the implementation of the program. She has carried out the data analysis and has written the article. VM-V: has applied the evaluation instruments and has implemented the program. She has collaborated in the design of the program and in the analysis of data.

### ACKNOWLEDGMENTS

Study financed by the University of the Basque Country (UPV/EHU) (PPG17/31), and by the Ministry of Economy, Industry and Competitiveness (PSI2017-90650-REDT).


Competence Training.' Swiss J. Psychol. 65, 125–135. doi: 10.1024/1421-0185. 65.2.125


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Garaigordobil and Martínez-Valderrey. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Characteristics, Structure, and Effects of an On-Line Tool for Improvement in Adolescents' Competency for Interaction With Alcohol: The e-ALADOTM Utility

#### Edited by:

Jesús de la Fuente1,2

Pablo Fernández-Berrocal, Universidad de Málaga, Spain

## Reviewed by:

Amâncio António De Sousa Carvalho, University of Trás-os-Montes and Alto Douro, Portugal Yang-Hsueh Chen, National Chengchi University, Taiwan

> \*Correspondence: Jesús de la Fuente jdlfuente@unav.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 13 May 2018 Accepted: 14 January 2019 Published: 26 February 2019

#### Citation:

de la Fuente J, Cubero I, Peralta FJ, Sánchez MC, Salmerón JL and Fadda S (2019) Characteristics, Structure, and Effects of an On-Line Tool for Improvement in Adolescents' Competency for Interaction With Alcohol: The e-ALADOTM Utility. Front. Psychol. 10:127. doi: 10.3389/fpsyg.2019.00127 Mari Carmen Sánchez<sup>2</sup> , Jose Luis Salmerón<sup>3</sup> and Salvatore Fadda<sup>4</sup> <sup>1</sup> School of Education and Psychology, University of Navarra, Pamplona, Spain, <sup>2</sup> Department of Psychology, University of

\*, Inmaculada Cubero<sup>2</sup>

Almería, Almería, Spain, <sup>3</sup> Educational Psychologist, Centro Educativo Agave, Almería, Spain, <sup>4</sup> Prevention Service, University of Sassari, Sassari, Italy

, Francisco Javier Peralta<sup>2</sup>

,

This research report aims to present the characteristics, structure and effects of a psychoeducational technological innovation (called the e-ALADO Program) for the prevention of alcohol intake in adolescents. Based on the Competency model for interaction with alcohol, this program consists of a total of 24 lessons that promote conceptual, procedural, and attitudinal learning, in ICT format (www.alado.es). The hypothesis of this validation study established that adolescents treated with the program would improve their levels of competence and their interaction behavior with alcohol, depending of their personal level of self-regulation. A total of 148 adolescents from 12 to 16 years of age from three Spanish educational centers with different sociocultural contexts participated. A quasi-experimental methodology with repeated measures and use of inferential analysis was used (ANOVAs and MANOVAs). The results show a main principal effect of the Treatment variable, of the Self-Regulation Level variable, and an interaction effect of Treatment × Self-regulation in the conceptual and attitudinal subcompetence for interaction with alcohol. The results are discussed in the face of new technological developments that allow the evaluation and intervention in the prevention of alcohol intake with adolescents. An important implication of this work is related to the importance of self-regulation as a psychological variable. Also, the suitability of psychoeducational interventions with new technological formats in the prevention of adolescents' alcohol intake as entrepreneurial activity.

Keywords: prevention of alcohol intake, competence model, adolescence, e-Program, technological utility

### INTRODUCTION

fpsyg-10-00127 February 23, 2019 Time: 18:29 # 2

### Preventing Alcohol Intake Through Online Systems in Adolescence Problems of Alcohol in Adolescence

Alcohol consumption among adolescents is an international problem (Elsayed et al., 2018; Foster et al., 2018; Kendler et al., 2018). In Spain, the age of contact with alcohol and intake of alcohol is increasingly premature. The Spanish Surveys on Drug Use in Secondary Education (Spanish Observatory of Drugs and Drug Addiction [Observatorio Español de la Droga y las Toxicomanías] (OEDT), 2014–2015) indicate that, despite a reduction in consumption rates, alcohol remains the most frequently consumed substance among Spanish adolescents. A high percentage of Spanish adolescents report regularly consuming alcohol and having started consumption at a very early age (Quiroga et al., 2018).

It's a fact that this problem requires bold and innovative intervention strategies that allow educating pre-adolescents with primary prevention strategies, before the problem requires intervention in secondary prevention. For this purpose, recent research has tried to show whether treatments based on ICT systems are more efficient than traditional systems (Bennett and Glasgow, 2009; Munson and Jaccard, 2018). This question is especially relevant for adolescent populations, which can already be considered as a digital natives (Collins and Halverson, 2009).

### The Importance of the Use of Online Technological Strategies

The use of online ICTs has been increasing in recent years, based to their versatility and their numerous possibilities for addressing different populations (White et al., 2010; Sundström et al., 2017; for more information, see the present Monographic). There have been quite a number of web-based interventions in adult populations (Johansson et al., 2017; Mellentin et al., 2017; Baldin et al., 2018; Han et al., 2018). Online interventions have also been carried out in populations of university students and youth (Bertholet et al., 2015; Berman et al., 2016; Fazzino et al., 2017).

However, utilities oriented to adolescent populations are much fewer in number (Arnaud et al., 2016; Caudwell et al., 2016). Moreover, many such online tools are based on psychosocial or clinical models of prevention, applied to the sphere of educational psychology—despite the recent development of theoretical and research models using the Competency Approach of Educational Psychology itself, and the evidence that has upheld the value of this approach (de la Fuente et al., 2017a). This Competency Approach assumes the importance of the acquisition of three levels of learning: knowledge, skills, and attitudes, in a manner consistent with the Gagné Educational Psychologist model (Gagné, 2013). Consequently, there is a need for evidencebased, online technological developments with an educational psychology approach. This aim of this paper is to present the characteristics and effects of the ALADO online utility, a technological tool that has already been tested in educational psychology interventions for the prevention of alcohol intake in adolescents. Other complementary results have already been published, in Spanish (Marcos, 2013).

### The e-ALADO Utility as a Psychoeducational Technological Innovation Foundations

### **The theory of Self- vs. Externally-Regulated Learning**TM **and the alcohol intake**

Brown (1998) conceptualized self-regulation (SR) from the tendency of individuals regarding their particular ability to plan and flexibly manage their behavior. SR has received particular attention in recent years as an essential factor to better comprehend health and disease, as a result of healthy habits and individuals' capacity to set and maintain healthy goals (Mann et al., 2013; Kitsantas et al., 2017). In fact, SR has been considered, based on recent evidence, as a variable or construct of the metabehavioral order, meaning, a meta-skill or skill to manage the cognitive, affective, and motivational abilities.

This Self-Regulated Learning vs. Externally-Regulated Learning (SRL vs. ERL) theoretical approach (de la Fuente, 2017) predicts that the individuals' may be characterized on a behavioral continuum: Self-regulatory, A-regulatory, or Dysregulatory behavior. In this case, despite the fact that the name of the Theory refers to Self-Regulated Learning (SRL), the construct it refers to is that of personal self-regulation (SR) as a more general construction, considering that SRL is a specific type of Self-Regulation. There is evidence that people may have different degrees of personal self-regulation (high–medium–low), alluding to the extent and to the number of practices they employ to exercise their health behavioral regulation (Zimmerman et al., 1999; Clark et al., 2001). Self-regulation (SR), or high in selfregulation, may be considered as the degree of a person's positive proactivity in its active and adequate management of the regulation of well-being and health (Brown, 1998). A-regulation (AR), or medium in self-regulation, may be defined conceptually as the lack of proactivity and so equivalent to the concept of behavior reactivity (Zimmerman and Labuhn, 2012). Dysregulation (DR), or low in self-regulation, may be defined as the degree of negative proactivity, that is, of active and inadequate management to regulate one's behavior. This de-regulation avoids the effort involved in proactive self-regulation of health, and of the procrastination (Clariana, 2013; Balkis and Duru, 2017).

This theoretical model considers the context as the set of situational stimuli that can make probable the directionality of a behavior, in interaction with the subject, being these of real, virtual or symbolic type. (1) External Self-Regulatory context (ESR) promotes positive or adequate proactivity, or clearly fosters self-regulation. In this context, there are numerous external signs or encouragements which promote and make self-regulated behavior more likely at the beginning, during, and at the end of all behavioral acts. Highly predictable of positive events are a feature of this context. (2) External A-Regulatory (EAR) context does not promote external self-regulation, or de-regulation. In this context, there are no external signs or encouragements to make self-regulated behavior or de-regulated behavior more

likely at the beginning, during, or at the end of the action. Highly unpredictable events are a feature of this context. (3) External Dys-regulatory (EDR) context, actively promoting deregulation. The context promotes non-positive, inadequate, or negative proactivity. In this context, there are many external signs which make de-regulated behavior more likely, favoring active de-regulation at the beginning, during, and at the end of the behavioral act. This kind of context means that the individual needs to make a great effort to engage in self-regulation. Highly predictable of negative events are a feature of this context. In a previous research report, examples of this theoretical formulation were presented (de la Fuente, 2017; pp. 3–4).

### Characteristics

The e-Alado is a technological online utility<sup>1</sup> , based on the ALADO program (de la Fuente et al., 2012), designed for adolescents The program provides assessment and intervention in different matters of learning. It can be used by students, teachers, and parents, but in this case it was applied only to students. The teacher's guide and instructions for use have been published.

#### Structure

The competency model for interacting with alcohol (de la Fuente et al., 2017a) is the foundation of this utility and its associated program, and is consistent with the SRL vs. ERL Theory (de la Fuente, 2017). Its three constituent types of learning (subcompetencies) have been defined theoretically. Prior research has shown the importance of each type of learning (de la Fuente et al., 2017a):

(1) Knowing (understanding or conceptual subcompetence), that is, having adjusted knowledge and not only information with reference to a given phenomenon, with facts, concepts and principles that have been well elaborated:


(2) Being able to (know-how or procedural subcompetence), referring to the meta-skill of self-regulation and decision-making, and to social interaction skills for behaving as one wishes:


<sup>1</sup>http://www.alado.es

(3) Mindset (knowing how to be or attitudinal subcompetence), referring to attitudes, values and habits toward alcohol as a substance:


In this Competence model for interaction with alcohol, it is assumed that: (1) a person is competent, when they have knowledge (know), have skills and meta-skills (know how to do) and want (know how to be); (2) the level of competence predicts the way in which the adolescent interacts with alcohol, that is, how he manages and decides in situations where alcohol is present and is capable of exercising counter-control and behavioral self-regulation, and finally, do not consume alcohol. It is assumed that there is a good interaction with the substance when it is not consumed during adolescence.

### **Teaching–learning content**

The teaching–learning content of the e-ALADO utility is as follows (see the demo<sup>2</sup> ):

(1) Facts, concepts, and principles content (see **Anex, Figure A1**).

• Facts from real life (cognitive branch of knowledge):

Alcohol in human culture Situations in which alcohol is present Personal and social consequences of inadequate alcohol use

• Concepts referring to alcohol (cognitive branch of knowledge):

Brain effects from alcohol intake 1 Brain effects from alcohol intake 2 Brain effects from alcohol intake 3 Brain effects from alcohol intake 4

• Principles of behavior (affective branch of knowledge; beliefs):

Principle of healthy recreation Principle of health and alcohol abstinence Principle of self-regulation and responsibility (self-determination)

(2) Procedures content: Meta-skills and skills (see **Anex, Figure A2**).

• Personal:

Improving as a person Self-regulation and personal self-determination (1) Self-regulation and personal self-determination (2) Self-regulation and personal self-determination (3) Strategies for coping with problems (1)

<sup>2</sup>http://alado.es/leccion\_demo.html

Strategies for coping with problems (2)

• Social-moral

Assertiveness and social skills Values clarification

• Cognitive-linguistic

fpsyg-10-00127 February 23, 2019 Time: 18:29 # 4

Making decisions about how to behave in complex situations

Imagining alternatives in complex situations

(3) Attitudes, values, and rule-following (habits) content (see **Anex, Figure A3**).

• Attitudes:

Exercising caution with alcohol Recreation under control Responsibility

• Values:

Personal honesty Respect and care for one's body Health and well-being

• Habits:

Habits of practicing sports Habits of recreational activities Habits at parties and celebrations Habits at difficult personal times

#### Methodology and Activities

The program was carried only with on-line activities of the same. The teacher-tutor only motivated the students to do so and after the activity dialog about the doubts. The duration was 8 months because the weekly time of the tutorial task to do it was only 1 h. Also to give time to a consistent attitudinal change and not remain as a punctual and intensive activity, without impact on adolescents. Eight learning modules over 8 months (October–May). Each month-long learning module includes three learning units of subcompetencies, corresponding to the conceptual, procedural, and attitudinal levels. These eight modules incorporate initial, process and product assessments, corresponding to the teaching–learning process. For each of the 24 lessons (three lessons per month), 8 for each conceptual, procedural, and attitudinal subcompetency (see content structure), there are different types of activities:


An example of the activities developed, in each lesson, could be found in the Teacher's Guide of Intervention (only in Spanish)<sup>3</sup> .

### Objectives and Hypotheses

Based on the foregoing assumptions, the objectives of this report were: (1) to explain the structure of the e-ALADO technological innovation; (2) gather evidence about its effects, following implementation. Our hypotheses established that the application of this online program: (1) would bring about a statistically significant increase in students' levels of conceptual, procedural, and attitudinal subcompetencies of interaction with alcohol, and improvement in their actual interaction with alcohol, compared to their prior levels (before program application); (2) This improvement would be moderated by students' pre-existing level of personal self-regulation, according to the exposed model and the existing theoretical evidence (de la Fuente et al., 2017a).

### MATERIALS AND METHODS

### Participants

The population under study were students from public secondary schools in a southern province of Andalusia (Spain). To gain variability in the sample, participants were not selected or grouped by any previous criteria (consumption, problems with alcohol). The sample was the natural one existing in the three centers. In order to include different types of schools in this investigation, schools were selected from three types of urban areas: (1) center of town, from a medium-high social stratum (n = 106), (2) surrounding neighborhoods, from a mediumlow social stratum (n = 101) and (3) outlying, marginalized population areas with compensatory education, from a low social stratum (n = 121). The homeroom teachers from every group participated voluntarily in the experiment, having been invited by the School Psychology adviser of the local Teacher Development Center. Initial participating students were between the ages of 12 and 17 years [12 (n = 51), 13 (n = 80), 14 (n = 97), 15 (n = 55), 16 (n = 35), and 17 (n = 10)] and were enrolled in compulsory secondary education (grades 7–10) at one of three public secondary schools. This age range was selected for methodological purposes, making it possible to form two groups, the 12- to 14-year-olds (n = 207), corresponding to the first stage of adolescence (puberty, or early adolescence), and the 15 to 17-year-olds (n = 121), corresponding to the second stage of adolescence (adolescence per se). The final sample size from which all measurements (pre–post) were taken contained 148 subjects. Of these, 80 were male (54% of the sample) and 64 were female (46% of the sample). The mean age of the sample was 13.82 ± 1.19 years old.

<sup>3</sup>https://www.researchgate.net/publication/319998580\_ALADO\_GUIA\_DE\_ INTERVENCION

### Instruments

### Conceptual Subcompetency

fpsyg-10-00127 February 23, 2019 Time: 18:29 # 5

The scale Evaluación de los Hechos, Conceptos y Principios sobre el Alcohol, EHCP (Assessment of facts, concepts, and principles about alcohol, AFCP) was used (Cubero and Sánchez, unpublished). The scale is composed of 38 items concerning the effects of alcohol use; psychometric analyses of this scale show reliability (α = 0.827) and consistent construct validity, with three factors: knowledge of facts, concepts, and principles concerning alcohol. Exploratory Factor Analysis (EFA) showed KMO = 0.801; Bartlett's Sphericity Test (df = 703) = 2767.595; p < 0.001. Confirmatory Factor Analysis (CFA) showed adequate indicators for the Default model: Chi-square = 1612.957, Degrees of freedom (779–117): 662, p < 0.001; all the variances are significant for p < 0.001; NFI = 0.865; RFI = 0.848; IFI = 0.914; TLI = 0.902; CFI = 0.913; RMSEA = 0.025; HOELTER model = 1095 (p < 0.05), 1138 (p < 0.01).

### Procedural Subcompetency

The SRQ, Self-Regulation Questionnaire (Brown et al., 1999; Carey et al., 2004; Neal and Carey, 2005) was used, in its 21-item abbreviated Spanish version, SRQ-21 (Pichardo et al., 2018). Its reliability (α = 0.826) and validity values are consistent, with two dimensions, planning and action control. EFA showed an index KMO = 0.985; Bartlett's Sphericity Test (df = 210) = 3603.882; p < 0.001. CFA showed adequate indicators for the Default model: Chi-square = 408.448, Degrees of freedom (252–64):188,

p < 0.001; all the variances are significant for p < 0.001; NFI = 0.894; RFI = 0.870; IFI = 0.940; TLI = 0.925; CFI = 0.9393; RMSEA = 0.022. However, this structure does not concur with others found in other samples (Pichardo et al., 2014; Artuch-Garde et al., 2017).

#### Attitudinal Subcompetency

The scale for Evaluación de las Actitudes ante el alcohol, EAA [Assessment of Attitudes toward Alcohol, AAA] was used


<sup>∗</sup>Place of alcohol intake in this theoretical model.

(1 = low; 2 = medium; 3 = high).

FIGURE 3 | Effects of Moment × Level of regulation on attitudes (1 = low; 2 = medium; 3 = high).

(Cubero and Sánchez, 2018). A total of eight items assess attitudes and values toward alcohol (α = 0.825). EFA showed KMO = 0.869; Bartlett's Sphericity Test (df = 28) = 529.335, p < 0.001. CFA showed adequate indicators for the Default model: Chisquare = 59.274, Degrees of freedom (44–24): 20, p < 0.001; all the variances are significant for p < 0.001; NFI = 0.913; RFI = 0.926; IFI = 0.934; TLI = 0.918; CFI = 0.924; RMSEA = 0.08; HOELTER model = 1215.

In the case of the interaction of Moment × Level of selfregulation, partial effects showed a significant effect on the dependent variable knowledge [F(2,184) = 4.661, p < 0.01 (Pillai's trace), η = 0.050, power = 0.775], and on interaction with alcohol [F(2,184) = 6.543, p < 0.01 (Pillai's trace), η = 0.068, power = 0.908]. Direct values are presented in **Table 1** and the effects in **Figures 1**–**4**.

#### TABLE 2 | Sequence of the e-ALADO intervention.

• INITIAL ASSESSMENT (OCTOBER).

#### October


#### November


#### December


#### January


#### February


#### March


#### April


#### May


#### • FINAL ASSESSMENT

C, conceptual subcompetence; P, procedural subcompetence; A, attitudinal subcompetence.

#### Adjusted Behavior in Interacting With Alcohol

We used the Escala de Ajuste en la interacción con el alcohol [Scale of Adjustment in interacting with alcohol], which contains four items (α = 0.915). This scale belongs to the Inventario de Evaluación de conocimientos, actitudes e interacción con el alcohol (Cubero and Sánchez, unpublished) [Inventory for Assessment of knowledge, attitudes and interaction with alcohol].

All instruments were validated with Spanish teenagers. The data and validation procedure of these instruments have already been presented in a previous research report (de la Fuente et al., 2017a).

### Procedure

Data collection instruments were applied over the course of the school year 2009–2010, within the framework of the Alado Project of Excellence (2007–2010), through an online utility created for this purpose (see footnote 1). Although these data are old, we consider that, given the characteristics of them, they still have a great importance for the variables analyzed (see **Table 2**).

Cooperation had been previously requested from the Teacher Development Center, from the students' parents and from the School Board, for student participation. The project was approved by the University Bioethics Commission (University of Almería; Ref. 2009; n◦ 028) and by the School Boards of the participating schools. The students participated voluntarily. The parents were informed in writing. As the participants in the Project were minors, both the parents and school administrators gave written informed consent for the study. The consent obtained was both informed and written. The data was protected in an archived and registered file, as indicated by the Spanish Data Protection Law.

### Data Analyses

An quasi-experimental single-group study, with repeated measures pre and post, was used. The independent variables considered were (1) Moment (Treatment) × (2) Level of Selfregulation. For levels of the independent variable self-regulation, cluster analysis was used, obtaining three levels: low, medium, and high. Inferential statistical analyzes (multivariate analysis, ANOVA) were carried out using SPSS (v.23.0 for Windows) to verify the effect of the independent variables on the dependent ones: the level of competence.

### RESULTS

### E-ALADO Program Effects

There was a significant main effect of the Moment factor [F(4,175) = 270.866, p < 0.001 (Pillai's trace), η = 0.861, power = 1.0; moment 2 > 1, p < 0.001], of Level of regulation [F(8,352) = 3.497, p < 0.001 (Pillai's trace), η = 0.074, power = 0.981; moment 2 > 1, p < 0.001], and of the Interaction of these two [F(8,352) = 3.497, p < 0.001 (Pillai's trace), η = 0.074, power = 0.981] on the set of dependent variables being analyzed (see **Table 3**).

Partial analysis showed the effects more precisely. The Moment factor showed a significant partial effect on each of the variables analyzed, with improvement registered in levels of variables; in knowledge [F(1,184) = 773.523, p < 0.001 (Pillai's trace), η = 0.813, power = 1.00; moment 2 > 1, p < 0.001]; in self-regulation [F(1,184) = 27.602, p < 0.001 (Pillai's trace), η = 0.134, power = 0.993; moment 2 > 1, p < 0.001]; in attitude [F(1,1784) = 5.515, p < 0.025 (Pillai's trace), η = 0.028, power = 0.993; moment 2 > 1, p < 0.001]; and in interaction with alcohol behavior [F(1,184) = 5.515, p < 0.025 (Pillai's trace), η = 0.028, power = 0.993].

The level of self-regulation factor showed a differential, significant partial effect on each of the variables analyzed. Regarding knowledge, no significant effect appeared [F(2,184) = 0.349, p < 0.706, ns (Pillai's trace), η = 0.004,

TABLE 3 | Effect of the e-ALADO utility in interaction with students' level of self-regulation.


<sup>∗</sup>p < 0.05, ∗∗p < 0.001.

power = 0.109]; on self-regulation [F(2,184) = 9.444, p < 0.001 (Pillai's trace), η = 0.096, power = 0.976; post hoc: H > L, p < 0.001; H > M, p < 0.05]; on attitude [F(2,184) = 3.974, p < 0.02 (Pillai's trace), η = 0.043, power = 0.765; post hoc: H > L, p < 0.001; H > M, p < 0.05]; on interaction with alcohol behavior [F(2,184) = 2.749, p < 0.05 (Pillai's trace), η = 0.030, power = 0.537; post hoc: n.s.].

### Summary

The experimentally manipulated variable, e-ALADO Program, has shown a significant effect in the increase of conceptual, procedural, and attitudinal learning, as well as abstinence (no alcohol consumption). The variable manipulated by selection, the Self-Regulation Level of the students, has produced a differential effect on these variables, especially in attitudinal change. There is evidence that, although high students in SR benefit more from the program, this educational experience benefits everyone.

### DISCUSSION

### Foundation and Structure

Following the conceptual scheme of the I + D + I value chain (de la Fuente et al., 2018a), the e-ALADO program can be considered as a Psychoeducational Technological Development with two benefits. The first, which is a conceptual consequence of assuming:


The second, which is a program in on-line format, which involves more interactivity and empowerment of meaningful learning in students, when addressing a problem of great importance in adolescence: the early prevention of alcohol consumption.

From the point of view of psychological practice, it is important to understand that educational evaluation and intervention programs – whenever possible – must evolve toward on-line technological supports and formats. Technological innovation is a necessity in the field of psychological evaluation and intervention (de la Fuente et al., 2018c; Garaigordobil and Martínez-Valderrey, 2018).

### Effects of Implementation

The hypotheses established that the application of this online program: (1) would bring about a statistically significant increase in students' levels of conceptual, procedural, and attitudinal subcompetencies, and improvement in their level of interaction with alcohol, compared to their prior levels (before program application); (2) This improvement would be moderated by students' pre-existing level of personal self-regulation, according to the exposed model and the existing theoretical evidence (de la Fuente et al., 2017a).

The results consistently confirmed -although with restrictions- certain of our hypothesis for the prevention of alcohol consumption in adolescence (Spear, 2015; Stigsdotter-Ekberg et al., 2016). Regarding Hypothesis 1, that the intervention would bring about a statistically significant increase in students' levels of conceptual, procedural, and attitudinal subcompetencies, and improvement in their actual interaction with alcohol, compared to their prior levels, the main effect results show an effect of Moment (that is, the effect of having used the utility). These results are partially maintained for the conceptual, procedural, and attitudinal subcompetencies, as well as for interaction with alcohol. The results contribute empirical evidence that the e-ALADO technological utility helps improve adolescents' competency for interaction with alcohol, as well as their actual interaction. This result is consistent with data that was presented for the model of alcohol intake competence, referred to the levels of conceptual, procedural, and attitudinal subcompetence are necessary and predict the level of interaction with alcohol (de la Fuente et al., 2017a).

Regarding Hypothesis 2, the results also produced evidence that students' low–medium–high level of self-regulation has its own weight in the general effect, just as is established in SRL vs. ERL Theory (de la Fuente, 2017). The level of self-regulation determined, by itself, significant differences in the level of procedural and attitudinal subcompetences, but in interaction with the treatment had a significant effect on conceptual subcompetences and the degree of adequacy of interaction with alcohol (or zero consumption). In other words, students' level of self-regulation (high vs. medium vs. low) would interact with the effect of the utility on the students. Just as the theory predicts, students who are high in self-regulation (selfregulating) would have greater likelihood of benefitting from the utility, while students low in self-regulation (dysregulatory) would be the least likely to benefit. This fact is consistent with prior research showing the importance of self-regulation as a meta-behavior that is involved in alcohol intake behavior (de la Fuente et al., 2017a) and the health (Garzón-Umerenkova et al., 2018). After the intervention, the participating students showed a significant increase in the conceptual competency (adjustment

in the level of facts, concepts and principles related to alcohol intake), but the largest increase occurred in students with a higher level of self-regulation. That also happened with attitudinal subcompetence. Likewise, a significant increase was produced in the procedural competency (personal self-regulation). This conceptual improvement is interactive with students' level of personal self-regulation. In other words, students who already have a higher level of personal self-regulation get more benefit from this learning experience. There is a significant decrease in the level of contact with alcohol (absence of intake), in comparison to the initial level.

Despite the findings of this study, we must analyze its limitations, so that these may be addressed in future research. The first limitation refers to the design: for reasons outside the researchers' control, it was not possible to include several equivalent control groups. This fact recommends caution in generalizing the results and is an inherent threat to the study's validity. For this reason, future research studies should revalidate the impact of this technological utility.

### CONCLUSION

Based on the foregoing, it seems reasonable to foster this type of technological utility in preventive, educational intervention where programs are established to develop personal selfregulation (Castillo and Dias, 2009; Castillo et al., 2012). Training should address the two sectors of the educational community that are directly involved, that is, both teachers and students, in compulsory and post-compulsory secondary education (Sánchez et al., 2007), as well as at university (Mauri et al., 2009). The e-ALADO utility is ready to be transferred to groups who work with adolescents in the prevention of alcohol intake, because it is well founded and has been validated with encouraging results. Given that the utility was produced at a public university, within a publically funded Research Project, the means of transfer is through R&D contracts with public or private organizations and institutions. Another desirable future line of development would be to adapt the utility to other technological formats that are more familiar to adolescents, such as Android or tablets.

However, we can not assume that the e-ALADO program has a linear learning effect, the same for all adolescents. Although the trend found is an effect of general improvement, it is very important to know and pay attention to adolescents with a low level of self-regulation. According to the results, we know that they are students that need more help and external regulation. Also, probably, those who benefit least from this program.

### Practical Implications

Consequently, from the above information, several practical implications are deduced to apply this program with adolescents and Spanish centers:

(1) It's important to have a minimum previous teacher training in the Competency model and in the Program Use Guide, which has not been done in our experimentation. This would improve the involvement of teachers in the program. It is not enough to provide a technological tool to apply it in a decontextualized way.


### AUTHOR CONTRIBUTIONS

JdlF design, analysis, and writing. IC ALADO Project director and text revision. FP center implementation and material design. MS data analysis and text review. JS implementation in centers. SF review of the preventive foundation of intervention.

### FUNDING

This study was funded by Ministerio de Economía y Competitividad, Spain (Grant No. PSI2015-64460-R).

### ACKNOWLEDGMENTS

ALADO Project (2008–2010). Neurobiological risk factors and alcohol use in adolescents: educating for prevention. A pilot experience in results transfer to an at-risk population. Reference: P06-cts-01350. Junta de Andalucía, Andalusian (Spain).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2019.00127/full#supplementary-material

### REFERENCES

fpsyg-10-00127 February 23, 2019 Time: 18:29 # 10


as a determinant of intake: the role of self-regulation. Front. Psychol. 8:1800. doi: 10.3389/fpsyg.2017.01800


d'Innovació i Recerca en Educació 2, 33–60. doi: 10.7453/gahmj.2012. 1.1.010


sistemasInformacion/sistemaInformacion/encuestas\_ESTUDES.htm [accessed December 20, 2018].


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 de la Fuente, Cubero, Peralta, Sánchez, Salmerón and Fadda. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Does the Use of Learning Management Systems With Hypermedia Mean Improved Student Learning Outcomes?

María Consuelo Sáiz-Manzanares1,2 \*, Raúl Marticorena-Sánchez<sup>3</sup> , José Francisco Díez-Pastor<sup>3</sup> and César Ignacio García-Osorio<sup>3</sup>

<sup>1</sup> Faculty of Health Sciences, Hospital Militar, University of Burgos, Burgos, Spain, <sup>2</sup> Department of Health Science, University of Burgos, Burgos, Spain, <sup>3</sup> Department of Civil Engineering, University of Burgos, Burgos, Spain

Learning management systems (LMSs) that incorporate hypermedia Smart Tutoring Systems and personalized student feedback can increase self-regulated learning (SRL), motivation, and effective learning. These systems are studied with the following aims: (1) to verify whether the use of LMS with hypermedia Smart Tutoring Systems improves student learning outcomes; (2) to verify whether the learning outcomes will be grouped into performance clusters (Satisfactory, Good, and Excellent); and (3) to verify whether those clusters will group together the different learning outcomes assessed in four different evaluation procedures. Use of the LMS with hypermedia Smart Tutoring Systems was studied among students of Health Sciences, all of whom had similar test results in the use of metacognitive skills. It explained 38% of the variance in student learning outcomes in the evaluation procedures. Likewise, three clusters that grouped the learning outcomes in relation to the variable 'Use of an LMS with hypermedia Smart Tutoring Systems vs. No use' explained 60.4% of the variance. Each cluster grouped the learning outcomes in the different evaluation procedures. In conclusion, LMS with hypermedia Smart Tutoring Systems in Moodle increased the effectiveness of student learning outcomes, above all in the individual quiz-type tests. It also facilitated personalized learning and respect for the individual pace of student-learning. Hence, modules for the analysis of supervised, unsupervised and multivariate learning should be incorporated into the Moodle platform to provide teaching tools that will undoubtedly contribute to improvements in student learning outcomes.

### HIGHLIGHTS


Keywords: hypermedia resources, Moodle, Smart Tutoring System, learning outcomes, Educational Data Mining

#### Edited by:

Jesus de la Fuente, University of Navarra, Spain

#### Reviewed by:

Michelle Hood, Griffith University, Australia Domenico Criscuolo, Genovax S.r.l., Italy

#### \*Correspondence:

María Consuelo Sáiz-Manzanares mcsmanzanares@ubu.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 21 January 2018 Accepted: 11 January 2019 Published: 12 February 2019

#### Citation:

Sáiz-Manzanares MC, Marticorena-Sánchez R, Díez-Pastor JF and García-Osorio CI (2019) Does the Use of Learning Management Systems With Hypermedia Mean Improved Student Learning Outcomes? Front. Psychol. 10:88. doi: 10.3389/fpsyg.2019.00088

## INTRODUCTION

fpsyg-10-00088 February 12, 2019 Time: 16:34 # 2

Over the past decade, a change in the teaching-learning context has been identified. Teacher and student interaction takes places with increasing frequency through learning management systems (LMSs), such as, for example, Moodle (Modular Object Oriented Developmental Learning Environment). Recent studies (Yamada and Hirakawa, 2015; Järvelä et al., 2016) have pointed out that collaborative learning in virtual environments improves learning outcomes. Nevertheless, the mere use of these interactive spaces is not sufficient to ensure that effective learning takes place (Sáiz et al., 2017a). If effective learning is to be guaranteed, the teacher must consider the following points (Mayer, 1998; Clark and Mayer, 2008; de Raadt et al., 2009; Bernard and Bachu, 2015):


In summary, LMS, if well-designed, will increase self-regulated learning (SRL), planning, and the use of metacognitive skills. All of those skills will facilitate increased motivation toward learning (Mayer, 1998). Likewise, those learning environments will provide the opportunity for students to develop a framework of key processes, which will foreseeably strengthen effective learning (Sáiz et al., 2017a).

### Feedback and Hypermedia Resources

An LMS facilitates flexible use of hypermedia resources, which helps the teacher to provide both formative and summary feedback, virtually in real time (Hattie and Timperley, 2007). Research in this field (Cerezo et al., 2016) has demonstrated that learning that uses the new Information and Communications Technology (ICT) helps build knowledge. However, for this process to take place, both the declarative and the procedural knowledge of the students must be strengthened through the use of SRL in increasingly challenging tasks (Azevedo, 2005). The stepped structure of the material to learn will assist the preparation of problem-solving strategies in the learner, as the learning goals are sequentially ordered (Winne, 2014; Höök and Eckerdal, 2015); all of which will increase motivation (Zimmerman and Moylan, 2009; Segedy and Biswas, 2015). Nevertheless, so that all of these benefits may be reaped, the use of hypermedia resources has to be included in a dynamic structure that adapts itself to the learning needs of each student to achieve effective learning. In other words, the teacher has to design the architecture of LMS teaching processes. Likewise, an analysis must be done of the different (student–student; student–professor; student–machine) interactions that take place on the platform, so as to redesign, if necessary, processes and procedures. Data-mining techniques are employed in the analysis of those interactions (Educational Data Mining -EDM-) (Romero et al., 2002, 2013). Their automatization was done through modules embedded in the LMS or from the web-service records (logs). Once the logs are transferred, the information has to be filtered, selecting only the relevant information that refers to the object of study, as the registers contain a lot of information, not all of which is applicable in each case. The data to be processed is typically represented in JSON (Java Script Object Notation) format (ECMA International, 2017). Once organized, the data may be analyzed by employing EDM techniques from statistical programs such as, for example, SPSS, R, Matlab or through programs that integrate WEKA libraries (in Java) or Pandas (in Python), because the platforms will not usually include complex modules for analysis (Luna et al., 2017).

In summary, the use of hypermedia resources facilitates the development of in-depth and better-quality learning (de Kock, 2016; Norman and Furnes, 2016). Likewise, it increases the use of metacognitive skills (Sáiz and Arnaiz, 2017), as it strengthens planning, supervision, control, and reflection on the object of learning. Active participation by the learner in the learning process is therefore increased. The whole process provides its own feedback in the form of a loop (Zimmerman and Moylan, 2009).

### Intelligent Tutoring Systems and Project Based Learning

Over recent years, work has proceeded along these lines for the design of Artificial Intelligence Systems (Cuba-Ricardo et al., 2015; Li et al., 2015). Those systems include object level and meta-level processes in the machine following the model of Nelson and Narens (1990). The use of the Project-Based Learning methodology has shown itself to be an effective means of developing those processes in Blended Learning (B-Learning) environments. An intrinsic part of this methodology is the planning and the construction of the learning process through carefully designed research questions (Markham et al., 2003). Recent research (Bannert et al., 2015; Dias et al., 2017) has found that students who use the Project Based Learning methodology in LMS, employ more metacognitive skills (Azevedo et al., 2011; Sáiz and Montero, 2016). A summary of the interaction process is shown in **Table 1**.

Over recent years, many investigations have focused on analyzing the effect of e-learning on learning. One relevant aspect is the inclusion of Smart Tutoring System modules. These modules strengthen the personalization of learning and the individualized follow-up of the student, which predicts the learning outcomes by 61.3% (Matwin and Mielniczuk, 2016; Sáiz et al., 2017b).

### Intelligent Tutoring Systems: Virtual Pedagogical Agents

At present, the use of Intelligent Tutoring Systems is increasingly frequent in what are known as Virtual Pedagogical Agents that provide feedback to students in real time on both learning processes and products. These systems facilitate the division of learning into sub-goals and, in consequence, the process of

Sáiz-Manzanares et al. Smart Tutoring and Data Mining

TABLE 1 | Cognitive and metacognitive orientation skills in the process of following a Project Based Learning.


regulating learning. This working methodology can be designed at different levels for pedagogic support to different types of interaction with what is called MetaTutor. A type of tutoring that has the objective of facilitating learning in real time and that is designed to augment the effectiveness of the human tutor (Externally Regulated Learning). There are also different forms of applying this type of intelligent tutoring, which can be done on an individual or a collaborative basis (Collaborative co-Regulated Learning) (Harley et al., 2017).

Along these lines, recent investigations have made it quite clear that Intelligent Tutoring Systems facilitate the development of SRL, because they broaden the skills of reflecting on cognitive and metacognitive processes. Students will not always learn these self-regulated behaviors in a natural way. The use of these systems with students will therefore facilitate:


The above-mentioned Intelligent Tutoring Systems include the use of hypermedia systems, that provide feedback on the TABLE 2 | Distribution of the groups and mean and standard deviation for the variables age and gender.


Mage <sup>=</sup> mean age, SDage = standard deviation age.

learning process and support to the student or group of students in real time. In addition, those systems can provide different types of regulation and can predict learning results (Lau et al., 2017).

Likewise, recent students have highlighted that the use of Intelligent Tutoring Systems in learning processes permit personalized instruction and respect the learning rhythms of students (Lajoie and Poitras, 2017). These authors indicate that a lot of data on the learning process is provided through these environments and that their analysis and study through data-mining techniques will provide information for the design of increasingly effective environments and better practices for instruction.

In summary, it may be pointed out that this field of work is in its infancy and, in principle, has broad potential. Nevertheless, variables such as attentional level and self-control, and the use of cognitive and metacognitive skills for learning and motivational and affective factors should be studied in the learners. LMSs, provide a lot of information that is recorded in interaction logs and Educational Data Mining techniques will allow us to analyze such information virtually in real time.

### Application of Data-Mining Techniques for the Analysis of the Results of Interaction on the Learning Management System

Another significant aspect in the learning process in LMS is the type of evaluation procedure that is used, because the evaluation procedure appears to be directly related with the learning behavior of the student on the platform (Sáiz and Montero, 2015; Cerezo et al., 2016). As mentioned, EDM techniques are used for the analysis of those behaviors (Romero et al., 2008). EDM techniques can be either supervised learning (classification and regression) or unsupervised learning



**100%** 


FIGURE 2 | Chronogram of six-monthly activities and process planning.



FIGURE 3 | Design of the Moodle platform in the experimental group. Activities were held with network videos, materials, articles and web-based materials of interest. The Flipped Classroom experience included videos prepared ad hoc that incorporated quizzes with feedback on the student response.

#### TABLE 3 | Skills in each of the ACRAr scales and of the different coefficients of validity.


(clustering) (Zacharis, 2015). The results of this analysis may be used to study both the pace and the trajectory of learning of each student. The structure of a personalized Smart Tutoring Systems in Moodle may be consulted in **Figure 1**. Studies have shown that the use of Smart Tutoring Systems modules are effective at increasing the motivation of students toward learning the subject matter and can thereby achieve effective learning (Kaklauskas et al., 2015).

As previously mentioned, the Research questions of this study are:

RQ1: Is the use of LMS with hypermedia Smart Tutoring Systems in Moodle a predictor of student learning outcomes?

RQ2: Will the learning outcomes be structured into different groups by performance when the LMS with hypermedia Smart Tutoring Systems in Moodle is and when it is not being used?

RQ3: Will the cluster groupings differentiate between the results of the different Learning Outcomes?

## METHODS

### Participants

A sample of 83 students from the third year of the Degree in Occupational Therapy was used, with 41 subjects in the Control Group (it is the group in which it does not apply Smart Tutoring System in Moodle) and 42 in the Experimental Group (it is the group in which it is applied Smart Tutoring System in Moodle). The descriptive statistics of each group in terms of gender may be seen in **Table 2**. The assignation of students to either the experimental or to the control group was done by convenience sampling.

FIGURE 5 | Individualized report following Student A.

### Instruments

### UBUVirtual Platform, Version 3.1

This platform incorporates a Moodle-based LMS that begins with a constructivist approach, developed through a modular

the day of delivery. Likewise, the teacher provided feedback on the process.

system, that permits progressive configurations. The versatility of modules and their activities facilitate flexible interaction between the users (students and teachers), which is the basis of interactive learning (Saeed et al., 2009). The subject module on which the

TABLE 4 | Distribution of the Evaluation Procedure and percentage of Total Mark.


TABLE 5 | Mann–Whitney U-test and Wilcoxon Signed-Rank test between the control group and the experimental group.


teaching was developed is based on a Project Based Learning design and systematic structuring of the learning contents, the tasks, and the evaluation procedures. At the beginning of the term, a timetable was made available to the students setting out the learning contents, the weekly activities, the evaluation tests, and the weighting of each test in the final mark (see **Figure 2**). Likewise, the architecture of the subject may be seen in the LMS. This structure was equally applied in the Control Group and in the Experimental Group, except for the, which were only developed in the Experimental Group. The differences were the use of hypermedia resources: (1) Quizzes with feedback to responses and Flipped Classroom experiences. Comprehension questions were included in these activities, after which feedback was given to the student on the response that had been given (**Figure 3**).

#### The Learning Skills Scales (ACRAr) of Román and Poggioli (2013)

This instrument has been widely tested in different Spanishspeaking populations (Carbonero et al., 2013). It identifies 32 skills at different points in the processing of information. The skills in each of the scales that constitute the ACRAr are listed in **Table 3**. Only the Metacognition scale was applied in this scale for which an α = 0.75 was obtained in the sample.

### Program of Intervention in the Experimental Group Through an Smart Tutoring Systems in Moodle Architecture

An architecture was designed within the Moodle platform with an individual and group tutoring system. The individual tutoring consisted in providing individual feedback for each of the responses that the students gave to each of the five self-evaluation quizzes on conceptual knowledge (see **Figure 4**). In addition, a report was drafted on the performance of each student showing comparisonswith the group average of the class (see**Figure 5**) after the completion of each quiz. Group tutoring was done through the analysis of the group productions in the presentation of the Project Based Learning; an example may be seen in **Figure 6**.

#### Flipped Learning Experience

Two supporting videos were prepared for the two final units (4 and 5) of the subject module (Sáiz and Arnaiz, 2017). Those videos were used in a Flipped Classroom experience and contained short quizzes to check understanding. Students had to answer the questions to be able to watch the video until the end, and each answer received feedback. These materials are available at the Institutional Repository of the University of Burgos under a Creative Commons license. Only the Experimental Group was affected by this experience.

Unit 4 http://hdl.handle.net/10259/4525 (in Spanish).

Unit 5 http://hdl.handle.net/10259/4526 (in Spanish).

#### Student Learning Outcomes

In Both the Experimental Group and the Control Group Groups were recorded in the following evaluation procedures (see **Table 4**).

(1) Learning Outcomes: Self-Evaluation Quizzes with a weight of 30% of the final mark; (2) Learning Outcomes: Practice (practice relating to the theoretical contents of each of the five thematic units) with a weight of 20% of the final mark; (3) Learning Outcomes in Development Project Based Learning with a weight of 25% of the final mark; Learning Outcomes Presentation of Project-Based Learning with a weight of 25% of the final mark; and, finally, Total Learning Outcomes, the final mark of which is the sum of the previously described weightings.

### Procedure

The research project was approved by the Ethics Committee of the University of Burgos. Previously, at the start of the project, the students were informed of the objectives and their participation was at all times on a voluntary basis. Likewise, the informed

TABLE 6 | Indicators of asymmetry and kurtosis in the Control Group and in the Experimental Group.


M, mean; SD, standard deviation; A, asymmetry; SEA, standard error of asymmetry; K, kurtosis; SEK, standard error of kurtosis.

consent of each participant was recorded in writing. The subject module, for the Control Group and for the Experimental Group, was structured into the following sections on the Moodle Platform: obligatory working material (theory), complementary material, practical activities (five), solution of the Project Based Learning and self-evaluation activities (quizzes) (see **Figure 4**). Both the practices and the project were done in groups (with either 3 or 5 students).

The difference between GC and GE is found in the use of LMS (see section "Program of Intervention in the Experimental Group Through an Smart Tutoring Systems in Moodle Architecture" and "Flipped Learning Experience"). In both groups, the subject module had a duration of 14 weeks and the type of learning was B-Learning [partly face-to-face and partly through the Moodle Platform]. However, in the Experimental Group, the teaching was structured on the basis of programmed and continuous use of the platform in a Replacement Blend (RB) mode, the interaction fundamentally taking place through deliveries and virtual feedback. A Supplemental Blend (SB) methodology was used in the Control Group, which implied face-to-face feedback. Before starting the teaching program, both groups of students were administered the ACRAr Learning Skills Scale (Román and Poggioli, 2013). The teaching was imparted by the same teacher during the different terms. Convenience sampling was used to assign students to either the Experimental or the Control Group.

### Design, Variables, and Statistical Analysis

These three elements of the study were defined as follows:

1. Designs: a quasi-experimental design was used with a control group equally skilled in the variable metacognitive skills, in order to respond to RQ1. And a descriptivecorrelational design was used, to respond to RQ2 and RQ3. 2. Variables: the independent variable was the use of an individual Smart Tutoring Systems in Moodle module (Replacement Blend-RB) and a Flipped Classroom experience v. no experience and the dependent variables were the learning outcomes in different evaluation procedures (see section "Student Learning Outcomes").

Statistical analyses: (1) analysis of the equivalence between the Control Group and the Experimental Group for the variable metacognitive skills before the intervention, for which the Mann– Whitney U-test and the Wilcoxon signed-rank tests were used; (2) analysis of asymmetry and kurtosis; (3) fixed effect ANOVA (use of an Smart Tutoring System in Moodle vs. no use), effect value (eta squared) and the Bonferroni test; (4) Cluster analysis for which the k-means clustering technique; (5) Crosstable and (6) Wilk's Lambda<sup>1</sup> and Canonical Discriminant Function.

### RESULTS

### Previous Statistical Analysis

Both groups were tested to find out whether they had a similar distribution according to the results of the ACRAr Scale of Metacognitive Skills (Román and Poggioli, 2013) before the study was carried out. To do so, the Mann–Whitney and the Wilcoxon Signed-Rank tests were applied, in which no significant differences were found between both groups in any of the skills (see **Table 5**), for which reason both groups were considered equivalent. If any differences had been found, this variable would have been considered as a covariable.

The indicators of asymmetry and kurtosis were determined, in order to test the characteristics of the distribution of

<sup>1</sup>Wilk's Lambda tests how well each level of independent variable contributes to the model. The scale ranges from 0 to 1, where 0 means total discrimination, and 1 means no discrimination.

the sample. In asymmetry, the highest values |2.00| indicate extreme asymmetry and the lowest values indicate a normal distribution (Bandalos and Finney, 2001). With regard to the values of kurtosis, values between |8.00|and|20.00| suggest extreme kurtosis (Arias, 2008; Arias et al., 2013). As may be seen in **Table 6**, the asymmetry and kurtosis values in both groups were within acceptable limits, for which reason a parametric statistic was used.

The number of records (logs) were also registered: 13.410 in the Control Group and 26.056 in the Experimental Group. These data indicate an increase of 12,646 logs to the platform by the Experimental Group. In other words, the interaction of the Experimental Group students with the platform was almost twice that of the Control Group. The interactions of the teacher numbered 437 with the Control Group and 516 with the Experimental Group, showing an increase in teacher activity of

TABLE 7 | Single-factor fixed effects ANOVA (use of a Smart Tutoring System in Moodle vs. no use).


Group = use of Smart Tutoring System in Moodle; p = significance; η <sup>2</sup> = eta squared; M = mean; SD = standard deviation. <sup>∗</sup>p < 0.01.

TABLE 8 | Distribution of students in the two clusters in relation to the control and experimental group.


TABLE 9 | Final cluster centers of k-means when k = 2 is used.


18%. Student activity in both the Experimental Group and the Control Group is presented below, in **Figure 7**.

### Confirmation of the Research Questions

A fixed-effects ANOVA (use of LMS with hypermedia Smart Tutoring System vs. no use) was applied to confirm RQ1 ("Is the use of LMS with hypermedia Smart Tutoring Systems in Moodle a predictor of student learning outcomes?"). The results showed that the use of an LMS with hypermedia Smart Tutoring System influenced the learning results of the students in all the evaluation tests, except in the practices, in which both groups of students obtained similar results (see **Table 7**). The highest effect values in Learning Outcomes: Total and in Learning Outcomes: Self-Evaluation Quizzes explained 38 and 21% of the variance, respectively.

The unsupervised learning technique was used, in order to test RQ2 ("Will the learning outcomes be structured into different groups by performance when the LMS with hypermedia Smart

TABLE 10 | Distribution of students in the three clusters in relation to the control and experimental group.


TABLE 11 | Final cluster centers of k-means when k = 3 is used.


TABLE 12 | Discriminant analysis between groups.


<sup>∗</sup>p < 0.05.

Tutoring Systems in Moodle is and when it is not being used?"), by grouping the sample of students around different variables, in this case in relation to Learning outcomes. The technique is at present widely used in Educational Data Mining and has shown its effectiveness at ascertaining the characteristics of the groups that yield the best results. It is of assistance to teachers in the improvement of the teaching design (Klösgen and Zytkow, 2002; Muldner et al., 2011; Bogarín et al., 2017, 2018). In particular, the k-means clustering was used to test RQ2.

Initially, we use the k-means algorithm using as inputs the variables related to learning outcomes. At first using a value of k = 2, it was expected that the composition of the two clusters would correspond to that of the two groups: control and experimental. Although practically all the students of the experimental group were grouped in the same cluster (cluster C1), 4 were left out, and in that there were also 20 other students from the control group (see **Tables 8**, **9**).

In a second step, a value of k = 3 was used, and this time, the clusters were more compact and interpretable (see **Tables 10**, **11**). Cluster C3, which we could associate with the group of excellent students, contains the bulk of the students of the experimental group and some students of the control group (sometimes the personal aptitudes of a student make their learning results good, regardless of the teaching technique used). Cluster C2, which we could associate with good students, contains the rest of the students of the experimental group and the bulk of students in the control group. Finally, in cluster C1, which could be matched with less bright students, there are only seven students and they are all from the control group.

Also, a discriminant analysis was conducted to study RQ3: "Will the cluster groupings differentiate between the results of the different Learning Outcomes?" The results pointed to a different behavior of the three clusters in the different evaluation procedures. Nevertheless, Wilks' Lambda was only of statistical significance in Learning Outcomes: Total (see **Table 12**). The behavior of the three clusters is shown below in **Figure 8**.

### DISCUSSION AND CONCLUSION

The fact that the clusters obtained by k-means, when using as input variables the learning outcomes, have this strong correspondence with the control and experimental groups is an additional indication that the use of Smart Tutoring System seems to increase learning outcomes in the students. A possible explanation is that the system helps to apply the metacognitive skills of orientation, planning, evaluation, and reflection to problem-solving tasks, which helps to define the problem-solving process through graduated steps of progressive difficulty (Azevedo, 2005; Winne, 2014; Höök and Eckerdal, 2015; Cerezo et al., 2016; Harley et al., 2017). Moreover, this process facilitates SRL (Cerezo et al., 2016; Sáiz and Montero, 2016; Lau et al., 2017) and the personalized feedback of the teacher in real time, which increases the motivation of the student toward the learning material (Hattie and Timperley, 2007;

Zimmerman and Moylan, 2009; Segedy and Biswas, 2015). The use of LMS that incorporate hypermedia Smart Tutoring Systems includes all these characteristics in the platform for the strengthening of object-level and meta-level structures (Cuba-Ricardo et al., 2015; Li et al., 2015). The effectiveness of this system architecture is complemented through the use of the Project-Based Learning methodology on the Moodle Platform (Bannert et al., 2015). It all means that the problem may be solved through progressive approximations to the goal (Azevedo et al., 2011) and it favors the use of metacognitive skills of planning and evaluation applied to both process and product in the learning activity (Sáiz and Montero, 2015). Likewise, if this form of personalized education in B-Learning environments is supported by the use of hypermedia resources, such as for example Flipped Classroom experiences (Sáiz and Arnaiz, 2017; Wang, 2017) and quizzes with interactive feedback on the responses in real time (Sáiz et al., 2017b), its effectiveness is all the greater. Therefore, the personalization of learning together with the use of the previously described methodological and technological resources is in step with the learning rhythm of the student (Matwin and Mielniczuk, 2016).

In summary, if the B-Learning environments use the LMS that incorporate hypermedia Smart Tutoring Systems, they appear to be more effective (Sáiz et al., 2017b). In addition, the student learning outcomes in different evaluation procedures appear to be related with the use of those modules, the ones that explain 57.8% of the variance in the learning outcomes, especially those related with the completion of self-evaluation quizzes. One explanation may be that those systems allow for individualized student follow up and that the individualized feedback strengthens the development of insight throughout the learning process. Another important preventive measure to identify at-risk students is to find the groupings in clusters, as they explain 60.4% of the variance in the learning outcomes. A map can be sketched from an analysis of those clusters for the prediction of performance in the various evaluation procedures. All of the above will foreseeably allow the correction of possible learning problems and thereby reinforce higher indicators of academic performance (Díez et al., 2017).

One possible explanation is that the LMS with hypermedia Smart Tutoring Systems permit the development of greater personalized learning that is more in keeping with the pace of learning of each student. In addition, the records that the interactions between those learners leave on the system permit a lot of information to be gathered that can be analyzed by using data-mining techniques. The teacher is therefore able to access information in real time that helps with the systematic regulation throughout the teaching-learning process practically

### REFERENCES

Arias, B. (2008). Desarrollo de un Ejemplo de Análisis Factorial Confirmatorio con LISREL, AMOS y SAS. Seminario de Actualización sobre Discapacidad SAID 2008. Salamanca: Instituto Universitario de Integración en la Comunicad.

Arias, B., Verdugo, M. A., Navas, P., and Gómez, L. E. (2013). Factor structure of the construct of adaptative behavior in children with and without intellectual in real time. Hence, strengthening the incorporation of analytical tools in Moodle that can generate automatic (supervised and unsupervised) learning techniques and multivariate analysis techniques in an easy way is an important issue for those in charge of universities. It will provide an analysis for the teacher in real time of the learning characteristics of their students throughout both the teaching and the learning process. It will also permit the teacher to ascertain the grouping around variables of both performance and learning behaviors that are recorded on the platform from the start of the course (Bogarín et al., 2017, 2018). All this information will facilitate the adjustment of the teaching to the needs of each student and of each one of the groups found with these clustering techniques that can detect similar groups in relation to a series of variables that the teacher may a priori define as significant, all of which will foreseeably increase the results of effective learning (Sáiz, 2017, unpublished).

### Limitations of This Study and Future Lines of Investigation

The conclusions of this study must be analyzed with caution with regard to any generalization of the results, due to various reasons such as the size of the sample and the origin of the students (from the same university and the same degree course), sample characteristics and type of sample. Future investigations will therefore be directed at enlarging the size of the sample and the number of degree courses.

### ETHICS STATEMENT

The Ethics Committee of the University of Burgos approved this study. Written informed consent was obtained from all participants.

### AUTHOR CONTRIBUTIONS

MCS-M performed the statistical analyses and data interpretation and prepared the manuscript. JFD-P and CIG-O supervised the statistical analyses and collaborated in the drafting of the conclusions of the paper. RM-S supervised the document structure, analyses, and results.

### FUNDING

This work was funded by The Research Funding Program 2018 of the Vice-Rectorate for Research and Knowledge Transfer of the University of Burgos.

disability. Int. J. Clin. Health Psychol. 13, 155–166. doi: 10.1016/S1697-2600(13) 70019-X

Azevedo, R. (2005). Using hypermedia as a metacognitive tool for enhancing student learning? The role of self-regulated learning. Educ. Psychol. 40, 199–209. doi: 10.1207/s15326985ep4004\_2

Azevedo, R., Johnson, A., Chauncey, A., and Graesser, A. (2011). "Use of hypermedia to assess and convey self-regulated learning," in Handbook of

Self-Regulation of Learning and Performance, eds B. J. Zimmerman and D. H. Schunk (New York, NY: Routledge), 102–121.



Education, eds D. J. Hacker, J. Dunlosky, and A. C. Graesser (New York, NY: Routledge), 299–315.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Sáiz-Manzanares, Marticorena-Sánchez, Díez-Pastor and García-Osorio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Satisfaction With the Self-Assessment of University Students Through e-Coping With Academic Stress UtilityT<sup>M</sup>

Jesús de la Fuente1,2 \*, José Manuel Martínez-Vicente<sup>3</sup> , Francisco Javier Peralta-Sánchez<sup>3</sup> , María Carmen González-Torres<sup>1</sup> , Raquel Artuch<sup>4</sup> and Angélica Garzón-Umerenkova<sup>5</sup>

<sup>1</sup> Department of Theory and Methods in Education and Psychology, School of Education and Psychology, University of Navarra, Pamplona, Spain, <sup>2</sup> Department of Psychology, Universidad Autónoma de Chile, Santiago, Chile, <sup>3</sup> Department of Psychology, University of Almería, Almería, Spain, <sup>4</sup> Universidad Internacional de la Rioja, Logroño, Spain, <sup>5</sup> Fundación Universitaria Konrad Lorenz, Bogotá, Colombia

#### Edited by:

Pablo Fernández-Berrocal, Universidad de Málaga, Spain

#### Reviewed by:

Melissa Christine Davis, Curtin University, Australia Jesús Nicasio García Sánchez, Universidad de León, Spain

> \*Correspondence: Jesús de la Fuente jdlfuente@unav.es

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 28 April 2018 Accepted: 19 September 2018 Published: 08 November 2018

#### Citation:

de la Fuente J, Martínez-Vicente JM, Peralta-Sánchez FJ, González-Torres MC, Artuch R and Garzón-Umerenkova A (2018) Satisfaction With the Self-Assessment of University Students Through e-Coping With Academic Stress UtilityTM. Front. Psychol. 9:1932. doi: 10.3389/fpsyg.2018.01932 The general purpose of this report is: (1) research was to check whether the degree of satisfaction with the self-assessment activity of university students was related to the scores obtained and the degree of different variables, associated with level of Self-Regulation; (2) to present the online utility, e-Coping with Academic StressTM, as a technological development in Educational Psychology; (3) analyze the possibilities of transfer of this technological innovation. A total of 929 university students, coming from a public university, participated in the use of this online utility. University students can use the tool's online inventories to make self-assessments in the different variables of Studying, Learning and Performing under Stress (SLPS Competency Model). Descriptives, correlational and inferential analyzes (ANOVAs and MANOVAs) were carried out. The results allowed to know the profile of competences of the analyzed university students, in addition to the degree of satisfaction with the self-evaluation. Finally, we communicate possible actions and options available for transfer of this resulting technology, through RD transfer contracts arranged directly or with other universities.

Keywords: SLPS Competency Model, university academic stress, e-technological development, innovation transfer, validation study

### INTRODUCTION

The value chain of RD & I (Research, Technological Developent, and Transfer of Innovation) in Educational Psychology is conceptualized as carrying out scientific research, for the production of technological development (processes, products or services) that ultimately gives rise to innovation (and its transfer) as the ultimate element of the process. The RD & I value chain can mean an advantage to the different activities of academics, research, and professional practice, with respect to processes, products and services that are generated in the sphere of psychology and education (de la Fuente et al., 2018a). Consistent with the previous conception, the purpose of this report is: (1) research was to check whether the degree of satisfaction with the self-assessment activity of university students was related to the scores obtained and the degree of different variables,

associated with level of self-regulation: (2) to present the online utility, e-Coping with Academic StressTM, as a technological development in Educational Psychology; (3) analyze the possibilities of transfer of this technological innovation.

The previous questions that this research has guided are: on what variables does the degree of satisfaction depend on the selfassessment of the students?, Does it depend on the degree of personal self-regulation?, Are there some variables that determine the degree of satisfaction positively or negatively?

### The Problem of Stress as Academic Emotion

Human response to stress has been studied extensively in several contexts, especially in clinical and healthcare fields (O'Donovan and Hughes, 2007; Hamdan-Mansour et al., 2009; Pettit and De Barr, 2011; Costarelli and Patsai, 2012; Gulewitsch et al., 2013; Schönfeld et al., 2017). In the educational field, however, despite significant progress in understanding cognitive and metacognitive processes, more effort is required to gain a clear understanding of the mechanisms that make up the stress response, especially as the negatively impacts these processes (Pintrich, 2004; de la Fuente, 2014a). Stressful study situations are learning and performance contexts that can trigger stress responses (cognitive, physiological and motor). This is particular true when pursuing a university degree or competing for employment through professional exams. If stress responses are severe, learning or performance may suffer. Such high-stress contexts are likely to appear over the course of academic and professional life for virtually every university student or graduate.

Recent research has revealed the importance of taking into account positive and negative emotions experienced during learning and performance processes at university (Andersson et al., 2009; Bardi et al., 2011; Hamaideh, 2011; Postareff et al., 2016; Pekrun et al., 2017). Metaphors of information processing and construction formed the basis of classic, firstgeneration cognitive models, and not enough attention was given to emotions and how they affect cognitive processes while learning. After this, motivational-affective models appeared as the second generation (Pintrich, 2004; Zimmerman, 2008), and insisted on the need to pay attention to and intervene in affective processes that operate during learning, because they affect and accompany the cognitive processes that support information processing, sometimes positively and sometimes in a negative, interfering way. Negative emotional experiences can take place when learning under stress at the university, and even more so when preparing for professional exams; this process must become fully understood (Regehr et al., 2013).

Evidence shows that, during the process of learning and study, the different manifestations of a stress response interfere in cognitive and motivational processes (Serlachius et al., 2007; Chou et al., 2011). We must understand the stress response if we seek to understand learning processes in general, and particularly study processes. Stress has been shown to interfere with processes of memory, attention, and information recall. The anxiety associated with stress can lead to more and more worry and negative emotionality. When negative thoughts and irrational beliefs replace positive thinking, a weakened motivationalaffective state ensues, and demotivation follows (Largo-Wight et al., 2005).

Consequently, effort must be made toward prevention of stress responses, students must be helped toward establishing the competencies of managing stressing and being ready for its appearance. However, this aspect is usually overlooked in preparatory programs for university students and exam candidates (Conley et al., 2013), which focus almost exclusively on academic content.

## Theoretical Foundation

### Model of Competency for Studying, Learning and Performing in Stressful Contexts

The Competence for Study, Learning and Performance with Stress model (SLPS Competency) is a multi-dimensional construct (de la Fuente, 2015a) inasmuch as it refers to stress factors for university students and exam candidates, and to stress management. This model is based on the 3P model (presageprocess-product) of Biggs (Biggs, 2001) and orders the variables to be analyzed within it. The presage variables refer to the predictor variables or previous experience of the student, which is not modifiable. The process variables are mediating variables and refer to the level of competence to studying, learning and performin inder stress situations (CSLPS model). The product variables refer to the level of stress experienced in the learning situation.

### **Presage variables: past experience**

This variable refers to how a subject assesses his or her past experience with professional examinations, inquiring into the subject's perception of examinations past, along with present and future expectations. Looking at university populations, recent reviews have analyzed factors that mediate between academic self-efficacy, created from past experiences, and later academic performance (Honicke and Broadbent, 2016), as well as the effect of prior experiences on creating expectations (Jones, 2017).

### **Process variables: competence for studying and learning under stress**

This type of competence evidently does not refer to a single skill or to one type of knowledge. The individual constructs a set of skills, knowledge, attitudes and habits that enable him or her to confront a certain assessment situation and succeed. The Multidimensional nature of the Competency for Studying, Llearning and Performing under Sstress, CSLUS (de la Fuente et al., 2013b; de la Fuente, 2015a) is characterized by assuming three levels of learning:

#### (1) Knowledge:

Facts (knowledge about the characteristics of the class subject or professional exam: job openings, percentage of candidates who pass, requirements)

Concepts (competitive exam system, requirements; type of examination, scoring, prior merits/credits, type of class subject)

Principles (beliefs about the professional exam or selection process)

### (2) Know how:

fpsyg-09-01932 November 7, 2018 Time: 17:12 # 3

Instrumental skills (written and oral skills; control over anxiety)

Learning and study skills (study skills and techniques)

Cognitive meta-skills for study (learning strategies), emotional meta- skills (coping strategies) and behavioral metaskills for managing stress (self-regulation strategies) (3) Mindset:

Attitudes and values (academic behavioral confidence, achievement motivation)

Study habits (time management, persistence, discipline)

#### **Product variables: positive vs. negative emotions**

Engagement has been defined as the emotional involvement that accompanies an intense experience. On one hand, it can be promoted by the dynamics of a situation or the immediate context, which creates situational interest (Hidi and Anderson, 1992). On the other hand, it refers to a multidimensional construct that involves three dimensions: (1) cognitive engagement (self-regulation); (2) behavioral engagement (putting forth effort); (3) emotional engagement (interest) (Friedricks et al., 2004).

Maslach et al. (1996) conceptualized the burnout síndrome as a response to chronic work stress, including a feeling of emotional exhaustion, attitudes of depersonalization (negative feelings toward the people that one works with), and a lack of personal fulfillment in work, through the appearance of processes that devalue one's own professional role. Recent research has offered evidence that the burnout syndrome in students comprises three dimensions (Shaufeli et al., 2002): (1) Exhaustion: A feeling of fatigue caused by academic demands; (2) Cynicism: A cynical, distant attitude regarding academic tasks; and (3) Perceived competence: Feeling ineffective as a student. In university students, burnout is manifest primarily through emotional fatigue, while manifestations of cynicism and lack of personal fulfillment are scarce in these populations. The importance of level of emotional fatigue in university students stems from its role as a modulating variable that significantly influences students'expectations of successfully completing their studies.

### Academic Self-Assessment From the Theory of SRL vs. ERL

Self-assessment is a behavioral activity that forms part of selfregulating behavior (Brown, 1998) and self-regulated learning (Zimmerman and Labuhn, 2012). In the academic sphere, it is an essential element of self-knowledge and self-improvement. Different repertories are involved: knowledge (knowing what to assess), skills (knowing how to assess) and attitudes (wanting to self-assess). From this point of view, the e-coping utility is a technological development that places these elements at the university student's disposal: (1) students are offered knowledge of the variables for self-assessment and better self-understanding, in order to study and learn under stressful conditions; (2) they are given a procedure so they know how to self-assess each variable, as well as information about levels of competence in each; and (3) a satisfactory experience with self-assessment is made possible (de la Fuente et al., 2016). According to Self-Regulated vs. Externally Regulated Learning Theory, SRL vs. ERL Theory (de la Fuente, 2017), however, the final, attitudinal variable will depend on the student's self-regulating characteristics, given that not every student has the same emotional experience with this activity:


In accordance with this typology, the SRL vs. ERL Theory predicts that: (1) students rated high in self-regulation (SR: selfregulation) would have a high degree of satisfaction with the selfassessment activity, since it offers them knowledge and tools for improvement, as well as reinforcement, given that their expected outcome and emotions are positive; (2) students with a medium rating (AR: a-regulation) would show medium satisfaction, given that the activity would involve a positive emotional sense in some cases but negative in others; (3) students with a low rating in regulation (DR: dysregulation) would have the lowest degree of satisfaction, since this activity is bothersome to them, they typically avoid it or they perform it improperly, due to expectations of a poor outcome and a negative type of emotion. This postulate is presented in **Appendix 1**.

### Aims and Hypotheses

Based on the previous assumptions, the general purpose of this report is: (1) research was to check whether the degree of satisfaction with the self-assessment activity of university students was related to the scores obtained and the degree of different variables, associated with level of Self-Regulation; (2) to present the online utility, e-Coping with Academic StressTM, as a technological development in Educational Psychology; (3) analyze the possibilities of transfer of this technological innovation. More specifically, in relation to the first objective, this research was to evaluate university students' degree of satisfaction with using the e-Coping online utility. The following was hypothesized: (1) students' satisfaction with the self-assessment

would significantly (positive or negative) correlate to the mean score obtained on the psychological variable being assessed; (2) the level of high-medium-low in Self-Regulation of students as stated in the previous SRL vs. ERL Theory-, will significantly determine the level of each variable analyzed; (3) students' high-medium-low level of satisfaction with self-asessment would be similar or interdependence to the high-medium-low level obtained on the each psychological variable under analysis, except for variables with reverse directionality.

### MATERIALS AND METHODS

### Participants

The study sample consisted of 929 undergraduate students from a public university of the southeast (Spain). The students were enrolled in Psychology, Primary Education, or Educational Psychology; 86.5% were female (n = 673) and 13.5% were male (n = 256). The age range was 19–25 years (19 years, n = 201; 20 years, n = 303, 21 years, n = 131; 22 years, n = 82: 23 years, n = 49; 24 years, n = 38; 25 years, n = 20) with a mean age of 23.08 (st = 4.4) years. By academic cycles of the Degree, there were a total of 576 students of 1st cycle and 353 of 2nd cycle.

The selection of the subjects was not probabilistic, since all the students who completed the e-Coping utility were studying the subject "Educational Psychology." The completion of the same was raised as an activity of self-evaluation and self-improvement of academic learning. Although they completed it voluntarily, by completing it they were given 2 points.

### Instruments

### Technological Development: The Utility e-Coping With Academic Stress

This utility is an intervention from Educational Psychology, based on the Competency Model for Studying, Learning and Performing under Stress (CMSLPS; de la Fuente, 2015b). Its purpose is to help university students or professional exam candidates better manage their study, learning and related stress. This utility continues in the line of other recent intervention programs (Regehr et al., 2013), including other online interventions (Day et al., 2013). It is preferable that the instruments be used with appropriate guidance from an Educational Psychologist, in order to avoid any erroneous interpretations and inferences. In case of doubt, it is best to consult a professional. Although a good number of assessment inventories are available, e-Coping with Academic Stress (de la Fuente, 2015b) offers a selection of inventories that make it easier to self-assess and to enhance desired behaviors. The inventories have been validated, and the different levels were established in prior samples of university students or professional exam candidates. In any event, the levels should not be considered absolute, but merely an indication to help the student or candidate make decisions about how to improve. A low level on any particular variable, for example, would indicate the student's need for considerable work to improve their competence in this aspect; a medium level would represent an average degree of competence, and should be improved to some extent; a high level reveals adequate competence, although certain behavioral aspects might be identified for further improvement. See **Table 1**.

For each variable of the model, the student is able to (1) self-assess, obtain his/her own score (low-medium-high); (2) obtain improvement strategies for that variable or level of subcompetency. The tool is currently available in Spanish- and English-language versions, but it can be implemented in other languages. See **Appendix 2** in **Supplementary Material**.

#### Instruments

#### **Meta-cognitive skills for learning and study**

Learning approach. The Revised Two-Factor Study Process Questionnaire, R-SPQ-2F (Biggs et al., 2001), in its validated Spanish version (Justicia et al., 2008), was used to measure this variable. Its 20 items pertain to four subscales (Deep Motive, Deep Strategy; Surface Motive, Surface Strategy) that measure two dimensions: Deep and Surface learning approaches, respectively. Items are scored on a 5-point Likert scale where 1 = rarely true of me and 5 = always true of me. The confirmatory factor structure of the Spanish version had a second factor structure with two factors (Chi-Square = 2645.77; df = 169, CFI = 0.95, GFI = 0.91, AGFI = 0.92, RMSEA = 0.07), which also yielded acceptable reliability coefficients (Deep, α = 0.81; Surface, α = 0.77), similar to those found by the original authors, using the AMOS Program. CFI and NFI values ranged from 0 (poor fit) to 1 (good fit; Bentler, 1990). These indices require values greater than 0.90 to represent good fit of a model. RMSEA was used because it accounts for model parsimony (i.e., goodnessof-fit values can be artificially inflated with greater numbers of parameters in the model). Our model of choice was the most parsimonious one, hence, specifying the model with a small number of parameters was preferable. RMSEA values greater than 0.08 reflect poor fit, values from 0.05 to 0.08 indicate acceptable fit, and values less than 0.05 reflect a good fit (MacCallum et al., 1996).

#### **Meta-emotional skills for learning and study**

Coping Strategies Scale. The EEC (Chorot and Sandín, 1987), in an abbreviated, validated Spanish version, EEC-Short (de la Fuente, 2014b), was used to measure this variable. While the original instrument contains 90 items, the validation revealed a first-order structure with 64 items and a second order with 10 factors and two dimensions, both of them significant and showing adequate fit values [Chi-square = 878.750; Degrees of freedom (77–34) = 43, p < 0.001; NFI = 0.901; RFI = 0.945; IFI = 0.903, TLI = 0.951, CFI = 0.903, RMSEA = 0.07]. Reliability measurements are Cronbach alpha of 0.93 (complete scale), 0.93 (first half) and 0.90 (second half), Spearman-Brown of 0.84 and Guttman of 0.80. Two dimensions are evaluated: (D1) Emotion-focused coping (α = 0.95); (D2) Problemfocused coping (α = 0.91). In relation to emotion-focused strategies, the factors were: (F1) Avoidant distraction (0.79); (F7) Reducing anxiety and avoidance (0.88); (F8) Preparing for the worst (0.80); (F9) Emotional venting and isolation (0.91); and (F10) Resigned acceptance (0.86). In relation to problem-focused strategies, the factors were: (F2) Seeking family help and counsel (0.92); (F5) Self-talk (0.82); (F10)

Positive reappraisal and firmness (0.87); (F12) Communicating feelings and social support (0.89); and (F13) Seeking alternative reinforcements (0.80).

#### **Meta-motivational variable**

Resilience. Resilience was assessed using the CD-RISC Scale (Connor and Davidson, 2003) in its validated Spanish version

TABLE 1 | Variables of self-assessment and psychometrics characteristics of the instruments, in the e-Coping utility (de la Fuente, 2015b).


Note: Variables used in this investigation are shown in boldface type.

(Mateu et al., 2010; Manzano-García and Ayala-Calvo, 2013). This inventory makes it possible to assess different aspects of being able to face difficulties and overcome them. It provides information on perceived competence, stress management, control and spirituality (Berry and York, 2010). Adequate reliability and validity values were found in Spanish samples, and a five-factor structure: F1: Persistence/tenacity, strong selfefficacy (TENACITY); F2: Emotional and cognitive control under pressure (STRESS); F3: Adaptability/ability to bounce back (CHANGE); F4: Perceived Control (CONTROL), and F5: Spirituality (SPIRITUALITY).

#### **Meta-behavioral skills for learning and study**

Personal self-regulation. The Short Self-Regulation Questionnaire (SSRQ) (Brown et al., 1999) was used to measure this variable. Having been previously validated in Spanish samples (Pichardo et al., 2014; Garzón-Umerenkova et al., 2017), it shows acceptable validity and reliability values, similar to the English version. The Short SRQ contains four factors (goal setting/planning, perseverance, decision making and learning from mistakes) and 17 items (all with saturations greater than 0.40), along with a consistent confirmatory factor structure (Chi-Square = 250.83, df = 112, CFI = 0.90, GFI = 0.92, AGFI = 0.90, RMSEA = 0.05). Internal consistency was acceptable for the total questionnaire (α = 0.86) and for three factors: goal setting/planning (α = 0.79), decision making (α = 0.72) and learning from mistakes (α = 0.72). The perseverance factor, however, showed low internal consistency (α = 0.63). Correlations were studied: between each item and its factor total; between the factors; and between each factor and the complete questionnaire. In all cases results were good, except in the case of decision making, which showed less correlation with the other factors (range: 0.41– 0.58). The correlations between the original version and the complete version, and between the original and the short versions (complete SRQ with 32 items and short SRQ with 17 items), with a Spanish sample, were better for the short version (shortoriginal: r = 0.85 and short-complete: r = 0.94; p < 0.01) than for the complete version (complete-original: r = 0.79; p < 0.01).

#### **Attitudes, values and habits for learning and study**

Action-emotion style. The Jenkins Activity Survey for students-Form H (JASE-H) was used. The Type-A Behavior Pattern (TABP) is measured with this scale; its student version is adapted (Bermúdez et al., 1990; Bermúdez et al., 1991, Unpublished) from the Jenkins Activity Survey T-version (Krantz et al., 1974). Its four factors are Impatience, Hostility, Competitiveness and Hardworking. There are 32 items in total, which are answered on a six-point Likert scale; the subject must choose the degree to which the item applies to him or her. A response of one means the item is not at all applicable to the respondent, and six means it is totally applicable to him or her. The JASE-H offers a global TABP score, which is the sum of scores assigned to all items, as well as specific measurements for each of the components comprising the TABP. The JASE-H possesses high internal consistency (alpha coefficient of 0.85 for the total scale; 0.81 for the Impatience-Hostility factor, 0.82 for Competitiveness and 0.70 for Hardworking) and high stability over time, both for the complete scale (0.68) and for its factors (0.61, 0.76, and 0.70, respectively). The authors report consistent Reliability and Validity measures. The statistics are Alpha = 0.832, and Guttman Split-Half = 0.803 (de la Fuente et al., 2013a).

Academic confidence. The Academic Behavioral Confidence Scale, ABC (Sander and Sanders, 2006, 2009) in a Spanish validated version (Sander et al., 2011). The ABC scale was developed from the idea that understanding students' confidence toward their studies could be important for making sense of students' expectations of teaching, learning and assessment. This psychometric tool assesses the confidence of under-graduate students in their own anticipated study behaviors. The ABC scale has four subscales that tap into crucially distinct aspects of students' academic behavior: Grades, Studying, Verbalizing and Attendance (Sander and Sanders, 2009). This variable is proven to be a predictor of academic performance and learning approach (de la Fuente et al., 2013b).

#### **Stress**

Emotional indicators of stress. The Marlach-Utrech Burnout-Engagement questionnaire (Shaufeli et al., 2002) assesses the student's level of engagement in the task vs. emotional exhaustion. This emotional dimension is an important correlate of subjective stress in these types of situations; there is plentiful evidence of the importance of positive vs. negative emotions during study, with particular importance given to the negative impact of burnout (Lorenz et al., 2010; Tavolacci et al., 2013).

### Procedure

On a voluntary basis, participants used an online platform e-Coping Stress (de la Fuente, 2015b; de la Fuente et al., 2015a) to complete the scales. A total of five specific teaching-learning processes were covered; they represent different university subjects that were taught over two academic years. Assessment of presage variables took place in September–October of 2014 and 2015, process variables in February–March of 2015 and 2016, and product variables in May–June of 2015 and 2016.

The university students used the E-Coping with Stress utility in their practicum for the subject of Educational Psychology (part of the degree programs in Psychology and Primary Education at different universities) in order to gain an understanding of the study variables through an experience with self-assessment and improvement in these variables. They completed one online questionnaire per week, at home, for 4 months. They were also assured of anonymity in data completion and data storage in a shielded database; the data were processed for the single purpose of this group investigation, never examined individually.

At the end of each inventory there is a Likert-type scale (1–5) to assess the student's degree of satisfaction with the assessment and improvement experience afforded by the e-utility for that variable. This scale is completed just after finishing each inventory, before learning one's score. The score on this final scale revealed users' subjective satisfaction with each assessment, as well as their specific experience with each inventory, making it possible to detect which one is most useful, as well as the optimal moment for application during university studies.

The students participated voluntarily an informed consent from the participants was obtained and was previously approved by the University Ethics Committee each university (Bioethics Committee of the University of Almería, and Commission of Research Ethics of the University of Navarra), in the context of R&D Project (2012-2015). The consent obtained was both informed and written. The data was protected in an archived and registered file, as indicated by the Spanish Data Protection Law.

### Data Analysis

Using an ex-post-facto design, preliminary ANOVAS were carried out to rule out an effect of age, gender and academic cycle on satisfaction with the self-assessment. First, a bivariate correlation of Pearson was analyzed. In a complementary way, the descriptive indexes of each variable were calculated (means, standard deviation, asymmetry and kurtosis). Second, ANOVAS and MANOVAS were carried for determination of effect of the level of Self-regulation (high-medium-low) on level of each variable. Third, low-medium-high groups were established for each of the variables through a K-means cluster analysis. Several ANOVAs were also carried out, to establish independence between low-medium-high levels of each variable and from the level of satisfaction with the self-assessment activity. Statistical suitability of these groupings was also established by ANOVAs, as well as the effects of the dependent variables, using SPSS v. 23.0.

### RESULTS

### Preliminary Results: Effect of Gender and Cycle on Satisfaction With Self-Assessment Activity

The previous ANOVAS carried out showed non-significant main effects of the gender variables [F(1,831) = 2.635, p < 0.123, η <sup>2</sup> = 0.003, power = 0.370] and academic cycle [F(1,831) = 1.769, p < 0.172, η <sup>2</sup> = 0.002, power = 0.264], in the satisfaction with the self-assessment carried out.

### Descriptive Values and Association Relations Between Satisfaction With the Self-assessment, and Variable Assessed (Hypothesis 1)

The results showed significant bivariate correlations of the positive dimensions of the variables analyzed and satisfaction with the self-assessment. A significant, positive bivariate correlation appeared between Satisfaction with Self-Asssessment (SAT) and the following: Self-Regulation (SR) (r = 0.275), Deep Approach (DA) (r = 0.324), Problem Coping (CP) (r = 0.178), Resilience (r = 0.312), the Competence-Hardworking (CHW) component (r = 0.538), Academic Confidence (AC) (r = 0.333) and Engagement (r = 0.275). Also, a significant negative relationship appeared between the negative dimensions of the variables in relation to Surface Approach (SA) (r = −0.114) and Burnout (r = −0.375). See **Table 2**.

### Effect of the High-Medium-Low Level of Self-Regulation on HML Level of Each Variable (Hypothesis 2)

The results showed that in most of the analysis, the level of selfregulation of the students (high-medium-low) determines, in a statistically significant way, the level of the variable analyzed. Initially it was found that the levels established in Self-Regulation (High-Medium-Low, HML) were significantly different.

First, for the meta-cognitives variables, a significant main effect of the HML level of SR appeared in the dimensions of the learning approaches for both dimensions, although in the opposite direction. The HML levels in SR determine the same levels of Deep Approaches (DA) and the inverse in Surface Approaches (SA). Fort the meta-emocional variables, in the case of coping strategies, although a significant main effect did not appear in the total number of strategies used, a significant partial effect did appear for the problem-centered strategies, for the former students and the media in self-regulation. Also, the HML level of SR determined the levels of meta-motivational variable Resilience.

In the case of attitudinal variables, the HML level of SR significantly determined the HML level of the emotion-action style (directly with respect to the CH dimension, and infiractly with respect to the IH dimension) and of academic confidence.

Finally, for the variables of stress, the HML level of SR positively determined the level of engagement and negatively the level of the burnout variable. The direct mean values and the specific effects are shown in **Table 3**.

### Effect of the High–Medium–Low Level of Each Variable on HML Satisfaction With the Self-Assessment Experience (Hypothesis 3)

The high–medium–low levels of the variables analyzed showed different significant effects on satisfaction with the selfassessment (SAT). Students' levels of self-regulation, as a metabehavioral variable, determined their level of SAT, and also their level of learning approaches, whether positively (DA) or negatively (SA). Likewise, the level of the meta-emotional variable coping strategies determined the level of SAT, especially in the case of problem-focused coping, but not in emotion-focused coping. Accordingly, the same occurred for levels of resilience (RES), action-emotion style (AES) and academic confidence (AC). Finally, the level of the variable burnout (BUR) determined the level of SAT; the inverse was true for level of engagement (ENG). See **Table 4**.

### DISCUSSION AND CONCLUSION

### Research

The results confirming the first hypothesis of our investigation contributed empirical support for the idea that satisfaction of self-assessment activities (SAT) is associated with the variables assessed. If we consider that self-assessment is a behavior typical of self-regulation in its different phases (before,



SAT, Satisfaction with Self-Asessment; DA, Deep Approaches; SA, Surface Approaches; SR, Self-Regulation; CTOT, Coping Total; EMOTC, Emotion Coping; PROBC, Problem Coping; RESIL, resilience; AES, Action-Emotion Style; CSL, Competivity-Hard Word; IH, Impatience-Hostility; AC, Academic Confidence; BURN, Burnout; ENGA, Engagement.

TABLE 3 | Effect of the level of Self-Regulation (high-medium-low) (IV) on level of each variable (DVs) (n = 929).


SR, Self-Regulation; DA, Deep Approaches; SA, Surface Approaches; CTOT, Coping Total; EMOTC, Emotion Coping; PROBC, Problem Coping; RESIL, Resilience; AES, Action-Emotion Style; CHW, Competivity-Hard Word; IH, Impatience-Hostility; AC, Academic Confidence; BURN, Burnout; ENGA, Engagement; <sup>∗</sup> significant major trend; ∗∗p < 0.001.

during, and especially after the behavior), these results are consistent with prior results that have consistently shown selfregulation to be linearly and positively predictive of flourishing and health, while a linear, negative prediction is found for reasons to procrastinate (Garzón-Umerenkova et al., 2018), and the use of motivational regulation strategies had significant


TABLE 4 | Differences according the level of the variable assessed (IV) on the level of Satisfaction with the Self-Assessment, SAT (DV) (n = 929).

SR, Self-Regulation; DA, Deep Approaches; SA, Surface Approaches; CTOT, Coping Total; EMOTC, Emotion Coping; PROBC, Problem Coping; RESIL, resilience; AES, Action-Emotion Style; CHW, Competivity-Hard Word; IH, Impatience-Hostility; AC, Academic Confidence; BURN, Burnout; ENGA, Engagement; <sup>∗</sup>Significant major trend; ∗∗p < 0.001.

positive indirect effects on students' academic performance and affective/cognitive well-being (Grunschel et al., 2016). There is also a significant positive predictive relationship between self-regulation and resilience, with the self-regulation factor learning from mistakes being the most predictive of resilience (Artuch-Garde et al., 2017). Likewise, a positive interdependence relationship is found between level of selfregulation, deep learning approaches, problem-focused coping strategies, resilience, and academic confidence, with a negative interdependence relationship for test anxiety (de la Fuente et al., 2017).

One relevant aspect of the results is the negative association found between the activity of self-assessment and surface learning approaches and between self-assessment and burnout. If students with these characteristics manifest dissatisfaction with the activity of self-assessment, it is probably because the activity itself can induce negative emotionality and stress when results are less than adequate or are indicative of the student's low skills for learning. This may be at the root of this association; however, this aspect should be further investigated. This matter should also be taken into account in university student advising and guidance processes. Help with self-assessment, or an online self-help program, is not sufficient; mentoring programs are needed that teach these students to continue self-improvement without becoming discouraged, and offer them external support.

The second hypothesis afirmed the level of high-mediumlow in Self-Regulation of students will significantly determine the level of each variable analyzed. This results they show empirical support, since it was proved that the HML levels of SR determine the HML levels of the analyzed variables, as stated in the previous SRL vs. ERL Theory (de la Fuente, 2017; see **Appendix 1**), and recent empirival evidence (de la Fuente et al., 2017). Once again, the importance of personal self-regulation as a meta-behavioral correlate of the subjects that influences the rest of the variables, metacognitive, meta-emotional, meta-motivational, and stress level is confirmed.

The third hypothesis asserted that high levels in the independent variables that form part of Self-Regulated behavior (SR) would be accompanied by high satisfaction with the performance of SAT. By contrast, low levels in the independent variables, representing low regulation or dysregulation (DR), would be accompanied by the lowest level of satisfaction with the self-assessment experience. In the case of self-regulating students (SR), as the data confirm, this system of self-assessment and self-help is optimal, considering that these students are characterized by personal work and they are able to take best advantage of an autonomous self-help system. In the case of students who are medium and -especially- low in self-regulation (dysregulatory), the evidence moves us closer to an explanatory mechanism for why such students avoid doing self-assessment, or simply put it off with procrastination (Garzón-Umerenkova et al., 2018) or distractive mechanisms; self-assessment generates further stress for them, in addition to what they already experience.

However, these results have the limitation that they are not generalizable to the entire population because they have not used probabilistic sampling and are limited to the students who participated in the use of the e-Utility. Future research should show if this effect is maintained with all types of university students, from different countries and cultures.

### Implications for Intervention

fpsyg-09-01932 November 7, 2018 Time: 17:12 # 10

Consequently, the "Matthew effect" (Merton, 1968) would be applicable in their case, where university students with low regulation or dysregulation gain no satisfaction from doing improvement-oriented self-assessments; leading them to even further dysregulation, considering that self-assessment is a mechanism that compiles critical information for the exercise of self-regulation. From our point of view, this represents an important limitation of this e-utility, given that when used alone, as is the case, without external regulation (in time and extrinsic motivation), there is a high likelihood that the self-assessment and improvement tasks will be dropped at some point. It is therefore more advisable to use them in a context of external regulation that engages the students and ensures their completion. Advisory support for their use is also of benefit, in order to ensure their profitability.

On the other hand, there is a growing academic interest in the development of preventive or remedial interventions, virtual or face-to-face, that support the academic quality of the institutions and the achievement of goals by the students. For example, for the management of academic procrastination (Rozental et al., 2014; De Paola and Scoppa, 2015; Glick and Orsillo, 2015), time management (Nadinloyi et al., 2013), stress reduction (Regehr et al., 2013), promotion of health behaviors (Webb et al., 2010) or improvement of academic performance (Lin and Tsai, 2016). The focus on cognitive-behavioral change through the different options offered by the internet is a source of research into R & D programs with social impact (Webb et al., 2010) that, as in the research presented, point to a path for the development of this type of evaluation or feedback tools with verifiable results.

For some years, the lack of application of the principles and results of educational research in the context of educational practice has been worrisome (Vanderlinde and van Braak, 2010; Levin, 2013). The development presented "e-utility" allows to facilitate the application of theoretical principles of education to improvements for practical purposes, in different academic contexts or even from other contexts in which the same principles can be transferred, such as working environments (Rabenu and Yaniv, 2017) or personnel selection processes (Sautelle et al., 2015).

### Theoretical Implications

When learning and study involve pressure and potentially stressful situations, emotional experiences are just as important as the cognitive processes used. Traditionally, in its effort to help students, the psychological assessment of study and learning has focused on cognitive skills and metacognitive strategies. This approach is reasonable when learning contexts are not stressful. In contexts of university teaching and learning, however, conditions often trigger stress responses. Such conditions include highly demanding tasks, high performance requirements, sustained effort and uncertainty about succeeding in one's objectives. In such cases, not only cognitive behaviors must be examined, but also emotional behaviors. In the case of professional examination candidates, work must be approached from a competency model for competing in professional examinations (de la Fuente, 2015a), integrating the conceptual, procedural and attitudinal levels of the subcompetencies. The utility e-Coping with Academic Stress (de la Fuente, 2015b) - based on the SLPS Competency Modeloffers students the opportunity to self-assess their achievement emotions, and subsequently work toward self-improvement, at different points of the teaching-learning process. Teachers may also benefit by understanding the levels of these variables that are represented in their students and their class groups, and by making suitable adjustments in the teaching-learning process.

### Implications for the Transfer of e-Utility

From the conception of the value chain RD & I, it is important to carry out activities of transfer of the empirically validated theoretical models and technological developments arising in the field of Educational Psychology. Different innovation transfer activities were carried out and are presented here. They are based on the SLPS Competency Model <sup>R</sup> , for learning how to learn, and the E-Coping with Academic Stress <sup>R</sup> utility:

### Transfer Seminars

This innovation transfer activity consists of presenting the model, the technology developed, and its possible applications in different healthcare and educational organizations. Several actions of this type have been carried out, in hospitals as well as in Spanish and other European universities.

#### **Catalog of Public Purchases in Innovation**

The e-Coping with Academic Stress <sup>R</sup> utility was included in the Catalog of Public Purchases in Innovation, prepared by the AMETIC Technology Platform and the Ministry of Health (Spain). The catalog includes technology developments typical of E-Healthcare and seeks to promote the use of new technology developments from companies in the sphere of public and private healthcare: http://ides.es/blog/cat%C3%A1logo-de-oferta-innov adora-en-tics-de-la-salud-vida-activa-e-independiente.

### R&D Technology Transfer Contracts

R&D transfer contracts of technological innovation were carried out with two European universities. In each case, the contract included general consulting and improvement of the R&D&I value chain, and in particular, use of the models and technological tools mentioned above.

### Transfer to UAL Spin-Off Company, Education and Psychology I + D + i

The university's office of research results transfer has transferred these products to the cited company for their exploitation.

#### Technological Demonstrations

fpsyg-09-01932 November 7, 2018 Time: 17:12 # 11

More recently, technological demonstrations have been held at conferences on Psychology and Technological Innovation and Entrepreneurship (de la Fuente et al., 2018b): http://cipi2018. copao.com/es/.

### AUTHOR CONTRIBUTIONS

JdlF: utility development, research design, data analysis, and text writing. JM-V: data analysis and text revision. FP-S: completion of the students. MG-T: completion of the students and general revision of the text. RA: sample collection and bibliographic review. AG-U: general revision of the English version in the text.

### REFERENCES


### FUNDING

The utility e-Coping with Academic Stress have been developed thanks to support from R&D Project ref. EDU2011-24805 (2012– 2015), Ministry of Science and Innovation (Spain) and the European Social Fund. For more information, please contact with the first author: jfuente@ual.es or jdlfuente@unav.es.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.01932/full#supplementary-material




**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 de la Fuente, Martínez-Vicente, Peralta-Sánchez, González-Torres, Artuch and Garzón-Umerenkova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

## APPENDIX 1

fpsyg-09-01932 November 7, 2018 Time: 17:12 # 14

TABLE A1 | Conceptual continuum and typologies of each Self-Regulatory Behavior (reproduced with permission: de la Fuente, 2017).


<sup>∗</sup>Place of self-assessment and positive vs. negative self-affect.

# Differential Effects of Transcranial Direct Current Stimulation (tDCS) Depending on Previous Musical Training

Ana Sánchez-Kuhn<sup>1</sup> , Cristian Pérez-Fernández<sup>1</sup> , Margarita Moreno<sup>1</sup> , Pilar Flores1,2† and Fernando Sánchez-Santed1,2 \* †

<sup>1</sup> Department of Psychology and CIAIMBITAL, CeiA3, University of Almería, Almería, Spain, <sup>2</sup> Instituto de Neurorehabilitación Infantil InPaula, Almería, Spain

Previous studies have shown that transcranial direct current stimulation (tDCS) facilitates motor performance, but individual differences such as baseline performance seem to influence this effect. Accordingly, musicians offer an inter-individual differences model due to anatomical and functional variances displayed among the motor cortex regions. The aim of the present work was to study if the baseline motor skill predicts whether tDCS can enhance motor learning. For that objective, we administered anodal (n = 20) or sham (n = 20) tDCS on the right primary motor cortex region of 40 right-handed healthy participants, who were divided into four groups: musicians (tDCS/sham) and nonmusicians (tDCS/sham). We measured the skill index (SI) presented in the sequential finger-tapping task (SEQTAP) at baseline, during three 20 min/2 mA stimulation sessions, and in follow-up tests after 20 min and 8 days. Depending on the normality of the data distribution, statistical differences were estimated by ANOVA and Bonferroni post hoc test or Kruskal–Wallis and U Mann–Whitney. Results showed that musicians scored higher in baseline performance than non-musicians. The non-musicians who received tDCS scored higher than the sham group in the first and second stimulation session. This effect was extended to the 20 min and 8 days follow-up test. In musicians, there was no effect of tDCS. The present method seems to be suitable for the achievement of positive and consolidated tDCS effects on motor learning in inexperienced participants, but not in musicians. These data may have an implication for the rehabilitation of motor impairments, contributing to more individualized stimulation protocols.

Keywords: transcranial direct current stimulation, motor cortex, sequential finger tapping task, musicians, individual differences

### INTRODUCTION

Transcranial direct current stimulation (tDCS) is a non-invasive neuromodulation technique that is garnering increasing interest as an innovative neurorehabilitation tool (Dubljeviæ et al., 2014; De Ridder et al., 2017). tDCS has shown to change the excitability of the underlying neurons of the stimulated cortical area (Nitsche and Paulus, 2000), while being safe and relatively easy to use (Bikson et al., 2016). There is an increasing body of evidence from tDCS studies reporting

#### Edited by:

Unai Diaz-Orueta, Maynooth University, Ireland

#### Reviewed by:

Javier Oltra-Cucarella, University of Alicante, Spain Richard A. P. Roche, Maynooth University, Ireland

\*Correspondence:

Fernando Sánchez-Santed fsanchez@ual.es †These authors have contributed equally to this work

#### Specialty section:

This article was submitted to Clinical and Health Psychology, a section of the journal Frontiers in Psychology

Received: 11 May 2018 Accepted: 25 July 2018 Published: 10 September 2018

#### Citation:

Sánchez-Kuhn A, Pérez-Fernández C, Moreno M, Flores P and Sánchez-Santed F (2018) Differential Effects of Transcranial Direct Current Stimulation (tDCS) Depending on Previous Musical Training. Front. Psychol. 9:1465. doi: 10.3389/fpsyg.2018.01465

**146**

positive effects on human motor function (Russo et al., 2017; Sánchez-Kuhn et al., 2017). tDCS has contributed to the motor rehabilitation of motor deficits derived from stroke (Boggio et al., 2007; Russo et al., 2017), dysphagia (Kumar et al., 2011), and Parkinson's disease (Fregni et al., 2006), and its applicability has been extended to cerebral palsy in children (Almeida et al., 2014; Aree-Uea et al., 2014). In healthy participants, it has been shown to enhance motor function in the upper limbs (Reis and Fritsch, 2011; Sriraman et al., 2014; Karok et al., 2017) and the lower limbs (van Asseldonk and Boonstra, 2016). Previous studies have confirmed that tDCS might afford the most substantial benefits when combined with motor training (Reis and Fritsch, 2011; Page et al., 2015; Pérez-Fernández et al., 2016).

Transcranial direct current stimulation is administered by a portable device that contains a 9-volt battery connected to two electrodes: the anode and the cathode. A constant mild electrical current flows between both electrodes, entering through the scalp and changing the excitability of the underlying neurons. This process generates depolarization or polarization of the neuronal membrane, depending on the anodal (a-tDCS) or cathodal (ctDCS) nature of the electrode, respectively, although these effects can be reversed depending on the intensity and the duration of the stimulation (Nitsche et al., 2008). The administration of tDCS has been shown to produce sustainable changes in the amplitude of the motor evoked potentials of the cortical stimulated area that last for up to 90 min after stimulation ends (Nitsche and Paulus, 2001). Functional near-infrared spectroscopy (Merzagora et al., 2010; Takai et al., 2016) and positron emission tomography (Lang et al., 2005) have shown tDCS to increase cerebral blood flow and oxyhaemoglobin concentrations. Moreover, functional magnetic resonance imaging (fMRI) has shown activation of the primary motor cortex (M1) after stimulation by a-tDCS (Kwon et al., 2008), confirming its cortical excitatory effects.

The excitatory effects of tDCS have been attributed to an important reduction in GABA activity and a N-methyl-Daspartate receptor (NMDAr)-mediated augmentation of synaptic strength via an increase in intracellular Ca2<sup>+</sup> levels (Liebetanz et al., 2002). Moreover, the alteration of the glutamatergic system could ultimately lead to the release of brain-derived neurotrophic factor (BDNF) (Clarkson et al., 2010). In fact, treatment with tDCS has been shown to change BDNF levels (Filho et al., 2016), promoting BDNF-dependent synaptic plasticity (Fritsch et al., 2010), which might be key in explaining the long lasting effects of tDCS. Consequently, tDCS has shown to produce greater motor learning, compared to a sham condition, with a maintenance of these effects for up to 3 months, highlighting this technique as a promising neurorehabilitation tool (Reis et al., 2009). However, literature also shows a large amount of variability among the corticospinal excitability reactions of stimulated participants (Wiethoff et al., 2014). A recent metaanalysis concluded that the application of tDCS enhances motor skills but with rather low effect sizes (Hashemirad et al., 2016). The literature has attributed this variability to the task (Saucedo-Marquez et al., 2013; Karok et al., 2017), duration of stimulation (Puri et al., 2016), or electrode montage (Tazoe et al., 2014). Previous works argue that these fairly contradictory results may be explained by individual differences among study participants (Bikson et al., 2012; Li et al., 2015), as the effects of tDCS appear to be brain state-dependent (Bikson et al., 2016). Thus, it seems to be critically determined by the previous psychological state of the stimulated participant, including baseline gamma-aminobutyric acid (GABA) levels, individual circadian rhythms, genetics, brain injury and the initial state of the motor and cognitive function (Li et al., 2015), which configured the focus of the present work.

Musicians are considered a human model of inter-individual differences for studying behavioral-cognitive processes and brain effects of acquiring, practicing, and maintaining specialized motor skills (Schlaug, 2015). Musical training seems to shape certain brain areas through neuroplasticity mechanisms, as neuroimaging techniques have demonstrated differences in structures and functions of the motor regions of musically trained individuals, especially those areas related to auditory and sensorimotor networks (Gaser and Schlaug, 2003a,b; Bengtsson et al., 2005; Bangert et al., 2006; Baumann et al., 2007; Hyde et al., 2009; Herholz and Zatorre, 2012; Steele et al., 2013; Zamorano et al., 2017), leading to a better motor performance (Scheurich et al., 2018). These differences lead to an enhanced motor function. For instance, when tapping a specific rhythm, musicians have shown a more synchronized and flexible tapping rate, as well an enhanced error correction mechanism than nonmusicians (Scheurich et al., 2018). In addition, musicians seem to learn faster during a motor sequence tapping task compared to a control group (Tucker et al., 2016) and have showed to be more precise than controls in a circle-drawing task (Janzen et al., 2014). Moreover, Spilka et al. (2010) showed that musicians were able to imitate hand movements during video clip watching more accurately than non-musicians (Spilka et al., 2010). Therefore, Gorniak et al. (2018) proposed that musicians are a unique population with respect to fine motor control of the hand that show cortical reorganization, and suggest that this population should be studied separately from typical healthy controls with respect to hand function (Gorniak et al., 2018).

The Sequential Finger Tapping task (SEQTAP) is one of the most used tasks to measure motor tapping in healthy subjects. The application of anodal tDCS over M1 during three consecutive days has previously been shown to reduce significantly the reaction time in the finger tapping task/serial reaction time task (SEQTAP/SRTT) (Hashemirad et al., 2016). Evidence supports the hypothesis that timely co-application of (hand/arm) training and tDCS to the contralateral M1 facilitates long-term memory formation, reflecting use-dependent plasticity (Rroji et al., 2015). Nevertheless, data on the effects of tDCS on the retention of the skill in the SEQTAP remain controversial. Saucedo-Marquez et al. (2013) found that intervention with tDCS on M1 during three consecutive days improved the performance in the SEQTAP task during the stimulation, but the effect of tDCS was diminished 1 week after the stimulation. Accordingly, Schambra et al. (2011) found offline effects of tDCS over the Sequential Visual Isometric Pinch Task (SVIPT), but with a low effect size. A recent metaanalysis concluded that the effects of a-tDCS over the nondominant M1 generally diminish after 24 h (Dissanayaka et al., 2017). Therefore, experimental studies are needed to define the stimulation protocols that produce the retention of the skill over sequence motor learning.

Hence, the aim of the present work was to evaluate whether previous musical training differentially impacts the effect of tDCS, and to evaluate the effects of tDCS on a motor sequencelearning task during and after stimulation. For this purpose, we applied a-tDCS over the right M1 during three consecutive sessions of performance of the SEQTAP and registered the skill index (SI) exhibited during the stimulation of musicians and non-musicians. In addition, we performed two follow-up measurements 20 min and 8 days after the stimulation in order to assess the effect of tDCS on the retention of the skill.

### MATERIALS AND METHODS

### Participants

Forty six healthy subjects participated in the study. The participants were all undergraduate students from the University of Almeria. Inclusion criteria were as follows: (1) right handedness; (2) no metallic implants on the head area; (3) no recent consumption of drugs or psychotropic medication; (4) no diagnosed psychopathology per the Diagnostic and statistical manual of mental disorders (5th Edition) (DSM-5); (5) no history of epilepsy, (6) naivety to the task and to tDCS, (7) and to score in the SEQTAP task between the 2nd and the 98th percentile. Participation was voluntary and academically rewarded. Two types of participants were recruited: musicians (M) and non-musicians (nM). Inclusion criteria for musicians were: (1) playing of a musical instrument at least once per week during a year; (2) the played instrument required the left hand; and (3) the instrument was practiced within the last 3 months. We counted in total with n = 19 musicians: seven piano players (5.14 ± 3.48 years of experience), nine guitar players (4.44 ± 4.18 years of experience), two drums players (4.00 ± 2.82 years of experience) and four saxophone players (10.50 ± 1.73 years of experience). Three out of 19 participants played two instruments.

Participants were randomly assigned to the tDCS or sham condition using Microsoft Excel software. After statistical analysis, four participants were detected as outliers: two participants scored above the 98th percentile and two participants scored below the 2nd percentile in one of the SEQTAP tests. Two participants presented missing data in at least one test. Therefore, the present study included 40 participants with an age range of 18–32 years (mean = 20.77 years, SD = 3.50), of which 70% were female, distributed into four groups: musicians-tDCS (M-tDCS) (n = 9) (19.36 ± 1.75 years old, 6 female), musicianssham (M-sham) (n = 10) (21.00 ± 3.26 years old, 5 female), non-musicians-tDCS (nM-tDCS) (n = 11) (22.10 ± 4.17 years old, 7 female) and non-musicians-sham (nM-sham) (n = 10) (20.77 ± 4.41 years old, 9 female).

Volunteers gave their informed consent to participate in the study, which was undertaken in accordance with the ethical standards of the World Medical Assembly (WMA) Declaration of Helsinki on the Ethical Principles for Medical Research Involving Humans. All personal information was handled under the Spanish personal data protection law of the 13th December 15/1999. The experimental procedure was approved by the Committee on Bioethics in Human Research (CIH) of the University of Almeria, Spain.

### Sequential Finger Tapping Task (SEQTAP)

The SEQTAP task is a commonly used motor sequence learning task in which the participant learns to type, as quickly and accurately as possible, a sequence of numbers with the nondominant hand. Participants respond to a series of five digits, ranging from 1 to 4 on a computer screen by pressing the corresponding button with the corresponding finger on a keyboard (Walker et al., 2002). We adapted the task from Saucedo-Marquez et al. (2013) with the following modifications: the time of each block was reduced from 40 to 20 s and the inter-block time was reduced from 20 to 10 s. Better performance in keyboard training has been found when training trials are distributed over time rather than blocked together (Baddeley and Longman, 1978). In addition, we attempted to reduce the fatigue of the participants by using a more distributed learning method. This structure permitted the participants to complete ˜2 blocks per minute over 20 min. The task was completed with the left (non-dominant) hand and was programmed with E-Prime Professional v. 2.0.8.74.

The measure registered during the SEQTAP task was the SI, obtained by dividing the percentage of correct sequences by the average time per trial (Cuypers et al., 2013):

> SI = % Correct Sequences mean respone time per 20s trial

### Transcranial Direct Current Stimulation (tDCS)

Transcranial direct current stimulation was administered with a Magstim DC-Stimulator Plus from neuroConn (Ilmenau, Germany) on the right M1 according to the 10–20 mm international EEG system. The selected area and both electrodes were soaked in physiological saline (∼20 ml per session). The excess of saline was eliminated with a clean dry towel. The anode (5 cm × 4 cm) was placed in the selected area (C4) by a tDCS cap and the cathode (5 cm × 4 cm) was placed in the contralateral trapeze and attached with hypoallergenic adhesive tape. Unicephalic stimulation was used, as outcome measures in motor sequence learning have reported no differences between unicephalic and bicephalic tDCS (Hashemirad et al., 2016). Moreover, the activity in the brainstem autonomic centers has not been shown to be modulated by the extracephalic location of the reference electrode (Vandermeeren et al., 2010). Stimulation was delivered at 2 mA (Iyer et al., 2005; Galea and Celnik, 2009), and thus applied with a current density of 0.10 mA/cm<sup>2</sup> , a level considered to be within the safe parameters (Iyer et al., 2005) which has previously shown to improve online performance gains and the offline maintenance for implicit motor sequence learning (Kantak et al., 2012; Sánchez-Kuhn et al., 2017). Stimulation was delivered during the complementation of the 3 SEQTAP sessions over 20 min (fade-in and fade-out of 30 s). In the sham condition, the stimulation lasted only for the first minute plus a fade-in

TABLE 1 | Shows the Mean (M) and Standard Deviation (SD) Skill Index scores (SI scores) obtained by each group in each test.


M-tDCS, Musicians tDCS; M-sham, Musicians sham; nM-tDCS, Non-musicians tDCS; nM-sham, non-musicians sham; BL, baseline test; S1, tDCS session 1; S2, tDCS session 2; S3, tDCS session 3; 20 min, 20 min follow-up test; 8th day, 8th day follow-up test; <sup>∗</sup>p < 0.05.

and fade-out of 30 s. Discomfort was monitored using verbal open-ended questions.

### Experimental Procedure

The experiment was conducted in an artificially lit room held at approximately 22◦C on a computer not connected to electricity for the duration of the experiment.

A previous pilot experiment was carried out on four participants: two males and two females. Possible side effects or causes of discomfort were noted. Mild itching was noted in one of four participants in the area of the electrode and seemed to be dependent on the volume of saline used. Therefore, the volume of saline for the sponge was adjusted to 20 ml per session, following Raw et al. (2016), as a higher volume of saline, but not oversaturated sponges, appear to be related to less discomfort (Woods et al., 2016). Hence, the excess of saline was dried to avoid dripping.

As depicted in **Figure 1**, each participant received tDCS or sham for three consecutive days during the performance of the SEQTAP in sessions 20 min in duration. The baseline test of the SEQTAP (1 min duration) was performed 5 min prior to session 1. Following session 3, follow-up tests were conducted 20 min and 8 days later (8th day follow-up test), each 3 min in duration. The experiment was conducted on afternoons between Monday and Thursday over 16 weeks.

### Statistical Analysis

In instances were data was normally distributed (Shapiro–Wilk test: p ≥ 0.05), we tested skewness and kurtosis to confirm a uniform distribution, and statistical differences were calculated by the Analysis of Variance (ANOVA) with Bonferroni post hoc. Effect size (η 2 p ) was calculated for all the significant results and interpreted following Cohen's classification: (0.1 – small size, 0.3 – medium size, and 0.5 – large size) (Cohen, 1973). If data was non-normally distributed, statistical differences were estimated using a Kruskal–Wallis and U Mann–Whitney test. Effect size (r) for U Mann-Whitney results was calculated for all the significant results, with the previously mentioned classification. Statistical significance was set up at p ≤ 0.05. Possible statistical differences among age and gender were assessed using Pearson's correlation (2-tailed). Analyses were executed using SPSS Version 24.0 software (IBM Corp, Armonk, NY, United States).

### RESULTS

The obtained results are displayed in **Table 1**.

In the baseline test (**Figure 2A**), results showed significant differences in the SI score between musicians and nonmusicians (0.023 ± 0.010 and 0.016 ± 0.010, respectively) [F (1, 38) = 11.755, p = 0.001; η 2 <sup>p</sup> = 0.24]. No previous differences

were found in the baseline test between the nM-tDCS group and the nM-sham group (U = 36, p = 0.49), nor between the M-tDCS group and the M-sham group (U = 29, p = 0.07).

Within the three sessions of tDCS results showed a group effect in session 1 [χ 2 (3) = 16.205, p = 0.001], session 2 [χ 2 (3) = 15.799, p = 0.001], and in session 3 [χ 2 (3) = 12.911, p = 0.001]. Specifically, in the non-musicians, analysis revealed statistically significant differences between the nM-tDCS group and the nM-sham group in session 1 (U = 16; p = 0.01; r = 0.42) and session 2 (U = 18; p = 0.02; r = 0.43). No statistically significant differences were seen in session 3 (U = 27, p = 0.15).

In the musicians, no statistically significant differences were found between the M-tDCS and the M-sham group in session 1 (U = 53, p = 0.91), session 2 (U = 52, p = 0.86), or session 3 (U = 52, p = 0.86). At the 20-min follow-up test (**Figure 2B**), there was a group effect [F(3, 36) = 4.839, p = 0.001], and post hoc analysis revealed significant differences between the nM-tDCS group and the nM-sham (p = 0.03; η 2 <sup>p</sup> = 0.20), but not between the M-tDCS group and the M-sham group (p = 1.00).

At the 8th day follow-up test (**Figure 2C**), there was a group effect [χ 2 (3) = 15.799, p = 0.001]. Specifically, there were statistically significant differences between the nM-tDCS group and the nM-sham group (U = 18, p = 0.02; r = 0.88). However, there were no significant differences between the M-tDCS group and the M-sham group (U = 52, p = 0.863).

There was a significant negative correlation between age and the SI scores of the baseline test (r = –0.494; n = 40; p = 0.001). No other significant correlation was found between age or gender and the SI scores of the baseline test, the tDCS sessions, the TABLE 2 | Shows the number of participants of each group that reported sensations during the stimulation.


Itch, Mild itching at the beginning of the stimulation (˜1 min); Heat, Heat feeling in the electrode placement.

20-min follow-up or the 8th day follow-up test. Five of 40 participants reported fatigue due to performance of the task in at least one test. The perceived sensations during the tDCS stimulation are reported in **Table 2**.

### DISCUSSION

The aim of the present study was to explore the effect of three sessions of tDCS over the motor cortex of musicians and nonmusicians during the performance of a motor learning task. Performance was also assessed in follow-up tests of 20 min and 8 days.

In the non-musicians, we found an effect of tDCS at the first and second session of stimulation, in the 20 min and in the 8th day follow-up test. In the musicians, we found no effect of tDCS in any of the tests. Therefore, results show that tDCS may help in

the learning and maintenance of newly acquired motor abilities but that the effect is dependent on previous musical training.

Musicians showed better performance in the SEQTAP compared to non-musicians in the baseline test. It might be possible that previous musical training shapes brain structures (Gaser and Schlaug, 2003a; Hyde et al., 2009), the myelinisation (Bengtsson et al., 2005; Steele et al., 2013) and consequently the behavioral outcomes (Xie et al., 2013) due to plasticity processes. Therefore, the present results confirm the condition of previous musical training as a suitable human model for studying individual differences among motor processes in healthy subjects (Schlaug, 2015).

During the three sessions of SEQTAP in which tDCS was administered, the principal effect of the stimulation was found in the nM group, as the nM-tDCS group scored significantly higher than the nM-sham group. This effect was seen in the first and in the second session, pointing toward a higher effect of tDCS at the beginning of the learning process. Ehsani et al. (2016) found similar effects of tDCS applied over M1, where the principal effect of tDCS was seen over the first blocks of a motor training task during a single stimulation session. Accordingly, Antal et al. (2004) found positive effects of tDCS on M1 in the early learning phase of a visuo-motor task. The present results support studies in which anodal stimulation strengthens newly formed associations (Orban de Xivry and Shadmehr, 2014), which seem to be primarily influenced by changes in membrane potential and GABAergic neurotransmission via interneurons of the neuronal network existing at a given time point (Nitsche et al., 2004, 2008). Previous studies have also suggested that the effects of tDCS are reduced as participants gain expertise, since participants may utilize different brain networks after the acquisition of the new task than before (Bullard et al., 2011; Clark et al., 2012). Thus, in order to optimize effectiveness in training progress, future studies should apply tDCS over different brain locations at different times during training.

The nM-tDCS group showed benefit from tDCS at the 20 min follow-up test and the 8th day follow-up. The results are in accordance with previous reports suggesting a role of M1 in the retention of newly acquired motor memories (Muellbacher et al., 2002; Hadipour-Niktarash et al., 2007; Galea and Celnik, 2009; Hunter et al., 2009). According to this assumption, the application of multiple sessions of tDCS over M1, compared to a single session tDCS, has been shown to induce significant changes in behavioral outcomes in the SEQTAP, particularly post-intervention (Hashemirad et al., 2016). This consolidation of newly acquired motor abilities in healthy participants over time is considered to be dependent on alterations in membrane potential and synaptic plasticity, specifically in glutamate and GABA signaling (Stagg and Nitsche, 2011). In contrast, Saucedo-Marquez et al. (2013) found no effects of tDCS in the SEQTAP retention test. The most notable difference between that study and the present work was the change in the length of the blocks in the SEQTAP, which were reduced from 40 to 20 s of tapping and 20 to 10 s of resting. These modifications could have reduced the fatigue of the participants, as only five of 40 participants reported fatigue in one or more sessions. Moreover, previous studies have shown better performance in keyboard training when training trials were distributed over time rather than grouped together (Baddeley and Longman, 1978), as spaced learning protocols might yield to better outcomes than massed ones (Smolen et al., 2016).

The fact that tDCS did not produce any differences between the stimulated and the sham group in the musicians supports the idea that inter-individual factors can vary the responses to tDCS (Li et al., 2015). One of the factors that could predict the effect of tDCS might be the baseline performance. The initial motor function state of the participant can have a meaningful impact on the effects of tDCS, as previous studies have shown that participants with poorer selective muscle activation improved more after the stimulation of tDCS on M1 (Uehara et al., 2015). In addition, Ciechanski et al. (2017) found a greater effect of tDCS over neurosurgical skill acquisition in low, rather than in high-skill trainees. These findings can be extended also to other study fields, such as lateralised visual detection task sensitivity. For instance, in a study carried out by Learmonth et al., participants were divided into "poor performers" (lower d') and "good performers" (higher d'). tDCS was applied to the left posterior parietal cortex (PPC) and only poor performers got benefitted from the stimulation (Learmonth et al., 2015). A further study involving musicians found that novice jazz players' musical performance was enhanced by tDCS applied over the right dorsolateral prefrontal cortex (r-DLPFC), while experienced jazz players' musical performance was unchanged and even deteriorated following stimulation (Rosen et al., 2016). A High-Definition tDCS (HD-tDCS) study also reported a baseline-dependent effect of stimulation, showing greater benefit for those participants with poorer baseline scores (Shen et al., 2016). The fact that participants with a higher level of baseline performance experience less benefit from tDCS should be taken into account in future tDCS studies. Future works should include other populations with high motor capacities such as gamers, who have been shown to have signs of brain reorganization due to motor expertise similar to professional musicians (Granek et al., 2010).

The ineffectiveness of tDCS over motor capacity in musicians has also been reported by Furuya et al. (2013), who found no apparent improvement in the fine control of finger movements in professional pianists, while untrained individuals benefited from tDCS. This finding has two possible explanations. One reason for this unsuccessful effect of tDCS has been attributed by previous studies to a ceiling effect displayed by highperforming musicians (Furuya et al., 2014). This explanation is not plausible for the present study, as the musician group show an improved performance over the three sessions. However, neuroimaging studies offer a second possible explanation for this finding: functional magnetic resonance imaging (fMRI) in professional piano players during complex finger movement task training showed significantly lower activation clusters in M1, supplementary motor area, premotor cortex, and superior parietal lobule when compared to a control group (Krings et al., 2000). Therefore, it might be possible that during motor learning, professional musicians display a reduced level of brain activity in areas required for the control of basic movement (Koeneke et al., 2004; Granek et al., 2010; Wright et al., 2012). In addition,

neuroimaging studies have pointed toward a larger use of other brain areas by musicians during learning and memorization, such as the superior parietal cortex, the supramarginal gyrus, and the cerebellum (Lotze et al., 2003; Stewart et al., 2003). These brain areas might be addressed in future clinical studies of tDCS involving musicians.

The limitations of the study are mostly related to the sample size and sample heterogeneity. However, in the present study, each participant was requested to assist four different days to the laboratory, which means a high time cost for the participant and made difficult the recruitment of the sample. One of the most remarkable limitations is the predominance of women in the composition of the subgroups. Although no differences were found in gender in the present study, future tDCS studies should address an equal gender distribution, as tDCS has shown to be sensible to individual differences such as this (Chew et al., 2015). Aiming at obtain a higher size effect, the present study needs further replication counting with a larger sample.

### CONCLUSION

In conclusion, the present study showed that tDCS applied over the right M1 had a positive effect on motor learning in healthy non-musician participants, enhancing motor performance at the first and second session of tDCS, showing a maintenance of this effect 20 min and 8 days after the intervention. Therefore, the current experiment offers a protocol that allows the study of both

### REFERENCES


online and offline effects of tDCS among healthy participants. These results enhance tDCS as a complementary technique for motor neurorehabilitation, particularly for its long-term potentiation effects. However, the beneficial effects of tDCS were not observable in musically trained participants. Consequently, it is plausible to conclude that tDCS on M1 has different effects depending on previous motor experiences, which highlights the importance of individual differences when considering the effects of tDCS.

### AUTHOR CONTRIBUTIONS

AS-K performed the collection, analysis, and interpretation of the data and drafted the paper. CP-F and MM reviewed the paper and provided critical intellectual output to the work. FS-S and PF performed the main conception and design of the work, interpretation of data, and reviewed and commented the paper. All authors read and approved the final version to be published.

### FUNDING

This work was supported by the grants from the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund (MINECO-FEDER) (Grants Nos. PSI2014- 55785-C2-1-R, PSI2015-70037-R, and PSI2017-86847-C2-1-R).




**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer RR and handling Editor declared their shared affiliation at the time of the review.

Copyright © 2018 Sánchez-Kuhn, Pérez-Fernández, Moreno, Flores and Sánchez-Santed. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# High Entrepreneurship, Leadership, and Professionalism (HELP): A New Resource for Workers in the 21st Century

### Letizia Palazzeschi, Ornella Bucci and Annamaria Di Fabio\*

Department of Education and Psychology, Psychology Section, University of Florence, Florence, Italy

World of work in the 21st century is characterized by instability, insecurity, and continuous change. To face these challenges of the post-modern era, workers are required to use their personal resources. A new construct called high entrepreneurship, leadership, and professionalism (HELP) is a preventive resource that helps maintain, improve, and find work in uncertain or dynamic conditions. This study aims to examine the personality correlates of HELP in Italian workers and identify different clusters based on HELP and other variables, such as workplace relational civility and flourishing. To this end, the following instruments were administered to 204 Italian workers: the HELP questionnaire, the Big Five Questionnaire, the Workplace Relational Civility Scale, and the Flourishing Scale. The personality correlates of HELP underscored the role of conscientiousness (and its subdimension perseverance) and extraversion (and its subdimension dominance). The cluster analysis identified three clusters characterized by high, average, and low HELP scores. Participants in the first cluster with high HELP scores appeared to possess higher perseverance, dominance, workplace relational civility, especially readiness, and higher flourishing than those in the other two groups. The present results can open new opportunities for future research and interventions in a primary prevention perspective to foster resources for workers and healthy organizations in the 21st century.

Keywords: entrepreneurship, leadership, professionalism, personality traits, workplace relational civility, flourishing, entrepreneurs, workers

### INTRODUCTION

World of work in the 21st century is characterized by instability, insecurity, and continuous change, and navigating this complex work scenario requires workers to develop their personal resources (Buunk et al., 2007; Peiró et al., 2010; Silla et al., 2010; Savickas, 2011; Guichard, 2013; Di Fabio and Kenny, 2016a,b; De la Fuente et al., 2017a,b). A new construct, high entrepreneurship, leadership and professionalism (HELP; Di Fabio et al., 2016) is considered a promising resource to actively construct career paths and help workers negotiate the challenges of the present liquid society (Bauman, 2000). Entrepreneurship, leadership, and professionalism have emerged as core constructs, crucial in the 21st century (Di Fabio et al., 2016). Although they have been studied separately in the past, only recently have they been examined as an integrated construct (Di Fabio et al., 2016) that reflects aspects of motivation, intention, and efficacy. Di Fabio et al. (2016) have developed a theoretically and empirically integrated framework for studying HELP.

#### Edited by:

Douglas F. Kauffman, Independent Researcher, Bedford, United States

#### Reviewed by:

Rita Chiesa, Università degli Studi di Bologna, Italy Michela Cortini, Università degli Studi G. d'Annunzio Chieti e Pescara, Italy

> \*Correspondence: Annamaria Di Fabio adifabio@psico.unifi.it

#### Specialty section:

This article was submitted to Organizational Psychology, a section of the journal Frontiers in Psychology

Received: 14 March 2018 Accepted: 26 July 2018 Published: 28 August 2018

#### Citation:

Palazzeschi L, Bucci O and Di Fabio A (2018) High Entrepreneurship, Leadership, and Professionalism (HELP): A New Resource for Workers in the 21st Century. Front. Psychol. 9:1480. doi: 10.3389/fpsyg.2018.01480

Entrepreneurship has been recently defined as "a process that evolves over time and includes different phases from forming an intention, starting-up, scaling-up, stabilizing, and managing the business, exit and potential re-entry" (Gorgievski and Stephan, 2016, p. 440). It is a highly valued concept in the 21st century because it can help create job opportunities and boost productivity and economic growth (Van Praag and Versloot, 2008). There are two approaches to defining entrepreneurship in the psychological literature (Gorgievski and Stephan, 2016): the first considers entrepreneurs as an occupational category that includes individuals who are self-employed and manage their own business; the second regards entrepreneurial action and processes as those that are implied in the individuation, construction, and implementation of opportunities (Shane and Venkataraman, 2000; Davidsson, 2015, 2016). The most recent psychological perspective on entrepreneurship (Gorgievski and Stephan, 2016) identifies three dimensions. The first dimension refers to within-individual processes, between-individual differences, and interactions between individual entrepreneurs and their immediate (teams, units, and organizations) and wider contexts (regions or countries). The second dimension refers to multi-level processes across different phases of the entrepreneurial process, and the third dimension focuses on different kinds of businesses in terms of the entrepreneur's identity, goals, and start-up motivations.

Leadership is related to the broader theme of human resource management (Hitt and Duane, 2002; Peiró and Rodríguez, 2008; Monzani et al., 2015). It regards the influence of group activities to achieve an objective (Rauch and Behling, 1984; House et al., 1999; Boyatzis, 2008; Boyatzis et al., 2015). Leadership has traditionally been considered a process of influence between the leader and group members toward achieving group aims (Hollander, 1992). Two well-known leadership styles are transactional and transformational (Burns, 1978; Bass, 1985). Transactional leadership is characterized by the leader demonstrating initiative to connect with the group members in their work context for the exchange of resources. On the other hand, transformational leadership involves changes in the beliefs, needs, and values of the collaborators.

Most recently, new leadership styles have been proposed. Sustainable leadership aims to avoid social and environmental damage (Hargreaves and Fink, 2003, 2004); servant leadership focuses on the personal growth of the subordinates (Ehrhart, 2004); benevolent leadership involves leaders treating followers as family members, showing concern for their well-being in both the work domain and private life (Cheng et al., 2004; Wang and Cheng, 2010); authentic leadership uses a transparent and ethical leadership style, emphasizing people's strengths rather than their weaknesses (Avolio et al., 2009); ethical leadership entails the pursuit of the right aims and focuses on empowering the organization's members (Gallagher and Tschudin, 2010); and mindful leadership involves paying attention to the present moment, recognizing feelings and emotions, and keeping them under control, especially under stress (George, 2012). Entrepreneurial leadership is the basis of HELP construct. It considers the ability to influence others for using resources in a strategic manner with the aim of promoting behaviors that seek opportunities and advantages (Ireland et al., 2003), including setting clear goals, creating opportunities, empowering people, and promoting mutual and organizational awareness (Cunningham and Lischeron, 1991).

Professionalism is the third aspect of HELP, after entrepreneurship and leadership (Di Fabio et al., 2016). It is traditionally defined as "an ongoing process through which an individual derives a cohesive sense of professional identity by integrating the broad-based knowledge, skills, and attitudes within psychology with one's values and interests" (Ducheny et al., 1997, p. 29). Professionalism is particularly relevant to entrepreneurial leadership, which refers to the ability to envision strategic scenarios that can facilitate the identification and implementation of value creation (Gupta et al., 2004).

In constructing this framework, Kanter's (1989) theory focused on the link between careers and economic, social, and political issues. Chan et al.'s (2012) framework adopted a person-centered perspective that highlighted subjective careers (Di Fabio et al., 2016). According to Kanter (1989) and Chan et al.'s (2012), in an unstable work environment, it is possible to recognize entrepreneurship, leadership, and professionalism (ELP) as three fundamental dimensions of a subjective career space. While Kanter (1989) maintained that these constructs are distinct, Chan et al. (2012) examined the motivation, intentions, and efficacy of each of these three different constructs. Subsequently, Di Fabio et al. (2016) proposed an integrated model where entrepreneurship, leadership, and professionalism were integrated into the HELP construct, along with the three aspects of motivation, intentions, and efficacy for each aspect. This integrated construct, HELP, seems particularly promising not only because it introduces a new preventive integrated perspective, but also because it is an increasable resource differently from personality traits that are considered substantially stable in the literature (Palazzeschi et al., 2018). From a primary preventive perspective (Hage et al., 2007; Kenny and Hage, 2009; Di Fabio and Kenny, 2016a), HELP can be seen as a resource for workers to manage the complex career challenges of the 21st century (Di Fabio et al., 2016). HELP can also promote healthy organizations (Lowe, 2010; Tetrick and Peiró, 2012; Di Fabio, 2016, 2017a; Di Fabio et al., 2016) where individuals have the resources to cope with the continuously changing work sphere through proactive and innovative solutions, which result in greater well-being.

### Aims and Hypotheses

The present study attempts to examine the personality correlates of HELP in Italian workers and identify different clusters based on HELP and other variables, such as workplace relational civility and flourishing, that are interesting variables in the framework of positive healthy organizations.

The three variables of entrepreneurship, leadership, and professionalism in relation to Big Five personality traits are examined in the literature considering them as separate constructs. We examined recent studies and reviews of the relationships between entrepreneurship and the Big Five personality traits. One meta-analysis that considered studies conducted between 1990 and 2010 pointed to the presence of

positive relationships between entrepreneurs' performance and Conscientiousness, Openness, Extraversion, and a negative relationship between entrepreneurs' performance and Neuroticism (Brandstätter, 2011). In Slovenian context, Antoncic et al. (2015) showed that entrepreneurship in terms of firm start-up activities is strongly and positively associated with Openness, less so with Extraversion, and inversely associated with Agreeableness.

Bono and Judge's (2004) meta-analysis examined the relationships between the Big Five personality traits and transformational leadership. They examined 384 correlations from 26 independent studies and found the strongest positive correlation between transformational leadership and Extraversion, and a negative correlation between transformational leadership and Neuroticism. Authentic leadership was positively correlated with Conscientiousness and Openness to experience (Komariah, 2016). Furthermore, ethical leadership was inversely related to Neuroticism and positively to Openness to experience, Agreeableness, and Conscientiousness. Notably, ethical leadership was not correlated with Extraversion (Özbag, 2016 ˘ ).

Extant research on the relationship between professionalism and personality traits is limited. One study showed a positive correlation between the professionalism of physician-assistant students and Conscientiousness (Moser and Dereczyk, 2012). A more recent study on anesthetist trainees also highlighted the association between professionalism and Conscientiousness (Sawdon et al., 2017).

In summary, previous research has regularly found a relationship between Extraversion, one of the Big Five traits and entrepreneurship, while Conscientiousness has been linked to leadership styles and professionalism. The other three Big Five personality traits—Agreeableness, Neuroticism, and Openness were not conclusively linked with any of the three HELP constructs. Further, in a recent Italian study (Palazzeschi et al., 2018), involving workers of different public and private organizations, Conscientiousness emerged as the best personality correlate of HELP, followed by Extraversion.

Workplace relational civility (WRC, Di Fabio and Gori, 2016a) is a new kind of relational style in the workplace "characterized by respect and concern for oneself and others, interpersonal sensitivity, personal education, and kindness toward others. It also includes civil behaviors such as treating others with dignity and respecting social norms to facilitate peaceful and productive cohabitation" (Di Fabio and Gori, 2016a, p. 2). The WRC construct comprises three dimensions: (1) relational decency at work, which refers to decency-based relationships characterized by respect for oneself and others, assertiveness, ability to express beliefs and opinion, and relational capacity; (2) relational culture at work, which refers to politeness, kindness, good education, courteousness; and (3) relational readiness at work, which refers to sensibility toward others (speed in understanding the feelings of others and exhibiting proactive sensibility), ability to understand the emotions of others, concerns for others, delicacy, attention to the responses of others, empathy, and compassion. It is important to emphasize that WRC can be evaluated using a "mirror" scale of measurement—the Workplace Relational Civility Scale (Di Fabio and Gori, 2016a). Participants are first asked to indicate their relationship with others within a given period and then evaluate others' relationships with them. This modality helps them recognize their self-importance in the process. WRC is an important positive variable that facilitates early intervention at the workplace. It is especially significant and innovative as previous studies have only examined the negative aspects of workplace incivility (Cortina et al., 2001).

Flourishing is an important positive variable that is a comprehensive measure of well-being, vital to workers in the 21st century. It is a form of eudaimonic well-being, perceived as success in relationships, purpose, and future optimism (Diener et al., 2010).

This study attempted to replicate an earlier study that examined the personality correlates of HELP among Italian workers in care organizations. On the basis of the previously described framework, the following hypotheses were formulated for this study.

Ha1: Among the Big Five personality traits, Conscientiousness will be most highly and positively correlated to HELP.

Ha2: Positive correlations will emerge between HELP and Extraversion.

Ha3: Agreeableness, Neuroticism, and Openness will not significantly correlate with HELP.

The second aim of the present study was to investigate the relationship of HELP also with workplace relational civility and flourishing, examining whether participants clustered in meaningful ways based upon HELP scores and personality traits, workplace relational civility, flourishing. Given the exploratory nature of this investigation, no specific hypotheses were advanced.

## MATERIALS AND METHODS

### Participants

The study participants comprised 204 Italian workers employed in care organizations in the Tuscany region (female = 63.71%, male = 36.29%; mean age = 40.36 years, SD = 13.00).

### Measures

### High Entrepreneurship, Leadership, Professionalism Questionnaire (HELP-Q)

The HELP-Q (Di Fabio et al., 2016) is a 9-item integrated scale that identifies entrepreneurship (E), leadership (L), and professionalism (P) in terms of motivations, intentions, and efficacy. The nine items—three for each area—are scored using a 5-point Likert-type scale (1 = not at all, 2 = a little, 3 = somewhat, 4 = much, 5 = a great deal). Examples of items include "To what extent is it important for me to look for new ideas on how to make a profit for entrepreneurship?" (Entrepreneurship). "To what extent is it important for me to become a leader or a manager?" (Leadership). "To what extent is it important for me to excel in my chosen area of study/work?" (Professionalism). Cronbach's alpha coefficients were 0.92 for entrepreneurship, 0.92 for leadership, 0.90 for professionalism, and 0.77 for the total score.

#### Big Five Questionnaire (BFQ)

fpsyg-09-01480 August 27, 2018 Time: 10:24 # 4

The BFQ (Caprara et al., 1993) comprises 132 items scored on a 5-point Likert scale, ranging from 1 = Absolutely false to 5 = Absolutely true. Cronbach's alpha coefficients for the five personality traits were: 0.81 for Extraversion (e.g., "I think that I am an active and vigorous person"); 0.73 for Agreeableness (e.g., "I understand when people need my help"); 0.81 for Conscientiousness (e.g., "I tend to be very thoughtful"); 0.90 for Emotional stability (e.g., "I do not often feel tense"); 0.75 for Openness (e.g., "I am always informed about what is happening in the world").

### Workplace Relational Civility Scale (WRCS)

The WRCS is a 26-item self-report mirror instrument (Di Fabio and Gori, 2016a), covering three dimensions: relational readiness (RR), relational culture (RCu), and relational decency (RD) at work. The sum of these dimensions gives an overall score of WRC as well as a score for part A and B of the WRCS. Part A is the analysis of an individual's self-perception as pertaining to a particular issue (e.g., "I was able to express my values and beliefs calmly to others"), and part B is the analysis of an individual's perception of others on the same issue (e.g., "Others were able to express their values and beliefs calmly to me"). The participants were asked to describe their general relationship with others during 3 months prior to the administration and then describe their perception of others' general relationship with them over the same period. Items were scored on a 5-point Likert scale (1 = not at all, 2 = a little, 3 = somewhat, 4 = much, 5 = a great deal). Cronbach's alpha coefficients for the three dimensions within Part A were as follows: Factor RR (α = 0.83), Factor RCu (α = 0.76), and Factor RD (α = 0.75). Cronbach's alphas for Part B were as follows: Factor RR (α = 0.86), Factor RCu (α = 0.88) and Factor RD (α = 0.85). The Cronbach's alpha coefficients for the total scores for Part A and Part B were α = 0.87 and α = 0.92, respectively.

### Flourishing Scale (FS)

The Italian version (Di Fabio, 2016) of the FS (Diener et al., 2010) was used to evaluate flourishing as a measure of eudaimonic well-being. The scale comprises eight items scored on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). Some examples of the items are as follows, "My social relationships are supportive and rewarding," "I lead a purposeful and meaningful life," "I am optimistic about my future." The FS has a unidimensional structure and good reliability (α = 0.88).

### Procedure and Data Analysis

Questionnaires were administered by trained psychologists to participants in groups. The order of administering was counterbalanced to control possible effects of a fixed order of presenting the questionnaires. The study assured to respondents anonymity and confidentiality. The questionnaire included a statement regarding the personal data treatment, in accordance with the Italian privacy law (Law Decree DL-196/2003). The workers authorized and approved the use of anonymous/collective data for possible future scientific publications. Because the data was collected anonymously and the research investigated psycho-social variables not adopting a medical perspective, ethical approval was not sought.

Descriptive statistics and Pearson's correlations were calculated. A cluster analysis (k-mean method) was also performed, and ANOVAs using Bonferroni post hoc tests were conducted.

Cluster analysis was carried out to individuate different groups on the basis of the different aspects of HELP and, therefore, to differentiate the groups with respect to the variables considered: personality traits, workplace relational civility, and flourishing. Specifically, we are interested in differentiating groups on the basis of combinations of scores on the three aspects of HELP and to analyze how they are different with respect to the studied variables. Cluster analysis identifies groups or types of individuals who share particular attributes or relations among attributes according to a person-center approach to scientific research (Bergman et al., 2003; Magnusson, 2003).

### RESULTS

The results identified two personality correlates of HELP: Conscientiousness, (and its subdimension Perseverance) and Extraversion, (and its subdimension Dominance) (see **Tables 1**, **2**).

Correlations among the dimensions of HELP are reported in **Table 3**.

Three groups were identified from the cluster analysis of HELP scores. Participants in Cluster 1, with high HELP scores, also had higher mean scores on each of the HELP aspect—entrepreneurship, leadership, and professionalism than participants in Clusters 2 and 3, with average and low HELP scores, respectively (see **Table 4**).

ANOVAs relative to personality traits, WRC, and flourishing showed significant differences in terms of external descriptor variables (see **Table 5**).

Cluster 1 participants appeared to have greater Conscientiousness and Extraversion, with particularly high scores in the Perseverance and Dominance subdimensions. Further, they had a higher perception of WRC, especially


HELP = High Entrepreneurship, Leadership, and Professionalism; BFQ = Big Five Questionnaire. <sup>∗</sup>p < 0.05; ∗∗p < 0.01.


TABLE 2 | Correlations between HELP and Big Five Sub-Dimension Scores (N = 204).

HELP = High Entrepreneurship, Leadership, and Professionalism; BFQ = Big Five Questionnaire. <sup>∗</sup>p < 0.05; ∗∗p < 0.01.

TABLE 3 | Correlations among the dimensions of HELP.


HELP = High Entrepreneurship, Leadership, and Professionalism; ∗∗p < 0.01.

readiness, and higher flourishing scores than the two other clusters. Cluster 2 participants had intermediate scores—in a range between those of participants from Clusters 1 and 3—in all dimensions. Finally, Cluster 3 participants scored the lowest on all dimensions.

### DISCUSSION

This study attempted to examine the personality correlates of HELP among Italian workers and identify different clusters of individuals on the basis of HELP and other variables important for fostering positive healthy organizations such as WRC, and flourishing. Regarding the first aim relative to the relationship between HELP and personality traits, the present study showed that individuals with higher HELP scores were more conscientious and also extraverts, confirming also on Italian workers in care organizations the results of Palazzeschi et al. (2018) study. Conscientiousness is a personality trait that has traditionally been associated with two aspects of the integrated HELP construct: leadership and professionalism. On the other hand, extraversion has been associated with entrepreneurship, which lends support to the significant results for conscientiousness and extraversion in this study. Further, with regard to the Big Five subdimensions, we found that individuals with higher HELP scores were more perseverant and dominant, which is consistent with previous findings (Palazzeschi et al., 2018). No significant correlations were observed for Agreeableness, Neuroticism, and Openness, which supports our third hypothesis and is consistent with the previous study in the Italian context (Palazzeschi et al., 2018). Thus, the HELP construct appears to call on the personality traits of Conscientiousness—in terms of Perseverance—in pursuing one's goals and Extraversion—in terms of dominance—which reflects the ability to influence, guide, and manage other people (Caprara et al., 1993; Di Fabio et al., 2016). These personality characteristics seem to be important for constructing the career paths of workers in the 21st century.

Regarding the second aim of the present study to investigate the relationship between HELP, and also WRC, and flourishing, the analysis identified the presence of three clusters. Cluster 1 participants had greater Conscientiousness and Extraversion (in terms of Perseverance and Dominance, respectively), higher WRC, particularly reflected in readiness (sensibility, ability to understand the emotions of others, concern for others, delicacy, attention to the responses of others, empathy, and compassion; Di Fabio et al., 2016), and higher flourishing in terms of perceived success in relationships, purpose, and future optimism (Diener et al., 2010). These results suggest that people with higher HELP levels are more conscientious (perseverant) and extroverted (dominant). They also pay more attention to others, which is expressed as a readiness to listen to others, and experience a form of well-being related to perceived success in relationships, a sense of purpose, and optimism. Thus, HELP represents a promising preventive resource in the 21st century and in its unpredictable world of work (Di Fabio and Palazzeschi, 2016) both for workers and healthy organizations (Lowe, 2010; Tetrick and Peiró, 2012; Di Fabio, 2016, 2017a,b; Di Fabio et al., 2016). It helps entrepreneurs and workers to proactively and innovatively manage these post-modern challenges (Di Fabio et al., 2017) while also considering aspects of WRC and flourishing.

Although this study examines the integrated construct of HELP and its personality correlates in depth and identifies different clusters of individuals on the basis of HELP scores, it is necessary to highlight some limitations of this study, particularly in relation to the characteristics of the participants. The participants were not representative of the national population because all the participants were from the Tuscany region of Italy. Future research could extend this study by including participants from different parts of Italy and from different organizations. This study can also be replicated in other international contexts. Future research could also

TABLE 4 | Three Emergent Clusters according to Overall HELP Scores and its Dimensions.


HELP = High Entrepreneurship, Leadership, and Professionalism. a = Cluster 1, b = Cluster 2, c = Cluster 3, ∗∗∗p < 0.001. Bonferroni post hoc tests.

TABLE 5 | ANOVAs for Cluster 1, Cluster 2, and Cluster 3 relative to HELP Scores, Personality Traits, WRC, and Flourishing.


HELP = High Entrepreneurship, Leadership, and Professionalism; BFQ = Big Five Questionnaire, WRC = Workplace Relational Civility. a = Cluster 1, b = Cluster 2, c = Cluster 3, ∗∗∗p < 0.001. Bonferroni post hoc tests.

continue to study in depth the relationships between the new integrated construct of HELP and personality traits to see if these results will be confirmed or will emerge specificities in relation to different categories and contexts. Notwithstanding the highlighted limitations, the results of this work offers new opportunities for future research and interventions from a primary prevention perspective (Hage et al., 2007; Kenny and Hage, 2009; Di Fabio and Kenny, 2016b; Di Fabio, 2017a) for entrepreneurs, workers, and healthy organizations in the 21st century (Tetrick and Peiró, 2012; Di Fabio and Palazzeschi, 2012; Di Fabio et al., 2016; Di Fabio and Kenny, 2016a; Di Fabio and Gori, 2016b; Dempsey and Kauffman, 2017; Di Fabio and Peiró, 2018). HELP can be considered as preventive individual resources to successfully face with the challenges of the current continuous changing labor market. The HELP highlights the value of personal entrepreneurship, leadership and professionalism as an integrated preventive core of characteristics that can adaptively build one's one personal and professional paths. The HELP can also prevent possible career decision-making problems or failures rather than concentrating just on remediation. Furthermore the HELP calls for early actions to enhance personal resources in a primary prevention perspective (Hage et al., 2007; Kenny and Hage, 2009; Di Fabio and Palazzeschi, 2015; Di Fabio and Kenny, 2016b) to help individuals prevent future career and life problems.

The integrated construct of HELP appears to be promising resource as it can be developed and it facilitates early intervention, unlike personality traits that are considered to be stable. HELP represents a new integrated resource in a preventive perspective because the aspects it covers can be cultivated through early training. A key advantage of this resource is that the measurement scale is brief and easy to administer. The unstable and unpredictable world of work calls for new preventive resources to cope with multiple, differing challenges. In this framework, HELP can be considered an opportunity to find, maintain, and improve

### REFERENCES


work in the 21st century (Bauman, 2000; Di Fabio et al., 2016).

### AUTHOR CONTRIBUTIONS

LP and ADF conceptualized the study, chose the theoretical framework and the measures, analyzed the data, and wrote the methods and results. OB helped in the collection of the data. All authors wrote the paper together and read and revised the manuscript several times.

resilience, learning approaches, and coping strategies to predict achievement in undergraduate students. Front. Psychol. 8:1039. doi: 10.3389/fpsyg.2017.01039



**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Palazzeschi, Bucci and Di Fabio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Moon-Ho Ringo Ho1 \*, Marilyn A. Uy <sup>2</sup> , Bianca N. Y. Kang1 and Kim-Yin Chan2*

*1School of Social Sciences, Nanyang Technological University, Singapore, Singapore, 2Nanyang Business School, Nanyang Technological University, Singapore, Singapore*

Our study focuses on the impact of systematic entrepreneurship training comprising both active and passive learning activities on entrepreneurial alertness and efficacy among adolescent youth. Reports from a two-wave online survey among 328 students from five secondary schools (aged 13–16 years; 34.8% male and 65.2% female) reveal that those who went through entrepreneurship training (treatment group, *N* = 142) had significantly higher entrepreneurial alertness and efficacy levels compared to those who did not go through training (control group, *N* = 186). We also find that even with gender effects accounted for, the higher entrepreneurial alertness and efficacy levels in the treatment group are due in part to both passive and active/hands-on elements of the program. Our study offers direct evidence that conducting entrepreneurship training programs among secondary school students could be an effective means to enhance entrepreneurial competencies among the youth. Specifically, our findings highlight the value of entrepreneurial training in improving age-appropriate competencies of entrepreneurial alertness and efficacy.

Keywords: entrepreneurship, entrepreneurial training, entrepreneurial alertness, entrepreneurial efficacy, youth

### INTRODUCTION

Entrepreneurship plays a critical role in boosting economic growth and development (Wong et al., 2005). As such, policymakers have been focusing their efforts on entrepreneurship promotion. Promoting entrepreneurship goes beyond assisting incumbent entrepreneurs and business owners; it also encompasses inculcating an enterprising spirit among young people because adolescents are the source of the next wave of entrepreneurs. Entrepreneurship training, which aims to equip participants with relevant knowledge and skills (Katz, 2007), is regarded as a practical means to promote entrepreneurship among young people (Peterman and Kennedy, 2003).

Meta-analytic findings indicate that entrepreneurship training is effective in promoting cognitive and motivational outcomes resulting in more start-ups (Martin et al., 2013). However, our theoretical understanding of how and why entrepreneurship training exerts a positive influence on entrepreneurial competencies is still lacking (Martin et al., 2013). Specifically, our knowledge regarding designing and improving training to promote entrepreneurship effectively is somewhat surprisingly limited (Gielnik et al., 2015). Importantly, according to the Global Entrepreneurship Monitor's special report on entrepreneurship training, we need more studies on entrepreneurship training particularly on whether and in what specific ways training makes a difference (Martinez et al., 2010).

#### *Edited by:*

*Jesus de la Fuente, University of Almería, Spain*

#### *Reviewed by:*

*Jesús Nicasio García Sánchez, Universidad de León, Spain Angela Jocelyn Fawcett, Swansea University, United Kingdom*

> *\*Correspondence: Moon-Ho Ringo Ho homh@ntu.edu.sg*

#### *Specialty section:*

*This article was submitted to Educational Psychology, a section of the journal Frontiers in Education*

*Received: 31 October 2017 Accepted: 15 February 2018 Published: 05 March 2018*

#### *Citation:*

*Ho M-HR, Uy MA, Kang BNY and Chan K-Y (2018) Impact of Entrepreneurship Training on Entrepreneurial Efficacy and Alertness among Adolescent Youth. Front. Educ. 3:13. doi: 10.3389/feduc.2018.00013*

As a tangible response to the various calls for more systematic studies on entrepreneurship training, our study aims to validate the value of promoting entrepreneurship training among adolescents, and specifically to examine whether activities that involve passive learning (e.g., classroom lessons, assembly talks, visits to firms) and those that involve more active, experiential learning (e.g., attachments and internships, product/prototype creation, learning from a mentor, etc.) differentially improve important entrepreneurial competencies and efficacy. The entrepreneurship training in our current study involves adolescent youth. Our focal outcomes are (1) entrepreneurial alertness, or the ability to recognize new opportunities that were previously not available (Baron and Ensley, 2006), and (2) entrepreneurial self-efficacy, or the level of confidence individuals have in their entrepreneurial capabilities (Chen et al., 1998). Alertness and efficacy are ageappropriate entrepreneurial competencies that lay the foundations of future entrepreneurial activity in the later adult stage (Obschonka et al., 2011). Entrepreneurial competencies are not only relevant to those who aspire to pursue their own business ventures in the future; they are also increasingly being recognized as critical occupational skills in navigating the world of work in the twenty-first century (Uy et al., 2015).

## RESEARCH QUESTIONS AND GOALS

Entrepreneurship training is a structured program that aims to equip participants with the necessary skillset and mindset for identifying and launching new business ventures (Cope, 2005; Katz, 2007). Previous research on entrepreneurship training include studies that examined the effect of training on university students' entrepreneurial intentions (Souitaris et al., 2007), a pre–post comparison of entrepreneurial career intentions among students in terms of desirability and feasibility (Peterman and Kennedy, 2003), and how entrepreneurial competencies mediate the linkage between entrepreneurial personality and entrepreneurial intention and alertness (Obschonka et al., 2017). Building on these studies, we used a quasi-experimental design to test whether entrepreneurship training (and particular features of it) would improve entrepreneurial competencies and ascertain the impact of training intervention on outcomes of entrepreneurial alertness and entrepreneurial self-efficacy.

The first outcome of entrepreneurial alertness has been argued to be vital for successful entrepreneurship (Obschonka et al., 2017). Entrepreneurship "is based on [the] discovering of opportunities and resources to exploit them" (Kaish and Gilad, 1991, p. 45). It is thus only natural that successful entrepreneurship will require that the entrepreneur be adept at inferring from his surroundings to notice and see value in opportunities not apparent to others (Kaish and Gilad, 1991; Tang, 2008). On a similar note, Kirzner (1973) argued that the element considered entrepreneurial in human action is the alertness to information rather than the possession of information.

Tang et al. (2012) conceived entrepreneurial alertness to be a construct with three dimensions. The first dimension of scanning and searching the environment broadens the domain-specific knowledge base of the entrepreneur. The second dimension of association and connection allows the entrepreneur to make sense of the linkages between outwardly unrelated occurrences and to approach situations with a novel perspective (Tang et al., 2012). The final dimension of evaluation and judgment enables the entrepreneur to be more situationally aware: to focus on the most important details and to discern if there is a possible opportunity (Tang et al., 2012). Individuals possessing high entrepreneurial alertness thus have a more precise perception of reality (Gaglio and Katz, 2001) as they tend to search for and notice environmental changes, and tend to have a more adaptive mental framework (Baron, 2004). Previous studies have found a positive relationship between entrepreneurial alertness and the probability of new venture creation (Langowitz and Minniti, 2007).

The second outcome is entrepreneurial self-efficacy. Selfefficacy shapes one's goals, the amount of effort one is willing to put into the goal, and one's level of persistence (Bandura, 1993). Being a domain-specific efficacy, entrepreneurial self-efficacy has stronger predictive power compared to general self-efficacy (Gist, 1987). Entrepreneurial self-efficacy has thus indeed been consistently shown to be a positive influence on an individual's intentions to become involved in entrepreneurship (e.g., Chen et al., 1998; Segal et al., 2005; Zhao et al., 2005; Cassar and Friedman, 2009). Among incumbent entrepreneurs, entrepreneurial selfefficacy had a positive link with the amount of personal, financial, and time poured into their respective enterprises (Cassar and Friedman, 2009).

Entrepreneurial self-efficacy has been found to have a strong influence on the career options considered by youths in middle and high school aged 11–18 years (Wilson et al., 2007). Similarly, Schröder and Schmitt-Rodermund (2013) found that for adolescents from families that run their own enterprises, the higher their perceived entrepreneurial competencies (analogous to entrepreneurial self-efficacy), the more intrinsically motivated they were to succeed in their respective family businesses in the future. Entrepreneurial self-efficacy is also a "significant driver of the decision to invest in discovering an entrepreneurial opportunity as well as to exploit an entrepreneurial opportunity" (Cassar and Friedman, 2009, p. 254) and has additionally been found to moderate the relationship between environmental munificence and entrepreneurial alertness (Tang, 2008).

Both entrepreneurial alertness and entrepreneurial selfefficacy have been recommended as behavioral competencies that should be promoted more comprehensively in entrepreneurial curricula (Morris et al., 2013). Entrepreneurial alertness has been postulated to be "an individual capability that can be learned and improved" (Tang et al., 2012, p. 91). Entrepreneurial self-efficacy appears to be moldable through entrepreneurship education (Zhao et al., 2005; Fayolle et al., 2006). Taken together, the first goal of this study is to test if entrepreneurship training can increase (a) entrepreneurial alertness and (b) entrepreneurial self-efficacy.

In congruence with scholarly recommendations to explore which teaching methods are effective in entrepreneurship education (e.g., Segal et al., 2007; Pihie and Bagheri, 2011), our study also examined features of the entrepreneurship training program. As entrepreneurship is action-oriented (Rasmussen and Sørheim, 2006), the experiential learning of entrepreneurship through activities has been encouraged by several scholars (e.g., Rasmussen and Sørheim, 2006; Morris et al., 2013). Educators in general believe in the importance of experiential learning in entrepreneurship training (Segal et al., 2007).

The activities in the training program examined in this study range from assembly talks and mentor guidance to internships and competitions. The activities are of various types, from experiential to verbal persuasion. According to Bandura's theory of self-efficacy, self-efficacy can be developed through experiences of mastery, modeling, verbal persuasion, and physiological states (Bandura, 1977). Likewise, for entrepreneurial alertness, Tang et al. (2012) argued that following social cognitive theory, alertness can be influenced "by actively engaging in behavior, cognition, action, and experiential learning" (p.91). Thus, our second goal for this study is to examine whether the features of entrepreneurship training in the form of passive and active/ hands-on activities account for the impact of entrepreneurship training on (a) entrepreneurial alertness; and (b) entrepreneurial self-efficacy.

In sum, our research seeks to add nuance to address the broad question—*does entrepreneurship training make a difference?*—by focusing on the adolescent youth context. Specifically, using a quasi-experimental design that allows us to compare differences in entrepreneurial competencies between a treatment or training group and the equivalent non-training group before and after the training, our research examines two questions: (1) Does systematic entrepreneurship training could effectively account for changes in secondary school students' entrepreneurial alertness and efficacy? (2) Do active/experiential and passive learning activities differentially improve entrepreneurial competencies and efficacy?

### METHOD

Our data collection effort was part of a larger study on entrepreneurship promotion and training among the youth in Singapore. As such, we report only the relevant measurements and results pertaining to the aims of the current study.

### Participants, Design, and Procedures

We recruited participants from five secondary schools in Singapore for this two time-point study. These five schools were sponsored by the government agency responsible for entrepreneurship promotion and development to conduct entrepreneurship training among their students. As this was a quasi-experimental study, random assignment was not employed. Instead, students who registered for the entrepreneurship training program were recruited to form the treatment group, while a corresponding number of students who did not register for the training program from the same school were recruited to form the control group. Participants in the treatment and control groups were comparable in age and educational levels.

In total, we recruited 365 participants at Time 1, the initial stage before the commencement of the entrepreneurship training program. Time 1 was to serve as the baseline measurement. There were 156 (42.7%) participants in the treatment group and 209 (57.3%) participants in the control group. Participants had a mean age of 14.53 years (SD = 1.13). There were 130 (35.6%) males and 235 (64.4%) females. The sample comprised of 106 (29.0%) Secondary 1, 41 (11.2%) Secondary 2, 157 (43.0%) Secondary 3 and 60 (16.4%) Secondary 4 students. One participant did not provide information about his/her year of study.

An information sheet about the study was disseminated to the parents 3 weeks before the administration of the first survey at Time 1. As the participants were secondary school students and thus minors, we sought for parental consent. Students whose parents consented to their participation were invited for the first survey at Time 1. They completed the online questionnaire administered by the research team in the computer laboratories of their respective schools. Participants who completed the first survey received a S\$10 stationery store voucher and light refreshment.

We administered another survey at Time 2, 3 months after Time 1. Time 2 thus occurred after the conclusion of the program and was taken to see if there were any changes after the program. Some participants from Time 1 were excluded from participating at Time 2 following these exclusion criteria: (1) missing more than two responses in any section of the survey at Time 1 and/or (2) giving the same responses within any one displayed page in the Time 1 survey. After collecting the data, we used the same two criteria again when cleaning the responses from Time 2. After taking into consideration the two exclusion criteria and attrition, we were left with 328 (89.9%) valid responses in Time 2, with 142 (43.3%) participants in the treatment group and 186 (56.7%) participants in the control group. The demographic characteristics of the sample did not change drastically; the mean age was 14.55 years (SD = 1.13), and there were 114 (34.8%) males and 214 (65.2%) females. There were 96 (29.3%) Secondary 1, 37 (11.3%) Secondary 2, 140 (42.7%) Secondary 3 and 54 (16.5%) Secondary 4 students. One participant did not provide information about his/ her year of study. Participants who completed the Time 2 survey received a notebook and a stationery store voucher worth S\$10.

### Entrepreneurship Training Program

The training program, which took place from around May to September 2015 was systematically implemented across the five schools mentioned in the earlier section on participants, design, and procedures. The program involved a structured curriculum consisting of about 15–21 sessions for students to acquire four main skills—interpersonal/personal, innovative thinking, financial, and marketing communications. The trainers comprised internal (i.e., teachers) and external (i.e., vendors) sources. The school teachers have a background in design and technology and/ or had previous experience organizing similar entrepreneurship programs. The external trainers were hired from an established financial literacy education provider that obtained government accreditation. Because of the structured curriculum, the core components and key activities of the programs conducted in the five schools were similar; for example, students learned how to market and pitch, were involved in design and prototyping, and went on learning journeys. Students also met with entrepreneurs who shared their experiences, and some entrepreneurs were involved as mentors to students. The schools also held internal showcases and selected students were sent to take part in external competitions.

### Variables and Measures

#### Passive and Active Entrepreneurial Activities

A list of 11 activities was used at Time 2 as a measure of student participation in passive and active learning activities. The list was created by building on an earlier list of activities reported in Uy et al. (2013), as these activities were common across entrepreneurship training programs in Singapore schools. Passive entrepreneurial activities refer to three activities that expose participants to entrepreneurship *via* a passive delivery of information. These activities included assembly talks, classroom lessons, and external visits to understand enterprise and innovation. Active entrepreneurial activities refer to eight activities that expose participants to entrepreneurship *via* active exchange and engagement with entrepreneurial experiences. These activities included attachments and internships, lessons involving product and prototype creation with the opportunity for hands-on experience acquiring entrepreneurship-related skills, and participation in business competitions within and outside of the school. The full list of activities is in Appendix A.

Participants were asked to indicate the activities they participated in. Two activity scores were calculated by summing the number of activities participated under each activity type. The composite passive entrepreneurial activities scores ranged from 0 to 3 (M = 2.18, SD = 1.05), while the active entrepreneurial activities scores ranged from 0 to 8 *(*M = 4.24, SD = 2.49). Higher activity scores indicate greater participation in the respective entrepreneurial activity type. **Table 1** provides a breakdown of treatment and control group participation in various activities in Time 2. The figures suggest that the treatment group had a significantly higher participation rate on all 11 activities (*p* < 0.05).

#### Entrepreneurial Efficacy

Entrepreneurial efficacy was measured at Time 1 and Time 2 using a 15-item scale. Out of the 15 items, five items were adapted from Chan et al. (2012) entrepreneurial efficacy scale, and 10 were new items created to reflect self-reported efficacy in both entrepreneurial skillsets and mind-set. Eleven items measured self-reported efficacy in entrepreneurial skillsets; examples of the items are "I am confident of developing a product using needs identification techniques" and "I am capable of conducting a market research by myself." Four items measured self-reported efficacy in an entrepreneurial mind-set; examples of the items are "I realize that starting and managing a profitable business requires plenty of hard work and sacrifice" and "I understand that even though the objective of running a business is to earn money, I should be guided by moral principles." The full list of items is in Appendix B.

Participants were asked to rate their confidence in completing tasks related to entrepreneurship on a 5-point Likert scale ranging from 1 (*not at all confident*) to 5 (*extremely confident*). The reliabilities at Time 1 were α = 0.94 for entrepreneurial skillset efficacy, α = 0.85 for entrepreneurial mindset efficacy, and α = 0.93 for the entrepreneurial efficacy scale as a whole. The reliabilities at Time 2 were α = 0.93 for entrepreneurial skillset efficacy, α = 0.86 for entrepreneurial mindset efficacy, and α = 0.92 for the entrepreneurial efficacy scale as a whole.

Composite entrepreneurial skillset efficacy (for Time 1, M = 3.13, SD = 0.81; for Time 2, M = 3.09, SD = 0.78) and entrepreneurial mindset efficacy (for Time 1, M = 3.84, SD = 0.78; for Time 2, M = 3.84, SD = 0.76) scores were computed by taking the mean scores across the items in each subscale. Entrepreneurial efficacy as a whole had a mean of 3.32 at Time 1 (SD = 0.72) and a mean of 3.29 at Time 2 (SD = 0.68). Higher mean composite scores indicated higher entrepreneurial efficacy.

#### Entrepreneurial Alertness

Entrepreneurial alertness was measured at Time 1 and Time 2 using the entrepreneurial alertness scale developed by Tang et al. (2012). These items were grouped into three dimensions of alertness: (1) *scan and search* (five items), (2) *associate and connect* (three items), and (3) *evaluate and judge* (four items). The *scan and search* dimension refers to the ability of entrepreneurs to attune to information about new business ideas. Items for the *scan and search* dimension include "I have frequent interactions with others to acquire new information" and "I always keep an eye out for new business ideas when looking for information." The *associate and connect* dimension refers to the ability to apply and


Ho et al. Entrepreneurship Training

extend logic to different pieces of information. For the *associate and connect* dimension, example items are "I see links between seemingly unrelated pieces of information" and "I often make novel connections and perceive new or emergent relationships between various pieces of information." The *evaluate and judge* dimension describes an individual's ability to assess opportunities and make judgments about the feasibility of their business ideas. Examples of the *evaluate and judge* dimension items include "I have an extraordinary ability to smell profitable opportunities" and "I have a gut feeling for potential opportunities."

Participants rated the extent to which they agreed with the items on a 5-point Likert scale from 1 (*strongly disagree*) to 5 (*strongly agree*)*.* The reliabilities at Time 1 were α= 0.76 for scan and search, α = 0.70 for associate and connect, α = 0.71 for evaluate and judge, and α = 0.88 for the entrepreneurial alertness scale as a whole. The reliabilities at Time 2 were α = 0.71 for scan and search, α= 0.69 for associate and connect, α= 0.65 for evaluate and judge, and α = 0.83 for the entrepreneurial alertness scale as a whole.

Composite scan and search (for Time 1, M = 3.50, SD = 0.62; for Time 2, M = 3.47, SD = 0.57), associate and connect (for Time 1, M = 3.47, SD = 0.63; for Time 2, M = 3.54, SD = 0.58), and evaluate and judge (for Time 1, M = 3.53, SD = 0.59; for Time 2, M = 3.53, SD = 0.56) scores were computed by taking the mean scores across the items in each dimension. Overall entrepreneurial alertness had a mean of 3.50 at Time 1 (SD = 0.55) and a mean of 3.51 at Time 2 (SD = 0.47). Higher mean composite scores indicated higher levels of entrepreneurial alertness in each respective dimension.

#### Analysis

To demonstrate the impact of the training program on the treatment group, we first conducted MANCOVA on the five dependent variables (i.e., entrepreneurial skillset and mindset efficacy, and the three dimensions of entrepreneurial alertness scores) collected at Time 1, controlling for gender, to ensure the treatment and control groups were comparable before examining the training effect. To test the presence of training effect, we again conducted MANCOVA to compare the treatment and control groups in entrepreneurial efficacy and alertness at Time 2, controlling for gender. Any significant difference indicated from MANOVA was followed by *post hoc* univariate ANCOVA to detect at which dependent variable the training effect was present (Pituch and Stevens, 2016). Gender was included as a covariate in all the aforementioned analyses because gender gap between males and females in entrepreneurship has been found in serveral past studies (e.g., Mueller, 2004; Wilson et al., 2007; Walter et al., 2013).

With the training effect established, we then tried to identify if the features of the training program that can improve the entrepreneurial outcomes of the adolescents. Amount of exposure to two set of features, passive and active entrepreneurial activities (see **Table 2**), were compared between the treatment and control groups, controlling for gender by MANCOVA. After confirming these features differed between the two groups, we then checked if the amount of exposure to passive and active entrepreneurial activities can account for the significant training effects on the entrepreneurial outcomes found earlier through MANCOVA, controlling for gender. *Post hoc* ANCOVA analyses were followed up to determine which type of activities was relevant in contributing the training effect for each entrepreneurial outcome.

### RESULTS

**Table 2** below shows the descriptive statistics, correlations, and reliabilities of all the measures mentioned in the Section "Materials and Methods."

### Training Effects on Entrepreneurial Outcomes

At Time 1, MANCOVA results indicated no significant difference between the treatment and control groups in the entrepreneurial efficacy and alertness scores, *F*(5, 321) = 0.507, *p* = 0.771. Therefore, no *post hoc* test was further conducted. This implies that adolescents in both groups were comparable in terms of their perceived efficacy in entrepreneurial skillset, mind-set, and alertness at the beginning of the study, when entrepreneurial training was either absent or minimal.

At Time 2, MANCOVA results indicated significant differences between the treatment and control groups, *F*(5, 321) = 2.275, *p*= 0.047, η<sup>p</sup> 2 (partial eta squared) = 0.034. To identify where group


differences were present, univariate ANCOVA was conducted on each of the five dependent variables (i.e., entrepreneurial skillset, entrepreneurial mindset, and three dimensions of entrepreneurial alertness). There was a significant diffeence in entrepreneurial skillset efficacy between the treatment (M = 3.19, SD = 0.83) and control (M = 3.02, SD = 0.73) groups, *F*(1, 325) = 4.01, *p* = 0.046, ηp 2 = 0.012. There was also a significant difference in entrepreneurial mindset efficacy between the treatment (M = 3.98, SD = 0.72) and control (M = 3.73, SD = 0.77) groups, *F*(1, 325) = 8.349, *p* = 0.004, η<sup>p</sup> <sup>2</sup> = 0.025. There was a significant difference in the scan and search dimension of entrepreneurial alertness between the treatment (M = 3.55, SD = 0.60) and control (M = 3.42, SD = 0.55) groups, *F*(1, 325) = 4.027, *p* = 0.046, η<sup>p</sup> <sup>2</sup> = 0.012. There was also a significant difference in the evaluate and judge dimension of entrepreneurial alertness between the treatment (M = 3.61, SD = 0.60) and control (M = 3.47, SD = 0.51) groups, *F*(1, 325) = 4.412, *p* = 0.036, η<sup>p</sup> <sup>2</sup> = 0.013. In short, adolescents in the treatment group scored significantly higher in entrepreneurial skillset efficacy, mindset efficacy, scanning, and searching, and evaluating and judging compared to the adolescents in the control group. There was no significant difference in the associate and connect dimension of entrepreneurial alertness between the treatment (M = 3.58, SD = 0.61) and control (M = 3.51, SD = 0.56) groups at Time 2, *F*(1, 325) = 0.915, *p* = 0.339.

Taken together, the results suggest that the entrepreneurial training program had a valuable impact in improving overall entrepreneurial outcomes among adolescents. Compared to adolescents who did not receive any entrepreneurship training, those who received training reported higher entrepreneurial skillset and mindset efficacies, and improved ability to scan and search for, and evaluate and judge entrepreneurial opportunities.

### Features of the Training Program

The subsequent set of analyses centered on identifying the features of the training program that improved the entrepreneurial outcomes of the adolescents. MANCOVA was conducted to compare the amount of exposure to (1) passive entrepreneurial activities and (2) active entrepreneurial activities in the treatment and control groups (see **Table 1**), controlling for gender. There was a significant difference between the two groups, *F*(2, 324) = 30.918, *p*= 0.000, η<sup>p</sup> <sup>2</sup> = 0.16. *Post hoc* ANCOVA revealed significant differences in the amount of passive activities between the treatment (M = 2.61, SD = 0.74) and control (M = 1.85, SD = 1.12) groups, *F*(1,325) = 49.323, *p* < 0.001, η<sup>p</sup> <sup>2</sup> = 0.132. There was also a significant difference in the amount of active/ hands-on activities between the treatment (M = 5.25, SD = 2.19) and control (M = 3.47, SD = 2.42) groups, *F*(1, 325) = 46.957, *p* < 0.001, η<sup>p</sup> <sup>2</sup> = 0.126. Thus, compared to those who did not receive entrepreneurial training, those who received training reported having exposure to more passive and active/hands-on entrepreneurship-related activities. The entrepreneurial training program effectively increased the exposure to entrepreneurshiprelated activities for adolescents in the treatment group. No significant gender difference was found for the amount of exposure in both active and passive entrepreneurial activities.

The next step taken was to check if the amount of exposure to passive and active entrepreneurial activities was responsible for accounting for the significant training effects on the entrepreneurial outcomes found earlier. To establish the roles played by passive and active activities in the observed differences at Time 2, we performed MANCOVA on the five entrepreneurial outcomes (i.e., entrepreneurial skillset, entrepreneurial mindset, and three dimensions of entrepreneurial alertness), including gender, passive and active activities as covariates.

With the inclusion of gender, passive and active activities as covariates, there was no significant difference between the treatment and control groups, *F*(5, 319) = 1.202, *p* = 0.308, though we found a significant gender effect: *F*(5, 319) = 5.432, *p* < 0.001. Also, significant effects of exposure to both active and passive activities were detected. *Post hoc* ANCOVA analyses (with the inclusion of gender, passive, and active activities as covariates) was then performed on each of the entrepreneurial outcomes separately.

Active activities contributed to the differences between the treatment and control group in (1) entrepreneurial skillset efficacy, *F*(1, 323) = 19.607, *p* < 0.001, η<sup>p</sup> <sup>2</sup> = 0.057; (2) the *scan and search* dimension of entrepreneurial alertness, *F*(1, 323) = 5.383, *p* = 0.021, η<sup>p</sup> <sup>2</sup> = 0.016; (3) the *evaluate and judge* dimension of entrepreneurial alertness, *F*(1, 323) = 5.540, *p*= 0.019, η<sup>p</sup> <sup>2</sup> = 0.017. Passive activities, on the other hand, contributed to the difference between the treatment and control group in entrepreneurial mindset efficacy, *F*(1, 323) = 6.832, *p* = 0.009, η<sup>p</sup> <sup>2</sup> = 0.021.

Gender was also a significant covariate for (1) entrepreneurial skillset efficacy, *F*(1, 323) = 22.621, *p* < 0.001, η<sup>p</sup> <sup>2</sup> = 0.065; (2) entrepreneurial mindset efficacy, *F*(1, 323) = 9.730, *p* = 0.002, ηp <sup>2</sup> = 0.029; (3) the *scan and search* dimension of entrepreneurial alertness, *F*(1, 323) = 5.170, *p* = 0.024, η<sup>p</sup> <sup>2</sup> = 0.016; and (4) the *evaluate and judge* dimension of entrepreneurial alertness, *F*(1, 323) = 4.374, *p* = 0.037, η<sup>p</sup> <sup>2</sup> = 0.013.

Taken together, the results show that besides gender, the program features of passive and active activities can account for the training effects on the entrepreneurial outcomes of entrepreneurial skillset, entrepreneurial mindset, and all three dimensions of entrepreneurial alertness. Our results suggest that entrepreneurial training is valuable and should be promoted among adolescents to enhance their entrepreneurial efficacy and alertness.

### DISCUSSION

Based on a recent report sponsored by the World Bank, many countries have increasingly recognized structural policies, such as entrepreneurship training, to be effective means of equipping their citizens—especially the youth—with necessary entrepreneurial competencies (Valerio et al., 2014). However, there is a dearth of research on entrepreneurial education in secondary schools (Sánchez, 2013; Moberg, 2014; Elert et al., 2015). Our findings offer insights that contribute to the development of a comprehensive theory of entrepreneurship education and training (Pittaway and Cope, 2007; Martin et al., 2013).

### Research Implications

Our study provides valuable evidence that entrepreneurship training programs among secondary school students can be an effective means for equipping youths with entrepreneurial competencies and favorable attitudes toward entrepreneurship. There were no significant differences between the treatment and control groups on any of the measures before the program commenced. Significant differences appeared after the treatment group had undergone training. These differences demonstrate, as mentioned by Morris et al. (2013), that dynamic competencies can be picked up and grown over time with practice and exposure.

Our study also uncovered that entrepreneurship training can increase the entrepreneurial alertness of secondary school students to some extent *via* the usage of active/hands-on activities, consistent with social cognitive theory (Tang et al., 2012). Valliere (2013) proposed a schematic model of entrepreneurial alertness, whereby alertness can be taught by (1) first helping individuals to obtain the needed schemata for entrepreneurial alertness by imparting relevant information, and then (2) conducting continued deliberate practice of activating and applying the schemata. Though only the ratings of *scan and search*, and *evaluate and judge* dimensions were significantly higher in the treatment group versus the control group, we note a similar, albeit smaller, difference between both groups in the *associate and connect* dimension. Perhaps, the active/hands-on activities in the program lent themselves better to the practice of scanning and searching as well as evaluating and judging, versus associating and connecting information. Alternatively, the items for *associate and connect* may have appeared to be broader in scope than the items from the other two dimensions, which were more specific to entrepreneurial behaviors, thus making it difficult for students to judge if the program had indeed improved their abilities in that area.

Next, our study shows that entrepreneurship training can increase the entrepreneurial self-efficacy of secondary school students *via* active/hands-on and passive activities embedded in the training program. The contribution of active/hands-on activities to the development of entrepreneurial efficacy (i.e., skillset) in the students is supported by arguments on the impact of experiential learning in entrepreneurship education (e.g., Gibb, 1987; Chen et al., 1998; Aronsson, 2004; Fuchs et al., 2008). Experience in performing a relevant and sufficiently challenging task has long been argued to be the most critical factor in developing high domain-specific self-efficacy (Erikson, 2003) The presence of three crucial elements of mastery modeling (Wood and Bandura, 1989) in active/hands-on activities may have contributed to their effectiveness in building self-efficacy in the entrepreneurial skillset. The interactions with mentors and lessons on product development may have helped impart entrepreneurial skills and affirmed the self-belief of students about their capabilities (Wood and Bandura, 1989). Group work, report presentations, and internships may have filled in the role of guided skill mastery, where students put to practice their recently developed skills in a "safe" space with feedback (Wood and Bandura, 1989). Internal and external school competitions may have played the final role of allowing students to take on directly increasingly difficult tasks that can build and stabilize their self-efficacy (Wood and Bandura, 1989).

The contribution of passive activities to the development of entrepreneurial mindset efficacy in the students demonstrates that though verbal persuasion has been classically viewed as more limited in its efficacy (Bandura, 1977), it may still have its place in entrepreneurship education. Experiential learning may not be suitable for all teaching contexts (Fayolle, 2008). Furthermore, realistic verbal persuasion is helpful in overcoming lack of selfconfidence, improving self-regulation, encouraging individuals to put in more effort in performing a task (Wood and Bandura, 1989), thus making it particularly effective in shaping the entrepreneurial mindset efficacy of students.

The significant finding of gender as a covariate is in line with the gender gap commonly noted in entrepreneurship research. In our results, we observe as well that female students who attended the program scored lower than male students in all dimensions of entrepreneurial self-efficacy and alertness. It has previously been suggested that the masculine discourse surrounding entrepreneurship (Ahl and Marlow, 2012) and societal stereotypes of entrepreneurship as predominantly masculine (Gupta et al., 2008) may lead to females being less likely to identify themselves as entrepreneurs, regardless of the types of activities in which they have been involved (Verheul et al., 2005). Other authors have found that for females, having a role model can help boost their entrepreneurial self-efficacy (Barnir et al., 2011). In any case, our study provides further support to account for gender heterogeneity in future work (Westhead and Solesvik, 2016).

### Limitations and Future Directions

We acknowledge that the quasi-experimental nature of the study may have contributed to the smaller effect sizes. It is also possible that the training program may have been presented slightly differently by the trainers and/or teachers from the various schools. Future studies can consider employing the same trainers across every program to establish uniformity across schools to have a better control over potential confounding trainer effects. The number of time-points can also be increased with longer followup periods to verify the stability and changes in the effects of the training program. A true experimental study can be conducted with all the confounding variables controlled in the future.

The lack of training effect on the *associate and connect* dimension of entrepreneurial alertness presents a possible avenue for future research. For example, future studies may consider looking at the sort of activities in a training program that can improve this dimension, or perhaps investigate if this dimension may be less malleable compared to the other two dimensions.

Future studies can also go beyond the passive/active grouping of training activities to examine more deeply which specific activities contribute most to the effectiveness of the program. Previous literature has suggested that teamwork activities may be especially potent for entrepreneurial learning (Huber et al., 2014). Moreover, there have been suggestions that the social interactions of adolescents with their peers and educators are important factors that influence the effectiveness of the training program (Man and Yu, 2007). In contrast, other authors have promoted for more solo-work activities (Sexton and Upton, 1984).

Our study also did not look into whether the development of entrepreneurial self-efficacy and entrepreneurial alertness would increase the likelihood of the students becoming actual entrepreneurs. Schoon and Duckworth's (2012) longitudinal study involving over 6,000 individuals tracked from birth until age 34 revealed that becoming an entrepreneur was associated with entrepreneurial career intent expressed at mid-adolescence (age 16). It will be important for future studies to monitor if youth who undergo training end up establishing their own business ventures at some point in their lifetime.

Finally, it may be useful for future studies to explore more outcomes alongside entrepreneurial self-efficacy and entrepreneurial alertness to refine policy-making decisions. For example, it has been found that self-control is another important factor for successful entrepreneurship: it ensures that the entrepreneurial goals set by individuals high in entrepreneurial self-efficacy are not too lofty and unachievable (Baron et al., 2016).

### Practical Implications

Our study highlights that the effectiveness of an entrepreneurship training program for youth relies heavily on the types of activities it provides. Experiential learning is important for the development of the more *hands-on* competencies of entrepreneurial skillset efficacy, *scanning and searching*, and *evaluating and judging*. The activities should ideally cover a range of difficulty and independency levels, from lessons on product development in school to competing in nation-wide entrepreneurship competitions. Students can then hone their self-efficacy and entrepreneurial alertness as they progress through the program.

Verbal persuasion, though less encouraged in the current literature for inclusion in entrepreneurship training programs, still appears to play a crucial role, at least in shaping the entrepreneurial mindset of students. The personal sharing of entrepreneurship experiences by individuals, some of whom are alumni of the respective schools, may have made the social persuasion particularly effective due to their perceived credibility and familiarity with entrepreneurship (Bandura, 1984, as cited in Gist, 1987). External visits to companies and meeting the heads of those companies may be effective in helping students and more realistically shape their ideas of entrepreneurship. Such visits give

### REFERENCES


students an opportunity to understand more deeply how they can utilize otherwise distant knowledge learned from the classroom within the constraints of real life.

Overall, the study shows that a healthy mix of activities is critical to the success of an entrepreneurship education program.

### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of the ethics guidelines for human biomedical research of the Institutional Review Board at Nanyang Technological University, which approved the study protocol. Parents or guardians provided written informed consent for the adolescents' participation. Assent from the adolescent participants were further sought before the study was conducted.

### AUTHOR CONTRIBUTIONS

M-HH and MU are responsible for the design of the study, data analysis, and manuscript preparation. BK is responsible for data analysis and manuscript preparation. K-YC is responsible for the design of the study and manuscript preparation.

### ACKNOWLEDGMENTS

We would like to thank Spring Singapore, Stephanie Tan Li Ting, and Jiang Weiting for their assistance in data collection, and the Ministry of Education for the financial support.

### FUNDING

This study was funded under the Singapore Ministry of Education's Tier 2 Academic Research Fund (Grant no: MOE 2013-T2-2-104).


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2018 Ho, Uy, Kang and Chan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

## APPENDIX A

### Activities List


#### Passive activities


#### Active activities


### APPENDIX B

### Entrepreneurial Efficacy

Based on your current abilities, how confident are you in performing the following tasks successfully now?

Key terms you will see

Partner(s) refers to an individual or company who is to some extent involved in your business dealings.

Instructions: Please read each statement carefully and choose the option that best describes your confidence level using the scale. There are NO right or wrong answers. Please answer honestly and frankly.


# I'll Follow the Minority: The Effects of Sales Level on Purchase Intention of Self-expressive Products

Xiaoyu Wan<sup>1</sup> , Tingyi Wang<sup>1</sup> and Jifei Wu<sup>2</sup> \*

*<sup>1</sup> School of Economics and Management, Chongqing University of Posts and Telecommunications, Chongqing, China, <sup>2</sup> Business School, Sun Yat-sen University, Guangzhou, China*

The present study focuses on the naive theory of exclusivity (vs.popularity) triggered by the sales level of self-expressive (vs. functional) products and introduces perceived self-image exclusivity and perceived face threat to explain the effect of self-expressive products' sales levels on consumers' purchase intention. Specially, about 900 young people participated in four experiments, which used T-shirts, pillows, cups, fashion coats and heating blankets as experimental materials. Through four studies, it is found that consumers are more likely to choose self-expressive (vs. functional) products with low sales (vs. high sales) level. In addition, the paper presents a serial mediation effect of perceived self-image exclusivity → perceived face threat, which can explain the "I will follow the minority" effect of self-expressive products. Finally, the study presents the theoretical and practical significance and future research direction.

#### Edited by:

*Douglas F. Kauffman, Independent Researcher, Bedford, United States*

#### Reviewed by:

*Yufang Zhao, Southwest University, China Andrea Scalco, University of Aberdeen, United Kingdom*

\*Correspondence: *Jifei Wu wujf8@mail2.sysu.edu.cn*

#### Specialty section:

*This article was submitted to Organizational Psychology, a section of the journal Frontiers in Psychology*

Received: *13 March 2018* Accepted: *13 June 2018* Published: *10 July 2018*

#### Citation:

*Wan X, Wang T and Wu J (2018) I'll Follow the Minority: The Effects of Sales Level on Purchase Intention of Self-Expressive Products Front. Psychol. 9:1135. doi: 10.3389/fpsyg.2018.01135* Keywords: naive theory, self-expressive products, perceived self-image exclusivity, perceived face threat, purchase intention

### INTRODUCTION

Imagine a consumer who is exploring the information of Amazon Best Sellers on amazon.com, intending to buy a coat and wet wipes there. How likely will he/she choose a coat with high sales level (namely, ranked as a best seller)? Will the choice differ from that concerning wet wipes?

Some online retail websites (e.g., American Amazon or China T-mall) show the sales level information (e.g., best seller rank or sales volume) of every product, which plays an important role during consumer decision making. Sales level, which reflects the number of buyers, will influence a consumer's choice of a product. This idea is generated by social naive theories because consumers often evaluate products or services based on common sense or naive theory (Raghunathan et al., 2006; Labroo and Mukhopadhyay, 2009; Yorkston et al., 2010; Deval et al., 2013). However, previous studies have found that consumers may hold contradictory naive beliefs about the same information (Deval et al., 2013; Steinhart et al., 2014). In some cases (e.g., buying functional products), consumers may choose products that many others like, driven by the naive theory of popularity, that is, preferring high sales level, and yet in other situations (e.g., buying self-expressive products), they will be attracted to products that few others are interested in, driven by the naive theory of exclusivity (Steinhart et al., 2014), that is, avoiding high sales level.

Follow previous research, sales level information's influence on the purchase intention of self-expressive products (vs. functional products) is driven by the naive theory of exclusivity (vs. popularity). This is because self-expressive products (vs. functional products) are better at expressing the image and self of their owners (Berger and Heath, 2007), while the high (low)

**173**

sales indicates a more commonplace (exclusive) quality for both the self-expressive products and their owners (Machleit et al., 2000), which is a negative (positive) signal for consumers who seek self-image exclusivity through self-expressive products.

It is worth noting that previous research has shown that people consciously make different choices from others to highlight their uniqueness in the group (Chan et al., 2012), and the reason people choose scarce products is to maintain their uniqueness (Steinhart et al., 2014). However, this study argues that the exclusive preference of consumers for self-expressive products (vs. functional products) cannot be completely explained by the need for uniqueness; face also plays an important role in the consumer choice due to its close relationship with a person's desired self-image (Goffman, 1967; Litt et al., 2014). Thus, this study draws on the naive theory associated with selfexpressive products to explore how the naive theory of exclusivity affects consumer choice considering sales level and reveals the underlying mechanism of face. Specifically, the present study addresses when consumers prefer to choose low (vs. high) sales level products and why they buy self-expressive products with low (vs. high) sales level. Findings revel that consumers prefer to choose or buy self-expressive (vs. functional) products with low sales and functional (vs. self-expressive) products with high sales; furthermore, the serial mediation effect of perceived self-image exclusivity → perceived face threat is proposed to explain the effect.

### THEORETICAL FRAMEWORK

### Naive Theories in Consumer Behavior

Naive theories are defined as informal, common sense explanations that people use in their daily lives to influence the environment, and they often differ from formal, scientific explanations of what really happens (Furnham, 1988; Deval et al., 2013). Because the application and activation of naive theories requires minimal cognitive effort, and consumers often rely on naive theories for making inferences about marketing messages, products and services (Kardes et al., 2004), marketers often emphasize some product features that induce consumers' naive beliefs to optimize product marketing strategy (Lynn, 1992).

Previous research has shown that naive theories may conflict with each other and that consumers' assessments of products vary with the inferred rules triggered by prior priming (e.g., the popularity and exclusivity in a social context; Deval et al., 2013; Steinhart et al., 2014). Specifically, when following the naive theory of popularity, consumers will infer the interest of many others as a positive attribute (Steinhart et al., 2014). This phenomenon is similar to the "bandwagon" and the "As Seen on TV" effects (Hellofs and Jacobson, 1999; Powell and Prasad, 2010), which occur when consumers make a positive assessment of a product simply based on the number of people who have purchased or used it. In contrast, the naive theory of exclusivity suggests that the interest of many others may mean diminished product uniqueness (Lynn, 1992), leading consumers to think that the product is commonplace (Machleit et al., 2000); this is consistent with the "loss of exclusivity" mentioned in previous studies (Hellofs and Jacobson, 1999).

Following the reasoning of previous studies (Deval et al., 2013; Steinhart et al., 2014), activating a compelling naive theory can lead consumers to make purchase decisions by processing contextual cues. Moreover, as Deval et al. (2013) have showed, activation could be achieved by manipulating product popularity or exclusivity cues through presenting the number of people who are interested in a particular product. Further, as previous studies (Steinhart et al., 2014) have showed, the product itself (self-expressive vs. functional products) activates different naive beliefs, and the interaction of the contextual cues (interest of others) with the product further increases the capability of naive beliefs. However, the paper focuses on the naive theory of exclusivity activated by contextual cues (i.e., sales level information) and self-expressive products and examine whether and how the sales level of self-expressive products influences consumers' behavior.

### The Impact of Sales Level on Purchase Intention

Sales level is an important contextual cue that reflects the consumer interest and market share of a product, and a higher sales level indicates that a product is more desirable in the market (He and Oppewal, 2017). Previous studies divided the sales level into higher and lower levels judging by how many people had bought or owned a product (He and Oppewal, 2017). Moreover, most previous studies demonstrated high sales level as a popularity cue, holding that a high sales level means that more people liked the product and had bought or owned the product; that is, the product was more popular in society than another product with a lower sales level (Wu and Lee, 2016). However, low sales level (namely, low market share) is a signal that the product has an exclusive image (Hellofs and Jacobson, 1999), meaning that only a few people own the same product, and so it is closely related to product uniqueness (Tian et al., 2001). For what role the sales level plays in consumer purchase, studies argue that it would positively influence people's choices through perceived product popularity (He and Oppewal, 2017) and that it would have different influences when the product was purchased for oneself or others (Wu and Lee, 2016); in particular, it would be a negative signal concerning a self-expressive product and a positive signal concerning a functional product (Steinhart et al., 2014).

This study focuses on the impact of sales level on purchase intention of self-expressive vs. functional products. Functional products and self-expressive products were distinguished according to the extent to which a product category is believed to signal the status of the owner, that is, the status signaling capability (SSC; Wang and Wallendorf, 2006; Chan et al., 2012. Usually, a self-expressive (functional) product is one can (cannot) signal the status or identity of the owner significantly (Chan et al., 2012; Steinhart et al., 2014). Previous studies have showed that a product was a part of the consumer, and people used products to express their identities and tastes (Berger and Heath, 2007). However, different products showed different capabilities to signal owners' status or identities, and so individuals used to use certain types of products to achieve their self-expression (Belk, 1981). Indeed, some products, that is, symbolic products (e.g., a T-shirt), rather than products that are more functional and less self-expressive (e.g., a stereo system; Shavitt, 1990), more easily communicated information about their owners (Escalas and Bettman, 2005). Overall, previous studies considered selfexpressive products and functional products as two opposing product types (Steinhart et al., 2014), and these studies held that it was the status signaling capability rather than the performance of self-expressive products was important, on the contrary, it was the performance rather than the status signaling capability of functional products was important (Berger and Heath, 2007; Steinhart et al., 2014). Specifically, self-expressive products, which tend to include scarce and differentiated products (Tian et al., 2001; Steinhart et al., 2014); e.g., a unique, customized product or a product that could not be owned by others at the same time), possess self-expressive features, and an individual's consumption of them depends more on the personal or social meaning of the products than their functional utility (Berger and Heath, 2007). In contrast, a functional product is an essential, utilitarian tool that enables the owner to achieve a goal or complete a practical task (Wertenbroch and Dhar, 2000).

Besides, it is worth noting that this study just discusses the difference between unbranded self-expressive products and unbranded functional products, and the influence of famous brand on status signaling capability of products is beyond the scope of this study. The research avoided arousing product branding in experiments because a famous brand will increase the sales of products because people think that the consumption of the products with famous brand is a symbol of status.

According to previous studies, consumers' evaluation of functional products was often triggered by the naive theory of popularity. Consumers followed most people's judgement because they held that the wisdom of the majority cannot be wrong (Steinhart et al., 2014); therefore, they had a significantly more positive evaluation of the functional products with high vs. low level of sales. However, it is important to note that self-expressive products evoke the naive theory of exclusivity. Consumers' evaluations of self-expressive products, in contrast to evaluations of functional products, are predicted to be enhanced when few rather than many people own such products (Steinhart et al., 2014); that is, consumers' assessment of self-expressive products will be more positive when the sales level is low rather than high because a low sales level implies a scarcity and exclusivity of the products. Therefore, the following hypothesis is proposed:

H1: Low vs. high level of sales will result in more purchase intention of self-expressive vs. functional products.

### The Serial Mediation Effect of Perceived Self-image Exclusivity and Perceived Face Threat

Self-image was treated as the actual self-concept, i.e., as a perception of oneself (Bellenger et al., 1976; Sirgy, 1982). One's self-image was influenced by his or her personality and image. Impression management theory proposed that people try to control how others perceive them (Leary and Kowalski, 1990); this study regarded exclusivity (Steinhart et al., 2014) as a factor that should be considered. Perceived self-image exclusivity, in the present study, was defined as the degree to which one infers that others share the same image, and the lower the perceived selfimage exclusivity is, the more likely one's image will be similar to others'.

Previous research showed that, as consumers' concerns for exclusivity increased, the products' increased market share reduced the product evaluations (Hellofs and Jacobson, 1999). Hence, from a theoretical perspective, the interest of few others elevates product evaluation, as consumers rationalize that the product is not accessible to everyone (Steinhart et al., 2014); that is, the owners' self-image of the product is exclusive to some degree. In the present study, sales level is proposed as another important factor leading to exclusivity; a low (vs. high) sales level represents the product's exclusivity (vs. popularity) and scarce (vs. commonplace) image (Hellofs and Jacobson, 1999; He and Oppewal, 2017), but the perception of self-image exclusivity varies between the consumption of self-expressive and functional products. Because the symbolic characteristics of self-expressive (vs. functional) products match consumers' inner needs of image management and social identity (Whan Park, 1986), the consumption of self-expressive (vs. functional) products can bring social or individual meanings to individuals. Therefore, when considering a self-expressive product with low (vs. high) sales level, that is, an exclusive (vs. popularity) product, individuals will happily consider (vs. hardly consider) their selfimage with the product as exclusive. In contrast, the consumption of functional products mainly meets the needs of quality and utility. Previous studies showed that consumers were mainly concerned about the popularity of functional products (Steinhart et al., 2014; Wu and Lee, 2016) to infer the quality and value; therefore, the product's exclusivity would not be associated with self-image. In conclusion, individuals usually hold a belief about self-expressive (vs. functional) products that high sales level indicates a commonplace self-image.

Face was defined as the "positive social value a person effectively claims for himself by his or her self-presentation" (Goffman, 1967). However, face threat is a situation that occurs when a person's desired image is challenged or undermined (Goffman, 1967; Cupach and Metts, 1994), and it can be generated by the self (e.g., one found himself/herself wearing the same coat as someone else) or others (e.g., one's coat got laughed at; Cupach and Metts, 1994). Following the definition of face threat from previous studies (Goffman, 1967; Cupach and Metts, 1994), the present study defined perceived face threat as an inner perception or inference about how much a situation will challenge or undermine a person's desired image.

Face represents a supportive social self-image (Litt et al., 2014); therefore, when others' consumption behavior or potential evaluation in the environment impedes the individual from maintaining his or her own social self-image, face threat is generated. As predicted by this study, a self-expressive product with high (vs. low) sales level triggers less (vs. more) perceived self-image exclusivity; that is, the user of a product considered commonplace might sense negative evaluations about his or her image, which is contrary to individuals' desired self-image, that is, in contrast to his or her targeted self-presentation. Thus, face threat is generated. In contrast, the consumption of functional products hardly concerns whether the product can achieve social self-image, and so the need for face is not significant, and the popularity from a high sales level will not be associated with less face for consumers, that is, face threat. Individuals would try to avoid face threat or save face through their own behavior because when experiencing face threat with different levels of severity (Petronio, 2002; Litt et al., 2014; Wohn and Spottswood, 2016), caused by imagining or perceiving negative evaluations from others, individuals would produce a series of negative emotions or reactions (Wohn and Spottswood, 2016). Thus, the present study argued that the lower perceived self-image exclusivity associated with high sales level will significantly increase perceived face threat and ultimately reduce the purchase intention of the self-expressive products, rather than of functional products with high sales level.

In conclusion, individuals usually hold the belief concerning self-expressive (vs. functional) products that high sales means commonplace self-image and negative evaluations from others. Therefore, the following hypothesis is proposed:

H2: The effect of sales level on individuals' purchase intention of a self-expressive (vs. functional) product is serially mediated by perceived self-image exclusivity and perceived face threat (visualized as **Figure 1**).

### Overview of Studies

Theoretical propositions were tested in a series of studies. First, Study 1A, conducted under the condition of accepting prizes, explored whether low (high) level of sales will result in more self-expressive (functional) products' choices. Second, Study 1B nearly replicated the experiment in Study 1A, but to enhance the generalizability of the present study, cups described as either a self-expressive or a functional product were the experimental stimuli. Afterwards, in the condition of online shopping, Study 2 tested the proposed serial psychological mechanisms via perceived self-image exclusivity and perceived face threat of self-expressive (vs. functional) products. Finally, with discounting the belief that "high sales means commonplace and negative evaluations from others," which is the cause of the serial mediation, Study 3 tested whether the effect of different sales level on self-expressive products would disappear. Several unbranded products were used as materials to exclude the impact of product brand on the consumer purchase intention.

All participants gave their informed consent before Study 1A, Study 1B, Study 2, and Study 3. All of studies were conducted under the approval of the Academic Committee of the Department of Economics and Management at Chongqing University of Posts & Telecommunications.

### Study 1A

Study 1A was conducted to examine the basic hypothesis that low vs. high level of sales will result in more choice of self-expressive vs. functional products.

### Methods

#### Participants and Design

Participants (N =160, Mage = 21.33, SDage = 0.84, 55% female), who were asked to complete an online questionnaire about their perceptions and preference about activity prizes, were undergraduate business students at a southwest university of China who participated for course credit. Participants were randomly assigned to one of the two experimental conditions (sales level: high or low).

#### Stimuli and Procedure

T-shirts and pillows were selected as self-expressive and functional products in the Study 1A. A pre-test was used to confirm a pair of self-expressive and functional products. Fiftytwo participants (Mage = 21.46, SDage =0.83, 56% female) were exposed to a pair of products on mobile screen: T-shirts and pillows. They were then asked to rate their agreement with the following four statements of the scale about the status signaling capability (Wang and Wallendorf, 2006): "T-shirts (or pillows) can convey one's personality to the people around him/her," etc., (1 = strongly disagree, 7 = strongly agree), Cronbach's α = 0.85 (see Appendix A). Usually, the higher (lower) the average score was, a product tended to be a self-expressive (functional) product. Specially, a product which scored 1 (7) point of each questions was completely a functional (self-expressive) product. In the pretest, participants perceived that T-shirts had significantly stronger capability to signal status than pillows (MT−shirts = 5.4, Mpillow = 3.6, p < 0.001). Therefore, Study 1A selected T-shirts and pillows as self-expressive and functional products.

All instructions and questionnaires were presented via desktop. Each participant was exposed to a scenario assuming that they had won third prize in an activity, and the optional prizes were pillows and T-shirts all priced at \$22. To manipulate sales level, participants in the low level of sales condition were told that two kinds of prizes had been chosen by very few people. In the high level of sales condition, participants were told that two kinds of prizes had been chosen by a large number of people.

Then, all participants were asked to make a choice between the two prizes. Finally, participants rated their agreement with the statements which was the perception of sales level "Many others are likely to own this product" and "Few others are likely to own this product" (Steinhart et al., 2014) on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree).

### Results

Fourteen of one hundred sixty participants were deleted for either inconsistent answers or incomplete answers. One hundred and forty six valid data points were used (Nhigh = 73, Nlow = 73).

### Manipulation Check

Participants in the high sales level condition more strongly agreed with the statement that many others were likely to own each product (Mhigh = 4.92) than participants in the low sales level condition [Mlow = 3.41; t(144) = 3.73, p < 0.001]. In addition, participants in the high sales level condition agreed significantly less with the statement that few others were likely to own each product [Mhigh = 2.87, Mlow = 4.25; t(144) = 3.65, p < 0.001]. The manipulation of sales level was successful.

### Choice

The results of the Chi-square test showed that participants in the low level of sales condition were more likely to choose a selfexpressive product than participants in the high level of sales (Mlow = 41%, Mhigh = 16%, χ <sup>2</sup> = 10.830, p < 0.005). Conversely, participants were more likely to select functional products with a high sales level (Mhigh = 84%, Mlow = 59%, χ <sup>2</sup> = 10.830, p <0.005) (see **Figure 2**), in support of H1. Furthermore, there was no significant difference in the influence of gender on choice (χ 2 =0.343, p = 0.558 > 0.05).

### Discussion

The results of Study 1A showed that participants were more willing to choose self-expressive (vs. functional) products at low (vs. high) sales levels. However, it might be considered that the two products in Study 1A were so different in terms of use that participants' valuation and a real need for them would result in

bias. Therefore, in Study 1B, participants were exposed to two similar products.

### Study 1B

Study 1B also examined whether consumers choose different products (described as functional or self-expressive) considering different sales levels. Individuals' choice of self-expressive products were expected to more likely correspond with the low sales level.

### Methods

### Participants and Design

Participants (N = 163, Mage = 21.5, SDage = 0.85, 53% female), who were invited to complete an online questionnaire about their preference and perceptions about a birthday gift, received compensation of 10 RMB. All participants were randomly assigned to one of two conditions (sales level: high or low).

### Stimuli and Procedure

Cups were selected as stimulus in this study: adiabatic cup (functional product) or distinctive cup (self-expressive product). In a pre-test with 35 participants (Mage = 20.33, SDage = 0.74, 46% female), participants were invited to use a 7-point scale same as Study 1A to rate their agreement with the products' capability to signal status (Wang and Wallendorf, 2006; Chan et al., 2012), Cronbach's α = 0.83. In the pre-test, participants perceived that the distinctive cup had a significantly stronger capability to signal status than the adiabatic cup (Mdistinctivecup = 5.2, Madiabaticcup = 4.1, p < 0.001); the selection of two products was suitable.

All instructions and questionnaires were presented via desktop as Study 1A. Each participant was exposed to a scenario: "your birthday is coming, and one of your good friends prepared to give you a birthday gift (a cup) and told you to use it by yourself. He/she allows you to choose one of two (either a distinctive cup or an adiabatic cup, with same price and same specifications)." Participants exposed to high (low) sales level were told that both cups were used by many (few) people. Then, participants chose one from two cups. Finally, participants' perception of sales level were measured on a 7-point scale as Study 1A.

### Results

Eight of one hundred sixty three participants were deleted for either inconsistent answers or incomplete answers. One hundred and fifty five valid data points were used (Nhigh = 79, Nlow = 76).

### Manipulation Check

Participants exposed to the high sales level more strongly agreed with the statement that many others were likely to own each product (M = 5.3) than those exposed to the low sales level condition [M = 2.13; t(153) = 3.13, p < 0.001]; In addition, participants exposed to the high sales level condition agreed significantly less with the statement that few others were likely to own each product [Mhigh = 2.55, Mlow = 5.15; t(153) = 3.12, p < 0.01]; the manipulation of sales level was successful in Study 1B.

### Choice

A Chi-square test of participants' reported choice showed that consumers were more likely to choose the distinctive cup with a lower sales level (61.8%, χ <sup>2</sup> = 7.915, p < 0.01) compared to the one with a high sales level (39.2%). Conversely, individuals were more likely to select an adiabatic cup with a high sales level (Mhigh = 60.8%, Mlow = 38.2%, χ <sup>2</sup> = 7.915, p < 0.01, (see **Figure 3**). That is, in line with Study 1 and H1, individuals had a stronger intention to accept self-expressive products (vs. functional products) with low (vs. high) sales level. Furthermore, there was no significant difference in the influence of gender on choice (χ <sup>2</sup> = 0.008, p = 0.927 > 0.05).

### Discussion

Study 1B replicated and strengthened the results of Study 1A. Different from Study 1A, Study 1B used products that were more similar than those in Study 1A as stimulus and presented a scenario of accepting a birthday gift. As expected, participants' choice did not differ from that in Study 1A, in supporting of H1 again. However, it might be doubted that participants have to choose between a self-expressive product and a functional product in either high or low sales level condition, which is different from the online purchase process, where they can choose one product with various sales levels.

### Study 2

In Study 2, the underlying mechanisms behind the effect of sales level on self-expressive products' purchase intention were explored initially. Specifically, how exposure to high (vs. low) sales level of self-expressive (vs. functional) products influences a consumer's purchase intention were examined. It was expected that consumers' purchase intentions to be higher when exposed a low (vs. high) sales level of self-expressive (vs. functional) products (H1). Moreover, perceived self-image exclusivity and perceived face threat were expected to serially mediate the effect of sales level on consumers' purchase intentions of self-expressive products (H2).

### Methods

#### Participants and Design

Participants (N = 224, Mage = 19.4, SDage = 1.21, 51% female), who received compensation of 10 RMB, were invited to complete an online questionnaire about their purchase intention and perceptions about two products they would need in the coming winter. Study 2 used a two factors (sales level: high vs. low) <sup>∗</sup> (product type: self-expressive product vs. functional product) between-subjects design. All participants were randomly assigned to one of four conditions.

#### Stimuli and Procedure

Fashion coats and heating blankets were selected as selfexpressive products and functional products, respectively. In a pre-test with 53 participants (Mage = 21.02, SDage = 1.13, 48% female), a same measurement of the products' capability to signal status as Study 1A & 1B was repeated to successfully demonstrate that the fashion coat (heating blanket) was an expressive (a

functional) product (Mfashion coat = 5.4, Mheating blanket = 2.9, p < 0.001). Furthermore, another stimulus was a picture of snow, which can enhance the desire to buy the above products.

All instructions and questionnaires of four conditions were presented via desktop as Study 1A and Study 1B. Each participant in the conditions of self-expressive products or functional products was exposed to the picture of snow and a scenario stating: "winter is coming; you decide to buy a fashion coat (a heating blanket) for yourself, so you start to choose." Then, participants exposed to high (low) sales level conditions expected to find a fashion coat or a heating blanket with good style, price and material, etc., in the list of products with high (low) sales level.

#### Measures

After exposure to the scenario, participants in each condition responded to measures about their purchase intentions and perception of different products with different sales levels. The following dependent measures were used: (1) purchase intention (Dodds et al., 1991): How likely would you be to buy the fashion coat (heating blanket)? (1=not at all, 7=very much); (2) perceived self-image exclusivity (made some changes from Steinhart et al., 2014): you infer that others around you will not have the same image as you when using the fashion coat (heating blanket). (1= strongly disagree, 7= strongly agree); (3) perceived face threat: seven items of the severity of face threat scale from Litt et al. (2014) were adopted and changed slightly: "using the fashion coat (heating blanket) will make me feel awkward," etc., (1 = strongly disagree, 7 = strongly agree), Cronbach's α = 0.904 (see Appendix A); (4) perceived uniqueness was measured to exclude the influence of uniqueness on the underlying mechanism: the extent to which others think the product reflects its user's uniqueness (1 = very low, 7 = very high); (5) sales level perception was measured as in Study 1A and Study 1B.

### Results

Sixteen of two hundred and twenty four participants were deleted for either inconsistent answers or incomplete answers. Two hundred and eight valid data points were used (Nhigh and self-expressive = 50, Nhigh and functional = 54, Nlow and self-expressive = 51, Nlow and functional = 53).

#### Manipulation Check

Participants exposed to the high sales level more strongly agreed with the statement that many others were likely to own each product (M = 5.6) than those exposed to the low sales level condition [M = 1.12; t(206) = 2.66, p < 0.01]; participants exposed to the high sales level condition agreed significantly less with the statement that few others were likely to own each product [Mhigh = 1.56, Mlow = 5.23, t(206) = 2.95, p < 0.01]; the manipulation of sales level was successful in Study 2.

#### Purchase Intentions

We initially conducted a 2 × 2 between-subject ANOVA analysis, with two sales level (high or low) and two product types (self-expressive or functional). The main effect of sales level on purchase intention was not found to be significant, F(1,204) = 0.180, p = 0.671 > 0.05, but the interaction effect between sales level and product types was significant, F(1,204) = 15.638, p < 0.001. Specifically, an ANOVA analysis showed that the effect of sales level on purchase intentions of selfexpressive products was significant, F(1,99) = 11.896, p < 0.005. Participants expressed higher intention (M = 5.1) to purchase the fashion coat with low sales level but lower intention (M = 4.1) to buy that with high sales level. In contrast, additional ANOVA analysis revealed that, for functional products, participants were significantly more likely to buy a heating blanket with a high sales level (M = 5.52) than one with a low sales level (M = 4.03), F(1,105) = 7.104, p < 0.01, (see **Figure 4**). Thus, H1 was supported again. Furthermore, there was no significant difference in the influence of gender on purchase intention of both fashion coat, F(1,102) = 0.466, p = 0.496 > 0.05, and heating blanket, F(1,102) = 0.002, p = 0.965 > 0.05.

#### Serial Mediation Test

To provide evidence for the underlying psychological mechanisms, analysis followed the steps suggested by Hayes' PROCESS procedure (Hayes, 2013). Respectively, mediation test on the conditions of self-expressive products' and functional products' purchase intention were conducted. Perceived uniqueness, gender and age were used as covariates throughout all the analysis. As shown in **Figure 5**, findings reveal that for self-expressive products' purchase, the serial mediation effect of perceived exclusivity→ perceived face threat explains the negative impact of high level sales on products' purchase intentions (B = −0.28, bootstrapped 95% CI: −0.7478, −0.0362). Given that the direct effect of sales level on purchase intention is not significant (B = −0.16, bootstrapped 95% CI: −1.0557, 0.7292), it can be concluded that the serial effect of perceived exclusivity→ perceived face threat fully mediates the effect of sales level on purchase intention for self-expressive products' purchase (H2 supported). In other words, for self-expressive products' purchase, low (vs. high) level sales significantly increases perceived self-image exclusivity (Mlow = 5.13, Mhigh = 2.24; t = −12.6, p < 0.001), which leads to weaker (vs. stronger) perceived face threat (Mlow = 3.51, Mhigh = 4.83;

t = 4.8, p < 0.001), and finally increases (vs. decreased) purchase intentions.

However, for functional products' purchase, the serial mediation effect of perceived exclusivity→ perceived face threat was not significant (B = −0.0014, bootstrapped 95% CI: −0.1219, 0.0914), as expected. That is, for functional products, low (vs. high) sales level neither significantly increases perceived selfimage exclusivity (Mlow = 5.31, Mhigh = 4.32; t = −6.335, p > 0.05), nor yields significantly different perceived face threat (Mlow =2.44, Mhigh = 2.83; t = −1.58, p > 0.05) considering two sales levels. Therefore, sales level could not influence consumers' purchase intention by the serial mediation effect of perceived exclusivity→ perceived face threat.

### Discussion

Study 2 shows that individuals respond to low sales level with a more positive purchase intention when purchasing selfexpressive products. Conversely, when purchasing functional products, individuals exhibit more positive purchase intention when sales level is high (vs. low) (supporting H1 once again). In addition, ruling out perceived uniqueness as the alternative explanation, it was demonstrated that, when purchasing selfexpressive products with low (high) sales level, increasing (decreasing) perceived self-image exclusivity drives weaker (stronger) perceived face threat, which in turn influences consumers' purchase intentions. However, when purchasing functional productions, the mechanism of self-expressive products' sales level on purchase intention is not a concern. H2 was supported.

However, it might be doubted that when purchasing selfexpressive products, there may be other explanations in addition to perceived self-image exclusivity, perceived face threat. In Study 3, H2 were tested by discounting the belief that "high sales means commonplace and negative evaluations from others," which the paper discussed as the cause of the serial mediation.

### Study 3

Study 3 aimed once again to verify the mechanism underlying the effect of sales level of self-expressive products on purchase intention, using moderation approaches. If consumers activate their perceived self-image exclusivity and perceived face threat to

judge a self-expressive product with low (high) sales level, then discounting the belief of "high sales means commonplace and negative evaluations from others" will attenuate the effect of low (high) sales level on increasing (decreasing) consumers' purchase intention of self-expressive products. To examine this prediction, participants' belief were manipulated through a priming task. Moreover, participants' activation of the belief were measured after the priming task and examined whether this belief mediated the effect on product purchase intention.

### Methods

#### Participants and Design

Participants (N = 186, Mage = 19.8, SDage = 1.22, 49% female), who received compensation of 10 RMB, were invited to complete an online questionnaire about their purchase intention and perceptions about a fashion coat they would need in the coming winter. All participants were randomly assigned to one of three conditions (sales level and discounting belief: high level and baseline vs. low level and baseline vs. high level and discounting belief).

#### Stimuli and Procedure

On the one hand, consistent with Study 2, fashion coat was the self-expressive product and a snowy picture was the stimuli in this study. On the other hand, a fictitious research report titled "High Sales Level Is Not a Big Deal" was another stimuli. The report used approximately 300 words to describe a research study showing that even if someone bought a self-expressive product with high sales level, it is hard to find someone using the same product because of the large population, and notably, two people who use the same self-expressive product would not be negatively evaluated by others. The key message of the report was that empirical evidence does not support a significant association between high sales level and low self-image exclusivity and stronger face threat. A pre-test (N= 51, 45% female) confirmed that participants who read the "High Sales Level Is Not a Big Deal" report, compared with those who just read a weather report in the baseline condition, were less likely to think high sales level of selfexpressive products was strongly associated with low self-image exclusivity and high face threat.

All instructions and questionnaires were presented via desktop as Study 2. First, participants were invited to complete a reading task, which was reading a report either about weather (high level and baseline, low level and baseline conditions) or "High Sales Level Is Not a Big Deal" (high level and discounting belief condition). Second, participants were guided to summarize the main idea of the report. Then, participants with the condition of high sales levels and participants with the condition of low sales level were presented with the same snowy picture and scenario of searching for fashion coats as used in Study 2, respectively.

#### Measures

After exposure to the scenario, participants in each condition responded to measures about their purchase intentions and perception toward fashion coats with different sales levels, just as the participants in Study 2 did. The following dependent variables were measured: (1) purchase intention; (2) perceived self-image exclusivity; (3) perceived face threat; (4) perceived uniqueness; (5) sales level perception.

### Results

Ten of one hundred eighty six participants were deleted for either inconsistent answers or incomplete answers. One hundred seventy six valid data points were used (Nhigh and baseline = 53, Nlow and baseline = 59, Nhigh and discount = 64).

#### Manipulation Check

Participants exposed to the high sales level with both baseline and discount belief conditions more strongly agreed with the statement that many others were likely to own the fashion coat (M = 5.3) than those exposed to the low sales level condition (M = 1.27; t(174) = 2.11, p < 0.01); in addition, they agreed significantly less with the statement that few others were likely to own the fashion coat [Mhigh = 1.89, Mlow = 5.41; t(174) = 1.96, p < 0.001]; the manipulation of sales level was successful in Study 3.

#### Purchase Intention

We initially conducted an ANOVA analysis among three conditions (high and baseline, low and baseline, high and discount). The differences on purchase intention among three conditions were found to be significant, F(2,173) = 13.005, p = <0.001. Specially, an ANOVA analysis between two conditions (high and baseline, low and baseline) showed that the effect of sales level on the purchase intention of fashion coat was significant, F(1,110) = 8.890, p < 0.005. Participants exposed to a low sales level had a higher intention (M = 5.1) to buy the coat, but those exposed to a high sales level had a lower intention (M = 4.23) to buy it. In line with Study 2, this further supported H1. Furthermore, there was no significant difference in the influence of gender on purchase intention of fashion coat, F(1,110) = 1.007, p = 0.302 > 0.05.

### Serial Mediation Test

To further provide evidence for the serial mediation of Study 2, analysis following the steps suggested by Hayes' PROCESS procedure (Hayes, 2013) to test the serial mediation effects of selfexpressive products (here a fashion coat). Perceived uniqueness, gender and age were used as covariates throughout all the analysis as Study 2.

First, an analysis between high sales level (coded as 1) and low sales level (coded as 0), both with baseline belief, (see Model 1 of Appendix B), showed that the serial mediation effect of perceived self-image exclusivity→ perceived face threat explains the negative impact of high (vs. low) sales level on products' purchase intentions (B = −0.27, bootstrapped 95% CI: −0.7130, −0.0526), H2 was further supported. In other words, low level (vs. high level) sales significantly increase perceived product exclusivity (Mlow = 5.15, Mhigh =2.34; t = −12.01, p < 0.001), which leads to weaker (vs. stronger) perceived face threat (Mlow = 3.74, Mhigh = 4.77; t = 4.17, p < 0.001), and finally drives increased (vs. decreased) purchase intentions.

Second, an analysis comparing high sales level with the baseline condition (coded as 1) and high sales level with the discounted belief condition (coded as 0) showed that the serial mediation effect of perceived self-image exclusivity→ perceived face threat explains the negative impact of high level sales with the baseline belief (vs. discount belief) on products' purchase intentions (B = −0.1211, bootstrapped 95% CI: −0.3292, −0.0304) (see Model 2 of Appendix B). Thus, it can be concluded that the manipulation of the discounted belief is legitimate. The high sales level with discounted belief (vs. baseline belief) significantly increases perceived selfimage exclusivity (Mdiscount belief = 3.66, Mbaseline belief = 2.34; t = −4.77, p < 0.001), which leads to weaker (vs. stronger) perceived face threat (Mdiscount belief = 3.80, Mbaseline belief = 4.77; t = 3.89, p < 0.001), and finally drives increased (vs. decreased) purchase intentions.

Third, an analysis comparing the conditions of low sales level with baseline belief (coded as 0) and high sales level with discount belief (coded as 1) revealed that not only was the serial mediation effect of perceived self-image exclusivity→ perceived face threat not significant (B = −0.0309, bootstrapped 95% CI: −0.1276, 0.0013) (see Model 3 of Appendix B), and the direct effect of sales level on purchase intention was also not significant (B = 0.2460, bootstrapped 95% CI: −0.2886, 0.7805). That is, the discounted belief with high sales level condition increased perceived selfimage exclusivity, which weakened perceived face threat and further increased purchase intention. This is similar to the effect on products with a low sales level condition (with the baseline belief).

In conclusions, findings revealed that when discounting the belief which causes the perceived self-image exclusivity and face threat, the effect of different sales level on self-expressive products' purchase intention would not significant. Moreover, consumers in discounted belief conditions, rather than those in baseline conditions, would have significant stronger intentions to buy self-expressive products with high sales level. Together, the results of Study 3 provided more powerful and favorable evidence for H2.

### Discussion

Again, Study 3 demonstrated that for self-expressive products, consumers hold stronger intention to buy given a low sales level. Furthermore, by priming a discounted belief, Study 3 provided powerful evidence of the serial mediation consistent with Study 2 for self-expressive products, which supported the proposed mechanism for the effect of self-expressive products' sales level on consumer purchase intention in H2.

### GENERAL DISCUSSION

This study demonstrated a link between self-expressive products and naive theory of exclusivity in different contexts of buying products and explored the connection from the perspective of face threat. That is, exposing individuals to a self-expressive (vs. functional) product triggers the naive theory of exclusivity (vs. popularity), and a low (vs. high) sales level will further activate the naive theory of exclusivity (vs. popularity). This supports the premise concerning self-expressive products with different sales level that consumers are more likely to choose a selfexpressive (vs. functional) product in low (vs. high) sales level condition (Studies 1A and 1B), and that when purchasing a selfexpressive (vs. functional) product, consumers are more likely to buy one with low (vs. high) sales level (Studies 2 and 3). Notably, unlike previous studies, which hold consumers' self-perceptions of uniqueness as a main factor of self-expressive products' purchase and contextual cues (Steinhart et al., 2014), this study, ruling out the influence of perceived uniqueness, found that the serial mediation effect of perceived self-image exclusivity → perceived face threat explains the "I'll follow the minority" effect of self-expressive products. That is, findings revealed that low sales level will induce higher perceived self-image exclusivity, thereby weaken individuals' perceived face threat, and ultimately increase the intention to purchase the self-expressive products. Conversely, high (vs. low) sales levels will reduce perceived selfimage exclusivity and hence consumers' perceived face threat, ultimately undermining the purchase intention (Study 2). More importantly, when discounting the belief that "high sales means commonplace and negative evaluations from others," individuals' negative attitude toward the high sales level of self-expressive products will be eliminated. That is, when discounting the belief, there was no significant difference between consumers' purchase intentions toward self-expressive products with high and low sales level, and there was a significant difference between the high sales level group whose belief was discounted and the high sales level group whose belief was not discounted (Study 3).

From a theoretical perspective, the present study considered the sales level as a contextual cue that triggered the naive theory of exclusivity. The study extended the work of Steinhart et al.(Steinhart et al., 2014) that which one of naive theories a product induced depends on whether the product expresses users' self-image, focusing on the influence of self-expressive products' sales level on purchase intention. Different from the previous explanation from the perspective of individuals' uniqueness (Steinhart et al., 2014), this study explained the naive theory (Furnham, 1988; Deval et al., 2013) from the perspective of perceived face threat for the first time and provided evidence that the sales level information will affect perceived self-image exclusivity and then affect perceived face threat, which influences consumers' purchase intentions. In addition, a new perspective was proposed about online consumer behavior, namely, face threat, which has been investigated in social media (Oeldorf-Hirsch et al., 2017) but should also receive attention in online consumption research.

From a managerial perspective, the findings help marketers consider two trends. First, when promoting new self-expressive products on online retail websites, marketers should display self-expressive products with lower sales level, rather than best sellers, on the home page of recommendations; conversely, the functional products recommended there should have higher sales level. Second, considering a good way to change consumers' belief that "high sales means commonplace and negative evaluations from others" (e.g., by adding descriptions to increase the product exclusivity or decrease the possibility of face threat) will be an opportunity for the further promotion of products that already sell well.

### REFERENCES


Despite these advances, future research should also consider the impact of other factors on the conclusions of the present study. Such factors might include product price and brand reputation. For example, a consumer might buy an expensive self-expressive product with high sales level because that person might be proud to use an expensive product. Similarly, a famous brand will be significant factor that influences the effect of sales levels on the consumer's perception and purchase intention of self-expressive products, that is, when perceiving a self-expressive product with high sales level to be a famous brand, consumers might buy it because it is a brand with high prestige which will help them signal their identity. It may also be interesting to explore characteristics of individuals as a possible moderator. Specifically, individuals with stronger (vs. weaker) face consciousness will pay more attention to the sales level of self-expressive products, which will create an opportunity for online retail websites to make personalized recommendations for different individuals. Besides, all subjects in the present research are young consumers, whether the research conclusion is applicable to older consumer groups remains to be tested in the future.

### AUTHOR CONTRIBUTIONS

XW study design, data collection, data analysis, paper revising. TW study design, data collection, data analysis, paper writing, paper revising. JW study design, data analysis, paper revising.

### ACKNOWLEDGMENTS

The authors gratefully thank the financial support provided by the State Program of National Social Science Foundation of China (Grant No. 13BGL021), Research Project of Chongqing Education Commission Humanistic and Social Science in 2016 (Grant No. 16SKGH058).

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2018.01135/full#supplementary-material


**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2018 Wan, Wang and Wu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.