Skip to main content

BRIEF RESEARCH REPORT article

Front. Psychol., 30 April 2019
Sec. Educational Psychology

The Utrecht Work Engagement Scale for Students (UWES–9S): Factorial Validity, Reliability, and Measurement Invariance in a Chilean Sample of Undergraduate University Students

  • 1Escuela de Psicología y Filosofía, Universidad de Tarapacá, Arica, Chile
  • 2Research Unit of Occupational & Organisational Psychology and Professional Learning, KU Leuven, Leuven, Belgium
  • 3Department of Social, Health and Organizational Psychology, Utrecht University, Utrecht, Netherlands
  • 4WANT Research Team, Universitat Jaume I, Castellón de la Plana, Spain

This brief report examines the within–network construct validity of the UWES–9S in a convenience sample of 1502 Chilean students (52% were female) ranging between 18 and 25 years old. The results of confirmatory factor analysis supported a solution with three related factors that fit significantly better than a one-factor solution. The three subscales (i.e., vigor, dedication, and absorption) and the overall UWES–9S showed satisfactory internal consistency. The results of multiple–group confirmatory factor analysis supported gender invariance. Overall, the UWES–9S was found to be a reliable and valid scale to assess academic engagement in Chilean undergraduate university students.

Introduction

The concept of work engagement –a positive, fulfilling, work related state of mind characterized by vigor, dedication, and absorption (Schaufeli et al., 2002b) – refers to the work activities employees perform. However, this construct has also been applied to the activities students perform, referred to as academic engagement (Schaufeli et al., 2002a). The reasoning is that, psychologically speaking, the activities students perform can also be considered as “work,” in the sense of goal–directed and structured activities, which are compulsory in nature. More specifically, vigor refers to high levels of energy and mental resilience while studying, the willingness to invest effort in studying, and persistence even when encountering difficulties; dedication refers to being strongly involved in one’s studies and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge; and absorption refers to being fully concentrated and happily engrossed in what one is studying, where time passes quickly and one finds it difficult to detaching him/herself from studying (Schaufeli, 2017).

After almost two decades of research, we now know that academic engagement is associated with personality traits (Sulea et al., 2015), personal resources (Siu et al., 2014), and positive emotions (Ouweneel et al., 2011). Moreover, it has positive consequences for students, such as greater involvement in their studies (Loscalzo and Giannini, 2018), higher levels of wellbeing (Tayama et al., 2018), and better academic performance (Salanova et al., 2003, 2010).

The research on academic engagement carried out so far is at pre-professional levels (i.e., undergraduate university students) and uses the Utrecht Work Engagement Scale for students (UWES–S; Schaufeli et al., 2002a). The UWES–S is the student version of the most widely used instrument to assess work engagement, the Utrecht Work Engagement Scale (UWES; Schaufeli et al., 2002b). Originally, the UWES included 17 items and three dimensions (i.e., vigor, dedication, and absorption). Later, for pragmatic reasons, the UWES was reduced, resulting in a 9–item version (UWES–9) that showed adequate psychometric properties as well (Schaufeli et al., 2006).

The UWES–9 has been validated in industrial–organizational settings in various countries (e.g., Brazil–Sinval et al., 2018; Finland–Seppälä et al., 2009; Italy–Balducci et al., 2010; Japan–Shimazu et al., 2008; Norway–Nerstad et al., 2010; Russia–Lovakov et al., 2017; Serbia–Petrovic et al., 2017; and South Africa–Storm and Rothmann, 2003; among others–Schaufeli et al., 2006). However, validity studies on the UWES–9S are still scarce with mixed evidence for its three–factor structure (Sánchez–Cardona et al., 2016; Loscalzo and Giannini, 2018). However, it cannot be ruled out that this is caused by flaws in sampling and research design (e.g., Parra and Pérez, 2010; Portalanza–Chavarria et al., 2017).

The UWES–9S is also popular in Spanish–speaking South American countries, where no rigorous psychometric evaluation has been carried out so far. Thus, aim of the present brief report is to fill this gap by examining the factorial validity, reliability, and gender invariance of the UWES–9S in a sample of undergraduate Chilean university students. Providing empirical evidence about the UWES–9S makes it possible to examine the continuum from academic to working life, which has hardly been addressed (Upadyaya and Salmela–Aro, 2013).

Based on the arguments presented, we hypothesize the following: The UWES–9S will demonstrate acceptable psychometric properties in a Chilean sample of undergraduate university students.

Materials and Methods

Participants

The sample consisted of 1502 Chilean undergraduate university students, aged between 18 and 25 years (M = 20.55; SD = 1.91; 48% male and 52% female). The students came from different fields of study: social sciences (34%; include law, psychology, and Anthropology), health sciences (36%; include Nursing and Medicine), and natural sciences (30%; include Physics, Math, and Chemistry). Of the 1502 participants, 23% were in the first year, 34% in the second year, 26% in the third year, 16% in the fourth year, and 7% in the fifth year of their studies. Finally, 15% correspond to low, 80% to medium, and 5% to high socio–economic levels.

Procedure

The data were collected in the context of a research project about the role of students’ well–being in academic achievement. The project was approved by the research ethics committee of the host university. The participants voluntarily filled out an online questionnaire during their usual class schedule.

Instrument

The UWES–9S (Schaufeli et al., 2006) is a nine–item self–report scale grouped into three subscales with three items each (see Table 1): vigor (VI), dedication (DE), and absorption (AB). All items are scored on a seven–point frequency rating scale ranging from 0 (never) to 6 (always). For the purposes of this study, the Spanish version of the UWES–9S1 was adapted slightly –following the guidelines of the International Test Commission for adapting tests across cultures (Muñiz et al., 2013)– in order to improve its comprehension by undergraduate Chilean students. Furthermore, before the data collection, the nine items were piloted in a small group of participants to verify their clarity (n = 10). None of the participants expressed problems with understanding the items or the answering format of the UWES–9S.

TABLE 1
www.frontiersin.org

Table 1. Descriptive information of the UWES–9S and factor loadings resulting from CFA.

Statistical Analyses

All analyses were conducted using SPSS AMOS 23.0 (IBM Corp, 2015) and JASP computer software (Jasp Team, 2018). First, we examined the descriptive statistic values, the internal consistency –using Cronbach’s alpha and McDonald’s omega indexes– and gender differences. To examine the factor structure, we performed confirmatory factor analysis (CFA) through the maximum likelihood estimation approach. To evaluate its goodness of fit, we computed the chi–square (χ2) and normed χ2, root–mean–squared error of approximation (RMSEA) with a confidence interval (90% CI), comparative fit index (CFI), and standardized root mean residual (SRMR). To help evaluate the cut–off and determine model fit, we followed the guidelines published by the European Journal of Psychological Assessment (Schweizer, 2010). To explore gender invariance –following Chen (2008)– we tested for configural (i.e., same structure across groups), metric (i.e., same factor loadings across groups), and scalar invariance (i.e., same intercepts across groups) through multiple–group CFA. To evaluate the magnitude of the change in the fit index, we followed the recommendations of Cheung and Rensvold (2002), who suggested that an absolute difference in CFI (i.e., ΔCFI) or SRMR (i.e., ΔSRMR) of less than 0.01 would indicate that the models are equivalent in terms of fit. In addition, although the chi–squared difference test may overstate non-substantive discrepancies –because it is sensitive to sample size– we also report this value.

Results

Preliminary Analysis and Reliably of the Scores

As Table 1 shows, the skewness of the score distribution of the UWES–9S items ranged between −0.027 and −0.507, and kurtosis values ranged between −0.298 and −0.117. According to Finney and DiStefano (2006), these values indicate that the assumption of normality has not been violated. In addition, the size of the correlations ranged between 0.65 and 0.89, revealing strong significant relationships between the subscales of the UWES. Independent–sample t-tests revealed that –in accordance with previous studies (Schaufeli et al., 2017)– female students (M = 3.510, SD = 1.253) scored significantly higher than male (M = 3.315, SD = 1.258) students, t (1499) = 3.007, p < 0.05, d = 0.155, 95% IC (0.054, 0.257). However, based on Cohen’s (1988) criterion, the effect size was small. Finally, the analysis of Cronbach’s alpha and McDonald’s omega indexes if an item is deleted allowed us to retain the nine items and obtain good internal consistencies for the subscales and the overall UWES–9S (Table 1).

Sources of Validity Evidence of Internal Structure

Based on previous research on the validity of the UWES (e.g., Schaufeli et al., 2006, 2017), we tested two models: one that assumes that all the engagement items load in one single factor (M1), and one that proposes three related factors (M2). The results of the CFAs show that the data fit of both models to the data is good, except that the RMSEA value exceeds the criterion of 0.08 (see M1 and M2 in Table 2). To decide whether the model needed re–specification, we inspected the modification indexes, which indicated that allowing the errors for two items from the absorption subscale to correlate could increase the model fit (items 8 and 9, r = 0.30, p < 0.005). The two resulting models (see M3 and M4 in Table 2) fitted the data significantly better (see Table 2, M1–M3 for the one–factor solution and M2–M4 for the three–factor solution). In addition, the comparison of the re–specified models shows that the three–factor model (M4) fits significantly better than the re–specified model with one factor (M3). Hence, M4 was used as the baseline model in the multiple–group CFA.

TABLE 2
www.frontiersin.org

Table 2. Fit indexes for single–group and multiple–group CFA.

Measurement Invariance Across Gender

The baseline model showed an acceptable fit, with support for configural invariance (see M5 in Table 2). In the next step, equality constraints were imposed on all factor loadings in order to examine metric invariance. The resulting model (M6 in Table 2) also achieved an acceptable fit. When comparing M5 and M6, the change in the chi–square difference test was not statistically significant. In addition, the absolute difference in CFI and SRMR was less than 0.001. Thus, we concluded that metric invariance across genders is supported. Next, equality constraints were imposed on all intercepts to test scalar invariance. This model (M7 in Table 2) also achieved an acceptable fit. Therefore, following the same reasoning described above, we concluded that scalar invariance across genders is supported. Taken together, we concluded that academic engagement has the same meaning in our male and female participants.

Discussion

Limited attention has been paid to examining the psychometric properties of the UWES–9S, particular in Spanish–speaking Latin American countries. Therefore, this brief report provides empirical evidence about the factor validity, reliability, and measurement invariance across male and female Chilean students.

Our results coincide with international research on the factor structure of the UWES–9 (Schaufeli et al., 2002a, b, 2006; Balducci et al., 2010; Sánchez–Cardona et al., 2016; Loscalzo and Giannini, 2018; Tayama et al., 2018). In other words, vigor, dedication, and absorption are satisfactorily explained by a solution with three related factors. In addition, reliability analysis –based on Cronbach’s alpha and McDonald’s omega indexes– of the subscales and the overall UWES–9S indicated high internal consistency. Gender invariance was also demonstrated, revealing its capacity to evaluate academic engagement in a similar way and with the same precision in male and female undergraduate university students. Together, these results lead us to conclude that the UWES–9S produces scores that could be of value, in terms of reliability and validity, in measuring academic engagement in Chilean undergraduate university students.

The main strength of the present study is the large sample used. However, there are also some limitations that should be mentioned. First, we used a convenience sample that does not represent all Chilean undergraduate university students adequately, or all academic fields. Thus, a more representative and diversified sample is needed to compare our results. Second, we only focused on a within–network validity approach. Hence, a nomological network approach is needed to explore significant relationships between the UWES–9S and other study–related variables, such as academic burnout, psychological capital, or academic performance, to test convergent and predictive validity. Third, modification indices have been used to allow two errors to correlate in order to improve the fit of the UWES–9S. This strategy is coherent with previous UWES research (e.g., Schaufeli, 2017; Loscalzo and Giannini, 2018), and it is justified by overlapping meaning or item content (MacCallum et al., 1992): “I am immersed in my studies” and “I get carried away when I am studying.”

In order to expand the future research on academic engagement, it would be interesting to examine the applicability of the UWES–9S at levels that have not yet been explored (e.g., class–level academic engagement). In addition, examining the predictive role of the UWES–9S –as an external report (e.g., peer rated by teachers and parents)– in students’ academic outcomes could be considered a fruitful future line of research. Finally, we want to mention two challenges that would help to increase the knowledge about the UWES in academic contexts. The first would be to examine the equivalence (i.e., measurement invariance) of the UWES–9S across countries, and the second to establish cut–off scores for high and low levels of academic engagement.

Ethics Statement

This study was carried out in accordance with the recommendations of Ethics Committee of the Universidad de Tarapacá, with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by Research Ethics Committee of the Universidad de Tarapacá.

Author Contributions

All authors contributed equally to the research design and wrote the manuscript.

Funding

This work was supported by Ministerio de Economía y Competitividad (Grant No. # PSI2015–64933–R), Universitat Jaume I (Grant No. # UJI–B2017–81), and Universidad de Tarapacá (Grant No. # UTA Mayor 3749/18).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

  1. ^ www.wilmarschaufeli.nl

References

Balducci, C., Fraccaroli, F., and Schaufeli, W. B. (2010). Psychometric properties of the italian version of the utrecht work engagement work engagement scale (UWES–9). a cross–cultural analysis. Eur. J. Psychol. Assess. 26, 143–149. doi: 10.1027/1015-5759/a000020

CrossRef Full Text

Chen, F. F. (2008). What happens if we compare chop sticks with forks? the impact of making in appropriate comparisons in cross–cultural research. J. Personal. Soc. Psychol. 95, 1005–1017. doi: 10.1037/a0013193

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness–of–fit index for testing measurement in variance. Struct. Equa. Model. 9, 233–255. doi: 10.1207/S15328007SEM0902_5

CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, 2nd Edn. Hillsdale, NJ: Lawrence Erlbaum.

Finney, S. and J., DiStefano, C. (2006). “Non–normal and categorical data in structuralequation modeling,” in Structural Equation Modeling: A Second Course, eds G. R. Hancock and R. O. Mueller (Greenwich, CT: Information Age), 269–314.

Google Scholar

IBM Corp (2015). IBM SPSS Statistics for Macintosh, Version 23.0. Armonk,NY: IBM Corp.

Jasp Team. (2018). JASP (Version 0.9) [Computer software].

Loscalzo, Y., and Giannini, M. (2018). Study engagement in Italian university students: a confirmatory factor analysis of the utrecht work engagement scale–student version. Soc. Indic. Res. 142, 845–854. doi: 10.1007/s11205-018-1943-y

CrossRef Full Text | Google Scholar

Lovakov, A. V., Agadullina, E. R., and Schaufeli, W. B. (2017). Psychometric propertiesof the russian version of the Utrecht Work engagement scale (UWES–9). Psychol. Russ. State Art 10, 145–162. doi: 10.11621/pir.2017.0111

CrossRef Full Text | Google Scholar

MacCallum, R. C., Roznowski, M., and Necowitz, L. B. (1992). Model modifications in covariance structure analysis: the problem of capitalization on chance. Psychol. Bull. 111, 490–504. doi: 10.1037/0033-2909.111.3.490

CrossRef Full Text | Google Scholar

Muñiz, J., Elosua, P., and Hambleton, R. K. (2013). International test commission guidelines for test translation and adaption: second edition. Psicothema 25, 151–157. doi: 10.7334/psicothema2013.24

PubMed Abstract | CrossRef Full Text | Google Scholar

Nerstad, C. G. L., Richardsen, A. M., and Martinussen, M. (2010). Factorial validity of the utrecht work engagement scale (UWES) across occupational groups in Norway. Scand. J. Psychol. 51, 316–333.

Google Scholar

Ouweneel, E., Le Blanc, P. M., and Schaufeli, W. B. (2011). Flourishing students: a longitudinal study on positive emotions, personal resources and study engagement. J. Posit. Psychol. 6, 142–153. doi: 10.1080/17439760.2011.558847

CrossRef Full Text | Google Scholar

Parra, P., and Pérez, C. (2010). Psychometric properties of utrecht work engagement scale for students, UWES–S (short form), in psychology students. Rev. de Educ. en Ciencias de la Salud 7, 128–133. doi: 10.1186/s13104-018-3388-4

PubMed Abstract | CrossRef Full Text

Petrovic, I. B., Vukelic, M., and Cizmic, S. (2017). Work engagement in serbia:psychometric properties of the serbian version of the utrecht work engagement scale (UWES). Front. Psychol. 8:1799. doi: 10.3389/fpsyg.2017.01799

PubMed Abstract | CrossRef Full Text | Google Scholar

Portalanza–Chavarria, C. A., Grueso–Hinestroza, M. P., and Duque–Oliva, E. J. (2017). Properties of the utrecht work engagement scale (UWES–S9): exploratory analysis of student in Ecuador. Innovar 27, 145–156. doi: 10.15446/innovar.v27n64.62374

CrossRef Full Text | Google Scholar

Salanova, M., Llorens, S., Cifre, E., Martinez, I. M., and Schaufeli, W. B. (2003). Perceived collective efficacy, subjective well–being, and task performance among electronic work groups: an experimental study. Small Group Res. 34, 43–73. doi: 10.1177/1046496402239577

CrossRef Full Text | Google Scholar

Salanova, M., Schaufeli, W. B., Martinez, I., and Bresó, E. (2010). How obstacles and facilitators predict academic performance: the mediating role of study burn out and engagement. Anxiety Stress Coping 23, 53–70. doi: 10.1080/10615800802609965

PubMed Abstract | CrossRef Full Text | Google Scholar

Sánchez–Cardona, I., Rodríguez–Montalbán, R., Toro–Alfonso, J., and Moreno, I. (2016). Psychometric properties of the utrecht work engagement scale–student (UWES–S) in university students in puerto rico. Rev. Mexic. de Psicol. 33, 121–134.

Google Scholar

Schaufeli, W. B. (2017). General engagement: conceptualization and measurement with the utrecht general engagement scale (UGES). J. Well Being Assess. 1, 9–24. doi: 10.1007/s41543-017-0001-x

CrossRef Full Text | Google Scholar

Schaufeli, W. B., Bakker, A. B., and Salanova, M. (2006). The measurement of work engagement with a short questionnaire: a cross–national study. Educ. Psychol. Measure. 66, 701–716. doi: 10.1177/0013164405282471

CrossRef Full Text | Google Scholar

Schaufeli, W. B., Martinez, I. M., Marques–Pinto, A., Salanova, M., and Bakker, A. (2002a). Burn out and engagement in university students: a cross–national study. J. Cross Cult. Psychol. 33, 464–481. doi: 10.1177/0022022102033005003

CrossRef Full Text | Google Scholar

Schaufeli, W. B., Salanova, M., Gonzales–Romá, V., and Bakker, A. B. (2002b). The measurement of engagement and burn out: a two–sample confirmatory factor analytic approach. J. Happ. Stud. 3, 71–92. doi: 10.1023/A:1015630930326

CrossRef Full Text | Google Scholar

Schaufeli, W. B., Shimazu, A., Hakanen, J., Salanova, M., and De Witte, H. (2017). An ultra–short measure for work engagement: the UWES–3 validation across five countries. Eur. J. Psychol. Assess. 1–15. doi: 10.1027/1015-5759/a000430

CrossRef Full Text | Google Scholar

Schweizer, K. (2010). Some guidelines concerning the modeling of traits and abilities in test construction. Eur. J. Psychol. Assess. 26, 1–2. doi: 10.1027/1015-5759/a000001

CrossRef Full Text | Google Scholar

Seppälä, P., Mauno, S., Feldt, T., Hakanen, J., Kinnunen, U., Tolvanen, A., et al. (2009). The construct validity of the utrecht work engagement scale: multi sample and longitudinal evidence. J. Happ. Stud. 10, 459–481. doi: 10.1007/s10902-008-9100-y

CrossRef Full Text | Google Scholar

Shimazu, A., Schaufeli, W. B., Kosugi, S., Suzuki, A., Nashiwa, H., Kato, A., et al. (2008). Work engagement in Japan: validation of the Japanese version of the utrecht work engagement scale. Appl. Psychol. 57, 510–523. doi: 10.1111/j.1464-0597.2008.00333.x

CrossRef Full Text | Google Scholar

Sinval, J., Paisan, S., Queiros, C., and Maroco, J. (2018). Brazil–Portugal transcultural adaptation of the UWES–9: internal consistency, dimensionality, and measurement invariance. Front. Psychol. 9:353. doi: 10.3389/fpsyg.2018.00353

PubMed Abstract | CrossRef Full Text | Google Scholar

Siu, O. L., Bakker, A. B., and Jiang, X. (2014). Psychological capital among university students: relationship with study engagement and intrinsic motivation. J. Happ. Stud. 15, 979–994. doi: 10.1007/s10902-013-9459-2

CrossRef Full Text | Google Scholar

Storm, K., and Rothmann, S. (2003). A psychometric analysis of the utrecht work engagement scale in the South African police service. SA J. Indust. Psychol. 29, 62–70. doi: 10.4102/sajip.v29i4.129

CrossRef Full Text | Google Scholar

Sulea, C., van Beek, I., Sarbescu, P., Virga, D., and Schaufeli, W. B. (2015). Engagement, boredom, and burn out among students: basic need satisfaction matters more than personality traits. Learn. Individ. Diff. 42, 132–138. doi: 10.1016/j.lindif.2015.08.0181041-6080

CrossRef Full Text | Google Scholar

Tayama, J., Schaufeli, W. B., Shimazu, A., Tanaka, M., and Takahama, A. (2018). Validation of a Japanese version of the work engagement scale for students. Japan. Psychol. Res. 61, 1–11 doi: 10.1111/jpr.12229

CrossRef Full Text | Google Scholar

Upadyaya, K., and Salmela–Aro, K. (2013). Engagement with studies and work: trajectories from post comprehensive school education to higher education and work. Emerg. Adulthood 1, 247–257. doi: 10.1177/2167696813484299

CrossRef Full Text | Google Scholar

Keywords: UWES, psychometric analyses, academic engagement, university students, invariance across gender

Citation: Carmona-Halty MA, Schaufeli WB and Salanova M (2019) The Utrecht Work Engagement Scale for Students (UWES–9S): Factorial Validity, Reliability, and Measurement Invariance in a Chilean Sample of Undergraduate University Students. Front. Psychol. 10:1017. doi: 10.3389/fpsyg.2019.01017

Received: 15 November 2018; Accepted: 16 April 2019;
Published: 30 April 2019.

Edited by:

Isabel Menezes, Universidade do Porto, Portugal

Reviewed by:

Eduardo Fonseca-Pedrero, University of La Rioja, Spain
Maria Paula Paixão, University of Coimbra, Portugal

Copyright © 2019 Carmona-Halty, Schaufeli and Salanova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Marcos A. Carmona-Halty, marcos.carmona.halty@gmail.com; mcarmonah@uta.cl

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.