Original Research ARTICLE
Transcultural Adaptation of the Oldenburg Burnout Inventory (OLBI) for Brazil and Portugal
- 1William James Center for Research, ISPA – Instituto Universitário, Lisbon, Portugal
- 2Faculty of Psychology and Education Sciences, University of Porto, Porto, Portugal
- 3Faculty of Philosophy, Sciences and Languages of Ribeirão Preto, University of São Paulo, São Paulo, Brazil
- 4Business Research Unit (BRU-IUL), Instituto Universitário de Lisboa (ISCTE-IUL), Lisbon, Portugal
During the last few years, burnout has gained more and more attention for its strong connection with job performance, absenteeism, and presenteeism. It is a psychological phenomenon that depends on occupation, also presenting differences between sexes. However, to properly compare the burnout levels of different groups, a psychometric instrument with adequate validity evidence should be selected (i.e., with measurement invariance). This paper aims to describe the psychometric properties of the Oldenburg Burnout Inventory (OLBI) version adapted for workers from Brazil and Portugal, and to compare burnout across countries and sexes. OLBI's validity evidence based on the internal structure (dimensionality, reliability, and measurement invariance), and validity evidence based on relationships with other variables (work engagement) are described. Additionally, it aims presents a revision of different OLBI's versions—since this is the first version of the instrument developed simultaneously for both countries—it is an important instrument for understanding burnout between sexes in organizations. Data were used from 1,172 employees across two independent samples, one from Portugal and the other from Brazil, 65 percent being female. Regarding the OLBI internal structure, a reduced version (15 items) was obtained. The high correlation between disengagement and exhaustion, suggested the existence of a second-order latent factor, burnout, which presented measurement invariance for country and sex. Confirmatory factor analysis of the Portuguese OLBI version presented good goodness-of-fit indices and good internal consistency values. No statistically significant differences were found in burnout between sexes or countries. OLBI also showed psychometric properties that make it a promising and freely available instrument to measure and compare burnout levels of Portuguese and Brazilian employees.
Work organizations and labor relations all over the world are undergoing significant changes, with an impact on workers' lives and health, since the demands of modern working life are increasing pressure to levels never seen before (International Labour Office, 2016). Thus, the workforce must deal with a new landscape where psychosocial risks at work must be addressed (European Agency for Safety and Health at Work, 2018). Stress is a risk which, at extreme levels, can lead to burnout. Burnout has become a global concern, and work-related stress is a big challenge to organizations' performance and to their workers' health. Burnout levels vary depending on country, occupation, and individual characteristics, among which sex, is one of the most important factors (Purvanova and Muros, 2010). Burnout can affect any worker, with consequences not only in terms of health, safety, and well-being, but also for productivity, quality of service, and cost-effectiveness to the organization (Poghosyan et al., 2010; Carod-Artal and Vázquez-Cabrera, 2013). It is a severe reaction to occupational stress, having in its symptomatology changes to physical and psychological health and behavioral-motivational aspects, expressed through a reduction in job satisfaction or even a change of profession (Marques-Pinto et al., 2003). It is a syndrome (psychological in nature) that may occur when workers chronically face a stressful working environment and feel low resources to face high job demands (Maslach et al., 2001; Bakker and Demerouti, 2007; Maslach, 2015). The definition of Burnout has been expanded from a concept associated with human services professions to a concept related to all kinds of professions that can be affected (Lindblom et al., 2006).
From a historical perspective, burnout was initially considered as a psychological phenomenon in the USA, beginning with studies by the psychologist Freudenberger (1974) and the psychologist Maslach (1976). Despite some criticism (Bianchi et al., 2017) and the existence of several other related constructs, such as karōshi (), meaning “death by overwork” (International Labour Organization, 2013) and karōjisatsu (), meaning “suicide from overwork” (Amagasa et al., 2005), burnout became a popular topic in occupational health (Marques-Pinto et al., 2008; Schaufeli, 2017). There is some discussion about burnout (Bianchi, 2015; Bianchi et al., 2017; Epstein and Privitera, 2017; Mion et al., 2018) in terms of its dimensionality. The names attributed to the constructs can vary (Simbula and Guglielmi, 2010; Larsen et al., 2017). The most commonly suggested structure is a tri-factor one (Maslach et al., 2001, 2016), comprising emotional exhaustion (or simply exhaustion), depersonalization (also known as cynicism or disengagement), and reduced sense of personal accomplishment (or professional efficacy) (Halbesleben et al., 2004). It is expected that if a worker has high levels of the first two dimensions, there should be low levels of the third dimension since it is measured in the opposite direction to the other two. Carod-Artal and Vázquez-Cabrera (2013) state that emotional exhaustion is the most important dimension of burnout syndrome—being referred to as a state of having feelings of being emotionally overextended and depleted of one's emotional resources—representing the individual stress component (Bresó et al., 2007). Depersonalization refers to cynical or excessively detached responses to others in the work context; this is the interpersonal component of burnout (Maslach, 1998). Finally, diminished personal accomplishment refers to the decreased sense of competence and of productivity, representing the component of self-evaluation (Maslach, 1998).
There is also a two-dimensional approach to burnout (Demerouti et al., 2000). Based on empirical evidence, some authors consider that disengagement and exhaustion are the core dimensions of burnout, while reduced personal accomplishment plays a less important role (Maslach et al., 2001; Shirom, 2002). In fact, it has been shown that the relation of reduced personal accomplishment to burnout outcomes and antecedents is weaker than the other two dimensions (Lee and Ashforth, 1996). Moreover, while emotional exhaustion leads to disengagement, reduced personal accomplishment develops individually in relation to the other two dimensions (Leiter, 1993). Cordes and Dougherty (1993) suggest that it is an individual difference similar to self-efficacy.
Currently, burnout is becoming increasingly prominent in the literature (Leiter and Maslach, 2017); it has been associated with multiple occupational groups, beyond human services (Maslach and Leiter, 2017). In fact, burnout's prevalence has increased in some occupations, such as physicians in the USA (Shanafelt et al., 2015); it shows a high prevalence in various occupations, including radiology residents (Guenette and Smith, 2017), midwives in Australia (Creedy et al., 2017), nurses in various countries (Gómez-Urquiza et al., 2017). One reason why burnout is so common is due to the high levels of stress and emotional demands present in the job (Demerouti et al., 2001). Job stress can increase absenteeism, affect family roles, productivity, and mental and physical health, and decrease job satisfaction, which, in turn, can lead to reduced personal accomplishment, depersonalization, and emotional exhaustion (Carod-Artal and Vázquez-Cabrera, 2013).
Burnout Measurement With the Oldenburg Burnout Inventory
Despite the existence of several instruments to measure burnout, the Maslach Burnout Inventory (MBI; Maslach et al., 2016) is the most used (Ahola et al., 2017) and is commercially available. However, there are other options, some of them free, such as the Copenhagen Burnout Inventory (Kristensen et al., 2005); the Burnout Measure (Pines and Aronson, 1988; Malach-Pines, 2005); the Educator Burnout Inventory (Wang et al., 2003), the Oldenburg Burnout Inventory (OLBI; Bakker et al., 2004); the Shirom-Melamed Burnout Measure (Shirom and Melamed, 2006); the Bergen Burnout Indicator (Salmela-Aro et al., 2011); the Karolinska Exhaustion Scale (Saboonchi et al., 2013); and the Spanish Burnout Inventory (Gil-Monte et al., 2017).
OLBI seems to be the most prominent alternative to MBI (Demerouti et al., 2000). It was originally developed by Demerouti and Nachreiner (1998), who suggested two burnout dimensions, disengagement and exhaustion, applicable to professionals outside human services occupations. OLBI's versions vary across occupational groups and countries (see Table 1). In some countries—Brazil and Portugal (Campos et al., 2012), Sweden (Dahlin et al., 2007; Rudman et al., 2014), Slovenia (Kogoj et al., 2014), South Africa (Mokgele and Rothmann, 2014), Germany and Greece (Reis et al., 2015), and Malaysia (Mahadi et al., 2018)—OLBI has a version for students. OLBI does not contain any factor correspondent to what the MBI calls “professional efficacy”; this dimension received criticism in some studies (Bresó et al., 2007; Marôco et al., 2014), and, in the opinion of various authors, it is not a core burnout dimension (Bakker et al., 2004; Demerouti and Bakker, 2008) but can be interpreted as a possible burnout consequence (Koeske and Koeske, 1989) related to personality characteristics (Cordes and Dougherty, 1993).
The exhaustion subscale of OLBI has eight items which relate to feelings of emptiness, work overload, the need to rest, and physical, cognitive, and emotional exhaustion (Demerouti et al., 2003). Differently from the exhaustion concept presented in the MBI, the OLBI approach to exhaustion covers cognitive, physical, and affective aspects of exhaustion, which may facilitate the use of the instrument with workers of different kinds of activity (Demerouti et al., 2003; Bakker et al., 2004). The disengagement subscale has also eight items which refer to distancing oneself from the work, together with negative and cynical behaviors and attitudes in relation to one's job (Demerouti and Bakker, 2008). The OLBI's concept of disengagement differs from MBI's depersonalization in terms of the amplitude of the distancing, since OLBI's concept is broader: it may refer to distancing oneself from work in general or, more specifically, to distancing oneself from the content and object, along with experiencing negative attitudes (Demerouti et al., 2003). Thus, disengagement offers a less restricted view of the lack of interest in work. It is important to note that González-Romá et al. (2006) suggested that two of the three work engagement dimensions measured by the Utrecht Work Engagement Scale (UWES), vigor and dedication (the third dimension is absorption), can be paired with emotional exhaustion and cynicism (burnout dimensions). One dimension, named “identification,” involved dedication and cynicism; the other, named “energy,” comprised vigor and exhaustion factors, this indicates that OLBI's negatively- and positively-worded items can be markers for work engagement and burnout (Halbesleben and Demerouti, 2005).
Table 1 summarizes the different OLBI versions used with different samples found through a search of Embase, Scopus, PubMed, Web of Science, and Google Scholar using the terms: “OLBI,” “Oldenburg Burnout Inventory,” “adaptation,” “version,” “validity,” and “psychometric properties.” The OLBI's total number of items changed since its original structure of 25 items (Demerouti and Nachreiner, 1998) to 15 (Demerouti et al., 2001); today's English language version has 16 items (Bakker et al., 2004; Halbesleben and Demerouti, 2005). It has positively and negatively worded items—an equal number of each kind in the two dimensions—something that is considered an advantage (Price, 1997) since it can diminish acquiescence bias despite diminishing the internal consistency of the instruments (Salazar, 2015). OLBI has been translated into many languages, although not always evaluated in its psychometric properties (Table 1). Some studies use OLBI without taking into consideration the recommended steps to adapt an instrument for a country or culture different from the one for which it was originally developed (van de Vijver, 2016). There is a certain lack of use of adequate guidelines when translating and adapting the instrument for a new sample (International Test Commission, 2018). As can be observed in Table 1, the majority of the new OLBI versions have not evaluated their psychometric properties with the appropriate technique (confirmatory factor analysis [CFA]; Brown, 2015). In fact, some of them have avoided both exploratory and confirmatory factor analysis. Usually, the original two-factor structure is the one with better goodness-of-fit indices—even when compared with the other two-factor structures (e.g., with positive items in one group and negatively-worded items in the other), and with one- or four-factor structures (Demerouti et al., 2001). The measurement invariance/equivalence of the instrument across different groups is essential to properly establish comparisons (Davidov et al., 2014). OLBI invariance has been addressed by Demerouti et al. (2001) in its 15-items version for three different professions, they observed metric invariance (same factor's loadings). Demerouti and Nachreiner (1998) stated that three different groups of professionals obtained a similar OLBI (25-items) structure after a principal component analysis for each of the groups. Others researchers have obtained measurement invariance between countries (Demerouti et al., 2003) and between workers and students (Reis et al., 2015). These findings suggest that burnout is not exclusive to human services professions (Demerouti and Nachreiner, 1998; Demerouti et al., 2001) since various studies have tested burnout levels using OLBI in other occupations (e.g., executive directors, white-collar employees, construction workers). Altogether, few studies tested the measurement invariance of the groups with which they established comparisons. Finally, regarding the reliability of the scores, the internal consistency estimates were acceptable to good in most of the studies, while almost all studies reported only the Cronbach's α (see Table 1).
This study aims to describe the psychometric properties of an OLBI version developed simultaneously for workers from Brazil and Portugal, its validity evidence based on the internal structure (dimensionality, measurement invariance, reliability), and the validity evidence based on the relationship with other variables (work engagement); and to compare burnout across sexes and countries. Additionally, the study seeks to present a revision of OLBI's different versions since this is the first version of the instrument developed simultaneously for Portugal and Brazil, adapting an important instrument for understanding burnout in relation to sexes in the organizations. It will be structured by presenting some considerations about burnout among sexes, followed by burnout measured by OLBI.
Following the recommendations of The Standards for Educational and Psychological Testing (American Educational Research Association, 2014), this paper aims to assess two types of validity evidence for the Portuguese version (PT-BR and PT-PT) of the OLBI (Bakker et al., 2004)—one related to the internal structure, the other based on the relations to other variables (work engagement). Since various studies have successfully confirmed the original two-factor structure of OLBI (Halbesleben and Demerouti, 2005; Peterson et al., 2011; Subburaj and Vijayadurai, 2016), it was hypothesized that the tested OLBI version would present a good fit confirming its original dimensionality of two factors (H1). Burnout has been hypothesized by some authors as a higher-order dimension (Taris et al., 1999; Shirom and Melamed, 2006; Marôco et al., 2008). Thus, a possible second-order latent factor, burnout, was tested for OLBI (H2). Through the review of the different versions (Table 1), the majority of the studies showed acceptable to very good reliability of the scores' evidence in terms of internal consistency (e.g., Demerouti and Bakker, 2008; Innstrand, 2016; Subburaj and Vijayadurai, 2016). Consequently, it was assumed that OLBI would present acceptable internal consistency reliability estimates (H3). Some studies found evidence of measurement invariance for OLBI between occupations (Demerouti et al., 2001; Demerouti and Bakker, 2008; Innstrand, 2016) and sex (Foster, 2015), but none investigated measurement invariance among workers of different countries. H4 hypothesized that OLBI will present evidence of measurement invariance between sexes and countries.
Research has found that burnout levels can vary among sexes, with females usually presenting slightly more exhaustion than males (Purvanova and Muros, 2010; Innstrand et al., 2011), females being more likely to experience burnout (Dimou et al., 2016). However, others suggest that research does not allow one to conclude any sex-specific risks (Seidler et al., 2014; Adriaenssens et al., 2015) considering that the burnout differences can be related with the levels of workload as well as care-load (Bekker et al., 2005; Langballe et al., 2011). Burnout can also vary among countries also (Poghosyan et al., 2010; Alexandrova-Karamanova et al., 2016; Jovanović et al., 2016). North American countries have a tendency to present higher exhaustion and disengagement levels than European countries—differences that can be related to cultural aspects (Maslach et al., 2001). However, regarding the Portugal-Brazil comparison, no differences were reported in a previous study (Dias et al., 2010). Occupations can play a substantial role in burnout levels (e.g., emotional challenges of working in the teaching or caregiving role) (Maslach et al., 2001). Altogether, it was hypothesized that burnout's latent means differ between sexes and countries (H5).
Work engagement is known to be a construct with strong correlations with burnout (Demerouti et al., 2010; Petrović et al., 2017), since both can be considered indicators of well-being (Bakker et al., 2014). Thus, the divergent validity evidence based on the relation to other variables, work engagement, was assessed (H6).
A total sample composed of 1,172 participants was collected by combining two independent samples: one sample of Brazilian workers in various occupations (n = 604), and one of Portuguese workers in various occupations (n = 568). Both samples completed the OLBI and the Utrecht Work Engagement Scale (Schaufeli and Bakker, 2003). Participation was anonymous and voluntary. The average age of the total sample was 35.47 years (SD = 9.95), with 65% being female. Workers' occupations were according to the International Standard Classification of Occupations ISCO-08 (International Labour Office, 2012)—mainly professionals or administrative support—and 73% of the sample were, at least, college graduates (Table 2). Regarding children, 59% had none; 45% reported being married or cohabiting.
A non-probabilistic convenience sampling was used. The inclusion criteria were: (1) all participants were workers with a contract or formal ties with their employers, (2) had easy access to a PC, smartphone, or tablet to access the online platform where the instruments were deployed, and (3) were literate.
The OLBI was used to assess burnout, through the development of a version transculturally adapted both for Brazil and Portugal (Table 3). The OLBI is a self-report five-point rating scale (1 = “Strongly disagree”; to 5 = “Strongly agree”) with eight questions within each of the two dimensions, disengagement and exhaustion (Demerouti et al., 2001). The disengagement factor refers to distancing from work in terms of both object and content, and to the development of cynical and negative attitudes and behaviors in relation to one's job (Bakker et al., 2004). Exhaustion refers to feelings of physical fatigue, the need to rest, and feelings of overtaxing and emptiness in relation to work (Demerouti and Bakker, 2008). To develop the Portuguese version (Table 3) the English version of the OLBI was used (Bakker et al., 2004) following The ITC Guidelines for Translating and Adapting Tests (International Test Commission, 2018), adapting the items to the Portuguese language according to the Orthographic Agreement signed by both Portugal and Brazil in 2009. The items were discussed with Portuguese and Brazilian psychologists and methodologists to create a version of the items that gathered the consensus of specialists regarding cultural, semantic, and idiomatic equivalence in the two countries. Finally, a small pilot test was done with 15 workers from each country; this did not suggest any modifications and the Portuguese adapted OLBI's 16 items were understood. The final single version (for both countries) had no other changes.
Work engagement refers to a positive motivational state and is composed of vigor, dedication, and absorption. This construct was measured with UWES-9 in its transculturally adapted version to both Brazil and Portugal (Sinval et al., 2018). It is a self-report instrument scored on a seven-point rating scale (0 = “Never”; 6 = “Always”), with three questions in each of its three dimensions. The UWES has shown good divergent validity evidence with the OLBI, since work engagement and burnout are moderately and negatively related (Goering et al., 2017). It was chosen not only for its good psychometric qualities for both countries, but also because it showed measurement invariance between both countries and it is a short instrument that allows for a robust work engagement measure with only a few items (Schaufeli and Bakker, 2003). It is a well-spread measure across many countries (Sinval et al., 2018) and is actually the most used instrument to measure work engagement. However, studies that investigated the relations between burnout and work engagement have mainly used the MBI for burnout and the UWES for work engagement (Schaufeli and de Witte, 2017). This study used OLBI together with UWES, trying to enrich the discussion about the two concepts, rather than just discussing instruments. The UWES dimensions are vigor, referring to the energy and resilience that one has in work; dedication, referring to being enthusiastic, inspired, and proud of one's work; and absorption, referring to being immersed in one's work without the perception of time passing (Schaufeli et al., 2002). It is expected that high levels of work engagement correspond to highly energized workers (Schaufeli and Bakker, 2010).
Data were gathered from 2015 to 2017, in both countries, in an effort to have a larger sample, since web surveys present low response rates (Massey and Tourangeau, 2013). Both samples completed the OLBI, a brief sociodemographic questionnaire, and the UWES-9. All the collected data were obtained online using LimeSurvey software (LimeSurvey GmbH, 2017) running on the website of two major universities in each country. Nearly 35 percent of the disseminated questionnaires were completed in both countries. Participants were both contacted individually and through companies which answered positively to the invitation to participate in the study. Before filling out the survey, participants were informed about the study, assuring them that the study was a research study and that the company would not access individual data and that companies simply helped the researchers disseminate the study. Informed consent was obtained online from all participants.
To allow comparative studies, the same procedures were used in both countries. The study was approved by the Ethics Committee of the University of Porto (on 03-18-2015), Portugal, and the University of São Paulo (on 01-09-2014; CAAE no. 33301214.2.0000.5407), Brazil, and followed the usual rules for online surveys, namely, no access of participants' companies to individual results and no direct contact between participants and researchers [A few used the email to clarify some details about access to individual data, but it is not possible to identify whether they participated in the study].
A confirmatory factor analysis (CFA) was conducted to verify if the original two-factor structure proposed by Bakker et al. (2004), presented an adequate fit to the study sample. Only complete data cases were considered. As goodness-of-fit indices, SRMR (Standardized Root Mean Square Residual), RMSEA (root mean square error of approximation), NFI (Normed Fit Index), CFI (Comparative Fit Index), and the TLI (Tucker Lewis Index) were used. The fit of the model was considered good for TLI, CFI and TLI values above 0.95; SRMR below 0.08; and RMSEA values below 0.08 (Hoyle, 1995; Boomsma, 2000; McDonald and Ho, 2002; Byrne, 2010). All statistical analyses were performed with R (R Core Team, 2018) and RStudio (RStudio Team, 2018). The descriptive statistics were obtained with the skimr package (McNamara et al., 2018), the standard error of the mean (SEM) was calculated with the plotrix (Lemon, 2006) package and the coefficient of variation (CV) was estimated with the package sjstats (Lüdecke, 2019). To assess multivariate normality, Mardia's multivariate kurtosis (Mardia, 1970) was used; it was calculated using the psych package (Revelle, 2018). The lavaan package (Rosseel, 2012) was selected to conduct the CFA analyses using the Weighted Least Squares Means and Variances (WLSMV) estimation method (Muthén, 1983).
To test the proposed structure for OLBI, the cross-validity evidence was assessed to give information about how well the new structure will fit an independent sample of the same population (Cudeck and Browne, 1983). To do so, the sample was randomly split into two sub-samples through the package minDiff (Papenberg, 2018). The workers' age was used as criteria variable for which it was desired to minimize differences between subsamples (Papenberg, 2018). The subsamples were generated using 1,000 repetitions in order to minimize the differences, since the most equal group assignment was selected. Having two independent subsamples with similar properties, one subsample can be used as calibration subsample, and another as validation subsample (Chin and Todd, 1995).
The convergent validity evidence was analyzed using the average variance extracted (AVE) which was estimated as described in Marôco (2014) and Fornell and Larcker (1981). The constructs' convergent validity evidence was assumed for values of AVE ≥ 0.5 (Hair et al., 2009).
The discriminant validity evidence was checked (Fornell and Larcker, 1981; Marôco, 2014) to verify whether the items that represent a dimension were strongly correlated with other dimensions (Marôco, 2014): for two factors, x and y, if AVEx and AVEy ≥ ρ2xy (squared correlation between the factors x and y), there is discriminant validity evidence. The Heterotrait-monotrait (HTMT) criterion (Henseler et al., 2015) was also used. Values above 0.85 were considered indicative of satisfactory discriminant validity evidence (Kline, 2016). The HTMT ratios of correlations were calculated using the semTools package (Jorgensen et al., 2018).
The reliability of the scores was assessed with various estimates of internal consistency as recommended (Irwing and Hughes, 2018): αordinal (Zumbo et al., 2007), and ωordinal (Bollen, 1980; Raykov, 2001) using the semTools package (Jorgensen et al., 2018), higher values were indicative of better internal consistency results. Also, the McDonald's hierarchical omega (ωH; Zinbarg et al., 2005) was estimated; a higher value of ωH indicates a stronger influence of the latent variable common to all factors, and that the observed scale scores generalize to scores for the common latent variable (Zinbarg et al., 2007). The omega hierarchical subscale (ωHS) was calculated for each specific factor, it reflects the reliability of each subscale after controlling for the variance due to the general factor (Reise et al., 2013). Both the ωH and the ωHS were used for calculating the internal consistency of the bi-factor model. There is some discussion about the use of αordinal (Revelle and Condon, 2018) as so we reported other estimates. The αordinal was calculated based on the polychoric correlations. However, the ωordinal and ωH accounts for both item covariances and item thresholds (Green and Yang, 2009). The ωordinal and the ωH are different in the denominator, the first assumes a congeneric factor model where measurement errors aren't correlated (Bollen, 1980), the second uses the observed covariance matrix instead of the model-implied covariance matrix (McDonald, 1999; Jorgensen et al., 2018). The CR was calculated by summing the z scores of the item scores. The second-order factor reliability was also calculated using the omega coefficient (Jorgensen et al., 2018). The proportion of observed variance explained by the second-order factor after controlling for the uniqueness of the first-order factor (ωpartial L1); the proportion of the second-order factor explaining the variance of the first-order factor level (ωL2); and the proportion of the second-order factor explaining the total score (ωL1) were also calculated. The reliability estimates were calculated with the semTools package (Jorgensen et al., 2018).
The measurement invariance of the higher-order model was assessed using the lavaan package (Rosseel, 2012), the categorical items were considered into account through theta-parameterization (Millsap and Yun-Tein, 2004) to compare a group of seven different models based on the recommendations of Millsap and Yun-Tein (2004) and on the second-order models' invariance specificities (Chen et al., 2005): (a) configural invariance; (b) first-order factor loadings; (c) second-order structural loadings; (d) thresholds of measured variables; (e) intercepts of first-order factors; (f) disturbances of first-order factors; and (g) residual variances of observed variables. Mean scores for burnout latent variable were compared within the structural equation modeling framework; effect sizes (Cohen's d) were determined (Cohen, 1988). The raw means, SDs and score percentiles were calculated using the doBy package (Højsgaard and Halekoh, 2018).
The results related to psychometric properties of the OLBI in terms of internal structure are presented first, followed by the latent means comparisons, and finally by the validity evidence based on the relations to other variables.
Validity Evidence Based on Internal Structure
Items' distributional properties
To judge distributional properties and psychometric sensitivity on the Portuguese and Brazilian samples, summary measures, skewness (Sk), kurtosis (Ku), and a histogram for each of the 16 items were used (Table 4). No strong deviations from the normal distribution (Finney and DiStefano, 2013) were considered for absolute values of Ku smaller than seven (7) and Sk smaller than three (3), assuring that they wouldn't compromise CFA results (Marôco, 2014). Mardia's multivariate kurtosis for the 16 items of OLBI was 48.88; p < 0.001. All possible Likert-scale answer values were observed on all items; no outliers were deleted. These items follow an approximately normal distribution in the normative population under study, since their distributional properties are indicative of appropriate psychometric sensitivity.
Factor-related validity evidence
To proceed with OLBI's transcultural adaptation to Brazil and Portugal, a cross-validity evidence approach was adopted. The sample was divided in two subsamples: calibration (n = 586) and validation subsamples (n = 586). The first was used to test which is OLBI's best solution in terms of fit to the data, and theoretical sense. The second subsample was used to assess cross-validity evidence of the proposed model. The two-factor OLBI fit to the data was mediocre ( = 720.764; p < 0.001; n = 586; CFI = 0.980; CFIscaled = 0.918; NFI = 0.977; TLI = 0.977; SRMR = 0.072; RMSEA = 0.101; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.094; 0.108[), since CFI, NFI, and TLI values were above 0.95 (good fit), SRMR values were bellow 0.08 (good fit), but RMSEA values were above 0.10 being indicative of poor fit (MacCallum et al., 1996). One item presented a very low loading (λitem 13 = 0.220): and thus, this item was deleted. Also, based on the analysis of the modification indices, four correlations between items' residuals of the same factor were added, since it seems reasonable that indicators from the same factor explain shared error variance (Kline, 2016). The reduced model of 15 items showed better goodness-of-fit indices (Figure 1; = 514.098; p < 0.001; n = 586; CFI = 0.986; CFIscaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[), which indicated an acceptable fit, all items presented loadings above or equal to 0.47 (p < 0.001). The Cheung and Rensvold (2002) criterion (ΔCFI ≤ 0.01) supported the preference for the reduced model (ΔCFIscaled = −0.019). Thus, H1 was accepted.
Figure 1. OLBI's two-factor reduced version (15-item) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Correlations between latent variables, residuals' correlations and factor loadings for each item are shown. = 514.098; p < 0.001; n = 586; CFI = 0.986; CFIscaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[.
Convergent validity evidence
To check if items contained within each factor are related to each other, the AVE was calculated for disengagement (AVE = 0.57), and for exhaustion (AVE = 0.50). These results suggest acceptable convergent validity evidence for the OLBI-15.
Discriminant validity evidence
The discriminant validity evidence between the two OLBI factors was unsatisfactory. These findings showed that the two factors are strongly related to each other, since AVEdisengagement = 0.57 and AVEexhaustion = 0.50 were smaller than r2DE = 0.69. Regarding the HTMT ratio of correlations (Henseler et al., 2015) the obtained value (0.80) is below the satisfactory threshold. These findings point to the fact that the two factors' correlation might be explained by a second-order latent factor, by a bi-factor model or by a unidimensional model.
A unidimensional model where the factor burnout loads on all 15 items was tested. The four residuals' correlations were maintained. This model assumes that the only latent factor that explains the manifest variables is burnout. As so, it assumes that the other two latent variables (i.e., disengagement and exhaustion) aren't meaningful by themselves since the discriminant validity evidence wasn't satisfactory. The content explained by them is similar, the unidimensional model tests if it is plausible to specify a single latent variable.
The OLBI's unidimensional model presented an mediocre fit (Figure 2; = 737.139; p < 0.001; n = 586; CFI = 0.979; CFIscaled = 0.913; NFI = 0.977; TLI = 0.975; SRMR = 0.077; RMSEA = 0.114; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.106; 0.121[). Based on the Cheung and Rensvold (2002) criteria (ΔCFI ≤ 0.01) the two-factor reduced model was found to have a statistically better fit to these data than the unidimensional model (ΔCFIscaled = −0.024). All the factor loadings and residuals' correlations were statistically significant (p < 0.001). Item 14 had the lowest factor loading (λitem 14 = 0.450).
Figure 2. OLBI's unidimensional reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Residuals' correlations and factor loadings for each item are shown. = 737.139; p < 0.001; n = 586; CFI = 0.979; CFIscaled = 0.913; NFI = 0.977; TLI = 0.975; SRMR = 0.077; RMSEA = 0.114; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.106; 0.121[.
A bi-factor model (Holzinger and Swineford, 1937; Holzinger and Harman, 1938) is a nested factor model (Gustafsson and Balke, 1993) or direct hierarchical model (Gignac, 2008) that specifies a single general factor among each measured variable that accounts for commonality shared by the related domains; and multiple specific orthogonal factors each of which account for unique variance above and over the general factor (Rios and Wells, 2014; Mansolf and Reise, 2017; Chen and Zhang, 2018). The bi-factor model has advantages (Canivez, 2016; Chen and Zhang, 2018), but also some limitations (Mulaik and Quartetti, 1997; Reise et al., 2010; Murray and Johnson, 2013) in comparison with higher-order models (e.g., second-order models), as so, the choice between them should be carefully weighted.
The OLBI's bi-factor model presented an acceptable fit (Figure 3; = 392.202; p < 0.001; n = 586; CFI = 0.990; CFIscaled = 0.937; NFI = 0.987; TLI = 0.986; SRMR = 0.056; RMSEA = 0.085; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.077; 0.093[). The ΔCFI ≤ 0.010 criterion (Cheung and Rensvold, 2002) didn't find meaningful differences between the two-factor reduced model and the bi-factor model (ΔCFIscaled = 0.000). All factor loading of the general factor (i.e., burnout) were statistically significant (p < 0.001), although the specific factors presented two non-significant loadings (α = 0.05), one on the disengagement subscale (λitem 11 = −0.001), and one on the exhaustion subscale (λitem 16 = 0.037).
Figure 3. OLBI's bi-factor reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Latent loadings for each factor; and factor loadings for each item are shown. = 392.202; p < 0.001; n = 586; CFI = 0.990; CFIscaled = 0.937; NFI = 0.987; TLI = 0.986; SRMR = 0.056; RMSEA = 0.085; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.077; 0.093[.
A second-order latent factor may be admissible when two factors have high correlations between them, or/and when exists a higher order construct which might explain the lower order factors (Chen et al., 2005; Marôco, 2014). Since the two OLBI factors did not present satisfactory discriminant validity evidence between them, a second-order model was tested. The higher-order construct was named as burnout. Having as a start point the reduced model, and since there were not enough degrees of freedom to test the second-order latent model, the two structural weights between the second-order factor and the first-order factors were constrained to be equal.
The OLBI's second-order latent factor model presented an acceptable fit (Figure 4; = 514.098; p < 0.001; n = 586; CFI = 0.986; CFIscaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[). The RMSEA value was mediocre, however its confidence interval was precise and point estimates for RMSEA have been shown to depend on sample size and model misspecification and model degrees of freedom (MacCallum et al., 1996; Chen et al., 2008). Nevertheless, other goodness-of-fit indices were used in conjunction to assess models' adequacy. SRMR values were acceptable, which seem to be generally accurate across all conditions (Maydeu-Olivares et al., 2018). The constrained structural weights from burnout to disengagement and exhaustion were high (γ = 0.91; p < 0.001). These results suggest that burnout is a higher order construct reflected on disengagement and exhaustion. The findings show that hypothesis 2 can be confirmed, since the paths from the second-order latent to the first-order ones were statistically significant (p < 0.001) and had high values.
Figure 4. OLBI's second-order factor reduced version (15 items) structure fit. A combined sample of Portuguese (n = 268) and Brazilian (n = 318) workers. Latent loadings for each factor; residuals' correlations and factor loadings for each item are shown. = 514.098; p < 0.001; n = 586; CFI = 0.986; CFIscaled = 0.937; NFI = 0.984; TLI = 0.983; SRMR = 0.064; RMSEA = 0.093; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.085; 0.101[.
The differences between the second-order model and the two-factor reduced model (ΔCFIscaled = 0.000) and the bi-factor model (ΔCFIscaled = 0.000) weren't meaningful based on the ΔCFI criteria (Cheung and Rensvold, 2002). After having in consideration all tested models (Table 5) the second-order model was selected, since it hadn't a worst fit than the bi-factor and the two-factor reduced model (based on the used criterion). This choice emerged as solution for the lack of evidence of discriminant validity of the two-factor reduced model. And also as a plausible option in theoretical terms as also suggested by other authors who have proposed a burnout second-order factor using MBI, CBI and OLBI (Marôco et al., 2008, 2014; Marôco and Campos, 2012). The bi-factor model presents equivalent fit, and some authors also proposed it as an alternative structure using MBI and the Job Burnout Scale (Wang and Gao, 2010; Mészáros et al., 2014; Morgan et al., 2014) although with problems in some cases (i.e., non-convergence, unsatisfactory unique proportion of variance explained of the observed scores). The second-order factor had very high structural weights, while the bi-factor model had only one factor loading above 0.50 on each for the two specific factors, pointing for clear insufficient proportion of the variance explained on the specific factors. Altogether, the obtained results seem to present evidence that favors the second-order model.
This structure also showed cross-validity evidence, since it presented a good fit to the data also when using the validation sample ( = 591.172; p < 0.001; n = 586; CFI = 0.985; CFIscaled = 0.934; NFI = 0.983; TLI = 0.982; SRMR = 0.068; RMSEA = 0.101; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.093; 0.109[). The structural weights were high (γ = 0.93; p < 0.001).
Reliability of the Scores: Internal Consistency Evidence
To estimate the reliability of the scores on the various models, the complete sample was used. The internal consistency values of the two-factor first-order model reduced were high for the three coefficients (Table 6). They suggest very good validity evidence in terms of the reliability of the scores.
The same was observed on the unidimensional model internal consistency estimates (CRburnout = 0.93; αordinal = 0.93; ωordinal = 0.91). Based on a bi-factor model, the hierarchical omega was high (ωH = 0.85) and omega hierarchical subscale (ωHS; Reise, 2012; Reise et al., 2013; Rodriguez et al., 2016b) were low (ωHS disengagement = 0.08; ωHS exhaustion = 0.20). The specific factors reliability score after controlling for the variance due to the general factor was clearly unsatisfactory, reinforcing the evidence in favor of the second-order model. The ωH value was high (Rodriguez et al., 2016a), suggesting a strong influence of the latent variable common to the two factors.
The proportion of observed variance explained by the second-order factor after controlling for the uniqueness of the first-order factor (ωpartial L1) was 0.93. The proportion of the variance of the first-order factors explained by the second-order factor (ωL2) was 0.91, and the proportion of the second-order factor explaining the total score (ωL1) was 0.86. Thus, the internal consistency of the second-order construct was indicative of very good values.
To verify if measurement invariance holds, the complete sample was used. The fit to the data of each individual group was globally acceptable. The Brazilian sample had an acceptable fit ( = 687.077; p < 0.001; n = 604; CFI = 0.982; CFIscaled = 0.922; NFI = 0.980; TLI = 0.978; SRMR = 0.073; RMSEA = 0.109; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.102; 0.117[) as so did the Portuguese sample ( = 494.338; p < 0.001; n = 568; CFI = 0.988; CFIscaled = 0.943; NFI = 0.985; TLI = 0.985; SRMR = 0.065; RMSEA = 0.092; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.084; 0.100[). The fit of the Females sample was slight better ( = 537.474; p < 0.001; n = 678; CFI = 0.989; CFIscaled = 0.944; NFI = 0.987; TLI = 0.986; SRMR = 0.059; RMSEA = 0.089; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.082; 0.096[) whereas the Males sample fit was marginally worse ( = 492.213; p < 0.001; n = 364; CFI = 0.979; CFIscaled = 0.922; NFI = 0.975; TLI = 0.974; SRMR = 0.081; RMSEA = 0.115; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.105; 0.125[).
To assess if the same second-order latent model holds in each sex and country, seven nested models with indications of equivalence were used (Marôco, 2014). Full uniqueness measurement invariance was supported for countries (Table 7) based on the Cheung and Rensvold (2002) criterion (absolute ΔCFIscaled ≤ 0.010) and on the Chen (2007) criterion (absolute ΔRMSEAscaled ≤ 0.015). The Δχ2 criterion (Satorra and Bentler, 2001) demonstrated the second-order metric invariance. Since the Δχ2 criterion is too restrictive, we opted for the ΔCFIscaled criterion. Results supported the structural invariance of the OLBI between Portugal and Brazil. The measurement invariance for OLBI among sexes (Table 7) was obtained, since full uniqueness measurement invariance was observed with the support of the Cheung and Rensvold (2002) criterion (absolute ΔCFIscaled ≤ 0.010) and of the Chen (2007) criterion (absolute ΔRMSEAscaled ≤ 0.015).
Sex's and Country's Burnout Latent Means Comparisons and Dimensions' Quartiles
Following the existence of full uniqueness measurement invariance, latent means can be compared. The results of the chi-square difference test suggest that weren't significant differences in burnout (Δχ2scaled (1) = 1.110; p = 0.292; d = 0.067) among countries. There weren't also statistically significant differences among countries in relation to the burnout dimension (Δχ2scaled (1) = 1.066; p = 0.302; d = 0.066). The quartiles, means, and SDs (raw values) for each sex within each country are presented in Table 8, these values are presented with the intent of providing population norms values.
Validity Evidence Based on the Relations to Other Variables
Burnout can be conceptualized as being the opposite of work engagement (Halbesleben and Demerouti, 2005). The UWES-9 second-order latent factor model presented a good fit ( = 175.820; p < 0.001; n = 1,104; CFI = 0.999; CFIscaled = 0.992; NFI = 0.999; TLI = 0.999; SRMR = 0.028; RMSEA = 0.076; P(rmsea ≤ 0.05) < 0.001; 90% CI ]0.065; 0.086[). All factor loadings were statistically significant as also one added residuals' correlation from item 1 and 2 (r = 0.76). The internal consistency reliability estimates were good both for the first-order model (ωvigor = 0.84; ωdedication = 0.92; ωabsorption = 0.88; ωtotal = 0.96) and for the second-order (ωpartial L1 = 0.96; ωL1 = 0.94; ωL2 = 0.97).
The first-order models of each instrument were used to establish correlations among the first-order latent variables (Table 9), and the second-order models of the instruments were used to analyze the correlation among the respective second-order latent variables (rburnout*work engagement = −0.85). The obtained correlation between the latent OLBI and UWES dimensions was negative and moderate to high, demonstrating the divergent validity of the measures obtained with OLBI (burnout) and the UWES-9 (work engagement).
Regarding the psychometric properties of the OLBI, results of this study provide evidence of the two-factor structure of the original instrument, having convergent validity evidence and good goodness-of-fit indices except for RMSEA value, which can be indicative of moderate errors of approximation in the population. Nevertheless, this value had a narrow confidence interval, that reflects a good precision of the model fit in the population (MacCallum et al., 1996). However, the discriminant validity evidence was not satisfactory (H1), which led us to test a possible second-order latent factor. Item 13 was removed from the tested version. Problems with item 13 also have been reported by Chevrier (2009). Item 13 has also been deleted from the OLBI-S proposal for Portuguese and Brazilian students (Campos et al., 2012) and OLBI's Malay version for students (Mahadi et al., 2018). In the Brazilian version for workers, that item was removed from the proposed reduced 13-item version (Schuster and Dias, 2018). Its removal was also suggested from the Italian version (Estévez-Mujica and Quintane, 2018). Other authors found that item 13's removal increased the disengagement internal consistency (Baka and Basinska, 2016). In the OLBI reduced Russian version (Smirnova, 2017), that item was the only one that lacked statistically significant loading, also being removed. In the Swedish version, it was the item with the lowest loading (λitem 13 = 0.38) (Peterson et al., 2011). Item 13's content, “This is the only type of work that I can imagine myself doing,” did not seem to make sense within today's economic and professional context, where careers are so uncertain and the number of different employers across a career is increasing (Savickas, 2012). Furthermore, the sample is composed mainly of younger workers, most of whom have a higher education. This can contribute to a perception of more control over their career and desire for more professional experiences rather than maintaining the same employer for a long period. Additionally, four correlations within pairs of residuals' belonging to the same factor were added. Such modifications mean that some unwanted or unexplained source of variance exist outside of the original model, which can be due to various reasons (e.g., lack of comprehension of the items or an unwanted theoretical trait present in that factor) and as such is speculative (Cote et al., 2001). Nevertheless, it seems plausible that theory can be slightly imprecise, and since the model (second-order) passed through the cross-validation approach and reinforced the validity evidence obtained with the calibration subsample. The correlation between the two OLBI's dimensions was high; this finding is shared by other studies which found correlation values between OLBI's factors similar to ours (Khan et al., 2016). Those findings also pointed to unsatisfactory discriminant validity evidence between disengagement and exhaustion.
Our internal consistency estimates presented very good values supporting H2, in line with most of the studies which evaluated this kind of estimate (Table 1). Still, this study goes one step further, since it obtained estimates that gave evidence about the reliability of the second-order latent factor and the reliability of a potential bi-factor model (ωH).
We admitted a possible second-order factor which was confirmed (H3) since the goodness-of-fit indices were not worse than the reduced version of the first-order model. The second-order model was also compared with a bi-factor model, which also didn't present meaningful better fit (i.e., ΔCFIscaled < 0.01) to the data than the second-order model. The bi-factor model has various advantages (Reise et al., 2010), although—in this study- those potential advantages weren't confirmed by the obtained results. Since the unique variance explained by the specific factors of (after controlling for the variance due to the general factor) wasn't satisfactory (Rodriguez et al., 2016b). Additionally, empirical observations strongly suggest the second-order model presented high values of structural weights both on the calibration as on the validation subsamples. Regarding the existent theory, some authors proposed burnout as a second-order factor (Taris et al., 1999; Shirom and Melamed, 2006; Marôco et al., 2008) the same applies to the bi-factor model with some authors preferring it in relation to hierarchical model (Mészáros et al., 2014). However, if there wasn't prior knowledge from the field regarding burnout conceptualization, exploratory bi-factor analysis (Jennrich and Bentler, 2011) or bi-factor exploratory structural equation modeling (Morin et al., 2015) cloud be also suitable analyses. If one desires to analyze bi-factor models outside the standard CFA procedures (e.g., setting to zero paths from latent constructs to indicators that are not theoretically associated) can adopt the already referred bi-factor exploratory structural equation modeling or the bi-factor Bayesian structural equation modeling (Golay et al., 2013) analyses. Marôco and Campos (2012) have proposed the a second-order interpretation of burnout, although using a different instrument, the MBI. In other study, Marôco and Campos (2012) made the same suggestion using the MBI, CBI and OLBI. This suggests that, besides the two OLBI first-order factors, there is a higher-order more general burnout dimension. The obtained findings contribute to the study of OLBI and the burnout dimensionality, maintaining the two-order structure, but also suggesting a second-order latent, which brings a novelty to this study, since this was proposed for the first time for OLBI using a sample of workers.
The OLBI presented measurement invariance for sex and country supporting H4. To the best of our knowledge, this was the first time measurement invariance was assessed across sexes for the full OLBI instrument, since Foster (2015) only tested measurement invariance among sex on the separate factors. No study has found testing measurement invariance among countries, bringing a novelty to this study.
No statistically significant differences were found between sexes for burnout (H5), which shows that each sex experiences that dimension in the workplace similarly. Females are known for having higher levels of negative emotional states than males (Kessler et al., 2005); in consequence, females usually score higher than males on exhaustion (Purvanova and Muros, 2010; Kumar and Mellsop, 2013; Pu et al., 2017; Schadenhofer et al., 2018). However, a meta-analysis (Purvanova and Muros, 2010) found that males usually have higher depersonalization levels than females. Altogether, the differences reported on previous studies seem to be annulling each other in terms of the general second-order factor. The relationship between burnout dimensions and sex is not always clear. O'Connor et al. (2018) suggested in a meta-analysis on burnout in health professionals, that the burnout dimensions and sex have an inconsistent relationship. A similar finding was reported by Estévez-Mujica and Quintane (2018), which found no relationship between exhaustion, disengagement, and burnout and sex in a sample of research and development workers. A recent study with Portuguese health professionals at the national level, found no significant differences among sexes in terms of burnout (Marôco et al., 2016). For the country factor, no significant differences were observed for the burnout latent variable (H5). These results are in line with the findings of a comparative study among Portuguese and Brazilian health professionals using the MBI (Dias et al., 2010), no significant differences were found between the burnout dimensions. Western European countries seem to present lower average burnout scores than in other parts of the world (Golembiewski et al., 1996). These differences can be due to cultural differences (Golembiewski et al., 1993; Maslach et al., 2001), In some countries, as Brazil, such differences can also occur between regions of the same country as a result of being such a big and culturally-mixed country (Hofstede et al., 2010b). Differences between Portugal and Brazil seem to exist at the individualism level, with Portugal being more collectivist and with smaller power distance (Hofstede et al., 2010a). In Brazil, employers seem to be more risk-taking than their Portuguese counterparts (Silva et al., 2009). In other words, Portugal has larger avoidance to uncertainty values (Hofstede et al., 2010a). In Brazil, organizations operate through general rule as much as through personal relationships (Garibaldi de Hilal, 2009); whereas in Portugal, the work relations appear to be more impersonal and formal (Dias et al., 2010). Brazil presents larger indulgence values than Portugal (Hofstede et al., 2010a); the Brazilian culture also reflects ambiguity and double-edged ethics (Garibaldi de Hilal, 2006). Brazilian organizations seem to perceive responsibility toward employees as one of the less important business priorities (Hofstede et al., 2010a), which can lead to poorer attention to work conditions.
Regarding the last hypothesis, the validity evidence based on the relations with other variables was good (H6), since the presented correlations were moderate to high between OLBI and UWES factors. Other studies found Pearson's correlation value −0.55 between the work engagement (UWES) score and OLBI's disengagement factor, and −0.48 between work engagement (UWES) and OLBI's exhaustion (Bosman et al., 2005). In the present study, a higher work engagement correlation with disengagement than with exhaustion was also found. The produced results were in line with the findings of Petrović et al. (2017), who found similar correlation values between the instruments' dimensions and a higher correlation between the pairs of variables previously referred to González-Romá et al. (2006). In other words, the correlations between vigor and exhaustion were higher than with the other UWES variables, and the same for the disengagement-dedication pair. This study's results are in the same direction, reinforcing the higher association between disengagement and dedication than between disengagement and vigor or absorption, and a higher association between exhaustion and vigor than between exhaustion and dedication and absorption.
Our results confirmed the four of the five hypotheses and gave us, globally, good validity evidence regarding this OLBI version. However, since H1 had unsatisfactory validity evidence in terms of discriminant validity, this study brought a novelty (in terms of OLBI dimensionality studies); therefore, a second-order latent model was proposed, which is admissible from a theoretical and practical perspective. Thus, it is suggested that OLBI can be used to compare burnout levels among samples with different occupations and sexes from Portugal and Brazil. However, since the proposed version is a reduced version, it must be said that the reduced version must be tested in independent samples of the same populations (Marôco, 2014).
Burnout seems to be a second-order factor that loads on two first-order factors, disengagement, and exhaustion—which are burnout's core dimensions. This underlines the expansion of the burnout domain beyond the exhaustion affective component (Halbesleben et al., 2004). Female and male workers experience similar levels of burnout, the same happens among Portuguese and Brazilian workers. The obtained results demonstrate that the absence of differences in burnout between sexes and countries suggest that the work experiences in terms of stress are similar in both samples. The observed differences between sexes reported on other studies are small (Purvanova and Muros, 2010) and might be related to other factors—namely family and workload (Bekker et al., 2005; Langballe et al., 2011).
To have instruments with good validity evidence, it is mandatory to have confidence in the obtained measures. OLBI can help to establish comparisons between sexes, and countries. However, one should be aware of the different versions and of the quality of the evidence provided in each study. Also, OLBI can be useful and practical, since it is a freely available self-report psychometric instrument which can contribute to studies where the impact of companies' interventions are studied (Gíslason and Símonardóttir, 2018). In fact, there is evidence that occupational stress can be reduced with specific interventions (Ruotsalainen et al., 2015). The development of a family-friendly work environment should be approached, allowing one a focus on the importance of a balance between work and life (Lo, 2003; Rubino et al., 2013). Giving workers paid sick days, medical and family leave insurance programs, and greater control over their schedule (Appelbaum et al., 2014) are good suggestions to improve the balance between work and home activities. However, any change in organizational practices without a corresponding change in social attitudes will not be enough (Field and Bramwell, 1998). Prevention strategies should consider the social and individual level of those that will receive them (Maslach and Leiter, 2017).
This study used two convenience samples. It had no other psychological measures besides burnout and work engagement, which would allow better assessment of the validity evidence based on the relationship between other variables—namely, predictive, concurrent, and discriminant evidence (American Educational Research Association, 2014). For example, a concurrent burnout measure which would allow verification of the concurrent validity evidence between different burnout instruments, as some studies have done (Demerouti et al., 2003; Marôco and Campos, 2012) would have been useful. Another limitation is that culture-specific aspects of stress were not assessed, which could explain some of the observed differences.
Further studies using OLBI should test its concurrent validity evidence with other burnout instruments (American Educational Research Association, 2014), something that has been tested with success in the Portugal-Brazil version for students (Campos et al., 2012) and in the Greek version for workers (Demerouti et al., 2003). The Portugal-Brazil OLBI version should be tested in samples of specific occupational groups (e.g., Armed Forces Occupations, Craft and Related Trades Workers, Skilled Agricultural, Forestry and Fishery Workers, Plant and Machine Operators and Assemblers, and Elementary Occupations), of whom there was not a satisfactory number in the sample collected for this study. The same applies to lower academic levels, which showed unsatisfactory frequency in this study. Future research should also assess the family load together with the workload to understand the family-work interaction regarding burnout.
Initially, burnout research was linked to human services occupations: thus, sex was not a concern since most employees in this area were female. This study compared burnout levels between sexes in two different countries, and simultaneously adapting a specific instrument that allows establishing direct comparisons between countries.
OLBI offers various advantages over other instruments that can measure burnout, and the obtained findings focus on the utility of this inventory to compare burnout among sexes and countries using samples from Portugal and Brazil. The instrument showed validity evidence based on the internal structure and on the relation with other variables (work engagement and its first-order dimensions). Altogether, the proposed OLBI version appears to be a valid alternative to assess burnout and establish rigorous comparisons between Portuguese and Brazilian workers. The differences between sexes seem to non-existent. Burnout differences reported in other studies seem to be related to other factors, such as work and family load, as previous research suggested (Bekker et al., 2005; Langballe et al., 2011).
All authors of this research paper have directly participated in the planning, execution, or analysis of this study. More specifically, JS wrote the first draft, and with JM performed all statistical analysis and its discussion. JS and SP discussed cross-cultural topics, and JS and CQ discussed theoretical framework.
The authors would like to thank the Portuguese National Occupational Health Program of the Directorate-General of Health (DGS) for the sponsorship. And to the University of São Paulo (USP) International grant conceded to JS (Grant agreement number: 2014.1.3766.1.8). This work was partially funded by the Center for Psychology at the University of Porto, Portuguese Science Foundation (FCT UID/PSI/00050/2013) and EU FEDER through COMPETE 2020 program (POCI-01-0145-FEDER-007294). This work was partially funded by the William James Center for Research, Portuguese Science Foundation (FCT UID/PSI/04810/2013).
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Adriaenssens, J., De Gucht, V., and Maes, S. (2015). Determinants and prevalence of burnout in emergency nurses: a systematic review of 25 years of research. Int. J. Nurs. Stud. 52, 649–661. doi: 10.1016/j.ijnurstu.2014.11.004
Ahola, K., Toppinen-Tanner, S., and Seppänen, J. (2017). Interventions to alleviate burnout symptoms and to support return to work among employees with burnout: Systematic review and meta-analysis. Burn. Res. 4, 1–11. doi: 10.1016/j.burn.2017.02.001
Al-Asadi, J., Khalaf, S., Al-Waaly, A., Abed, A., and Shami, S. (2018). Burnout among primary school teachers in Iraq: prevalence and risk factors. East. Mediterr. Heal. J. 24, 262–268. doi: 10.26719/2018.24.3.262
Alexandrova-Karamanova, A., Todorova, I., Montgomery, A., Panagopoulou, E., Costa, P., Baban, A., et al. (2016). Burnout and health behaviors in health professionals from seven European countries. Int. Arch. Occup. Environ. Health 89, 1059–1075. doi: 10.1007/s00420-016-1143-5
Al-shuhail, A. S., Al-saleh, S., Al-saleh, S. S., Elhassan, I., and Wajid, S. (2017). Prevalence of burnout among National Guard Health Affairs physicians in Dammam, Saudi Arabia–a cross-sectional study. Asian J. Pharm. 11, S924–S929. doi: 10.22377/ajp.v11i04.1738
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association.
Ananthram, S., Teo, S. T. T., Connell, J., and Bish, A. (2017). Control and involvement HR practices in Indian call centres: still searching for answers. Asia Pacific J. Hum. Resour. 56, 196–215. doi: 10.1111/1744-7941.12153
Appelbaum, E., Boushey, H., and Schmitt, J. (2014). The Economic Importance of Women's Rising Hours of Work: Time to Update Employment Standards. Center for American Progress and the Center for Economic and Policy Research. Available online at: https://www.americanprogress.org/wp-content/uploads/2014/04/WomensRisingWorkv2.pdf
Baka, Ł. (2011). Relationship between work – family conficts and burnout: mediating role of personal resources. Psychol. Społeczna 4, 367–374. Available online at: http://spbulletin.com/wp-content/uploads/2016/11/Baka_PS_4_2011.pdf
Baka, Ł., and Cieślak, R. (2010). Self-efficacy and social support and the effects of job stressors on job burnout and work engagement in teachers. Stud. Psychol. 48, 5–19. Available online at: https://www.researchgate.net/profile/Lukasz_Baka/publication/281209412_Zaleznosci_miedzy_stresorami_w_pracy_a_wypaleniem_zawodowym_i_zaangazowaniem_w_prace_w_grupie_nauczycieli/links/55db818608aec156b9afecc2/Zaleznosci-miedzy-stresorami-w-pracy-a-wypaleni
Bakker, A. B., Demerouti, E., and Sanz-Vergel, A. I. (2014). Burnout and work engagement: the JD–R approach. Annu. Rev. Organ. Psychol. Organ. Behav. 1, 389–411. doi: 10.1146/annurev-orgpsych-031413-091235
Barbier, M., Peters, S., and Hansez, I. (2009). Measuring positive and negative occupational states (PNOSI): structural confirmation of a new Belgian tool. Psychol. Belg. 49, 227–247. doi: 10.5334/pb-49-4-227
Bekker, M. H. J., Croon, M. A., and Bressers, B. (2005). Childcare involvement, job characteristics, gender and work attitudes as predictors of emotional exhaustion and sickness absence. Work Stress 19, 221–237. doi: 10.1080/02678370500286095
Bosman, J., Rothmann, S., and Buitendach, J. H. (2005). Job insecurity, burnout and work engagement: the impact of positive and negative affectivity. SA J. Ind. Psychol. 31, 48–56. doi: 10.4102/sajip.v31i4.199
Bresó, E., Salanova, M., and Schaufeli, W. B. (2007). In search of the “third dimension” of burnout: efficacy or inefficacy? Appl. Psychol. An Int. Rev. 56, 460–478. doi: 10.1111/j.1464-0597.2007.00290.x
Buitendach, J. H., Bobat, S., Muzvidziwa, R. F., and Kanengoni, H. (2016). Work engagement and its relationship with various dimensions of work-related well-being in the public transport industry. Psychol. Dev. Soc. J. 28, 50–72. doi: 10.1177/0971333615622895
Campos, J. A. D. B., Carlotto, M. S., and Marôco, J. (2012). Oldenburg Burnout Inventory–student version: cultural adaptation and validation into Portuguese. Psicol. Reflex. Crít. 25, 709–718. doi: 10.1590/S0102-79722012000400010
Canivez, G. L. (2016). “Bifactor modeling in construct validation of multifactored tests: implications for understanding multidimensional constructs and test interpretation,” in Principles and Methods of Test Construction: Standards and Recent Advances, eds K. Schweizer, and C. DiStefano (Gottingen: Hogrefe), 247–271.
Carod-Artal, F. J., and Vázquez-Cabrera, C. (2013). “Burnout syndrome in an international setting,” in Burnout for Experts: Prevention in the Context of Living and Working, ed S. Bährer-Kohler (Boston, MA: Springer), 15–35.
Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., and Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociol. Methods Res. 36, 462–494. doi: 10.1177/0049124108314720
Chen, F. F., Sousa, K. H., and West, S. G. (2005). Teacher's corner: testing measurement invariance of second-order factor models. Struct. Equ. Model. A Multidiscip. J. 12, 471–492. doi: 10.1207/s15328007sem1203_7
Chen, F. F., and Zhang, Z. (2018). “Bifactor models in psychometric test development,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: Wiley), 325–345.
Chernoff, P., Adedokun, C., O'Sullivan, I., McManus, J., and Payne, A. (2018). Burnout in the emergency department hospital staff at Cork University Hospital. Ir. J. Med. Sci. 1–8. doi: 10.1007/s11845-018-1871-5. [Epub ahead of print].
Creedy, D. K., Sidebotham, M., Gamble, J., Pallant, J., and Fenwick, J. (2017). Prevalence of burnout, depression, anxiety and stress in Australian midwives: a cross-sectional survey. BMC Pregnancy Childbirth 17, 1–8. doi: 10.1186/s12884-016-1212-5
Delgadillo, J., Saxon, D., and Barkham, M. (2018). Associations between therapists' occupational burnout and their patients' depression and anxiety treatment outcomes. Depress. Anxiety 35, 844–850. doi: 10.1002/da.22766
Demerouti, E., and Bakker, A. B. (2008). “The Oldenburg Burnout Inventory: a good alternative to measure burnout and engagement,” in Handbook of Stress and Burnout in Health Care, ed J. R. B. Halbesleben (New York, NY: Nova Science), 65–78. Available online at: https://pdfs.semanticscholar.org/7cb5/c694cb9ad8c38e63db5d6458e34cd4fef5ce.pdf
Demerouti, E., Bakker, A. B., Vardakou, I., and Kantas, A. (2003). The convergent validity of two burnout instruments: a multitrait-multimethod analysis. Eur. J. Psychol. Assess. 19, 12–23. doi: 10.1027//1015-5722.214.171.124
Demerouti, E., Mostert, K., and Bakker, A. B. (2010). Burnout and work engagement: a thorough investigation of the independency of both constructs. J. Occup. Health Psychol. 15, 209–222. doi: 10.1037/a0019408
Demerouti, E., and Nachreiner, F. (1998). Zur spezifität von burnout für dienstleistungsberufe: Fakt oder artefakt? [The specificity of burnout in human services: Fact or artifact?]. Z. Arbeitswiss. 52, 82–89.
Dias, S., Queirós, C., and Carlotto, M. S. (2010). Burnout syndrome and associated factors among health professionals: a comparative study between Brazil and Portugal. Aletheia 32, 4–21. Available online at: http://www.redalyc.org/articulo.oa?id=115020838002
European Agency for Safety and Health at Work (2018). Management of Occupational Health and Safety in European Workplaces — Evidence From the Second European Survey of Enterprises on New and Emerging Risks (ESENER-2). Luxembourg: Publications Office of the European Union.
Field, S., and Bramwell, R. (1998). An investigation into the relationship between caring responsibilities and the levels of perceived pressure reported by female employees. J. Occup. Organ. Psychol. 71, 165–170. doi: 10.1111/j.2044-8325.1998.tb00670.x
Finney, S. J., and DiStefano, C. (2013). “Non-normal and categorical data in structural equation modeling,” in Structural Equation Modeling: A Second Course, eds G. R. Hancock, and R. O. Mueller (Charlotte, NC: Information Age Publishing), 439–492.
Ford, V. F., Swayze, S., and Burley, D. L. (2013). An exploratory investigation of the relationship between disengagement, exhaustion and turnover intention among IT professionals employed at a university. Inf. Resour. Manag. J. 26, 55–68. doi: 10.4018/irmj.2013070104
Foster, G. C. (2015). Measurement Invariance of Burnout Inventories Across Sex. Available online at: https://etd.ohiolink.edu/!etd.send_file?accession=bgsu1428162452&disposition=inline
Garibaldi de Hilal, A. V. (2009). “Cultural mythology and global leadership in Brazil,” in Cultural mythology and Global Leadership, eds E. H. Kessler and D. J. Wong-MingJi (Cheltenham, UK: Edward Elgar), 93–107.
Gignac, G. E. (2008). Higher-order models versus direct hierarchical models: g as superordinate or breadth factor? Psychol. Sci. Q. 50, 21–43. Available online at: http://www.psychologie-aktuell.com/fileadmin/download/PschologyScience/1-2008/04_Gignac.pdf
Gil-Monte, P. R., Viotti, S., and Converso, D. (2017). Psychometric properties of the Spanish Burnout Inventory (SBI) in a sample of Italian health professionals: a gender perspective. Liberabit 23, 153–169. doi: 10.24265/liberabit.2017.v23n2.01
Goering, D. D., Shimazu, A., Zhou, F., Wada, T., and Sakai, R. (2017). Not if, but how they differ: a meta-analytic test of the nomological networks of burnout and engagement. Burn. Res. 5, 21–34. doi: 10.1016/j.burn.2017.05.003
Golay, P., Reverte, I., Rossier, J., Favez, N., and Lecerf, T. (2013). Further insights on the French WISC–IV factor structure through Bayesian structural equation modeling. Psychol. Assess. 25, 496–508. doi: 10.1037/a0030676
Golembiewski, R. T., Scherb, K., and Boudreau, R. A. (1993). “Burnout in cross-national settings: Generic and model-specific perspectives,” in Professional Burnout: Recent Developments in Theory and Research, eds W. B. Schaufeli, C. Maslach, and T. Marek (London: Taylor & Francis), 217–236.
Gómez-Urquiza, J. L., De la Fuente-Solana, E. I., Albendín-García, L., Vargas-Pecino, C., Ortega-Campos, E. M., and Cañadas-De la Fuente, G. A. (2017). Prevalence of burnout syndrome in emergency nurses: a meta-analysis. Crit. Care Nurse 37, e1–e9. doi: 10.4037/ccn2017508
González-Romá, V., Schaufeli, W. B., Bakker, A. B., and Lloret, S. (2006). Burnout and work engagement: Independent factors or opposite poles? J. Vocat. Behav. 68, 165–174. doi: 10.1016/j.jvb.2005.01.003
Guenette, J. P., and Smith, S. E. (2017). Burnout: Prevalence and associated factors among radiology residents in New England with comparison against United States resident physicians in other specialties. Am. J. Roentgenol. 209, 136–141. doi: 10.2214/AJR.16.17541
Halbesleben, J. R. B., and Demerouti, E. (2005). The construct validity of an alternative measure of burnout: investigating the English translation of the Oldenburg Burnout Inventory. Work Stress 19, 208–220. doi: 10.1080/02678370500340728
Halliday, L., Walker, A., Vig, S., Hines, J., and Brecknell, J. (2017). Grit and burnout in UK doctors: a cross-sectional study across specialties and stages of training. Postgrad. Med. J. 93, 389–394. doi: 10.1136/postgradmedj-2015-133919
Henseler, J., Ringle, C. M., and Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. J. Acad. Mark. Sci. 43, 115–135. doi: 10.1007/s11747-014-0403-8
Hofstede, G., Garibaldi de Hilal, A. V., Malvezzi, S., Tanure, B., and Vinken, H. (2010b). Comparing regional cultures within a country: lessons from Brazil. J. Cross. Cult. Psychol. 41, 336–352. doi: 10.1177/0022022109359696
Hofstede, G. G. J., Hofstede, G. G. J., and Minkov, M. (2010a). Cultures and Organizations: Software of the Mind: Intercultural Cooperation and Its Importance for Survival. 3rd Edn. New York, NY: McGraw-Hill.
Højsgaard, S., and Halekoh, U. (2018). doBy: Groupwise Statistics, LSmeans, Linear Contrasts, Utilities (R package version 4.6-2) [Computer software]. Available online at: https://cran.r-project.org/package=doBy
Innstrand, S. T., Langballe, E. M., and Falkum, E. (2012). A longitudinal study of the relationship between work engagement and symptoms of anxiety and depression. Stress Heal. 28, 1–10. doi: 10.1002/smi.1395
Innstrand, S. T., Langballe, E. M., Falkum, E., and Aasland, O. G. (2011). Exploring within–and between-gender differences in burnout: 8 different occupational groups. Int. Arch. Occup. Environ. Health 84, 813–824. doi: 10.1007/s00420-011-0667-y
International Labour Office (2016). Workplace Stress: A Collective Challenge. Geneva: International Labour Organization. Available online at: http://www.ilo.org/safework/info/publications/WCMS_466547/lang–en/index.htm (accessed July 1, 2017).
International Labour Organization (2013). Case Study: Karoshi: Death From Overwork. Available online at: http://www.ilo.org/safework/info/publications/WCMS_211571/lang–en/index.htm (accessed October 6, 2017).
Irwing, P., and Hughes, D. J. (2018). “Test development,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: John Wiley & Sons), 3–47.
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., and Rosseel, Y. (2018). semTools: Useful Tools for Structural Equation Modeling (R package version 0.5-1) [Computer software]. Available online at: https://cran.r-project.org/package=semTools
Jovanović, N., Podlesek, A., Volpe, U., Barrett, E., Ferrari, S., Rojnic Kuzman, M., et al. (2016). Burnout syndrome among psychiatric trainees in 22 countries: Risk increased by long working hours, lack of supervision, and psychiatry not being first career choice. Eur. Psychiatry 32, 34–41. doi: 10.1016/j.eurpsy.2015.10.007
Kessler, R. C., Chiu, W. T., Demler, O., and Walters, E. E. (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 62, 617–627. doi: 10.1001/archpsyc.62.6.617
Khan, A., and Yusoff, R. B. M. (2016). Psychometric testing of Oldenburg Burnout Inventory among academic staff in Pakistan. Int. Rev. Manag. Mark. 6, 683–687. Available online at: www.econjournals.com
Khan, F., Khan, Q., and Naz, A. (2016). Female academicians are burnout in Pakistan universities? Gomal Univ. J. Res., 157–167. Available online at: http://www.gu.edu.pk/new/GUJR/Issues-2017/ONLINE_SpecialIssueI/16_FemaleAcademiciansareBurnoutinPakistanUniversities.pdf
Ko, C. (2017). Exploring how affectivity influence on job burnout through organizational politics perception. Int. J. Organ. Innov. 10, 86–94. Available online at: http://www.ijoi-online.org/attachments/article/54/FINAL%20ISSUE%20VOL%2010%20NUM%202%20OCTOBER%202017%20-%20SECTION%20A.pdf
Koeske, G. F., and Koeske, R. D. (1989). Construct validity of the Maslach Burnout Inventory: a critical review and reconceptualization. J. Appl. Behav. Sci. 25, 131–144. doi: 10.1177/0021886389252004
Kogoj, T. K., Cebašek-Travnik, Z., and Zaletel-Kragelj, L. (2014). Role of stress in burnout among students of medicine and dentistry–A study in Ljubljana, Slovenia, Faculty of Medicine. Coll. Antropol. 38, 879–887. Available online at: http://ezproxy.library.usyd.edu.au/login?url=http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=medl&AN=25420369%5Cnhttp://dd8gh5yx7k.search.serialssolutions.com/?sid=OVID:medline&id=pmid:25420369&id=doi:&issn=0350-6134&isbn=&volume=38&i
Kristensen, T. S., Borritz, M., Villadsen, E., and Christensen, K. B. (2005). The Copenhagen Burnout Inventory: a new tool for the assessment of burnout. Work Stress 19, 192–207. doi: 10.1080/02678370500297720
Langballe, E. M., Innstrand, S. T., Aasland, O. G., and Falkum, E. (2011). The predictive value of individual factors, work-related factors, and work-home interaction on burnout in female and male physicians: a longitudinal study. Stress Heal. 27, 73–87. doi: 10.1002/smi.1321
Larsen, A. C., Ulleberg, P., and Rønnestad, M. H. (2017). Depersonalization reconsidered: an empirical analysis of the relation between depersonalization and cynicism in an extended version of the Maslach Burnout Inventory. Nord. Psychol. 69, 160–176. doi: 10.1080/19012276.2016.1227939
le Roux, A. M. (2004). The Validation of Two Burnout Measures in the South African Earthmoving Equipment Industry. Master mini-dissertation, North-West Univ. Potchefstroom, South Africa. Available online at: http://dspace.nwu.ac.za/bitstream/handle/10394/674/leroux_am.pdf
Leiter, M. P. (1993). “Burnout as a developmental process: Consideration of models,” in Professional Burnout: Recent Developments in Theory and Research, eds W. B. Schaufeli, C. Maslach, and T. Marek (Washington, DC: Taylor & Francis), 237–250.
Lekutle, M., and Nel, J. A. (2012). Psychometric evaluation of the Utrecht Work Engagement Scale (UWES) and Oldenburg Burnout Inventory (OLBI) within a cement factory. J. Psychol. Africa 22, 641–647. doi: 10.1080/14330237.2012.10820580
Lemon, J. (2006). Plotrix: a package in the red light district of R. R-News 6, 8–12. Available online at: https://www.r-project.org/doc/Rnews/Rnews_2006-4.pdf
LimeSurvey GmbH (2017). LimeSurvey: An Open Source Survey Tool [Computer software]. Available online at: http://www.limesurvey.org
Lindblom, K. M., Linton, S. J., Fedeli, C., and Bryngelsson, I.-L. (2006). Burnout in the working population: relations to psychosocial work factors. Int. J. Behav. Med. 13, 51–59. doi: 10.1207/s15327558ijbm1301_7
Lundkvist, E., Stenling, A., Gustafsson, H., and Hassmén, P. (2014). How to measure coach burnout: an evaluation of three burnout measures. Meas. Phys. Educ. Exerc. Sci. 18, 209–226. doi: 10.1080/1091367X.2014.925455
MacCallum, R. C., Browne, M. W., and Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychol. Methods 1, 130–149. doi: 10.1037/1082-989X.1.2.130
Mahadi, N. F., Chin, R. W. A., Chua, Y. Y., Chu, M. N., Wong, M. S., Yusoff, M. S. B., et al. (2018). Malay language translation and validation of the Oldenburg Burnout Inventory measuring burnout. Educ. Med. J. 10, 27–40. doi: 10.21315/eimj2018.10.2.4
Marôco, J., and Campos, J. A. D. B. (2012). Defining the student burnout construct: a structural analysis from three burnout inventories. Psychol. Rep. 111, 814–830. doi: 10.2466/14.10.20.PR0.111.6.814-830
Marôco, J., Marôco, A. L., and Campos, J. A. D. B. (2014). Student's academic efficacy or inefficacy? An example on how to evaluate the psychometric properties of a measuring instrument and evaluate the effects of item wording. Open J. Stat. 04, 484–493. doi: 10.4236/ojs.2014.46046
Marôco, J., Marôco, A. L., Leite, E., Bastos, C., Vazao, M. J., and Campos, J. A. D. B. (2016). Burnout in Portuguese healthcare professionals: an analysis at the national level. Acta Med. Port. 29, 24–30. doi: 10.20344/acta%20med%20port.v29i1.6460
Marques-Pinto, A., Lima, M. L., Silva, A. L., and da (2008). “Delimitação do conceito de burnout,” in Burnout e engagement em contexto organizacional: Estudos com amostras portuguesas, eds A. Marques-Pinto, and M. J. Chambel (Lisbon: Livros Horizonte), 15–52.
Marques-Pinto, A., Lima, M. L., and Silva, A. L. da. (2003). Stress profissional em professores portugueses: Incidência, preditores e reacção de burnout. Psychologica 33, 181–194. Available online at: https://www.academia.edu/6286459/Stress_profissional_em_professores_portugueses_Incid%C3%AAncia_preditores_e_reac%C3%A7%C3%A3o_de_burnout
Maslach, C., Jackson, S. E., and Leiter, M. P. (2016). Maslach Burnout Inventory Manual. 4th Edn. Menlo Park, CA: Mind Garden. Available online at: https://www.mindgarden.com
Maslach, C., and Leiter, M. P. (2017). “Understanding burnout: New models,” in The Handbook of Stress and Health: A Guide to Research and Practice, eds C. L. Cooper, and J. C. Quick (Chichester: John Wiley & Sons), 36–56.
Maydeu-Olivares, A., Shi, D., and Rosseel, Y. (2018). Assessing fit in structural equation models: a Monte-Carlo evaluation of RMSEA versus SRMR confidence intervals and tests of close fit. Struct. Equ. Model. A Multidiscip. J. 25, 389–402. doi: 10.1080/10705511.2017.1389611
McNamara, A., Arino de la Rubia, E., Zhu, H., Ellis, S., and Quinn, M. (2018). skimr: Compact and Flexible Summaries of Data (R package version 1.0.3) [Computer software]. Available online at: https://cran.r-project.org/package=skimr
Mészáros, V., Ádám, S., Szabó, M., Szigeti, R., and Urbán, R. (2014). The bifactor model of the Maslach Burnout Inventory-Human Services Survey (MBI-HSS): An alternative measurement model of burnout. Stress Heal. 30, 82–88. doi: 10.1002/smi.2481
Morgan, B., de Bruin, G. P., and de Bruin, K. (2014). Operationalizing burnout in the Maslach Burnout Inventory–Student Survey: personal efficacy versus personal inefficacy. South African J. Psychol. 44, 216–227. doi: 10.1177/0081246314528834
Morin, A. J. S., Katrin Arens, A., and Marsh, H. W. (2015). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Struct. Equ. Model. 23, 116–139. doi: 10.1080/10705511.2014.961800
Murray, A. L., and Johnson, W. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence 41, 407–422. doi: 10.1016/j.intell.2013.06.004
O'Connor, K., Muller Neff, D., and Pitman, S. (2018). Burnout in mental health professionals: a systematic review and meta-analysis of prevalence and determinants. Eur. Psychiatry 53, 74–99. doi: 10.1016/j.eurpsy.2018.06.003
Olinske, J. L., and Hellman, C. M. (2017). Leadership in the human service nonprofit organization: the Influence of the board of directors on executive director well-being and burnout. Hum. Serv. Organ. Manag. Leadersh. Gov. 41, 95–105. doi: 10.1080/23303131.2016.1222976
Papenberg, M. (2018). minDiff: Minimize Differences Between Groups (R package version 0.01-3) [Computer software]. Available online at: https://github.com/m-Py/minDiff
Peterson, U., Bergström, G., Demerouti, E., Gustavsson, P., Åsberg, M., and Nygren, Å. (2011). Burnout levels and self-rated health prospectively predict future long-term sickness absence: a study among female health professionals. J. Occup. Environ. Med. 53, 788–793. doi: 10.1097/JOM.0b013e318222b1dc
Peterson, U., Bergström, G., Samuelsson, M., Asberg, M., and Nygren, Å. (2008). Reflecting peer-support groups in the prevention of stress and burnout: randomized controlled trial. J. Adv. Nurs. 63, 506–516. doi: 10.1111/j.1365-2648.2008.04743.x
Petrović, I. B., Vukelić, M., and Cizmić, S. (2017). Work engagement in Serbia: psychometric properties of the Serbian version of the Utrecht Work Engagement Scale (UWES). Front. Psychol. 8:1799. doi: 10.3389/fpsyg.2017.01799
Poghosyan, L., Clarke, S. P., Finlayson, M., and Aiken, L. H. (2010). Nurse burnout and quality of care: cross-national investigation in six countries. Res. Nurs. Health 33, 288–298. doi: 10.1002/nur.20383
Pu, J., Zhou, X., Zhu, D., Zhong, X., Yang, L., Wang, H., et al. (2017). Gender differences in psychological morbidity, burnout, job stress and job satisfaction among Chinese neurologists: a national cross-sectional study. Psychol. Health Med. 22, 680–692. doi: 10.1080/13548506.2016.1211717
Qiao, H., and Schaufeli, W. B. (2011). The convergent validity of four burnout measures in a Chinese sample: a confirmatory factor-analytic approach. Appl. Psychol. 60, 87–111. doi: 10.1111/j.1464-0597.2010.00428.x
R Core Team (2018). R: A Language and Environment for Statistical Computing (version 3.5.1) [Computer software]. Available online at: https://www.r-project.org/
Reis, D., Xanthopoulou, D., and Tsaousis, I. (2015). Measuring job and academic burnout with the Oldenburg Burnout Inventory (OLBI): factorial invariance across samples and countries. Burn. Res. 2, 8–18. doi: 10.1016/j.burn.2014.11.001
Reise, S. P., Bonifay, W. E., and Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. J. Pers. Assess. 95, 129–140. doi: 10.1080/00223891.2012.725437
Reise, S. P., Moore, T. M., and Haviland, M. G. (2010). Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores. J. Pers. Assess. 92, 544–559. doi: 10.1080/00223891.2010.496477
Revelle, W. (2018). psych: Procedures for Psychological, Psychometric, and Personality Research (R package version 1.8.12) [Computer software]. Available online at: https://cran.r-project.org/package=psych
Revelle, W., and Condon, D. M. (2018). “Reliability,” in The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development, eds P. Irwing, T. Booth, and D. Hughes (Hoboken, NJ: John Wiley & Sons), 709–749.
Rodriguez, A., Reise, S. P., and Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. J. Pers. Assess. 98, 223–237. doi: 10.1080/00223891.2015.1089249
Rogala, A., Shoji, K., Luszczynska, A., Kuna, A., Yeager, C., Benight, C. C., et al. (2016). From exhaustion to disengagement via self-efficacy change: findings from two longitudinal studies among Human Services Workers. Front. Psychol. 6:2032. doi: 10.3389/fpsyg.2015.02032
Rosnah, I., Idris, M. A., Azmi, M. T., and Noor Hassim, I. (2017). Job stress on abdominal obesity: the moderating effects of anger and overeating behaviour. Malays. J. Nutr. 23, 437–448. Available online at: https://www.scopus.com/record/display.uri?eid=2-s2.0-85040452397&origin=inward&txGid=21158a61fcf5cd5de411607df70ea1d0
RStudio Team (2018). RStudio: Integrated Development for R (version 1.1.463) [Computer software]. Available online at: http://www.rstudio.com/
Rudman, A., Gustavsson, P., and Hultell, D. (2014). A prospective study of nurses' intentions to leave the profession during their first five years of practice in Sweden. Int. J. Nurs. Stud. 51, 612–624. doi: 10.1016/j.ijnurstu.2013.09.012
Ruotsalainen, J. H., Verbeek, J. H., Mariné, A., and Serra, C. (2015). Preventing occupational stress in healthcare workers. Cochrane Database Syst. Rev. 18:CD002892. doi: 10.1002/14651858.CD002892.pub5
Rzeszutek, M. (2013). Burnout syndrome in male and female gestalt and cognitive-behavioral psychotherapists. Rocz. Psychol. 16, 155–161. Available online at: https://www.ceeol.com/search/article-detail?id=110122
Saboonchi, F., Perski, A., and Grossi, G. (2013). Validation of Karolinska Exhaustion Scale: psychometric properties of a measure of exhaustion syndrome. Scand. J. Caring Sci. 27, 1010–1017. doi: 10.1111/j.1471-6712.2012.01089.x
Sales, B., Macdonald, A., Scallan, S., and Crane, S. (2016). How can educators support general practice (GP) trainees to develop resilience to prevent burnout? Educ. Prim. Care 27, 487–493. doi: 10.1080/14739879.2016.1217170
Salmela-Aro, K., Rantanen, J., Hyvönen, K., Tilleman, K., and Feldt, T. (2011). Bergen Burnout Inventory: reliability and validity among Finnish and Estonian managers. Int. Arch. Occup. Environ. Health 84, 635–645. doi: 10.1007/s00420-010-0594-3
Scanlan, J. N., and Still, M. (2013). Job satisfaction, burnout and turnover intention in occupational therapists working in mental health. Aust. Occup. Ther. J. 60, n/a-n/a. doi: 10.1111/1440-1630.12067
Schadenhofer, P., Kundi, M., Abrahamian, H., Stummer, H., and Kautzky-Willer, A. (2018). Influence of gender, working field and psychosocial factors on the vulnerability for burnout in mental hospital staff: Results of an Austrian cross-sectional study. Scand. J. Caring Sci. 32, 335–345. doi: 10.1111/scs.12467
Schaufeli, W. B. (2017). “Burnout: A short socio-cultural history,” in Burnout, Fatigue, Exhaustion: An Interdisciplinary Perspective on a Modern Affliction, eds S. Neckel, A. K. Schaffner, and G. Wagner (Cham: Springer), 105–127.
Schaufeli, W. B., and Bakker, A. B. (2010). “Defining and measuring work engagement: bringing clarity to the concept,” in Work Engagement: A Handbook of Essential Theory and Research, eds A. B. Bakker, and M. P. Leiter (New York, NY: Psychology Press), 10–24.
Schaufeli, W. B., Salanova, M., González-Romá, V., and Bakker, A. B. (2002). The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. J. Vocat. Behav. 3, 71–92. doi: 10.1023/A:1015630930326
Schuster, M. da S., Dias, V. da V., and Battistella, L. F. (2015). Burnout Inventory of Oldenburg: the influence of gender, occupation and supervision. Rev. Eletrôn. Mestr. Prof. Adm. Univ. Potiguar 8, 9–18. Available online at: Revista Eletrônica do Mestrado Profissional em Administração da Universidade Potiguar.
Schuster, M. d. S., Dias, V. d. V., Battistella, L. F., and Marquetto, M. F. (2013). “Oldenburg burnout inventory-validação de uma nova forma de mensurar burnout no Brasil,” in Paper Presented at the IV Encontro de Gestão de Pessoas e Relações de Trabalho (Brasília), 1–11.
Sedlar, N., Šprah, L., Tement, S., and Sočan, G. (2015). Internal structure of an alternative measure of burnout: Study on the Slovenian adaptation of the Oldenburg Burnout Inventory (OLBI). Burn. Res. 2, 1–7. doi: 10.1016/j.burn.2015.02.001
Seidler, A., Thinschmidt, M., Deckert, S., Then, F., Hegewald, J., Nieuwenhuijsen, K., et al. (2014). The role of psychosocial working conditions on burnout and its core component emotional exhaustion – a systematic review. J. Occup. Med. Toxicol. 9, 1–13. doi: 10.1186/1745-6673-9-10
Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J. A., et al. (2015). Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin. Proc. 90, 1600–1613. doi: 10.1016/j.mayocp.2015.08.023
Silva, M. A. O. M., da Gomes, L. F. A. M., and Correia, M. F. (2009). Entrepreneurial culture: a comparative study of entrepreneurs in Brazil and Portugal. Rev. Adm. Contemp. 13, 57–71. doi: 10.1590/S1415-65552009000100005
Simbula, S., and Guglielmi, D. (2010). Depersonalization or cynicism, efficacy or inefficacy: what are the dimensions of teacher burnout? Eur. J. Psychol. Educ. 25, 301–314. doi: 10.1007/s10212-010-0017-6
Sinval, J., Pasian, S., Queirós, C., and Marôco, J. (2018). Brazil-Portugal transcultural adaptation of the UWES-9: internal consistency, dimensionality, and measurement invariance. Front. Psychol. 9:353. doi: 10.3389/fpsyg.2018.00353
Smirnova, A. Y. (2017). The Oldenburg Burnout Inventory: diagnostics of state of mind's change of the employ on a continuum: work engagement – professional burnout. Philos. Psychol. Pedagog. 17, 211–218. doi: 10.18500/1819-7671-2017-17-2-211-218
Subburaj, A., and Vijayadurai, J. (2016). Translation, validation and psychometric properties of Tamil version of Oldenburg Burnout Inventory (OLBI). Proc. Soc. Behav. Sci. 219, 724–731. doi: 10.1016/j.sbspro.2016.05.067
Suyi, Y., Meredith, P., and Khan, A. (2017). Effectiveness of mindfulness intervention in reducing stress and burnout for mental health professionals in Singapore. Explor. J. Sci. Heal. 13, 319–326. doi: 10.1016/j.explore.2017.06.001
Taris, T. W., Schreurs, P. J. G., and Schaufeli, W. B. (1999). Construct validity of the Maslach Burnout Inventory-General Survey: a two-sample examination of its factor structure and correlates. Work Stress 13, 223–237. doi: 10.1080/026783799296039
Turtulla, S. (2017). Examining levels of job burnout among teachers working in Kosova in terms of different variables: proposing a psychological and counselling group program for managing. Eur. J. Multidiscip. Stud. 6, 285–297. doi: 10.26417/ejms.v6i1.p285-297
van de Vijver, F. J. R. (2016). “Test adaptations,” in The ITC International Handbook of Testing and Assessment, eds F. T. L. Leong, D. Bartram, F. M. Cheung, K. F. Geisinger, and D. Iliescu (New York, NY: Oxford University Press), 364–376.
Wang, G., Liu, C., and Wu, X. (2003). Development of Educator Burnout Inventory. Psychol. Dev. Educ. 3, 82–86. Available online at: http://en.cnki.com.cn/Article_en/CJFDTOTAL-XLFZ200303014.htm
Westwood, S., Morison, L., Allt, J., and Holmes, N. (2017). Predictors of emotional exhaustion, disengagement and burnout among improving access to psychological therapies (IAPT) practitioners. J. Ment. Heal. 26, 172–179. doi: 10.1080/09638237.2016.1276540
Yanos, P. T., Vayshenker, B., DeLuca, J. S., and O'Connor, L. K. (2017). Development and validation of a scale assessing mental health clinicians' experiences of associative stigma. Psychiatr. Serv. 68, 1053–1060. doi: 10.1176/appi.ps.201600553
Zinbarg, R. E., Revelle, W., Yovel, I., and Li, W. (2005). Cronbach's α, Revelle's β, and Mcdonald's ωH: their relations with each other and two alternative conceptualizations of reliability. Psychometrika 70, 123–133. doi: 10.1007/s11336-003-0974-7
Keywords: Oldenburg Burnout Inventory (OLBI), burnout, measurement invariance, Brazil, Portugal, multi-occupational, validity evidence
Citation: Sinval J, Queirós C, Pasian S and Marôco J (2019) Transcultural Adaptation of the Oldenburg Burnout Inventory (OLBI) for Brazil and Portugal. Front. Psychol. 10:338. doi: 10.3389/fpsyg.2019.00338
Received: 05 September 2018; Accepted: 04 February 2019;
Published: 12 March 2019.
Edited by:Simon Surguladze, King's College London, United Kingdom
Reviewed by:Marlene Sophie Penz, Technische Universität Dresden, Germany
Nikolaos Tsigilis, Aristotle University of Thessaloniki, Greece
Copyright © 2019 Sinval, Queirós, Pasian and Marôco. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: João Marôco, firstname.lastname@example.org