Introduction
In a recent volume of this journal, Schaffran et al. (2019) introduced the Burnout Prevention Questionnaire for Coaches (BPQ-C). Although we recognize the worthwhile efforts of Schaffran et al., we believe that there are several issues associated with this instrument. This commentary aims to expand on why we think the BPQ-C should not currently be used by practitioners and researchers to screen for burnout.
Lack of Theoretical Foundations
With regards to theory, we have four concerns. First, Schaffran et al.'s definition (and subsequent measurement) of burnout is inadequate. Although several definitions of burnout exist, most researchers agree that burnout is a multidimensional construct and models share—at a minimum—an exhaustion component (Maslach et al., 1996; Raedeke and Smith, 2001; Shirom, 2005). This was recently highlighted by the World Health Organization (2018). While not everyone agrees with one exact definition, no well-recognized definition includes general stress, social stress, and amotivation (Maslach et al., 1996; Raedeke and Smith, 2001; World Health Organization, 2018).
Second, Schaffran et al. (2019) seem to conflate antecedents, outcomes, and the construct of burnout itself. This point is exemplified by the fact that chronic job stress has historically been defined as an essential burnout antecedent and not a defining characteristic of burnout (Maslach et al., 1996; Shirom, 2005). Therefore, using stress as a symptom of burnout and not as an antecedent is conceptually incorrect. Similarly, the dimension labeled “pre-burnout” includes fatigue. This is confusing because of the evident conceptual overlap with exhaustion, which is known as a defining characteristic of burnout (Grossi et al., 2015).
Third, an underlying assumption for the BPQ-C seems to be that there exist sequential events in the burnout process. It is therefore peculiar that the authors relied exclusively on cross-sectional data when they created and tested their instrument (Leiter and Maslach, 1999, 2003; Hendrix et al., 2000). Especially since the complexity regarding temporality between burnout dimensions has been discussed before both in sport and more generally (Shirom and Melamed, 2006; Martinent et al., 2016).
Our final conceptual concern is the reliance on statistical considerations (from analysis of one sample), instead of also considering theoretical aspects, when deciding on the inclusion/exclusion of certain BPQ-C dimensions (Study 2). This exploratory approach has been criticized in the burnout literature previously and heightens risks for conceptual overlap with other constructs (see Shirom and Melamed, 2006), as well as misunderstanding of temporal relations with emotional exhaustion (Moore, 2000). The problem with this procedure becomes particularly apparent in relation to the resources dimension which defines sleep as a resource that may buffer potential burnout. However, with the same logic, poor sleep quality could be seen as a part of “pre-burnout” since poor sleep quality is a well-known antecedent of burnout symptoms (Söderström et al., 2012). Thus, although the authors briefly acknowledge the complexity of sleep in relation to burnout in their discussion, they do not provide any guidelines based on sound empirical work and theory to explain how sleep scores in the BPQ-C should be interpreted.
Utility as a Screening Tool
The authors claim that the BPQ-C has the potential to be used as a screening tool in practical settings to detect early signs of burnout, and to guide prevention strategies before a coach develops more serious burnout symptoms. Although a commendable goal, one well-known problem is that there is often no clear distinction between normal functioning and psychological illness. Evaluations of symptoms should be related to the context in which they appear as well as to contemporary values and economic forces present in society and politics (Kinderman et al., 2017). Furthermore, Schaffran and colleagues do not provide evidence that the BPQ-C can be used to adequately screen for clinical burnout. To do so, analyses such as receiver operating characteristic analysis (ROC), which tests the ability of measures to discriminate between individuals with and without a certain clinical characteristic, is necessary (Kraemer, 1992). Comparisons between a clinical and a healthy sample were not a part of the validation of the BPQ-C. In fact, no descriptive statistics of burnout are reported. This makes it even harder to determine variation and representability of the samples that were used. In addition, even when a clinical sample is compared with a healthy sample, discriminating between healthy and ill individuals is challenging as they show substantial overlap (Lundgren-Nilsson et al., 2012). Therefore, using screening tools without clinical interviews creates substantial risks of both false positives and negatives, a matter that has been neglected in the sport psychology literature and is not considered by Schaffran et al. in their article.
Questionnaire Development
Our last point of criticism concerns the development and validation of the BPQ-C. Several guides on how to develop a psychological instrument are available. Although they differ, there are some common grounds when it comes to item development, scale development, and scale evaluation (MacKenzie et al., 2011; Boateng et al., 2018). Schaffran et al., however, only communicate a few of these suggested steps. When such a step is communicated, like confirmatory factor analysis (CFA), the high degree of similarity in subscale content suggests a redundancy of these factors. This issue is compounded by the fact that parallel analysis suggested a one-factor model (see Sample 1). A stepwise evaluation would have easily eliminated this in the item developing phase. Lastly, in the three-factor CFA, several residuals are correlated (see Figure 1 in Schaffran et al., 2019). The reasons for this needs to be communicated since correlating residuals comes with several risks (with capitalization of chance being the most prominent one), making replication in another sample unlikely (Landis et al., 2009).
Discussion
This commentary provides several arguments as to why, in our opinion, the BPQ-C fails to meet several theoretical, clinical, and psychometric standards. Substantial work is therefore needed before the BPQ-C can be considered a valid measure. Instead, we recommend the use of existing validated measures of burnout. One example of such a measure is the Shirom Melamed Burnout Questionnaire (SMBQ; Kushnir and Melamed, 1992), which is based on Hobfoll's Conservation of Resources (COR) theory (Hobfoll, 1989). The SMBQ has been validated in both clinical and healthy samples (Lundgren-Nilsson et al., 2012). Another alternative would be the Maslach Burnout Inventory (Maslach et al., 1996), which follows the theoretical underpinnings provided by Maslach and colleagues (Maslach et al., 1996). The MBI has shown reasonable psychometric properties in previous research, and for which at least some evidence exists that the instrument is able to discriminate between clinical and healthy individuals. However, the latter only applies to the exhaustion subscale (Kleijweg et al., 2013). Here, we note, however, that using self-report measures as the only source for clinical screening may not be successful because of the overlap between clinical and healthy individuals. We therefore recommend that, wherever possible, self-report screening should be combined with clinical interviews (Lundgren-Nilsson et al., 2012; Kleijweg et al., 2013) and that the interpretation of self-reported burnout measures with clinical cut-offs should be made with caution.
Statements
Author contributions
EL is the main author who initiated the commentary, wrote the first draft, and has been editing all other contributors' drafts and comments. HG was involved in the design of the paper and has been commenting and editing the paper throughout the whole process. MG has been commenting on the paper and has contributed uniquely to the lack of theoretical foundations part. CL has been commenting on the paper and has contributed uniquely to the clinical screening tool part. AI has been commenting on the paper and has contributed uniquely on the part called questionnaire development. DM was involved in the initial scoping of the paper, has done English editing, and contributed uniquely on the lack of theoretical foundations part.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1
BoatengG. O.NeilandsT. B.FrongilloE. A.Melgar-QuiñonezH. R.YoungS. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health6:149. 10.3389/fpubh.2018.00149
2
GrossiG.PerskiA.OsikaW.SavicI. (2015). Stress-related exhaustion disorder–clinical manifestation of burnout? A review of assessment methods, sleep impairments, cognitive disturbances, and neuro-biological and physiological changes in clinical burnout. Scand. J. Psychol.56, 626–636. 10.1111/sjop.12251
3
HendrixA. E.AcevedoE. O.HebertE. (2000). An examination of stress and burnout in certified athletic trainers at Division IA universities. J Athletic Train.35:139.
4
HobfollS. E. (1989). Conservation of resources: a new attempt at conceptualizing stress. Am. Psychol.44:513.
5
KindermanP.AllsoppK.CookeA. (2017). Responses to the publication of the American Psychiatric Association's DSM-5. J. Hum. Psychol.57, 625–649. 10.1177/0022167817698262
6
KleijwegJ. H.VerbraakM. J.Van DijkM. K. (2013). The clinical utility of the Maslach Burnout Inventory in a clinical population. Psychol. Assess.25:435. 10.1037/a0031334
7
KraemerH. C. (1992). Evaluating Medical Tests: Objective and Quantitative Guidelines, Vol. 26. Newbury Park, CA: Sage publications.
8
KushnirT.MelamedS. (1992). The Gulf War and its impact on burnout and well-being of working civilians. Psychol. Med.22, 987–995.
9
LandisR.EdwardsB.CortinaJ. (2009). Correlated residuals among items in the estimation of measurement models, in Statistical and Methodological Myths and Urban Legends: Doctrine, Verity, and Fable in the Organizational and Social Sciences, eds LanceC. E.VanderbergR. J. (Abingdon: Taylor & Francis), 195–214.
10
LeiterM. P.MaslachC. (1999). Six areas of worklife: a model of the organizational context of burnout. J. Health Hum. Serv. Admin.21, 472–489.
11
LeiterM. P.MaslachC. (2003). Areas of worklife: a structured approach to organizational predictors of job burnout, in Emotional and Physiological Processes and Positive Intervention Strategies, eds GansterD. C.PerrewéP. L. (Bingley: Emerald Group Publishing Limited), 91–134.
12
Lundgren-NilssonÅ.JonsdottirI. H.PallantJ.AhlborgG. (2012). Internal construct validity of the Shirom-Melamed Burnout questionnaire (SMBQ). BMC Public Health12:1. 10.1186/1471-2458-12-1
13
MacKenzieS. B.PodsakoffP. M.PodsakoffN. P. (2011). Constructmeasurement and validation procedures in MIS and behavioral research: integrating new and existing techniques. MIS Q.35, 293–334. 10.2307/23044045
14
MartinentG.LouvetB.DecretJ.-C. (2016). Longitudinal trajectories of athlete burnout among young table tennis players: a 3-wave study. J. Sport Health Sci. 10.1016/j.jshs.2016.09.003. [Epub ahead of print].
15
MaslachC.JacksonS. E.LeiterM. P. (1996). Maslach Burnout Inventory Manual, Vol. 4. Palo Alto, CA: Consulting Psychologists Press.
16
MooreJ. E. (2000). Why is this happening? A causal attribution approach to work exhaustion consequences. Acad. Manage. Rev.25, 335–349. 10.5465/amr.2000.3312920
17
RaedekeT. D.SmithA. L. (2001). Development and preliminary validation of an athlete burnout measure. J. Sport Exerc. Psychol.23, 281–306. 10.1123/jsep.23.4.281
18
SchaffranP.KleinertJ.AltfeldS.ZeppC.KallusK. W.KellmannM. (2019). Early risk detection of burnout: development of the burnout prevention questionnaire for coaches. Front. Psychol.10:714. 10.3389/fpsyg.2019.00714
19
ShiromA. (2005). Reflections on the study of burnout. Work Stress19, 263–270. 10.1080/02678370500376649
20
ShiromA.MelamedS. (2006). A comparison of the construct validity of two burnout measures in two groups of professionals. Int. J. Stress Manage.13:176. 10.1037/1072-5245.13.2.176
21
SöderströmM.JedingK.EkstedtM.PerskiA.ÅkerstedtT. (2012). Insufficient sleep predicts clinical burnout. J. Occup. Health Psychol.17:175. 10.1037/a0027518
22
World Health Organization (2018). ICD-11 International Classification of Diseases for Mortality and Morbidity Statistics (11th Revision). Available online at: https://icd.who.int/browse11/l-m/en
Summary
Keywords
burnout, fatigue, exhaustion disorder, clinical burnout, burnout measurement
Citation
Lundkvist E, Gustafsson H, Gerber M, Lundqvist C, Ivarsson A and Madigan DJ (2019) Commentary: Early Risk Detection of Burnout: Development of the Burnout Prevention Questionnaire for Coaches. Front. Psychol. 10:2721. doi: 10.3389/fpsyg.2019.02721
Received
10 October 2019
Accepted
18 November 2019
Published
04 December 2019
Volume
10 - 2019
Edited by
Maicon Rodrigues Albuquerque, Federal University of Minas Gerais, Brazil
Reviewed by
Daniel Alvarez Pires, Federal University of Pará, Brazil
Updates
Copyright
© 2019 Lundkvist, Gustafsson, Gerber, Lundqvist, Ivarsson and Madigan.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Erik Lundkvist erik.lundkvist@gih.se
This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.