Scales Used to Measure Job Stressors in Intensive Care Units: Are They Relevant and Reliable? A Systematic Review

Background: Many studies have been conducted in intensive care units (ICUs) to identify the stress factors involved in the health of professionals and the quality and safety of care. The objectives are to identify the psychometric scales used in these studies to measure stressors and to assess their relevance and validity/reliability. Methods: All peer-reviewed full-text articles published in English between 1997 and 2016 and focusing on an empirical quantitative study of job stressors were identified through searches on seven databases and editorial portals. Results: From the 102 studies analyzed, we identified 59 different scales: 17 “all settings scales” (16 validated scales), 20 “healthcare settings scales” (13 validated scales), and 22 “ICU settings scales” (two validated scales). All these scales used measured stressors from at least one of the following eight broad categories: High job demands, Problematic relationships with other professionals, Lack of control over work situations and career, Lack of organizational resources, Problematic situations with users and relatives, Dealing with ethical- and moral-related situations, Risk management issues, and Disadvantages in comparison to other occupational situations. The “all settings scales” and “healthcare settings scales,” the most often validated, did not measure, or only slightly measured, the stressors most specific to ICUs. Where these were taken into account, the authors were forced to develop their own tools or modify existing scales without testing the validity of the tool used. Conclusions: This review highlights the lack of a tool that meets both the criteria of validity and relevance with regard to the specificity of work in ICUs. Future research must focus on developing reliable/valid tools covering all types of relevant stressors to ensure the quality of the studies carried out in this field.


INTRODUCTION
Intensive care unit (ICU) professionals must deal with patients with serious medical conditions that require complex diagnostic and therapeutic procedures. This often requires a considerable level of coordination of human resources. Furthermore, end-oflife decisions are frequent and contribute to an intense emotional charge (Teixeira et al., 2014;Flannery et al., 2016). The ICU is thus fertile ground for the emergence of professional stressors (Donchin and Seagull, 2002;Embriaco et al., 2007a). Assessed by individuals as situations that weaken or are beyond their resources (Lazarus and Folkman, 1984), work-related stressors impact the mental and physical health of workers and the quality and safety of care (Sochalski, 2004;de Cássia Pereira Fernandes et al., 2016;Krämer et al., 2016;Dragano et al., 2017;Vandevala et al., 2017).
To assess these professional stressors, various surveys have been developed that cover either generic scales addressing all professional activities or more specific scales focusing primarily on the healthcare field or on a specific healthcare sector. These healthcare scales are used to measure specific stressors such as "end-of-life decisions" (Ozden et al., 2013;Teixeira et al., 2014), "conflicts, " or the "health culture" (Profit et al., 2014;Garrouste-Orgeas et al., 2015) particularly studied for their involvement in burnout or anxiety-depression.
While the identification of professional stressors is in line with the promotion of well-being at work and the quality of healthcare, the reliability of the psychometric tools is also of great importance. However, it appears that the multiplication of stress assessment tools makes it difficult to choose the most appropriate scale. In this sense, Bonneterre et al. (2008) denounce the use of unsuitable tools or the use of tools with insufficient psychometric qualities, which make the predictive validities between stressors and epidemiological indicators unreliable. Therefore, when researchers or clinicians wish to evaluate the stressors present in the ICU, what types of tools can they find in the literature? First, are these tools psychometrically valid? In addition, are they able to measure all relevant stressors, including those most specific to ICUs? This latter question deals with the issue of ecological validity. The ecological validity concept examined whether a study and its findings are representative of real-life situations (Brewer and Crano, 2014). In particular, ecologically valid studies used material and procedure that satisfy three parameters (Schmuckler, 2001): they reflect situations or events that can actually occur in participants' everyday lives, reproduce or they refer to features of participants' current living environments, and stimulate from participants reactions that are already available in their response repertoire. Applied to the measurement of stressors in ICUs, an ecologically valid tool should cover all types of situations actually occurring in ICUs that are likely to cause stress to professionals, with item content that refers to their current working environment (e.g., service functioning, patients and families, tasks to perform). Thus, the ideal tool would respect these principles in addition to present good psychometric properties (e.g., reliability, construct validity). But is it possible to find such an ideal tool in peerreviewed journals?
To answer these questions, our literature review was guided by the following research objectives: 1) To comprehensively identify the scales and questionnaires that have been used to date to measure perceived job stressors in the ICU (Are there few or numerous measurement tools in the peer-reviewed literature?). 2) To determine the most frequently and least frequently used scales/questionnaires (Is there a consensual use of one or several tools? If so, what may explain this?). 3) To critically examine the ecological validity of these studies, i.e., their ability to take into account all stressors relevant to ICU settings; this means specific stressors as well as more general stressors such as job demands, lack of social support, etc. 4) To critically examine their basic psychometric reliability/validity as evidenced by the use of suitable methodological and statistical procedures such as factor analysis, Cronbach's alpha, etc.

Study Design
We considered all peer-reviewed full-text articles reporting an empirical study, a literature review, or a meta-analysis published in English between 1997 and 2016. The choice of article in English corresponds to the objective of reporting on the most commonly used stress scales in intensive care at an international level. In this sense, English publications, the common language of international researchers and practitioners, include the largest number of journals with high international visibility. Only the empirical studies that used a quantitative methodology to explicitly measure job/occupational stress factors for healthcare professionals in ICUs were selected. Noncompliant articles (commentaries, case reports, posters, and editorials) simply reporting perceived stress or focusing on patients instead of healthcare providers were excluded.

Search Strategy and Study Eligibility
The search for and selection of articles were conducted between March and May 2017 (according to PRISMA guidelines). Seven databases and editorial portals (Medline via Pubmed; PsycInfo and Psychology, and behavioral science collection via EBSCOhost; Elsevier's ScienceDirect, SpringerLink, Sage Journals, and Wiley Online Library) were screened using the following terms in the title, the abstract, or the keywords: (intensive OR critical) AND care AND (professional OR job OR work OR occupational) AND (stress OR stressors OR burnout). The references of 10 literature review articles (Dunser et al., 2006;Embriaco et al., 2007b;Fassier and Azoulay, 2010;Adriaenssens et al., 2015;Karanikola et al., 2015;Van Mol et al., 2015;Chuang et al., 2016;Flannery et al., 2016) were manually screened to identify other relevant studies that had not been initially retrieved.
The initial search, conducted by MG, identified 1,330 records (Figure 1), and 34 were added following a manual search in the references of the articles retrieved. Duplicates (n = 230), articles in a language other than English (n = 328), or those not conforming to the search criteria (n = 110) were removed. The abstracts were then screened for eligibility separately by two authors (MG and FL). When disagreement or the need for further analysis arose, the entire article was (n = 96) examined and discussions were held between AL, FL, MG, MM, MB, and AP to reach an agreement. From the 696 articles thus selected, 594 were excluded as they did not focus on stress in the ICU (n = 207), did not measure stressors (n = 276), did not include healthcare providers (n = 33), did not use a quantitative methodology (n = 77), or could not be retrieved (n = 1). In total, our review focused on 102 empirical studies (references in Supplementary Material).

Data Extraction and Criteria Used to Assess the Quality and the Relevance of Existing Scales in Intensive Care Unit Contexts
Following the selection of the 102 relevant articles, data were extracted and coded by AL and FL for each scale according to five criteria: (a) Identification of the scale used and the number of articles that used it. (b) The origin and metrological reliability/validity. At that level, we made a distinction between ante hoc and ad hoc scales, with subdivisions within these two broad categories of scales. This is because "previously-validated scales are generally preferable to ad hoc scales" (Furr, 2011, p. 8) given that the quality assessment of ad hoc scales rarely goes beyond face validity as perceived by the researchers (i.e., lack of independent evidence of validity from other sources such as experts or participants). Furthermore, the use of ad hoc measures reduces the comparability of studies as it introduces a possible confounding factor. Moreover, they generally lack a complete inspection of other forms of validity and reliability, as estimated by internal consistency coefficients (e.g., Cronbach's alpha), high test-retest correlation coefficients, theory-consistent and interpretable factor structure without problematic loadings, or correlations with similar measures (convergent validity) or logically related phenomena (nomological validity). In this respect, "rigorously developed measures have a lower probability of being based on chance or method variance than ad hoc measure" Peter and Churchill (1986, p. 3). Finally, ante hoc scales offer more the guarantee that they were elaborated independently from any hypothesistesting purpose (i.e., absence of hypothesis confirmation bias in the generation of items). In addition, we also consider whether ante hoc validated scales were modified-or notby ICU researchers. In fact, as pointed out by Furr (2011, p. 9) "well-validated original scales are preferable to modified scales. Because a modified scale's psychometric properties and quality might differ from those of the original scale, the modified scale is-to some degree-an ad hoc scale. As such, its psychometric properties and quality are unclear and suspect." Nevertheless, because some authors using an ad hoc scale have provided validity-related information regarding the origin of items and/or basic statistics (generally Cronbach's alpha and sometimes factor analysis results), we differentiated ad hoc scales according to the presence or absence of such information. As a result of the combination of these criteria, we differentiated five types of scales ranging from the probably more reliable/valid to the less reliable/valid: ante hoc validated scales (Type 1); ante hoc validated scales with ad hoc modifications (items removed or rewritten) (Type 2); ad hoc scales with validity-related information regarding the origin of items and basic statistics (Type 3); ad hoc scales with validity-related information regarding the origin of items only or report of basic statistics only (Type 4); ad hoc scales without report of any validityrelated information (Type 5). (c) The target population and the level of generality or specificity according to three levels: (1) scales designed to measure stressors in all work settings (i.e., all settings scales); (2) scales specific to healthcare providers from several types of clinical units/settings (i.e., healthcare settings scales); (3) scales tailored to measure stressors specific to ICU settings. (d) The number of items and, when relevant and available, their subscales. (e) The types of stressors measured. An in-depth assessment of the subscales and items enabled us to develop a typology of all stressors that had been measured by the different scales. FL, AL, and MG examined in detail each scale with regard to its structure (presence and labels of subscales) and item content and subsequently made iterative comparisons between them to identify the broad types and subtypes of stressors measured when considering all scales together. The choice of an inductive methodology to define the typology of stressors made it possible to allow oneself not to be limited to a typology preconceived by a theoretical model, which would risk not covering all the stressors explored in the literature. This typology was then critically examined by AL, GC, and GB and revised subsequently to reach a consensus. Then, each scale was assessed using a binary format "yes/no presence" for each type and subtype of stressors. This criterion was especially considered to assess the ecological validity of scales. Because ICU professionals can be jointly exposed to widely common stressors (e.g., workload, lack of support from colleagues), stressors shared with many other health professions (e.g., lack of recognition from patients and families, administrative hassles), and stressors more typical of ICUs (e.g., end-of-life decisions, constant monitoring of critically ill patients), we consider that the more a scale covers exhaustively the different types and subtypes of stressors, the more it was considered as ecologically valid.

RESULTS
What Types of Scales Are Used in the Intensive Care Unit?
From the 102 studies analyzed in our literature review, we identified 59 different scales according to three main categories ( Table 1): 17 "generic scales" used in all sectors of professional activities, 20 "healthcare scales" used in the field of human health, and 22 scales specific to ICU settings. Only 28 out of the 59 scales were Type 1 scales (ante hoc validated). The majority of them were generic scales (13/17) or healthcare settings scales (13/20); there were only two ICU-specific scales (2/22). Overall, among the 102 studies seeking to identify stressors in the ICU, 36 used a more or less problematic scale in terms of validity (Types 2-5).

Types of Stressors Assessed in Intensive Care Unit Studies
Eight major types of stressors, grouping 58 subtypes, have been identified (see Table 2; for the coding of each scale, see Table 1): (1) High job demands (40 scales); (2) Problematic relationships with other professionals (39 scales); (3) Lack of control over work situations and career (31 scales); (4) Lack of organizational resources (29 scales); (5) Problematic situations with users and relatives (20 scales); (6) Dealing with ethical and moral-related situations (19 scales); (7) Risk management issues (14 scales); (8) Disadvantages in comparison to other occupational situations (11 scales). Two types of stressors do not fall within these eight main categories and have been classified as "other" (measured by three different scales).
The High job demands category refers to taxing task-, workflow-, and role-related situations and distinguishes between eight subtypes. The problematic relationships with Frontiers in Psychology | www.frontiersin.org Frontiers in Psychology | www.frontiersin.org other professionals category includes 11 subtypes of stressors related to the lack of support, communication/collaboration problems, or conflicts with colleagues, superiors, or other departments. The lack of control over work situations and career category includes the different control and skill-related factors which may prevent professionals from being responsive, efficient, and committed. This category of stressors includes eight subtypes. The lack of organizational resources category includes nine subtypes of resources the healthcare organization might fail to sufficiently provide to promote the quality of life and the quality of care of ICU professionals. The problematic situations with service users and relatives category includes six subtypes of stressors which all involve one or more people whose behavior or health condition is likely to exceed the resources of healthcare professionals. The dealing with ethical-and moral-related situations category refers to eight subtypes of situations where patients' well-being and safety are likely to be harmed by unsuitable decisions/actions of the ICU team. The risk management issues category refers to risky situations for both patients and professionals with a substantial probability of making critical errors (three subtypes included). Lastly, the disadvantages in comparison to other occupational situations category of stressors concerns the extent to which the professional considers his/her current occupational situation is disadvantaged in comparison to other career choices (e.g., other jobs, other employers, other professions) (three measured subtypes).

Ability of Scales to Take Into Account the Specificity of Intensive Care Unit Stressors
There are important differences concerning the extent to which the three types of scales (i.e., all settings, healthcare settings, and ICU settings scales) address the broad range of the types and subtypes of stressors mentioned above. To highlight these differences, we compared the proportion of each type of scale that measured a given stressor relative to the overall number of scales that measured it. However, given that there were more ICU settings scales (N = 22) than healthcare settings scales (N = 20) and all settings scales (N = 17), we applied a correction to make the comparisons valid 1 . As illustrated by Figure 2, high job demands (52%), disadvantages in comparison to other occupational situations 1 The formula was: (N scales of a given type that measure the stressor/Overall N scales that measure the stressor)/(N all settings scales that measure the stressor/17) + (N healthcare settings scales that measure the stressor/20) + (N ICU settings scales that measure the stressor/22). For instance, there were nine scales that measured the "taxing work environment (noisy, hectic, crowded, heated, etc.)" stressor type, including three all settings scales, three healthcare settings scales, and three ICU settings scales. Without the correction, this stressor appeared to be measured to the same extent by the three types of scales (3/9). However, this conclusion is incorrect because the ratio 3/17 (17.65%) for all settings scales is higher than the ratio of 3/20 (15%) for healthcare settings scales and higher than the ratio of 3/22 (13.63%) for ICU settings scales. After the correction which took into account the unequal number of scale types (17 vs. 20 vs. 22), it emerged that this stressor was measured slightly more frequently by all settings scales [(3/17) (17, 20, and 22) to allow a valid comparison of row percentages. The formula was (N scales of the target type that measured the stressor/Overall N scales that measured the stressor)/((N all settings scales that measured the stressor/17) + (N healthcare settings scales that measured the stressor/20) + (N ICU settings scales that measured the stressor/22)).
FIGURE 2 | Comparison of the three types of scales regarding their propensity to measure the eight broad types of stressors (% is the corrected proportion of scales of a type covering at least one stressor in the target category).
(55%), and lack of control over work situations and career (53%) were more frequently measured by the all settings scales than by the other two scales (ranging from 15 to 32%). However, all settings scales were rarely used to measure risk management issues (24%) and problematic situations with service users and relatives (18%) and were hardly used to measure dealing with ethical-and moral-related situations (3%); conversely, ICU settings scales were used more frequently to measure these broad types of stressors (51, 45, and 72%, respectively). Healthcare settings scales were used to measure problematic situations with service users and relatives (37%) almost as often as ICU settings scales, but less frequently to measure risk management issues (25%) and dealing with ethical-and moral-related situations (26%). The three types of scales measured lack of organizational resources (30-38%) and problematic relationships with other professionals (31-37%) to the same extent. Beyond these eight broad types of stressors, there were a number of important differences concerning their subtypes. For instance, some stressors were more frequently measured by ICU scales than by the other two scales: decisional dilemmas/uncertainty regarding patients' survival and end-oflife care (100%), unsuitability of care (futility or over/under aggressiveness of therapeutics) (86%), conflict and inappropriate expectations or behavior from relatives (82%), or constant alert and sudden emergencies due to the patient's condition (65%).
In addition, in terms of item content, ICU settings scales and, to a lesser extent, healthcare settings scales generally refer more to precise and tailored features of the working environment of ICU professionals than all settings scales.
An analysis of the 59 scales reveals that no scale covered all the main stressors in ICUs (Table 2), not even those with satisfactory metrological properties. For instance, the JCQ, the most used across the world, is a Type 1 scale (ante hoc with satisfying validity-related statistics) which has primarily been used to measure the following types of stressors: lack of control over work situations and career, high job demands, and problematic relationships with other professionals. However, this scale does not cover stressors such as dealing with ethical-and moral-related situations, problematic situations with service users and relatives, and risk management issues and measures only one stressor in the broad category of the lack of organizational resources (note: the full 49-item JCQ also measures Lack of job security, a stressor associated with the disadvantages in comparison with other occupational situations category, but this version was not used in the retrieved ICU studies). Furthermore, four of the 17 studies retrieved combined it with a healthcare settings scale [NSS, NWI, Nine Equivalent of nursing Manpower use (NEMS)] or an ICU settings scale such as the "Perceived ethical environment questionnaire" and the "Perceived inappropriateness of care questionnaire" developed by Piers et al. (2011).
The second most frequently used scale in ICU studies-the MDS-also presents certain limitations regarding its coverage of the different types of stressors. It is a Type 1 scale, but it is tailored to measure stressors relevant in many healthcare settings which focus primary on stressors related to dealing with ethicaland moral-related situations. It also measures stressors relating to the lack of control over work situations and career, as well as one stressor relating to high job demands. However, this scale does not measure stressors relating to the following categories: problematic relationships with other professionals, problematic situations with service users and relatives, lack of organizational resources, risk management issues, and disadvantages in comparison to other occupational situations. This led authors of two out of the 16 studies which used the MDS to combine it with either a healthcare settings scale (Varjus et al., 2003) or an ad hoc ICUspecific questionnaire (Hamric and Blackhall, 2007) to measure additional stressors.
Similar observations can be made with regard to the third most frequently used scale, i.e., the NSS. This Type 1 scale specifically targets nurses (i.e., it is unsuitable for other healthcare professions). It primarily covers stressors from four broad types: high job demands, lack of organizational resources, problematic relationships with other professionals, and problematic situations with service users and relatives. However, it measures one stressor from the lack of control over work situations and career category to a limited extent and none from the categories dealing with ethical-and moral-related situations, risk management issues, and disadvantages in comparison with other occupational situations.
The same comments were made for all other scales. No scale covered the eight broad types of stressors in a comprehensive manner, and many of them were problematic in terms of validity/reliability. It is worth mentioning that 14 articles analyzed combined two or three scales to increase stressor coverage.

DISCUSSION
From the 102 studies analyzed, we identified 59 different scales. Only 28 out of the 59 scales were validated (Type 1), and two ICU settings scales out of 22 were validated. Our review of the literature highlights the wide variability across scales used to identify stressors in the ICU with regard to their level of generality/specificity (scales for all types of professional contexts or scales targeting healthcare contexts or more specifically ICU professionals), their psychometric qualities (five levels of validity/reliability), and the type of stressors covered by scales.
This variability sheds light on the constraints that appear to guide investigators' methodological choices. The advantage of using a generic scale validated internationally is that authors are able to carry out epidemiological studies allowing interprofessional and international comparisons (El Khamali et al., 2018). However, generic scales, primarily used in studies, appear ill adapted to measure the stressors more specific to the professional activity in ICUs. Indeed, the stressors in the categories Dealing with ethical-and moral-related situations and Risk management issues, represented primarily in specific scales, did not appear or were relatively few within the generic and healthcare scales.
For instance, the scale most commonly used by the authors was the JCQ scale, despite the fact that no factor related to dealing with ethical-and moral-related situations or risk management issues was covered. Many authors, however, raised the question of the importance of the issues related to risk management and patient safety in relation to high-tech care and the severity of the pathology (Aiken and Patrician, 2000;Adriaenssens et al., 2015). The difficulties associated with ethical dimensions have also been extensively studied in the ICU context in relation to end-of-life situations (Laurent et al., 2017). The SAQ-ICU scale combines interesting criteria to identify stress factors in ICU settings. The scale has been validated and covers factors relative to the problematic relationships with other professionals and lack of organizational resources categories. However, factors relating to stressors such as job disadvantages, problematic situations with users and relatives, high job demands, and risk management issues are absent, despite the fact that the latter two factors of stress have been widely reported by ICU professionals (Pastores et al., 2019).
Thus, investigators who chose to use a scale reflecting these more specific stressors of ICUs were forced to develop their own tools whose psychometric qualities were yet to be tried and tested (tools based on literature reviews, interviews with professionals, or items from different scales). Beyond the measurement level, these observations highlight the crucial limitation of the uncritical use of general theories of occupational stress (e.g., the job strain model of Karasek, the effort-reward imbalance model of Siegrist) to analyze job stressors in ICUs. Indeed, as advocated by a number of other scholars (Borteyrou et al., 2014), these theories/models of stress must be contextualized to enhance their ecological validity as they only account for generic stressors in professional settings and not the specific ones.
Our systematic review shows that a number of factors are absent from all the scales, for instance, diagnostic/admission decisions, the training and supervision of students, the lack of space related to rooms for break or family discussions (Blanch et al., 2016;Trevick et al., 2016;Pastores et al., 2019). We noted that the healthcare and ICU scales targeted either all healthcare professionals or nurses and that there was no scale developed specifically for physicians.
It is worth nothing that some scales allow the assessment of the effects on outcomes (e.g., job burnout) of the interaction between some types of stressors. This is the case of the JCQ and its variants which can estimate whether resources like job control (decision latitude) and social support (from colleagues and supervisors) moderate the effect of job demands. This is also the case of the ERI scale, which is based on the postulate that the imbalance (interaction) between the rewards obtained in exchange of job efforts is critical to explain outcomes, more than their isolated main effects. However, accounting for these interaction effects generally did not increase the explained variance of outcomes in empirical studies (e.g., Brough and Biggs, 2015;Gorgievski et al., 2019).
This study has a number of limitations. Several studies, and therefore stress scales, may not have been considered in our review if the objective of identifying ICU stressors was not clearly reflected in the article title, keywords, or abstract. In addition, we have focused mainly on studies published in English. Thus, scales developed at a more local level may have been excluded from our review, and a more extensive search seems warranted to cover more the important issue of international, intercultural, and health-system comparisons in terms of stressor identification and measurement.

CONCLUSION
Our review of the literature clearly raises the question of the relevance of the scales used in studies measuring stressors in intensive care settings. Indeed, no available tool meets both the criteria of metrological validity and of ecological validity (i.e., covering all relevant stressors in ICUs, particularly those that are the most specific to them). Thus, researchers and practicians currently face a methodological dilemma, as they are forced to make a choice between the two or to make some unfortunate "bricolage, " such as ad hoc elaboration, removal, or modification of items or the combination of different scales.
There is an urgent need to propose a validated tool capable of taking into account the whole professional reality of ICU settings. This tool would make it possible to compare the respective impacts of generic vs. specific stressors in the etiology of outcomes (e.g., burnout, job satisfaction, turnover intentions). A better identification of stress factors should make it possible to define a more appropriate care policy in particular to prevent burnout and its associated effects such as depression, suicidal tendencies, addictive behaviors, and physical impairment (e.g., Lheureux et al., 2016;Vandevala et al., 2017). Finally, the more comprehensive and thus more able to approximate the reality of the activities undertaken by professionals the tool is, the easier it will be to use the factors identified to implement effective training needs and target the necessary prevention and support measures.

DATA AVAILABILITY STATEMENT
All datasets generated for this study are included in the article/Supplementary Material.

AUTHOR CONTRIBUTIONS
AL: abstracts screened for eligibility, screened full text sectioned, examination in detail of each scale, examination of typology, and writing of the article. FL: abstracts screened for eligibility, examination in detail of each scale, and writing of the article. MG: initial search and examination in detail of each scale. MM: screened full text sectioned and proofreading. MB: screened full text sectioned and proofreading. AP: screened full text sectioned and proofreading. GB: examination of typology and proofreading. GC: examination of typology and proofreading.