ORIGINAL RESEARCH article
Sec. Quantitative Psychology and Measurement
Volume 11 - 2020 | https://doi.org/10.3389/fpsyg.2020.00537
Factor Structure of the 10-Item Perceived Stress Scale and Measurement Invariance Across Genders Among Chinese Adolescents
- 1The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, China
- 2School of Sociology and Psychology, Southwest Minzu University, Chengdu, China
- 3College of Teacher Education, Dali University, Dali, China
- 4Huaxi MR Research Center, Department of Radiology, West China Hospital of Sichuan University, Chengdu, China
Background: Chinese adolescents encounter a lot of stressors, such as academic burden and parental pressure. However, little is known about their perception of stress. The 10-item Perceived Stress Scale (PSS-10) is a widely used instrument to measure individuals’ appraisal of global stress in academic research and clinical practice. The current study aimed to evaluate the best-fit factor structure model of the PSS-10 and the measurement invariance across genders in Chinese adolescents.
Methods: A total of 1,574 Chinese senior high school students completed the PSS-10 (mean age = 15.26 ± 0.56 years, female = 54%). Confirmatory factor analysis (CFA) was conducted to determine the factor structure of the PSS-10. Multigroup CFA was carried out to test the measurement invariance of the PSS-10 across genders. A subsample (N = 1,060) answered additional questionnaires measuring stressful life events, anxiety, and depression to examine the convergent and concurrent validity of the PSS-10.
Results: The two-factor model was supported [i.e., χ2 (34) = 332.224, p < 0.001; non-normal fit index (NNFI) = 0.901, comparative fit index (CFI) = 0.925, root mean square error of approximation (RMSEA) = 0.075, standardized root mean square residual (SRMR) = 0.051]. Importantly, the model exhibited strong measurement invariance across female and male groups. Furthermore, the PSS-10 had adequate convergent validity for stressful life events (number: r = 0.13, p < 0.001; impact: r = 0.23, p < 0.001) and could explain incremental variance in predicting anxiety (ΔR2 = 0.13, β = 0.38, p < 0.001) and depression (ΔR2 = 0.16, β = 0.41, p < 0.001), suggesting excellent concurrent validity.
Conclusion: A two-factor model best fits the structure of PSS-10 among Chinese adolescents, with strong measurement invariance between gender groups, demonstrating its validity for assessing perceived stress among Chinese adolescents.
It has been argued that adolescence is a transition period in one’s life in which one undergoes biological, social, and psychological changes (McNamara, 2000; Tsai et al., 2015a, b, 2018; Lin and Tsai, 2016; Strong et al., 2016; Lee et al., 2017; Foulkes and Blakemore, 2018; Blakemore, 2019). Many psychological disorders, such as anxiety and depression, begin at adolescence and persist into adulthood (Blakemore, 2019). One of the most important reasons for the increased vulnerability to psychological health problems during this period is that adolescents encounter increasingly more stressors (i.e., academic stress, peer pressure, and parental pressure) (Kouzma and Kennedy, 2002, 2004; Deb et al., 2015; Seiffge-Krenke, 2019), and many adolescents may not have well-developed coping strategies because of limited life experiences (Moeini et al., 2008). When there is an imbalance between environmental demands and adolescents’ coping capacity, people will perceive stress (Cohen et al., 1997), which may be linked not only to unhealthy behaviors such as cigarette smoking and alcohol use (Wills and Shiffman, 1985; Cohen and Lichtenstein, 1990) but also to psychological health problems, including anxiety and depression (Lupien et al., 2009). Specifically, it has been suggested that adolescents under East Asian culture (e.g., China) are more likely to show higher depression and lower life satisfaction than the European counterparts due to the emphasis on academic performance (Stankov, 2013). The academic stress among Chinese adolescents comes from a heavy workload and fierce competition for university entrance (Cai et al., 2010) and parents’ high expectations on their academic achievement (Woo et al., 2004; Chan et al., 2017), which undermines their psychological health (Tsai et al., 2013). Hence, it is essential to evaluate Chinese adolescents’ perceived stress to be able to relate this measure to their health. Given a lack of research about their subjective perception of stress, the present study sought to evaluate the 10-item version of the Perceived Stress Scale (PSS-10; Cohen, 1988), one of the most widely used measure of global perceived stress, in a large sample of Chinese senior high school students (N > 1,000), in order to confirm whether this scale could be utilized in Chinese adolescents.
The PSS was originally developed as a 14-item scale (PSS-14) that assesses how respondents found their lives unpredictable, uncontrollable, and overloaded, which was designed for use in community samples with at least a junior high education (Cohen et al., 1983). Since the PSS items focus on the general nature of feelings and thoughts about stress rather than specific events or experiences, this scale is suggested to measure the global level of stress in any population (Cohen et al., 1983, 1994). Specifically, many studies have utilized the PSS to measure self-reported stress in various adolescent samples such as middle school students (e.g., Yosipovitch et al., 2007; Edwards et al., 2014) and clinical adolescents (e.g., Martin et al., 1995; Siqueira et al., 2000). While the PSS-14 shows adequate validity and reliability, the shorter 10-item version (PSS-10) was reported to show superior psychometric properties and has been recommended for use in future research (Cohen, 1988; for a review, see Lee, 2012). The PSS-10 was derived from the 10 items with high factor loadings from the original PSS-14 based on a sample of 2,387 US residents (Cohen, 1988). Since its development, the PSS-10 has been translated into over 25 languages and has been validated all over the world (Cohen, 2014). The PSS-10 possesses adequate internal consistency with Cronbach’s alpha coefficients ranging from 0.67 to 0.91 (e.g., Roberti et al., 2006; Siqueira Reis et al., 2010; Ng, 2013; Taylor, 2015; Denovan et al., 2019; Kaya et al., 2019), moderate convergent validity with stressful life events (e.g., Mitchell et al., 2008), and good concurrent validity with mental health problems such as depression and anxiety (e.g., Örücü and Demir, 2009; Perera et al., 2017; Baik et al., 2019). In particular, the PSS has been translated in Chinese and shown acceptable reliability and validity in adult samples such as postnatally distressed women (Chen et al., 2000), urban residents (Yang and Huang, 2003), working adults (Chu and Kao, 2005), police women (Wang et al., 2011), cardiac patients (Leung et al., 2010), elderly service workers (Ng, 2013), and university students (Lu et al., 2017). However, validations of the PSS-10 only focused on adult samples in previous studies, and no study has evaluated the PSS-10 in adolescents. To the best of our knowledge, the present study is the first to validate the PSS-10 in Chinese adolescents.
The factorial structure of the PSS-10 remains controversial. Although studies have consistently identified a two-factor structure of the PSS-10 with factor distinction corresponding to item directionality (i.e., negatively phrased items versus positively phrased items) (e.g., Cohen, 1988; Golden-Kreutz et al., 2004; Roberti et al., 2006; Siqueira Reis et al., 2010; Wang et al., 2011; Barbosa-Leiker et al., 2013; Taylor, 2015; Lu et al., 2017; Baik et al., 2019), researchers hold different opinions about the understanding of the distinction between the two factors. Cohen (1988) originally posited that the distinction only related to statement directionality (negative versus positive) and was irrelevant, thus the PSS-10 assessed a single construct. However, other researchers have argued that the distinction reflects separate components of the stress experience, thus measuring two factors. Specifically, negatively phrased items were labeled as “Perceived Helplessness” or “Perceived Distress,” and positively phrased items were labeled as “Perceived Self-Efficacy” or “Perceived Coping” (Golden-Kreutz et al., 2004; Roberti et al., 2006; Barbosa-Leiker et al., 2013). Another important issue concerns whether the factor structure underlying the PSS-10 is equivalent across gender groups. Previous studies have indicated that women scored higher on both the PSS-10 total scores and subscale scores than men (Leung et al., 2010; Andreou et al., 2011; Lesage et al., 2012; Barbosa-Leiker et al., 2013). However, some believe that these gender differences are due to measurement bias, that is, the idea that women are more inclined to score higher on negatively phrased items (Gitchel et al., 2011), instead of true gender differences (Lavoie and Douglas, 2012). To rule out this possibility, an increasing number of studies have tested the measurement invariance of the PSS-10 across genders and have found that the scale does not exhibit measurement bias against males and females (e.g., Barbosa-Leiker et al., 2013; Taylor, 2015; Reis et al., 2017; Denovan et al., 2019). However, whether this measurement invariance in the PSS-10 holds for Chinese adolescent is still unknown. Therefore, the goal of this study was to evaluate the factor structure of the PSS-10 and examine whether the measurement structure is equivalent across gender groups in Chinese adolescent students.
To address these questions, we first carried out confirmatory factor analysis (CFA) to test the factor structure of the PSS-10. Second, to examine whether the PSS-10 measures the same latent structure in females and males, we conducted a multigroup CFA to test measurement invariance (i.e., configural, metric, and scalar invariance) across gender groups. In addition, we validated its internal consistency reliability and validity in the current sample. Specifically, the convergent validity of the PSS-10 was measured for stressful life events [Adolescent Self-Rating Life Events Checklist (ASLEC)] because perceived stress may increase as the result of cumulative stressful life events, and the impact of such events may reflect stressor appraisal measured by the PSS-10 (Cohen et al., 1983). Additionally, concurrent validity was examined by testing whether PSS-10 scores could predict anxiety [Screen for Child Anxiety Related Emotional Disorders (SCARED)] and depression [Depression Self-Rating Scale for Children (DSRSC)] beyond stressful life events, age, and gender, given that the perception of stress may be an early symptom of anxiety and depression (Cohen et al., 1983).
Materials and Methods
Participants and Procedure
A total of 1,614 Chinese senior high school students in 10th grade participated in this study. Forty participants (2.48%) were excluded from analyses because values for the PSS-10 were missing completely at random [χ2 (112) = 103.323, p = 0.709] (Little, 1988; Schlomer et al., 2010; Kang, 2013). Thus, 1,574 participants were included in the analyses (856 females, 718 males; age = 13–17 years, mean age = 15.26 ± 0.56 years). The average age for the male group (mean age = 15.34 ± 0.59 years) was higher than that for the female group (mean age = 15.20 ± 0.52 years) [t(1,572) = 5.02, p < 0.001]. These participants were part of a larger longitudinal project investigating the behavioral and neural mechanisms underlying adolescents’ mental health, academic achievement, and social cognition in Chengdu, China (Wang et al., 2017, 2018; Li et al., 2018a, b). All participants were from three general local public high schools and were recruited via a class presentation to introduce the project by researchers from West China Hospital of Sichuan University and Chengdu Education Bureau. Because the participants completed a battery of surveys across several waves in the project, only part of the data including PSS measurement was included in the present study. In addition to the PSS, several other questionnaires were administered to measure stressful life events (ASLEC), anxiety (SCARED), and depression (DSRSC). Of all 1,574 participants, 1,060 participants (601 females, 459 males; mean age = 15.26 ± 0.54 years) completed all the measures. All tests were written in simplified Chinese in a pencil-and-paper format, and each participant completed the tests in his/her classroom. To make the participants feel confident about the confidentiality of their responses, two research assistants supervised the test sessions, and teachers were not present. The current study was approved by the local research ethics committee of the West China Hospital of Sichuan University. Written informed consent was obtained from all participants and their guardians prior to testing.
The PSS-10 is a self-reported scale to measure the global level of perceived stress (Cohen, 1988). This scale includes two factors (Golden-Kreutz et al., 2004; Roberti et al., 2006; Örücü and Demir, 2009; Barbosa-Leiker et al., 2013): Factor 1 (Perceived Helplessness) is made of negatively phrased items (i.e., items 1, 2, 3, 6, 9, and 10; e.g., “In the last month, how often have you felt nervous and stressed”); and Factor 2 (Perceived Self-Efficacy) is made of positively phrased items (i.e., items 4, 5, 7, and 8; e.g., “In the last month, how often have you felt that things were going your way”). The items of the PSS-10 used in this study were selected from the Chinese version of the PSS-14 that was translated by Chu and Kao (2005) and has shown high internal consistency and associations with negative mental health such as anxiety and depression in Chinese adults (Chu and Kao, 2005; Chu, 2010). Furthermore, words and expressions were slightly modified to suit our sample. Particularly, to ensure the accuracy and readability of each item, five 10th-grade students were first asked to answer the revised scale and then interviewed to indicate the readability and suitability of each item. The interviews were conducted by a group including the first author, the last author, and a class teacher. The group had a discussion after the interviews and reached a consensus that each item is readable and suitable for high school students. Participants were required to rate how often they felt a certain way over the past month on a five-point Likert scale (1 = never, 2 = rarely, 3 = sometimes, 4 = often, and 5 = always). Scores for positively phrased items were reversed to obtain the total score and subscale score for Factor 2 (Perceived Self-Efficacy). The total score of the PSS-10 ranged from 10 to 50, and a higher score indicated a higher level of perceived stress. Previous studies have suggested that the PSS-10 shows adequate internal consistency, test–retest reliability, and validity across different populations (for a review, see Lee, 2012).
Adolescent Self-Rating Life Events Checklist
To evaluate the convergent validity of the PSS-10, we incorporated the ASLEC to measure stressful life events experienced over the past 12 months (Liu et al., 1997). This measure includes 27 stressful life events from multiple domains: family pressure, academic pressure, interpersonal conflicts, physical health problems, and others. Participants were first instructed to answer “yes” or “no” to the question asking whether the given event had ever happened to them. Then, they were asked to rate the impact of each experienced event on a five-point Likert-type scale ranging from 1 (not at all) to 5 (extremely severe). The number of stressful life events for each participant was obtained by counting all experienced events. The impact of stressful life events was obtained by summing scores for all events. Of note, the impact of unexperienced events was coded as 1 (not at all). The ASLEC has been reported to have satisfactory validity and reliability in previous studies (Liu et al., 1997) and has been widely used to assess the number and impact of stressful life events in Chinese adolescents (e.g., Liu et al., 2000; Liu and Tein, 2005; Zheng et al., 2012). In the current study, the Cronbach’s alpha coefficient for the ASLEC was 0.92, indicating excellent internal reliability.
Screen for Child Anxiety Related Emotional Disorders
To examine the concurrent validity of the PSS, we employed the SCARED (Birmaher et al., 1997; Muris et al., 1998), which is a self-reported instrument measuring symptoms of Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV)-linked anxiety disorders in children 9–18 years old. The SCARED contains 41 items across five factors: somatic/panic, general anxiety, separation anxiety, social phobia, and school phobia (Birmaher et al., 1999). Participants were instructed to rate how frequently they have experienced each symptom in the last 3 months using a three-point scale, in which 0 represents “never,” 1 represents “sometimes,” and 2 represents “often.” The SCARED total score and subscale scores were computed by summing up the corresponding items, and a higher score indicated a higher level of anxiety. The Chinese version of the SCARED has proven to have good psychometric properties (Wang et al., 2002). In the current study, the overall Cronbach’s alpha coefficient of the SCARED was 0.94. For the five factors, the coefficients were 0.89, 0.84, 0.74, 0.84, and 0.68, respectively, indicating adequate internal reliability.
Depression Self-Rating Scale for Children
To examine the role of the PSS score as a predictor of depression, the DSRSC (Birleson et al., 1987) was used. This unidimensional instrument has 18 items and covers the major areas of mood disturbance. Participants were asked to rate how frequently they had experienced certain feelings in the previous week, ranging from 0 (never) to 2 (often). The total score of the DSRSC was obtained by summing the ratings of all items, with a higher total score indicating a higher level of depression. The Chinese version of the DSRSC has demonstrated satisfactory reliability and validity and is applicable to assess depressive symptoms in Chinese children and adolescents (Su et al., 2003; Yin et al., 2012). The Cronbach’s alpha coefficient for the DSRSC in this study was 0.81, indicating satisfactory internal reliability.
First, descriptive statistics of each PSS-10 item [i.e., mean, standard deviation (SD), skewness, kurtosis, and discrimination index (DI)] were calculated by SPSS (IBM, SPSS version 20, 2011). The absolute skewness and kurtosis values smaller than 1 indicate a normal distribution of the scores of items (Marcoulides and Hershberger, 2014). The DI is a measure of how well an item is able to distinguish between respondents with high scores and low scores. This measure is generally computed by subtracting the mean score of participants in the lower group (27%) from the mean score of those in the upper group (27%) and dividing the result by the maximum possible discrimination score (Kelley et al., 2002). DI values of 0.40 and above are regarded as being very good items, and those of 0.20 or less are regarded as being poor items (Ebel, 1954; Hopkins et al., 1990; Hu and Xie, 2012).
Second, we carried out a CFA to test whether the two-factor model can be replicated in the current sample using Amos 22.0 with maximum likelihood estimation method. We used multiple indices to evaluate the model’s goodness of fit as recommended by Kline (2005) and Byrne (2001): non-normal fit index (NNFI), comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). An acceptable model fit is indicated by an NNFI of >0.90, a CFI of > 0.90, an RMSEA of < 0.08, and an SRMR of <0.08 (Hu and Bentler, 1999; Byrne, 2001; Kline, 2005). Notably, to further determine the superiority of the two-factor model, we also evaluated a one-factor model and compared it with the two-factor model by using a chi-squared difference test (Duckworth and Quinn, 2009; Li et al., 2018b).
Third, measurement invariance concerns whether items and structural factors mean the same thing to different groups (Cheung and Rensvold, 2002). To evaluate measurement invariance across genders, we tested the configural, metric, and scalar invariance using a multigroup CFA in Amos 22.0. These tests are hierarchically nested by increasing levels of cross-group equality constraints (Gregorich, 2006; Kuhn and Holling, 2009). Configural invariance tested whether the factor structure of the PSS-10 is the same across gender groups. Metric invariance tested whether the intercorrelations of the PSS-10 items were equivalent across groups by constraining the factor loadings to be equivalent. Scalar invariance tested if the group mean differences were due to differences in the construct by fixing the intercepts to be equal across groups. Having change values of CFI and RMSEA (ΔCFI and ΔRMSEA) smaller than 0.01 is recommended as an appropriate criterion indicating measurement invariance (Cheung and Rensvold, 2002; Chen, 2007).
Additionally, we calculated the Cronbach’s alpha coefficients to evaluate the internal reliability of the PSS-10. We then evaluated the convergent validity of the PSS-10 by examining Pearson correlation coefficients between the total and subscale scores of the PSS-10 and the ASLEC. Finally, the concurrent validity of the PSS-10 was verified by testing its role in predicting anxiety (SCARED) and depression (DSRSC) based on hierarchical regression analyses. We only used the total score of the PSS-10 to index perceived stress in regression analyses to simplify analysis, given that the two subscale scores showed similar results. These analyses were conducted with SPSS (IBM, SPSS Version 20, 2011).
Table 1 presents the descriptive statistics for each item. As shown, the mean values ranged from 2.59 to 3.10, and the SD values ranged from 0.78 to 0.93. The scores of all items were normally distributed (skewness: −0.32∼0.51; kurtosis: −0.44∼0.52). In addition, all PSS-10 items had good discrimination, with DI values ranging from 0.46 to 0.57.
Confirmatory Factor Analysis
Confirmatory factor analysis was used to determine the factor structure of the PSS-10. All fit indices suggested a good data fit for the two-factor model [i.e., χ2 (34) = 332.224, p < 0.001; NNFI = 0.901, CFI = 0.925, RMSEA = 0.075, SRMR = 0.051] but not for the one-factor model [i.e., χ2 (35) = 1,056.412, p < 0.001; NNFI = 0.670, CFI = 0.743, RMSEA = 0.136, SRMR = 0.101]. By comparing these two models, we found that the two-factor model demonstrated a better fit than the one-factor model, as indicated by a significant chi-squared difference [Δχ2 (1) = 724.118, p < 0.001]. The factor loadings ranged from 0.40 to 0.75 for negatively phrased items (Factor 1: Perceived Helplessness) and from 0.57 to 0.67 for positively phrased items (Factor 2: Perceived Self-Efficacy), and all loadings were significant at the 0.001 level (Figure 1). In summary, CFA supported the two-factor model of the PSS-10.
Figure 1. Standardized factor loadings for the two-factor model of the 10-item Perceived Stress Scale (PSS-10) (N = 1,574).
Measurement Invariance Across Gender Groups
Measurement invariance tests were subsequently conducted using a multigroup CFA to determine whether the PSS-10 can represent the two-factor structure across gender groups.
First, configural invariance was performed by analyzing the two gender groups without constraining equality across the groups. The result showed that the configural invariance model (M1) fitted the data very well (CFI = 0.919, NNFI = 0.904, RMSEA = 0.050) (Table 2), and all factor loadings were significant (p < 0.005). These results indicated that the two-factor structural patterns are similar across gender groups.
Next, a metric invariance model (M2) was tested by constraining the factor loadings to be equal across groups. The results showed a good fit (CFI = 0.916, NNFI = 0.904, RMSEA = 0.052) (Table 2). Although Δχ2 (p < 0.01) was significant when compared to M1, the change value of CFI (ΔCFI = 0.003) and an RMSEA (ΔRMSEA = 0.002) smaller than 0.01 provide evidence to support the metric invariances.
Third, we conducted a scalar invariance test by restricting the item intercepts to be invariant across gender groups. As shown in Table 2, the model fit indices of the scalar invariance model (M3) are adequate (CFI = 0.908, NNFI = 0.907, RMSEA = 0.051). Compared to M2, although Δχ2 (p < 0.001) was significant, no significant changes occurred in the CFI (ΔCFI = 0.008) or RMSEA (ΔRMSEA = 0.001) based on the cutoff, which was 0.01 for ΔCFI and ΔRMSEA, suggesting that the means of the two constructs were equivalent across gender groups.
Taken together, the results of measurement invariance indicated that the structural factors of the PSS-10 were equivalent across gender groups.
The Cronbach’s alpha coefficients for the PSS-10 and the two subscales (Perceived Helplessness and Perceived Self-Efficacy) were 0.79, 0.80, and 0.71, respectively.
Convergent and Concurrent Validity
We examined the relationship between the PSS-10 and the ASLEC and found that the total PSS-10 score was significantly related to both the number (r = 0.13, p < 0.001) and the impact (r = 0.23, p < 0.001) of the ASLEC. We also tested the correlations between the two subscales of the PSS-10 and the ASLEC, and the results showed that Factor 1 (Perceived Helplessness) was significantly related to both the number (r = 0.12, p < 0.001) and the impact (r = 0.21, p < 0.001) of the ASLEC, whereas Factor 2 (Perceived Self-Efficacy) was only associated with the impact (r = 0.11, p < 0.001) but not the number (r = 0.05, p = 0.10) of the ASLEC. Importantly, for the total and subscale scores of the PSS-10, the associations with the impact of the ASLEC were all significantly higher than those with the number of the ASLEC (total score: Steiger’s Z = 4.89, p < 0.001; Factor 1: Steiger’s Z = 4.38, p < 0.001; Factor 2: Steiger’s Z = 2.89, p < 0.01). These results suggest that perceived stress was more highly correlated with conceptually similar constructs (i.e., subjective feeling of the impact of stressful life events) than with objective concepts (i.e., the number of stressful life events), indicating good convergent validity.
To verify the concurrent validity of the PSS-10, correlation analyses were first conducted to test the relationship between perceived stress and anxiety and depression. Scores of the PSS-10 and its two subscales were correlated with scores of the SCARED and its five subscales, as well as the DSRSC (Table 3). Next, hierarchical regression analyses were conducted to examine the incremental validity of perceived stress on anxiety and depression, after variance related to other variables has been explained. The results revealed that perceived stress explained an additional 13% of the variance when predicting anxiety (ΔR2 = 0.13, β = 0.38, p < 0.001) and an additional 16% of the variance when predicting depression (ΔR2 = 0.16, β = 0.41, p < 0.001) beyond the variance already explained by age, gender, ASLEC number, and ASLEC impact (Table 4). Altogether, the PSS-10 has satisfactory concurrent validity in predicting anxiety and depression.
In this study, we evaluated the factor structure, measurement invariance, reliability, and validity of the PSS-10 in a large sample of Chinese senior high school students. CFA supported a two-factor model of the PSS-10. Multigroup CFA further indicated that the two-factor structure of the PSS-10 was upheld across genders. Additionally, the PSS-10 had acceptable internal consistency, satisfactory convergent validity for stressful life events, and good concurrent validity in predicting anxiety and depression. To our knowledge, this is the first study to evaluate the PSS-10 in Chinese adolescents and to demonstrate measurement invariance across genders. Overall, our results suggested that the PSS-10 could be applied to Chinese adolescents to measure perceived stress.
The two-factor structure found in the current sample was in line with findings of previous studies on the use of the PSS-10 in other populations, such as university students (e.g., Roberti et al., 2006; Örücü and Demir, 2009; Lu et al., 2017), general adult samples (e.g., Siqueira Reis et al., 2010; Wang et al., 2011; Ng, 2013; Taylor, 2015; Baik et al., 2019), and clinical patients (e.g., Golden-Kreutz et al., 2004; Leung et al., 2010). Most researchers have argued that the distinction between the two factors may represent separate components of the stress experience (Baik et al., 2019). The first factor has been interpreted as “Perceived Helplessness” or “Perceived Distress” (negatively phrased items), which reflects a lack of control and negative emotions; the second factor has been interpreted as “Perceived Self-Efficacy” or “Perceived Coping” (positively phrased items), which taps positive feelings and perceptions of confidence (e.g., Golden-Kreutz et al., 2004; Roberti et al., 2006; Barbosa-Leiker et al., 2013). Nevertheless, given that Cohen (1988) originally posited that the distinction between the two factors was irrelevant and only reflected item directionality, and the two-factor may arise from the problem of method biases (Schmitt, 1994), we suggest that the subscale scores should be cautiously used and interpreted in future studies.
Based on the configural, metric, and scalar invariance tests, we found that the two-factor model of the PSS-10 was equivalent for male and female adolescents, suggesting that the PSS-10 scores may not be affected by gender bias. This is similar to the results of previous studies with English versions (e.g., Barbosa-Leiker et al., 2013; Taylor, 2015; Denovan et al., 2019) and German versions (e.g., Reis et al., 2017) of the PSS-10. The present study, to the best of our knowledge, is the first to establish measurement invariance in the Chinese PSS-10 across genders, which provides support for using the PSS-10 to assess gender differences in Chinese adolescents.
Further analyses supported the internal consistency, reliability, and validity of the PSS-10 in assessing Chinese adolescents. The internal consistency of the present study is comparable to that from previous studies in Chinese populations (e.g., Cronbach’s alpha: 0.67–0.87; Wang et al., 2011; Ng, 2013; Lu et al., 2017), indicating adequate internal reliability. Convergent validity was tested for stressful life events. Cohen et al. (1983) originally investigated the relationship between the PSS-14 and stressful life events and found that there was a small to moderate correlation between the number and impact of stressful life events and the PSS-14 in three American college student samples. Previous studies have also reported good convergent validity of the PSS-10 with other conceptually similar measures, such as the Life Event Scale (Mitchell et al., 2008). Consistently, the present study revealed that PSS-10 scores were significantly related to both the number and the impact of the ASLEC. Further regression analyses supported the concurrent validity of the PSS-10 for assessing anxiety and depression. Previous studies demonstrated correlations between the PSS-10 scores and anxiety/depression in Chinese university students (Lu et al., 2017) and policewomen (Wang et al., 2011). In addition to the correlational results, our study uses hierarchical regression analyses to further reveal the incremental validity of the PSS-10 beyond predictors of stressful life events, suggesting the distinctness of perceived stress beyond other theoretical related constructs that may be predictive of anxiety and depression.
The present study had two limitations that should be considered. First, the sample in this study only covered senior high school students, with a relatively narrow age range. Although this sample achieved sufficient statistical power to observe existing effects (Mackey et al., 2015), further investigations are needed to validate our findings in other adolescent students (e.g., junior high school students). Second, we only tested gender invariance. Future studies should test measurement invariance across other groups, such as across age groups or cultural groups. Despite these limitations, the present study adds to the literature by demonstrating that the PSS-10 is a sound measure of perceived stress among Chinese senior high school students with a stable two-factor structure across female and male groups, and with acceptable reliability and validity. Future research and clinical practice could use this scale to measure perceived stress in this population and to compare gender differences in the scores.
Our findings provide evidence to use the PSS-10 in future studies to investigate the perceived stress in Chinese adolescents and to compare the gender differences. Future directions in research may consider the cross-cultural measurement invariance in adolescents. Moreover, our findings support the use of the PSS-10 in clinical settings to determine the severity of the load of the stressful events among Chinese adolescents and guide effective interventions to decrease their stress symptoms, especially those related to the educational system.
Data Availability Statement
The datasets generated for this study are available on request to the corresponding author.
The studies involving human participants were reviewed and approved by the local research ethics committee of the West China Hospital of Sichuan University. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
SW and JL conceived the research. SW and YZ performed the research. XL, JD, and XW analyzed the data. XL wrote the manuscript.
This study was supported by the National Natural Science Foundation of China (Grant No. 31800963), the China Postdoctoral Science Foundation (Grant No. 2019M653421), the Postdoctoral Interdisciplinary Research Project of Sichuan University, and the Science & Technology Department of Sichuan Province, China (2017JY0031 to JD).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors would like to thank all the participants and their guardians.
Andreou, E., Alexopoulos, E. C., Lionis, C., Varvogli, L., Gnardellis, C., Chrousos, G. P., et al. (2011). Perceived stress scale: reliability and validity study in Greece. Int. J. Environ. Res. Public Health 8, 3287–3298. doi: 10.3390/ijerph8083287
Baik, S. H., Fox, R. S., Mills, S. D., Roesch, S. C., Sadler, G. R., Klonoff, E. A., et al. (2019). Reliability and validity of the perceived stress scale-10 in Hispanic Americans with English or Spanish language preference. J. Health Psychol. 24, 628–639. doi: 10.1177/1359105316684938
Barbosa-Leiker, C., Kostick, M., Lei, M., McPherson, S., Roper, V., Hoekstra, T., et al. (2013). Measurement invariance of the perceived stress scale and latent mean differences across gender and time. Stress Health 29, 253–260. doi: 10.1002/smi.2463
Birleson, P., Hudson, I., Buchanan, D. G., and Wolff, S. (1987). Clinical evaluation of a self-rating scale for depressive disorder in childhood (Depression Self-Rating Scale). J. Child Psychol. Psychiatry 28, 43–60. doi: 10.1111/j.1469-7610.1987.tb00651.x
Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., and Baugher, M. (1999). Psychometric properties of the screen for child anxiety related emotional disorders (SCARED): a replication study. J. Am. Acad. Child Adolesc. Psychiatry 38, 1230–1236. doi: 10.1097/00004583-199910000-00011
Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., et al. (1997). The screen for child anxiety related emotional disorders (SCARED): scale construction and psychometric characteristics. J. Am. Acad. Child Adolesc. Psychiatry 36, 545–553. doi: 10.1097/00004583-199704000-00018
Byrne, B. M. (2001). Structural equation modeling with AMOS, EQS, and LISREL: comparative approaches to testing for the factorial validity of a measuring instrument. Int. J. Test. 1, 55–86. doi: 10.1207/S153275741JT0101-4
Cai, Y., Liang, B. Y., and Zhou, Y. J. (2010). Relationship among stress of entrance examination, resilience and stress distress of middle school students. Chin. J. Clin. Psychol. 18, 180–183. doi: 10.16128/j.cnki.1005-3611.2010.02.038
Chan, Y., Chan, Y. Y., Cheng, S. L., Chow, M. Y., Tsang, Y. W., Lee, C., et al. (2017). Investigating quality of life and self-stigma in Hong Kong children with specific learning disabilities. Res. Dev. Disabil. 68, 131–139. doi: 10.1016/j.ridd.2017.07.014
Chen, C. H., Tseng, Y. F., Chou, F. H., and Wang, S. Y. (2000). Effects of support group intervention in postnatally distressed women: a controlled study in Taiwan. J. Psychosom. Res. 49, 395–399. doi: 10.1016/S0022-3999(00)00180-X
Chu, L., and Kao, H. S. R. (2005). The moderation of meditation experience and emotional intelligence on the relationship between perceived stress and negative mental health. Chin. J. Psychol. 47, 157–179. doi: 10.6129/CJP.2005.4702.05
Cohen, S. (1988). “Perceived stress in a probability sample of the United States,” in The Claremont Symposium on Applied Social Psychology. The Social Psychology of Health, eds S. Spacapan and S. Oskamp (Thousand Oaks, CA: Sage Publications, Inc), 31–68.
Cohen, S. (2014). PSS: Frequently Asked Questions. Available online at: http://www.psy.cmu.edu/~scohen/ (accessed March 20, 2019).
Cohen, S., Kamarck, T., and Mermelstein, R. (1994). “Perceived stress scale,” Measuring Stress: A Guide for Health and Social Scientists, eds S. Cohen, R.C. Kessler, and L.U. Gordon (New York: Oxford University Press)
Deb, S., Strodl, E., and Sun, J. (2015). Academic stress, parental pressure, anxiety and mental health among Indian high school students. Int. J. Psychol. Behav. Sci. 5, 26–34. doi: 10.5923/j.ijpbs.20150501.04
Denovan, A., Dagnall, N., Dhingra, K., and Grogan, S. (2019). Evaluating the perceived stress scale among UK university students: implications for stress measurement and management. Stud. High. Educ. 44, 120–133. doi: 10.1080/03075079.2017.1340445
Edwards, M., Adams, E. M., Waldo, M., Hadfield, O. D., and Biegel, G. M. (2014). Effects of a mindfulness group on Latino adolescent students: examining levels of perceived stress, mindfulness, self-compassion, and psychological symptoms. J. Spec. Group Work 39, 145–163. doi: 10.1080/01933922.2014.891683
Gitchel, W. D., Roessler, R. T., and Turner, R. C. (2011). Gender effect according to item directionality on the perceived stress scale for adults with multiple sclerosis. Rehabil. Couns. Bull. 55, 20–28. doi: 10.1177/0034355211404567
Golden-Kreutz, D. M., Browne, M. W., Frierson, G. M., and Andersen, B. L. (2004). Assessing stress in cancer patients: a second-order factor analysis model for the perceived stress scale. Assessment 11, 216–223. doi: 10.1177/1073191104267398
Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Med. Care 44(11 Suppl. 3), S78–S94. doi: 10.1097/01.mlr.0000245454.12228.8f
Hu, L. T., and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6, 1–55. doi: 10.1080/10705519909540118
Kaya, C., Tansey, T. N., Melekoglu, M., Cakiroglu, O., and Chan, F. (2019). Psychometric evaluation of Turkish version of the perceived stress scale with Turkish college students. J. Ment. Health 28, 161–167. doi: 10.1080/09638237.2017.1417566
Lavoie, J. A., and Douglas, K. S. (2012). The Perceived Stress Scale: evaluating configural, metric and scalar invariance across mental health status and gender. J. Psychopathol. Behav. Assess. 34, 48–57. doi: 10.1007/s10862-011-9266-1
Lee, C. T., Tsai, M. C., Lin, C. Y., and Strong, C. (2017). Longitudinal effects of self-report pubertal timing and Menarcheal age on adolescent psychological and behavioral outcomes in female youths from Northern Taiwan. Pediatr. Neonatol. 58, 313–320. doi: 10.1016/j.pedneo.2016.04.004
Lesage, F. X., Berjot, S., and Deschamps, F. (2012). Psychometric properties of the French versions of the Perceived Stress Scale. Int. J. Occup. Med. Environ. Health 25, 178–184. doi: 10.2478/S13382-012-0024-8
Leung, D. Y., Lam, T. H., and Chan, S. S. (2010). Three versions of Perceived Stress Scale: validation in a sample of Chinese cardiac patients who smoke. BMC Public Health 10:513. doi: 10.1186/1471-2458-10-513
Li, J., Lin, L., Zhao, Y., Chen, J., and Wang, S. (2018a). Grittier Chinese adolescents are happier: the mediating role of mindfulness. Pers. Individ. Dif. 131, 232–237. doi: 10.1016/j.paid.2018.05.007
Li, J., Zhao, Y., Kong, F., Du, S., Yang, S., and Wang, S. (2018b). Psychometric assessment of the short grit scale among Chinese adolescents. J. Psychoeduc. Assess. 36, 291–296. doi: 10.1177/0734282916674858
Lin, C.-Y., and Tsai, M.-C. (2016). Effects of family context on adolescents’ psychological problems: moderated by pubertal timing, and mediated by self-esteem and interpersonal relationships. Appl. Res. Qual. Life 11, 907–923. doi: 10.1007/s11482-015-9410-2
Liu, X., Kurita, H., Uchiyama, M., Okawa, M., Liu, L., and Ma, D. (2000). Life events, locus of control, and behavioral problems among Chinese adolescents. J. Clin. Psychol. 56, 1565–1577. doi: 10.1002/10974679(200012)56:12<1565::aid-7<3.0.co;2-u
Liu, X., Liu, L. Q., Yang, J., and Zhao, G. F. (1997). Reliability and validity of the adolescents self-rating life events checklist. Chin. J. Clin. Psychol. 5, 34–36. doi: 10.1016/j.eurpsy.2016.07.008
Lu, W., Bian, Q., Wang, W., Wu, X., Wang, Z., and Zhao, M. (2017). Chinese version of the Perceived Stress Scale-10: a psychometric study in Chinese university students. PLoS One 12:e0189543. doi: 10.1371/journal.pone.0189543
Lupien, S. J., McEwen, B. S., Gunnar, M. R., and Heim, C. (2009). Effects of stress throughout the lifespan on the brain, behaviour and cognition. Nat. Rev. Neurosci. 10, 434–445. doi: 10.1038/nrn2639
Mackey, A. P., Finn, A. S., Leonard, J. A., Jacoby-Senghor, D. S., West, M. R., Gabrieli, C. F., et al. (2015). Neuroanatomical correlates of the income-achievement gap. Psychol. Sci. 26, 925–933. doi: 10.1177/0956797615572233
Martin, R. A., Kazarian, S. S., and Breiter, H. J. (1995). Perceived stress, life events, dysfunctional attitudes, and depression in adolescent psychiatric inpatients. J. Psychopathol. Behav. 17, 81–95. doi: 10.1007/BF02229205
Mitchell, A. M., Crane, P. A., and Kim, Y. (2008). Perceived stress in survivors of suicide: psychometric properties of the Perceived Stress Scale. Res. Nurs. Health 31, 576–585. doi: 10.1002/nur.20284
Moeini, B., Shafii, F., Hidarnia, A., Babaii, G. R., Birashk, B., and Allahverdipour, H. (2008). Perceived stress, self-efficacy and its relations to psychological well-being status in Iranian male high school students. Soc. Behav. Pers. Int. J. 36, 257–266. doi: 10.2224/sbp.2008.36.2.257
Muris, P., Merckelbach, H., Schmidt, H., and Mayer, B. (1998). The revised version of the screen for child anxiety related emotional disorders (SCARED-R): factor structure in normal children. Pers. Individ. Dif. 26, 99–112. doi: 10.1016/S0191-8869(98)00130-5
Perera, M. J., Brintz, C. E., Birnbaum-Weitzman, O., Penedo, F. J., Gallo, L. C., Gonzalez, P., et al. (2017). Factor structure of the Perceived Stress Scale-10 (PSS) across English and Spanish language responders in the HCHS/SOL sociocultural ancillary study. Psychol. Assess. 29:320. doi: 10.1037/pas0000336
Reis, D., Lehr, D., Heber, E., and Ebert, D. D. (2017). The German version of the Perceived Stress Scale (PSS-10): evaluation of dimensionality, validity, and measurement invariance with exploratory and confirmatory bifactor modeling. Assessment 26, 1246–1259. doi: 10.1177/1073191117715731
Roberti, J. W., Harrington, L. N., and Storch, E. A. (2006). Further psychometric support for the 10−item version of the perceived stress scale. J. Coll. Couns. 9, 135–147. doi: 10.1002/j.2161-1882.2006.tb00100.x
Siqueira Reis, R., Ferreira Hino, A. A., and Romélio Rodriguez Añez, C. (2010). Perceived stress scale: reliability and validity study in Brazil. J. Health Psychol. 15, 107–114. doi: 10.1177/1359105309346343
Strong, C., Tsai, M.-C., Lin, C.-Y., and Cheng, C.-P. (2016). Childhood adversity, timing of puberty and adolescent depressive symptoms: a longitudinal study in Taiwan. Child Psychiatry Hum. Dev. 47, 347–357. doi: 10.1007/s10578-015-0570-y
Tsai, M. C., Chou, Y. Y., Lin, S. J., and Lin, S. H. (2013). Factors associated with adolescents’ perspectives on health needs and preference for health information sources in Taiwan. Arch. Dis. Child 98, 9–15. doi: 10.1136/archdischild-2012-301629
Tsai, M.-C., Hsieh, Y.-P., Strong, C., and Lin, C.-Y. (2015a). Effects of pubertal timing on alcohol and tobacco use in the early adulthood: a longitudinal cohort study in Taiwan. Res. Dev. Disabil. 36, 376–383. doi: 10.1016/j.ridd.2014.10.026
Tsai, M.-C., Lee, C.-T., Strong, C., Chen, W.-T., and Lin, C.-Y. (2018). Longitudinal impacts of pubertal timing and weight status on adolescent internet use: analysis from a cohort study of Taiwanese youths. PLoS One 13:e0197860. doi: 10.1371/journal.pone.0197860
Tsai, M.-C., Strong, C., and Lin, C.-Y. (2015b). Effects of pubertal timing on deviant behaviors in Taiwan: a longitudinal analysis of 7th- to 12th-grade adolescents. J. Adolesc. 42, 87–97. doi: 10.1016/j.adolescence.2015.03.016
Wang, S., Dai, J., Li, J., Wang, X., Chen, T., Yang, X., et al. (2018). Neuroanatomical correlates of grit: growth mindset mediates the association between gray matter structure and trait grit in late adolescence. Hum. Brain Mapp. 39, 1688–1699. doi: 10.1002/hbm.23944
Wang, S., Zhou, M., Chen, T., Yang, X., Chen, G., Wang, M., et al. (2017). Grit and the brain: spontaneous activity of the dorsomedial prefrontal cortex mediates the relationship between the trait grit and academic performance. Soc. Cogn. Affect. Neurosci. 12, 452–460. doi: 10.1093/scan/nsw145
Wang, Z., Chen, J., Boyd, J. E., Zhang, H., Jia, X., Qiu, J., et al. (2011). Psychometric properties of the Chinese version of the Perceived Stress Scale in policewomen. PLoS One 6:e28610. doi: 10.1371/journal.pone.0028610
Woo, B. S. C., Chang, W. C., Fung, D. S. S., Koh, J. B. K., Leong, J. S. F., Kee, C. H. Y., et al. (2004). Development and validation of a depression scale for Asian adolescents. J. Adolesc. 27, 677–689. doi: 10.1016/j.adolescence.2003.12.004
Yin, H., Xu, G., Zhen, L., Yang, F., and Han, T. (2012). Norms and reliability and validity of the depression self-rating scale for children in rural children of Henan province. J. Clin. Psychiatry 22, 310–312.
Yosipovitch, G., Tang, M., Dawn, A. G., Chen, M., Goh, C. L., Chan, Y. H., et al. (2007). Study of psychological stress, sebum production and acne vulgaris in adolescents. Acta Derm. Venereol. 87, 135–139. doi: 10.2340/00015555-0231
Keywords: Perceived Stress Scale, confirmatory factor analysis, measurement invariance, Chinese adolescents, stress
Citation: Liu X, Zhao Y, Li J, Dai J, Wang X and Wang S (2020) Factor Structure of the 10-Item Perceived Stress Scale and Measurement Invariance Across Genders Among Chinese Adolescents. Front. Psychol. 11:537. doi: 10.3389/fpsyg.2020.00537
Received: 10 November 2019; Accepted: 06 March 2020;
Published: 09 April 2020.
Edited by:Mengcheng Wang, Guangzhou University, China
Reviewed by:Cristina Sechi, University of Cagliari, Italy
Chung-Ying Lin, Hong Kong Polytechnic University, Hong Kong
Copyright © 2020 Liu, Zhao, Li, Dai, Wang and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.