Reliability and Validity of the Chinese Version of the Scale for Assessing the Stigma of Mental Illness in Nursing

Background: The prevalence of mental illness continues to increase in China, but research on stigma is still in its infancy, and there are even fewer studies on stigma among nurses. A comprehensive, effective and reliable tool is needed to assess stigma in nursing so that it can be reduced or eliminated to improve nursing quality. This study aimed to translate a 20-item scale for assessing the stigma of mental illness in nursing into Chinese and evaluate its reliability and validity. Methods: An improved Brislin translation model was used to translate the nursing mental illness stigma assessment scale into Chinese. Content and face validity were determined by a panel of experts. A convenience sample of 501 nursing students was chosen. Confirmatory factor analysis, concurrent validity and known group comparison were used to evaluate the scale's structural validity. The reliability was evaluated based on the internal consistency reliability and 2-week retest reliability. Results: The content validity index was 0.90. Confirmatory factor analysis showed that this study supported the three-factor model. The moderate correlation between the Chinese version of the Scale for Assessing the Stigma of Mental Illness in Nursing and the Perceived Devaluation Discrimination Scale suggested acceptable concurrent validity. Cronbach's α (0.863) and the retest coefficient (0.839) were indicative of internal consistency. Conclusion: The Chinese version of the Scale for Assessing the Stigma of Mental Illness in Nursing has acceptable concurrent validity, marginal factor validity, and satisfactory reliability in China. Therefore, the three-factor structure of the Chinese scale should be considered. Relevance to Clinical Practice: The Chinese version of the Scale for Assessing the Stigma of Mental Illness in Nursing can be used to understand the degree of mental illness stigma in nursing.


INTRODUCTION
Stigma, a concept first proposed in 1963 by sociologist Goffman, is defined by him as follows: "the situation of the individual who is disqualified from full social acceptance" (1). Mental illness stigma, which is mainly produced in response to the abnormal behavior of people with mental illness, is an exclusionary attitude and leads to patients with mental disease patients feeling shame, inferiority and emotional distress (2,3). Studies have shown that stigma negatively affects empowerment, social inclusion, quality of life, help seeking behaviors and adherence to treatment in people with mental illness (4,5). Nursing staff, who have close contact with patients in the treatment of mental illness, can also discriminate against patients, stereotyping those with mental illness as incompetent, unpredictable, destructive, and violent, which undermines nursing quality (6,7). This stigma can lead to a decline in the quality of nursing, and patients may experience isolation, abuse in healthcare, diagnostic overshadowing and exclusion (8,9). It can also hinder the rehabilitation and social integration of patients with mental disorders and reduce the treatment effect in patients (10). Associated stigma may also affect nursing staff (11), leading them to perpetuate the cycle of exclusion in their clinical practice (12) or even to act in an extreme way (13). Stigma also affects nursing staffs' job satisfaction, interpersonal relationships, patient self-stigma, and nursing staff ' professional identity (14). Correct assessment of nurses' stigma toward mental illness and effective adjustment to reduce and weaken this stigma can promote the recovery of patients and improve their clinical symptoms (15).

BACKGROUND
The World Health Organization reported in 2014 that one out of every four people may suffer from mental illness (16). Globally, an estimated 264 million people suffer from depression, 45 million suffer from bipolar disorder, and more than 20 million suffer from schizophrenia (17). In China, the number of people with mental illness is on the rise, with the prevalence rising from 3.2% in the 1970s to 15.56% (18). According to one study, China registered 4.3 million severely mentally ill patients in 2014 (19) while another study showed that in 2013, the number of such patients reached 16 million (20). Eight million were diagnosed with schizophrenia in 2018 (21). Mental health has become a prominent public health and social problem hindering China's economic development. At present, most of the research on mental illness stigma focuses on measuring the degree of stigma in the public and its related factors. Questionnaires used in these studies included the Struening Devaluation Scale (22) the Mass Discrimination Scal (23), the Link Perceived Devaluation Discrimination Scale (24) the Psychiatric Attitude Questionnaire (25,26), and the Attitude Style Questionnaire (27). The objects of measurement of these questionnaires were the general public. Some questionnaires were used to measure the self-stigma of patients (28,29) and the associated stigma of family members (30,31). Only in a few articles were nursing undergraduates and doctors the objects of study (32,33). There is no specific assessment scale for caregivers. At the same time, since the three principles of stigmatizing interventions (contact, education, and protest) are now integrated into mental health programs (34,35), it is particularly important to have reliable tools that can objectively assess the effectiveness of these interventions (36). Based on the above situation, it is necessary for China to adopt the Mental illness Stigma assessment scale for Chinese nurses and verify its reliability and validity. Spanish scholars Dr. Sastre-Rus, Meritxell et al. developed a new tool for assessing the stigma of mental illness in nursing in 2020 (37). Therefore, the purpose of this study was to translate and localize the new scale, which was based on Peplau's psychodynamic nursing theory, and to measure its reliability and validity. The study was also to provide a way to quickly and accurately measure the degree of stigma of mental illness among nursing staff in China.

Study Design
This was a cross-sectional and observational survey designed to test the reliability and validity of the new Chinese version of the Scale for Assessing the Stigma of Mental Illness In Nursing tool, also known as the SASMIN.

Setting
The participants in this research were senior college nursing students from the Nursing College of Jinzhou Medical University in China, and 501 students were recruited as volunteers.

SASMIN
The SASMIN is a 20-item scale developed by Dr. Sastre-Rus, Meritxell et al. to comprehensively assess the stigma of mental illness among nurses. SASMIN includes three dimensions: Factor 1: Violence/Dangerousness (8 items), Factor 2: Disability (5 items), and Factor 3: Irresponsibility/Lack of Competence (7 items). Both positive and negative wording is used (items 2, 3, 5, 10, 13, and 15 are forward scored, with the rest of the items are reverse scored). Each item is rated on a five-point Likert scale (5 = totally agree; 4 = somewhat agree; 3 = neutral; 2 = somewhat disagree; 1 = totally disagree). The total and dimension scores are the sum of the scores of each item, with the total score being between 20 and 100. A higher score infers a weaker degree of stigma. The original scale has acceptable internal consistency, with a Cronbach's alpha of 0.825 for the overall scale and 0.626-0.731 for the subscales.

Perceived Devaluation Discrimination
In this study, the Perceived Devaluation Discrimination (PDD) Scale was used to assess criterion validity (24,38). The PDD mainly measures the perceived disparagement and discrimination mental patients and their family members receive from the general public to evaluate the perceived stigma of patients and their families. The scale also measures the level of public and social perceptions of stigmatization and discrimination against people with mental illness. The Chinese version of the scale, which was translated and localized by Xu Hui (39) in 2007, has been verified to have good reliability and validity. This scale is a self-rating scale consisting of 2 dimensions and 12 items. The author took the midpoint of 2.50 as the standard and compared the average score with it. Those with a score higher than 2.50 were considered to have stigma, while those with a score lower than 2.50 were considered to have no stigma.

Translation
We followed a modified Brislin translation model (40) consisting of (A) forward translation, (B) back translation and (C) revision. The English version of the scale was independently translated by two nursing graduate students who were proficient in English and had passed the CET-6. The Chinese version was independently translated into English by two nursing researchers with PhDs who were not familiar with the original SASMIN and were also proficient in English. Finally, two other bilingual experts were invited to perform a comparative analysis between the two translated versions and the original scale, and the items with large differences were re-translated and translated back. After the development of the third version, 10 students were recruited to conduct a pre-investigation on this version to ensure the quality of the scale before the reliability and validity tests. According to the subjects' understanding of the contents of the scale, information about the time required to complete the questionnaire, possible problems and suggestions, and ambiguous and difficult items was collected, and the researchers continued to make timely modifications and adjustments to produce the final Chinese version of the SASMIN.

Data Collection
The researchers themselves distributed paper questionnaires to nursing students at Jinzhou Medical University individually from November to December 2020 and explained the content, purpose and significance of the questionnaires to the students before issuing them, reminding them to fill out the questionnaires carefully. After obtaining informed consent from the students, the questionnaires were distributed to the students on the spot, and the students completed the questionnaires independently. It took 5-10 min to complete the questionnaires, and the questionnaires were returned on the spot after completion. The researchers checked whether there were omissions and errors in the completed questionnaires and confirmed or revised the questionnaires with the students at once to ensure the completeness and quality of the questionnaire completion. Then, 31 randomly selected students were invited to complete the questionnaires again 2 weeks later to test the reliability of the retest. The retest interval is generally 10-14 days (41), so this interval was 14 days to ensure the reliability of the retest (41).

Ethical Considerations
Ethical approval for this study was obtained from our institutional ethics committee. The SASMIN scale was authorized by the original author, and participants' informed consent was obtained. Data confidentiality was ensured during the survey. All the students signed a written informed consent form, and the study data were kept in a completely confidential way.

Statistical Analysis
EpiData data entry software was used to double check the data entry, and SPSS 26.0 and AMOS 23.0 were used for statistical analysis (IBM).
An expert team consisting of 6 relevant experts (1 psychological expert, 1 doctor of nursing, 2 clinical nursing experts, and 2 nursing education experts) was invited to evaluate the content validity of the SASMIN scale. They had been working for 27 years on average. They rated each item with a four-point Likert scale (1 = not relevant, 2 = weakly relevant, 3 = strongly relevant, 4 = highly relevant). The content validity index (I-CVI) for each project was calculated based on the percentage of protocols rated 3 or 4 out of the total number. When the I-CVI is ≥0.78, it indicates that the overall content validity of the scale is good (42).
Descriptive analysis was used for the general information of the participants, and confirmatory factor analysis (CFA) was conducted to investigate the underlying factor structure of the translated scale. To confirm the replicability of the firstorder three-factor structure of the SASMIN (43), the following indexes must be good: χ 2 /DF, goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), incremental fit index (IFI), Tucker-Lewis index (TLI), and comparative fit index (CFI). Generally, χ 2 /DF is required to be <3, while all of other values are required to be >0.9, indicating good adaptability of the model; however, a value >0.8 indicates that the model is acceptable. In addition, the root mean square error of approximation (RMSEA) should be <0.08, indicating good adaptability and good model fit.
By calculating the Cronbach's α coefficient, the retest reliability and the revised item-total correlation, the internal consistency reliability of the scale was tested. The minimum acceptable Cronbach's α coefficient was set to 0.7 (44). The adjusted itemtotal correlation represents the correlation between each item and the sum of the other items in the scale, which is evaluated using the standard of 0.3 (45).
Calibration correlation refers to using a recognized valid scale as a standard to test the degree of correlation between the measured scale and the standard scale. In this study, the PDD scale was used as the calibration standard for evaluation. Pearson correlation analysis was used to test the correlation between the Chinese version of the SASMIN scale and the PDD scale. When the correlation coefficient r > 0.7, the validity of the test is high. When 0.4 < r < 0.7, the validity of the test was moderate. When r < 0.4, the validity of the test is low (46).
For the discriminant validity, according to the total score of the SASMIN scale, the top 27% of scores were grouped as high and the bottom 27% as low, and the scores of both groups were analyzed using the two-tailed independent samples t-test. If the scores of the two groups reached the level of significance (P < 0.05), the discriminant validity was good.
To assess the degree of stigma, the total stigma score of each participant was calculated and then converted to a Z-score in the data (N = 501). The Z-score is also called the standard score, which is the difference between an original score and the mean divided by the standard deviation. Stigma was divided into four grades, namely, severe stigma, moderate stigma, mild stigma and no stigma, so each grade contained an interval of 1.5 standard deviations. A Z score below −1.5 is considered to indicate that there is very high stigma. A Z score between −1.5 and the mean is considered to indicate moderate stigma. A Z score that is above average but below 1.5 is considered to indicate mild stigma. Finally, a Z score higher than 1.5 is classified as no stigma. Therefore, the standardized scores were divided according to the principle that the higher the score is, the weaker stigma.

Participant Characteristics
A total of 518 questionnaires were sent out, and 501 remained after the invalid questionnaires were removed, for a recovery rate of 96.72%. The respondents were college nursing students aged 18-25 years, and the average age was 20.94 ± 1.368 years. According to the results of the survey, the majority of respondents were female (442/501), which was mainly due to the lack of male students in the nursing student population. Eleven percent (55/501) of the students participated in mental illness-related extracurricular or community activities in college, 34.3% (172/501) of the students had contact with patients with mental illness, and 24.6% (123/501) of the students wanted to become psychiatric nurses. The detailed results are shown in Table 1.

Content Validity
The contents validity index (I-CVI) of the items ranged from 0.830 to 1.000, and the contents validity index (S-CVI) of the scale was 0.975. Correlation analysis results showed that the scores of each item were positively correlated with the total scores, and the differences were statistically significant (P < 0.01).

Construct Validity Analysis
In the model fitness index, the Chi-square degree of freedom was 2.388, the adjusted GFI was 0.917, the incremental fit index (IFI) was 0.899, the TFI was 0.881, the CFI was 0.898, and the root mean square error of approximation (RMSEA) was 0.053, which proved that the structural equation model of the Chinese version of the scale was valid and the Chinese data fit the original three-dimensional structural model well. The standardized factor load of each item was >0.5 (shown in Table 2), indicating that each item could explain its dimension, except items 1 (a14) and 5 (a17). Item 1 is that people with a mental disorder area burden on their family and society. Item 5 is that caring for a patient with a mental disorder is no more burden than caring for other patients. The standardized three-factor structural model of the SASMIN (n = 501) is shown in Figure 1.

Criterion Validity
In this study, the Chinese version of the PDD scale was used as the calibration scale to analyse its correlation with the total score and the dimensions of the SASMIN scale. The results showed that the total scores of the two scales was positively correlated (r = 0.401, P < 0.01). The significant correlation coefficients of the different dimensions and the PDD scale were 0.296, 0.306, and 0.355, respectively (P < 0.01).

Reliability Analysis
The reliability analysis results showed that the Chinese version of the SASMIN had ideal internal consistency, with an overall Cronbach's alpha coefficient of 0.863 and Cronbach's alpha coefficients of 0.901, 0.820, and 0.875 for the three factors. In addition, the item-total correlation coefficients ranged from 0.327 to 0.600, which were all higher than 0.3. Therefore, the 20 items were all integrated to the questionnaire ( Table 3).

Retest Reliability
The intragroup correlation coefficient (ICC) was used to evaluate the retest reliability of the scale. The sample size was 31, the questionnaire recovery rate was 100%, the ICC value of the total scale was 0.760, and the ICC values of each dimension were 0.766 (P < 0.000), 0.468 (P < 0.008), and 0.575 (P < 0.001).

Discriminant Validity
Discriminant validity is based on the total score of the SASMIN scale, which was ranked from high to low; the top 27% of the scores were grouped into the high-score group, and the bottom 27% of the scores were grouped into the low-score group. In this study, the critical value scores were 52 and 63, respectively, and the scores of each item in the 2 groups were analyzed by using a two-tailed independent samples t-test. The results showed that the score difference of each item in the 2 groups reached the level of significance (P < 0.05). The detailed results are shown in Table 4.

The Degree of Stigma
The Z score range of the Chinese version of the SASMIN was −3.44-3.05, with an average of 0. Specifically, 6.19% (31/501), 46.51% (233/501), 41.32% (207/501), and 5.99% (30/501) of participants were in the no stigma, low level of stigma, mild level of stigma and high stigma groups, respectively. Table 5 shows the average SASMIN score, which were normally distributed according to the skewness and kurtosis graphs. The total mean

DISCUSSION
This study was the first to translate the SASMIN into Chinese and test its reliability and validity. In the process of forming the Chinese version of the SASMIN, we strictly implemented the relevant process. Testing showed that the scale has good structural validity, discriminant validity and reliability and that it is reliable and effective in assessing the degree of nursing mental illness stigma in China. It can be used to measure nursing students' stigma of mental illness and could provide a reference for nursing educators for teaching plan formulation. It can also be used as a tool for nursing managers to assess nurses' stigma of mental illness, develop timely measures and interventions, improve nursing quality, and optimize nursing services to achieve the purpose of nursing, namely, "promoting patient health." The original Spanish scale was tested on students. The subjects in this study were also nursing students, which were from Jinzhou Medical University in northern China. These students were about to become nursing workers, making them a representative sample.
The content validity of the Chinese version assessed by six nursing experts resulted in an S-CVI ranged from 0.830 to 1.000; on average, the I-CVI was 0.975, so the scale items show good content validity.
For the CFA, the results showed that most items had factor loads higher than 0.5, except for items 1 and 5, which had loads close to 0.3. This may be because nursing undergraduates have not had in-depth contact with patients or have less contact with patients with mental diseases. Most nursing undergraduates' understanding of patients is influenced by the positive narration of teachers and the negative narration of society or family, which leads to the unstable attitude of nursing students toward mental diseases. The IFI was 0.899, the TFI was 0.881, and the CFI was 0.898. These results showed that the current three-factor model was within the acceptable range, although it did not achieve good fitting results. This was consistent with the results of the original author, so the dimensions of the scale were maintained, and further investigation will be conducted with a sample of clinical nursing staff to verify the results of this analysis.
We also chose the PDD scale for the assessment of the concurrent validity index. The Chinese version of the PDD has good validity and is widely used in China for measurement of stigma in the general public. Therefore, the use of the PDD for the concurrent validity index is justified. According to the Pearson correlation analysis, the total correlation between the two scales was 0.401, showing a moderate positive correlation.
According to the results on the degree of stigma, 47.21% of the students had moderate or higher stigma regarding mental illness, which indicated that the mental illness stigma of Chinese nursing students was serious. This may be due to the fact, shown in Table 1, that most nursing students (89%) did not participate in activities related to mental illness, and 65.7% had no contact with mentally ill people. They were therefore not familiar with patients. The results in Table 1 also show that only 24.6% of students were willing to be psychiatric nurses. Therefore, nursing educators and administrators should be reminded to add related academic training or lectures to actively guide nursing students' attitudes toward mental illness and help them establish a correct mindset.

Limitations
However, several limitations should be taken into account when interpreting the results of this study. First, the research subjects were not nursing staff but senior nursing students who were easy to recruit. Therefore, the representativeness of the sample may be somewhat lacking. Second, due to the self-report nature of the survey, bias was inevitable. In addition, due to the lack of research on this topic, it was difficult to develop a proper discussion and compare the results with previous studies. However, this also highlights the novelty, interest, and readability of this study.

CONCLUSION
Following a rigorous translation and validation process, the SASMIN demonstrated acceptable construct validity in terms of concurrent validity, discriminant validity, and factorial validity and acceptable internal consistency and test-retest reliability in China. The results of this study also showed that nursing mental illness stigma is prevalent in China, suggesting that measures need to be established to reduce mental illness stigma in nursing and improve the quality of care.

Relevance to Clinical Practice
The SASMIN can identify the degree of stigma associated with mental illness among nursing staff in China and can be used as a tool by nursing managers to measure the degree of stigma and improve quality of care and patient satisfaction. Nursing educators should also take the measurement results of the scale as a reference, adjust the curriculum in a timely manner, improve the quality of education, and cultivate outstanding nursing talent.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

AUTHOR CONTRIBUTIONS
WW: identify research content, research route design, survey, data analysis, and draft writing. HC: study route design, provide resources, and revise draft. WZ: review draft and provide statistical guidance. XX: collect data and organize data. HD: data collection and statistical analysis. All authors contributed to the article and approved the submitted version.