The Psychometric Properties of a Short UPPS-P Impulsive Behavior Scale Among Psychiatric Patients Evaluated in an Emergency Setting

Objective: Impulsivity is a multidimensional construct that has an important role for the understanding of diverse psychopathologies and problematic behaviors. The UPPS-P impulsive behavior scale, measuring five distinct facets of impulsivity, has been subject to several studies. No study has investigated the clinical utility of this questionnaire amongst an unstable psychiatric population. The aim of the current study is to examine the psychometric properties of the short version of this scale in a psychiatric emergency unit. Method: The S-UPPS-P was administered to 1,097 psychiatric patients in an emergency setting, where a subgroup of 148 participants completed a follow-up. The internal consistency, the construct validity, the test-retest reliability, and correlations with a substance misuse measure were examined. Results: Confirmatory factor analyses supported a five-factor solution. Results indicated good psychometric properties across psychiatric diagnoses and gender. The S-UPPS-P was partially invariant across sexes. The authors have found differences on the loading of one item and on the thresholds of two items from lack of premeditation and positive urgency subscales. Conclusion: This validation study showed that the UPPS-P conserved good psychometric properties in an unstable psychiatric sample, indicating that the instrument can be utilized in such settings.


INTRODUCTION
The nature of impulsivity, now known to be heterogeneous (1), is generally characterized by deficits in delaying gratification, impulse and urge control, decision making, and maladaptive behaviors (2). Its cross-cutting nosology component, in line with the Research Domain Criteria framework (e.g., RDoC Cognitive Systems) (3,4), plays a major role in the understanding of diverse psychopathologies and problematic behaviors. In fact, it is not surprising that impulsivity is one of the most common diagnostic criteria in the fifth edition of the Diagnostic and Statistical Manual for Mental Disorders (5) (e.g., Attention Deficit Hyperactivity Disorder and Conduct Disorder, Borderline Personality Disorder, Antisocial Personality Disorder, Substance Use Disorders, Bipolar Disorder).
However, it is interesting to note that impulsivity has been subject to terminological and conceptual confusion. To clarify the jingle-jangle fallacy 1 of impulsivity (6), exploratory and confirmatory factor analyses have been performed using the Urgency-Premeditation-Perseverance-Sensation Seeking-Positive Urgency (UPPS-P) impulsive behavior scale which revealed five specific facets of impulsivity (7). These dimensions are sensation seeking (tendency to seek out novel and thrilling experiences), lack of premeditation (tendency to not take into account the consequences of actions), lack of perseverance (tendency to have difficulty staying focused on a task that can be long, boring or difficult), negative urgency (the tendency to act rashly while in an intense negative mood), and positive urgency (the tendency to act rashly while in an intense positive mood).
The transdiagnostic characteristic of impulsivity is highly relevant for clinicians regarding the understanding and the treatment of different forms of psychopathologies and problematic behaviors. In this sense, impulsivity constitutes a key target for clinical interventions (27). The UPPS-P model has attempted to clarify issues regarding the heterogeneity of impulsivity by presenting a strong and stable factorial structure. Considering the fact that the five dimensions of the UPPS-P are all specifically associated with distinct psychopathologies and problematic behaviors, the short version of the UPPS-P scale could be a suitable solution to overcome difficulties concerning the identification and the management of the broad spectrum of impulsivity-related problems in psychiatric patients. Along these lines, to our knowledge, no prior study has examined the psychometric properties of the UPPS-P scale amongst unstable psychiatric populations such as those in emergency department (ED). The objective of this current study is to examine the psychometric properties of the French version of the S-UPPS-P (23) in a large sample of adult psychiatric patients evaluated in a psychiatric ED. More precisely, we aimed to investigate the internal consistency, the construct validity, the test-retest reliability of the S-UPPS-P, as well as correlations between its subscales and a substance misuse measure, in this particular setting across psychiatric diagnoses and gender.

Sample Description
The sample was taken from the Signature Bank of the Institut Universitaire en Santé Mentale de Montréal (IUSMM). Research nurses approached 1,862 eligible participants from the psychiatric emergency of the IUSMM. Of this number, 1,218 agreed to participate in the study. In the current validation study, 1,097 patients have accepted to participate. This is referred to as Time 1 (T1). French version of the S-UPPS-P used in the current study is taken from Bilieux et al. (23). These authors used forward-, consensus, and back-translation steps to assess the quality of the translation. Only 25 participants answered the questionnaire in the original English version. A subsample of 148 participants answered the questionnaire a second time within a 30-day interval right before leaving the hospital. This is referred to as T2. For details on the characteristics of patients at T1 and T2, please refer to Supplementary Tables 1, 2.
Our sample was mainly characterized by individuals with substance use disorders (SUDs) (N = 83), psychotic disorders (N = 429), mood disorders (N = 350), anxiety disorders (N = 104), personality disorders (PDs) (N = 115), and other psychiatric disorders (N = 16). Over half were men (N = 655 [59.7%]) with a mean (SD) age of 40.4 (14.1) years. All participants signed a detailed consent form, and the study was approved by the local ethics committee in accordance with the Declaration of Helsinki.

Impulsive Behavior Scale
The Impulsive behavior scale (S-UPPS-P) is an instrument composed of 20 items rated on a four-point Likert scale (23): (1) disagree strongly, (2) disagree some, (3) agree some, and (4) agree strongly. Five scales were computed by adding the 4 items corresponding to each scale: (1) negative urgency (NU), (2) positive urgency (PU), (3) sensation seeking (SS), (4) lack of perseveration (PE), and (5) lack of premeditation (PR). Each scale ranges from 4 to 16. The description of each scale by main psychiatric diagnosis and sex is indicated in Supplementary Table 3. The S-UPPS-P questionnaire is shown in Appendix 1.

Drug Abuse Screening Test (DAST-10)
The Drug abuse screening test is a 10-item check list. The items are dichotomous: (0) no, (1) yes. The total scale is the sum of the 10 items resulting in a scale of 0-10.

Statistical Analyses
Analyses were all performed in R v3.3.0 (29). We used the psych package (30) for reliability analyses and the lavaan package (31) for Structural Equation modeling (SEM).

Internal Consistency
To evaluate the internal consistency of the S-UPPS-P, Cronbach alphas (32) were estimated for each of the five subscales: negative urgency (NU), positive urgency (PU), sensation seeking (SS), lack of perseveration (PE), lack of premeditation (PR). We tested first for all participants and then by sex and main psychiatric diagnosis.

Construct Validity
Following the work made by Billieux et al. (23), confirmatory factor analyses using a probit link were performed to confirm the structure of the S-UPPS-P. Four factorial models were compared: (1) a single-factor model (Model 1) in which all 20 items loaded on a unique "impulsivity factor, " (2) a five factor model (Model 2) representing the five subscales of the S-UPPS-P (see Supplementary Figure 1), (3) a three factor model (Model 3) represented by three latent variables: urgency, sensation seeking and lack of conscientiousness and this three factor model, adding hierarchical structure: urgency (positive and negative urgency as lower order factors), sensation seeking and lack of conscientiousness (lack of premeditation and perseverance as lower order factors). Membership in one of the 4 Likert categories was assigned using three thresholds and a latent variable for each items of the UPPS-P. Model were adjusted using lavaan package (31) in R (29). Group analyses by gender were examined to test for measurement invariance across sexes with ordered categorical variable (33).

Test-Retest Reliability
As described previously in the sample description, a subsample of 148 participants answered the UPPS-P questionnaire a second time just before their release from the hospital. Only the participants who answered the questionnaire within 30 days of their admissions were selected. The mean (± SD) time between emergency admission (T1) and hospital release (T2) An association has been regularly observed between SUDs and impulsivity, specifically sensation seeking (34). By using a confirmatory factor analysis, we tested for relationships between the impulsivity measures and substance use (as measured with the DAST-10), since both constructs are theoretically and empirically related.

Comparisons of Mean Differences of S-UPPS-P Subscales Between Diagnostic Categories
To better understand differences in S-UPPS-P subscales between diagnostic categories, we performed analysis of variance (ANOVA) post hoc pairwise comparisons tests, corrected with Tukey test for multiple comparisons.

Internal Consistency
Cronbach's alpha for all the participants varied between 0.70 and 0.81 indicating an acceptable to a good reliability of the S-UPPS-P questionnaire. As illustrated in Table 1, each dropped item did not improve the Cronbach's alpha. Table 2 shows the results on the reliability of the UPPS-P by sex and main psychiatric diagnosis. In most subgroups, the 95% confidence intervals intersected with each other or are above the total alpha. Only the subgroup composed of women with psychotic disorders had a lower Cronbach's alpha. Figure 1 displays graphically the different confidence intervals. Some confidence intervals were very large due to a smaller prevalence of certain psychiatric disorders, such as SUDs and other psychiatric illnesses (F00-F09, F50-59, and F70-F99 in ICD-10). These results showed that the reliability of the UPPS is constant across psychiatric diagnoses and sexes. Table 3 Table 3, the two other models provided worse model fits.

Construct Validity
The 5-factor model, as shown in Figure 2, is formed by negative urgency (NU), positive urgency (PU), sensation seeking (SS), lack of perseverance (PE) as well as lack of premeditation (PR). We also performed a multi-group analysis by sex to test for measurement invariance. Measurement invariance was tested for sex using the procedure by Millsap and Yun-Tein (33) for ordered categorical variable. First comparison between the configural model and the equal loadings model using scaled chisquared differences (36) gave a statistically significant differences χ 2 (15) = 27.2, p = 0.0275 but the difference in CFI and TLI was very small (∼0.001). By freeing loadings of item 6 (PR: "My thinking is usually careful and purposeful"), and item 10 (PU: "When overjoyed, I feel like I can't stop myself from going overboard") the difference with the configural models were no longer statistically significant χ 2 (18) = 18.1, p = 0.1533. When we compared the partially freed loadings model to the fixed thresholds model the difference was statistically significant χ 2 (35) = 67.3, p < 0.001. We stopped invariance testing at this step because of the large chi-square differences. The difference was mostly driven by the sex difference on the second and third threshold of item 10 which was respectively 0.333 and 0.346. This indicates that men over reported item 10 even when conditioned on the positive urgency factor.

Test-Retest Reliability
Some participants were followed through their clinical pathways. We analyzed responses from participants who answered within 30 days of their original assessment just before they were released from the hospital. Results are presented in Table 4. The total score had a good test-retest reliability according to the correlation and intra-class correlation coefficients which were both around 0.60 and above. The scores ranged from 0.4 to 1.0 unit lower at Time 2.

Correlations With a Substance Misuse Measure (DAST-10)
We tested correlations between the five subscales of the S-UPPS-P and a substance use measure (DAST-10). We expected that patient's impulsivity score, specifically the sensation seeking subscale score, would be associated with the DAST total score. This was tested using a CFA with the factor structure described earlier. Results of the CFA of the 5 dimensions of the S-UPPS-P regressed on the DAST-10 showed that sensation seeking correlated with substance use (r = 0.37, p < 0.001). A trend was observed concerning a positive association between Lack of premeditation and substance use (r = 0.13, p = 0.053). These results remained statistically significant after the inclusion of age and sex as adjustment variables (see Figure 3).

Comparisons of Mean Differences of S-UPPS-P Subscales Between Diagnostic Categories
Results suggest significant differences between diagnostic categories across every S-UPPS-P subscale, as shown in Table 5. More specifically, pairwise post hoc tests showed that participants with a PD scored significantly higher in Negative Urgency,

DISCUSSION
The present study examined the psychometric properties of the S-UPPS-P questionnaire (23) which was administered in a psychiatric ED. Generally, results demonstrated that the S-UPPS-P has good psychometric properties across psychiatric diagnoses and sexes in this specific setting. Moreover, Cronbach' alphas supported acceptable to good internal consistency within dimensions: negative urgency (0.79), positive urgency (0.70), lack of perseveration (0.81), lack of premeditation (0.79), and sensation seeking (0.77). The internal consistency reliability was found to be constant across psychiatric diagnoses and sexes but was lower in the subgroup composed of women with psychotic disorders. The reasons for this latter observation are elusive. Furthermore, CFA confirmed similar structures consistent with previous studies (five-factor model: 25, 26 and three-factor hierarchical model: 23). Correlations and intra-class correlation coefficients indicated that the S-UPPS-P had a good testretest reliability. While below the usual cut-off (>0.70), results were good considering the psychological instability of patients admitted in a psychiatric ED (T1) in comparison to their release (T2). Finally, by testing correlations with a substance misuse measure, we observed a strong association between substance use and sensation seeking. These latter findings are similar to those of previous studies on impulsivity in SUD populations (34), and further justify the use of factors rather the use of a broad UPPS total score. Additionally, our results suggest that the distinction between impulsive behavior factors seems to be determinant in the comprehension of specific psychopathology and inadequate behaviors such as drug misuse. A significant finding in our study is that we were able to replicate the validity of the theory-driven factor structure of the UPPS-P model across both psychiatric diagnoses and  gender within a psychiatric ED. Indeed, both structures has been identified in general population samples using French (23), English (25), Spanish (24), and Italian (26) versions of the S-UPPS-P. The five distinct, yet interrelated factors (positive urgency, negative urgency, lack of premeditation, lack of perseverance, and sensation seeking) have already shown valuable implications for psychiatric research since each dimension was strongly associated with specific psychopathologies. In fact, our study has shown that participants with a PD had significantly higher scores than those with a psychotic, a mood, or an anxiety disorder on almost every subscales of the S-UPPS-P, thus reflecting that impulsivity and its subcomponents are core features of personality disorders (8,10,16). However, patients with PD did not significantly differ from patients with a SUD on the sensation seeking or the urgency's subscales. In fact, earlier studies suggested that impulsivity is a shared risk factor for developing a PD and/or a SUD (37,38). While PDs and SUDs may share subcomponents of impulsivity, the non-statistically significant results could be explained by the fact that both disorders are highly heterogeneous and intercorrelated (39,40). Thus, different subtypes of individuals with a PD and/or a SUD may be associated with distinct subcomponents of impulsivity (41,42). Nonetheless, our results suggest that the short version of the UPPS-P scale preserved its good psychometric properties in a sample of unstable psychiatric patients. Therefore, research on psychiatric unstable patients could use the S-UPPS-P to better understand psychopathologies and their comorbidities. While the S-UPPS-P displayed overall psychometric properties that were satisfactory, we found that the construct validity of the instrument was only weakly invariant across sexes. Although this result needs to be interpreted cautiously, it may have implications for future studies on the psychometric properties of the UPPS as well as on gender differences in impulsivity. Recently, a meta-analysis found that there were gender differences in impulsivity (43). Nevertheless, it has been subsequently argued by Cyders (44) that conclusions concerning gender differences in impulsivity might be premature, because the measurement invariance of impulsivity scales across sexes is largely unknown. This author investigated this issue regarding the UPPS-P in a sample of undergraduate and found a measurement invariance across sexes but found a higher reported level of sensation seeking in men when compared to woman (44). Our results suggest that the S-UPPS-P is partially invariant across sexes when administered to psychiatric patients in an ED. More specifically, we found that item 10 of the positive urgency subscale was overrated by men. This suggest that, in this particular setting, there could be a bias in positive urgency response between sexes. Therefore, future studies on sex differences regarding impulsivity should pay special attention to these subtle methodological issues.
Limitations in the current study need to be acknowledged. First, psychiatrist on the ward did not establish the diagnosis of psychiatric disorders with a validated instrument such as the Structure Clinical Interview for DSM-V. Also, we did not include another impulsivity scale (e.g., Barratt Impulsiveness Scale), nor measures related to the five subscales of the UPPS-P model. Therefore, future studies should seek to test the convergent validity of the UPPS-P. Finally, due to small sample sizes across diagnostic categories, we did not examine measurement invariance across diagnostic categories. However,  this should be considered in future studies in order to assess the stability of the psychometric properties of the S-UPPS-P across psychiatric disorders. On the other hand, the strengths of the current study include the administration of the S-UPPS-P to a large sample of patients in a specific clinical environment (e.g., psychiatric ED), the examination of several psychometric properties (e.g., internal consistency, construct validity, test-retest reliability, and correlations with a substance misuse measure), and the verification of the measurement invariance of the instrument across and sexes.

CONCLUSION
In conclusion, the current study showed that the S-UPPS-P is a valid questionnaire for psychiatric research in general, and for research on unstable psychiatric population (e.g., psychiatric ED). However, researchers should be aware that the S-UPPS-P might not be the scale with the most optimal psychometric properties for evaluating gender differences in sensation seeking.

AUTHOR CONTRIBUTIONS
AD and SP designed the study and wrote the protocol. JD and OP managed the literature searches. C-ÉG undertook the statistical analysis. JD wrote the first draft of the manuscript. All authors contributed to and have approved the final manuscript.