Latent Trait or Sum Score: Addressing Measurement Challenges in the Prediction of Self-Rated Symptom Outcomes in Psychological Treatment

Nils Hentati Isacsson^1,2*Magnus Johansson^1,2,3Viktor Kaldo^1,2,4

• ¹Center for Psychiatry Research, Stockholm, Sweden
• ²Karolinska Institutet, Stockholm, Sweden
• ³RISE Research Institutes of Sweden AB, Gothenburg, Sweden
• ⁴Linneuniversitet Fakulteten for halso och livsvetenskap, Kalmar, Sweden

Objective: Reliable and accurate measurement is fundamental to scientific progress, yet dominant measurement practices in psychology, clinical psychology, and in prediction research often lack rigor. Improving measures using Rasch Measurement Theory (RMT) offers advantages by the psychometric properties of: unidimensionality, local independence of items, ordering of response categories, and invariance. The ordinal level sum scores can be transformed into interval level latent trait scores improving measurement precision. However, the impact of using psychometrically superior questionnaires with the latent trait scores versus traditional sum scores in predictive models is unclear. This study evaluates whether using latent trait scores as predictors and outcomes, improves predictive performance compared to using traditional sum scores for predicting treatment outcome in psychological treatment. Methods: Self-rated symptom data from three different questionnaires, collected during the first four weeks of psychological treatment from 6,464 patients undergoing a 12-week treatment program, were used to predict post-treatment outcomes on the same questionnaires. This was done either 1) using sum scores as the questionnaires were originally developed or 2) using a more psychometrically robust version of the questionnaires based on Rasch analysis, which were also shorter. Prediction models used were Linear Regression, Bayesian Ridge Regression, and Random Forest. Multiple imputation was used to address missing data, and nested cross-validation was used for hyperparameter tuning and scoring. Results: Latent scores calculated using the psychometrically optimized, shorter version, comprising 23% of the full scale, showed similar predictive performance compared with the sum score of the full scale. Overall, there was a statistically significant but practically negligible difference of 0.007-0.008 Root Mean Squared Error between using the original sum score compared to latent trait scores. Conclusion: Initial findings comparing psychometrically improved questionnaires to original ordinal sum scores in a predictive framework indicate that using latent trait scores from this improvement showed similar predictive performance as the sum score of the full scale. This suggests that the improved versions are valuable by their improved psychometric qualities and reduced response burden by using considerably fewer items. Further research is needed to explore the uses of latent trait scores compared to ordinal sum scores in predictive research.