Frontiers in Psychiatry is proud to present our Psychometrics series. This series of Article Collections will deal with the definition and measurement of clinical constructs. The Psychometrics in Autism Article Collection aims to collect original research and review articles on new developments and applications of psychometrics and psychiatric assessment in Autism. Articles can include, but are not limited to, the following topics: Clinical psychometrics The challenge of quantitative psychiatry and measurement Application, validity, and reliability of psychiatric scales Measuring and modeling

University of Salento

Stella Maris Foundation (IRCCS)

Örebro University

Hospitaller Brothers of Saint John of God Linz

University of Education Upper Austria

Exploring advancements in psychiatric scales for autism measurement.

. Research into the psychosocial aspects of neurodevelopmental disorders has yielded a number of valuable insights: <ul> <li>The Mock Interview Rating Scale adapted for research with autistic transition-age youth has strong internal consistency, inter-rater reliability, and test–retest reliability, as well as initial support for its construct, divergent, and predictive validities.</li> <li>The Assessment of Concerning Behavior (ACB) is a reliable and valid tool to assess mental health and problematic/risky behaviors in Iranian children and young people with ASD.</li> <li>Auditory problems are the most distressing sensory issues experienced by individuals with neurodevelopmental disorders.</li> <li>The Daily Living Skills Scale (DLSS) is a new, relatively brief (53-item) open-source informant-reported instrument for capturing key aspects of daily living skills (DLS) in individuals with autism and other neurodevelopmental conditions.</li> </ul>

Frontiers in Psychiatry

Autism

Psychometrics in Psychiatry 2022:  Autism

IntroductionThe precise epidemiological burden of autism is unknown because of the limited capacity to identify and diagnose the disorder in resource-constrained settings, related in part to a lack of appropriate standardised assessment tools and health care experts. We assessed the reliability, validity, and diagnostic accuracy of the Developmental Diagnostic Dimensional Interview (3Di) in a rural setting on the Kenyan coast.</sec>MethodsUsing a large community survey of neurodevelopmental disorders (NDDs), we administered the 3Di to 2,110 children aged between 6 years and 9 years who screened positive or negative for any NDD and selected 242 who had specific symptoms suggestive of autism based on parental report and the screening tools for review by a child and adolescent psychiatrist. On the basis of recorded video, a multi-disciplinary team applied the Autism Diagnostic Observation Schedule to establish an autism diagnosis. Internal consistency was used to examine the reliability of the Swahili version of the 3Di, tetrachoric correlations to determine criterion validity, structural equation modelling to evaluate factorial structure and receiver operating characteristic analysis to calculate diagnostic accuracy against Diagnostic Statistical Manual of Mental Disorders (DSM) diagnosis.</sec>ResultsThe reliability coefficients for 3Di were excellent for the entire scale {McDonald’s omega (ω) = 0.83 [95% confidence interval (CI) 0.79–0.91]}. A higher-order three-factor DSM-IV-TR model showed an adequate fit with the model, improving greatly after retaining high-loading items and correlated items. A higher-order two-factor DSM-5 model also showed an adequate fit. There were weak to satisfactory criterion validity scores [tetrachoric rho = 0.38 (p = 0.049) and 0.59 (p = 0.014)] and good diagnostic accuracy metrics [area under the curve = 0.75 (95% CI: 0.54–0.96) and 0.61 (95% CI: 0.49–0.73] for 3Di against the DSM criteria. The 3Di had a moderate sensitivity [66.7% (95% CI: 0.22–0.96)] and a good specificity [82.5% (95% CI: 0.74–0.89)], when compared with the DSM-5. However, we observed poor sensitivity [38.9% (95% CI: 0.17–0.64)] and good specificity [83.5% (95% CI: 0.74–0.91)] against DSM-IV-TR.</sec>ConclusionThe Swahili version of the 3Di provides information on autism traits, which may be helpful for descriptive research of endophenotypes, for instance. However, for accuracy in newly diagnosed autism, it should be complemented by other tools, e.g., observational clinical judgment using the DSM criteria or assessments such as the Autism Diagnostic Observation Schedule. The construct validity of the Swahili 3Di for some domains, e.g., communication, should be explored in future studies.</sec>

Introduction: The precise epidemiological burden of autism is unknown because of the limited capacity to identify and diagnose the disorder in resource-constrained settings, related in part to a lack of appropriate standardised assessment tools and health care experts. We assessed the reliability, validity, and diagnostic accuracy of the Developmental Diagnostic Dimensional Interview (3Di) in a rural setting on the Kenyan coast.Methods: Using a large community survey of neurodevelopmental disorders (NDDs), we administered the 3Di to 2,110 children aged between 6 years and 9 years who screened positive or negative for any NDD and selected 242 who had specific symptoms suggestive of autism based on parental report and the screening tools for review by a child and adolescent psychiatrist. On the basis of recorded video, a multi-disciplinary team applied the Autism Diagnostic Observation Schedule to establish an autism diagnosis. Internal consistency was used to examine the reliability of the Swahili version of the 3Di, tetrachoric correlations to determine criterion validity, structural equation modelling to evaluate factorial structure and receiver operating characteristic analysis to calculate diagnostic accuracy against Diagnostic Statistical Manual of Mental Disorders (DSM) diagnosis.Results: The reliability coefficients for 3Di were excellent for the entire scale {McDonald’s omega (ω) = 0.83 [95% confidence interval (CI) 0.79–0.91]}. A higher-order three-factor DSM-IV-TR model showed an adequate fit with the model, improving greatly after retaining high-loading items and correlated items. A higher-order two-factor DSM-5 model also showed an adequate fit. There were weak to satisfactory criterion validity scores [tetrachoric rho = 0.38 (p = 0.049) and 0.59 (p = 0.014)] and good diagnostic accuracy metrics [area under the curve = 0.75 (95% CI: 0.54–0.96) and 0.61 (95% CI: 0.49–0.73] for 3Di against the DSM criteria. The 3Di had a moderate sensitivity [66.7% (95% CI: 0.22–0.96)] and a good specificity [82.5% (95% CI: 0.74–0.89)], when compared with the DSM-5. However, we observed poor sensitivity [38.9% (95% CI: 0.17–0.64)] and good specificity [83.5% (95% CI: 0.74–0.91)] against DSM-IV-TR.Conclusion: The Swahili version of the 3Di provides information on autism traits, which may be helpful for descriptive research of endophenotypes, for instance. However, for accuracy in newly diagnosed autism, it should be complemented by other tools, e.g., observational clinical judgment using the DSM criteria or assessments such as the Autism Diagnostic Observation Schedule. The construct validity of the Swahili 3Di for some domains, e.g., communication, should be explored in future studies.

Utility of the 3Di short version in the identification and diagnosis of autism in children at the Kenyan coast

BackgroundEmployment is a major contributor to quality of life. However, autistic people are often unemployed and underemployed. One potential barrier to employment is the job interview. However, the availability of psychometrically-evaluated assessments of job interviewing skills is limited for autism services providers and researchers.</sec>ObjectiveWe analyzed the psychometric properties of the Mock Interview Rating Scale that was adapted for research with autistic transition-age youth (A-MIRS; a comprehensive assessment of video-recorded job interview role-play scenarios using anchor-based ratings for 14 scripted job scenarios).</sec>MethodsEighty-five transition-age youth with autism completed one of two randomized controlled trials to test the effectiveness of two interventions focused on job interview skills. All participants completed a single job interview role-play at pre-test that was scored by raters using the A-MIRS. We analyzed the structure of the A-MIRS using classical test theory, which involved conducting both exploratory and confirmatory factor analyzes, Rasch model analysis and calibration techniques. We then assessed internal consistency, inter-rater reliability, and test–retest reliability. Pearson correlations were used to assess the A-MIRS’ construct, convergent, divergent, criterion, and predictive validities by comparing it to demographic, clinical, cognitive, work history measures, and employment outcomes.</sec>ResultsResults revealed an 11-item unidimensional construct with strong internal consistency, inter-rater reliability, and test–retest reliability. Construct [pragmatic social skills (r = 0.61, p < 0.001), self-reported interview skills (r = 0.34, p = 0.001)], divergent [e.g., age (r = −0.13, p = 0.26), race (r = 0.02, p = 0.87)], and predictive validities [competitive employment (r = 0.31, p = 0.03)] received initial support via study correlations, while convergent [e.g., intrinsic motivation (r = 0.32, p = 0.007), job interview anxiety (r = −0.19, p = 0.08)] and criterion [e.g., prior employment (r = 0.22, p = 0.046), current employment (r = 0.21, p = 0.054)] validities were limited.</sec>ConclusionThe psychometric properties of the 11-item A-MIRS ranged from strong-to-acceptable, indicating it may have utility as a reliable and valid method for assessing the job interview skills of autistic transition-age youth.</sec>

Background: Employment is a major contributor to quality of life. However, autistic people are often unemployed and underemployed. One potential barrier to employment is the job interview. However, the availability of psychometrically-evaluated assessments of job interviewing skills is limited for autism services providers and researchers.Objective: We analyzed the psychometric properties of the Mock Interview Rating Scale that was adapted for research with autistic transition-age youth (A-MIRS; a comprehensive assessment of video-recorded job interview role-play scenarios using anchor-based ratings for 14 scripted job scenarios).Methods: Eighty-five transition-age youth with autism completed one of two randomized controlled trials to test the effectiveness of two interventions focused on job interview skills. All participants completed a single job interview role-play at pre-test that was scored by raters using the A-MIRS. We analyzed the structure of the A-MIRS using classical test theory, which involved conducting both exploratory and confirmatory factor analyzes, Rasch model analysis and calibration techniques. We then assessed internal consistency, inter-rater reliability, and test–retest reliability. Pearson correlations were used to assess the A-MIRS’ construct, convergent, divergent, criterion, and predictive validities by comparing it to demographic, clinical, cognitive, work history measures, and employment outcomes.Results: Results revealed an 11-item unidimensional construct with strong internal consistency, inter-rater reliability, and test–retest reliability. Construct [pragmatic social skills (r = 0.61, p &lt; 0.001), self-reported interview skills (r = 0.34, p = 0.001)], divergent [e.g., age (r = −0.13, p = 0.26), race (r = 0.02, p = 0.87)], and predictive validities [competitive employment (r = 0.31, p = 0.03)] received initial support via study correlations, while convergent [e.g., intrinsic motivation (r = 0.32, p = 0.007), job interview anxiety (r = −0.19, p = 0.08)] and criterion [e.g., prior employment (r = 0.22, p = 0.046), current employment (r = 0.21, p = 0.054)] validities were limited.Conclusion: The psychometric properties of the 11-item A-MIRS ranged from strong-to-acceptable, indicating it may have utility as a reliable and valid method for assessing the job interview skills of autistic transition-age youth.

Psychometric properties of the mock interview rating scale for autistic transition-age youth

IntroductionAssessment of Concerning Behavior (ACB) was introduced by Tarver et al. (2021) to evaluate mental health and problematic/risky behaviors in children and young people with autism spectrum disorder (ASD).</sec>MethodsThis study examined the psychometric validation of the Assessment of Concerning Behavior (ACB) in an Iranian sample of parents of children and young people with ASD. Confirmatory factor analysis was conducted to examine the structure of ACB in a sample of 303 parents.</sec>ResultsThe data supported the two factor structure, all factor loadings were significant and scale structure was confirmed similar to the original scale. The results supported the two-factor structure for ACB that included internalizing and externalizing problems scales. The two factors of ACB are positively correlated with Aberrant Behavior Checklist scores which showed that the validity of two factors is satisfactory. The reliability of the two subscales was reasonable as well.</sec>Conclusion:The study suggests that the ACB could be an operational tool to assess the mental health and problematic/risky behaviors in Iranian children and young people with ASD.</sec>

Introduction: Assessment of Concerning Behavior (ACB) was introduced by Tarver et al. (2021) to evaluate mental health and problematic/risky behaviors in children and young people with autism spectrum disorder (ASD).Methods: This study examined the psychometric validation of the Assessment of Concerning Behavior (ACB) in an Iranian sample of parents of children and young people with ASD. Confirmatory factor analysis was conducted to examine the structure of ACB in a sample of 303 parents.Results: The data supported the two factor structure, all factor loadings were significant and scale structure was confirmed similar to the original scale. The results supported the two-factor structure for ACB that included internalizing and externalizing problems scales. The two factors of ACB are positively correlated with Aberrant Behavior Checklist scores which showed that the validity of two factors is satisfactory. The reliability of the two subscales was reasonable as well.Conclusion:: The study suggests that the ACB could be an operational tool to assess the mental health and problematic/risky behaviors in Iranian children and young people with ASD.

Psychometric validation of concerning behavior scale in Iranian children and young people with autism spectrum disorder

IntroductionIndividuals with neurodevelopmental disorders, such as autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), and specific learning disorders (SLD) have various types of sensory characteristics.</sec>MethodsThis study investigated sensory issues in individuals with neurodevelopmental disorders using a web-based questionnaire for qualitative and quantitative analysis, categorized the contents of their three most distressful sensory issues, and evaluated their order of priority.</sec>ResultsAuditory problems were reported as the most distressing sensory issue among the participants. In addition to auditory problems, individuals with ASD frequently reported more tactile problems, and individuals with SLD reported more visual problems. Among the individual sensory issues, in addition to aversion to sudden, strong, or specific stimuli, some participants reported confusions regarding multiple stimuli presenting concurrently. Additionally, the sensory issues related to foods (i.e., taste) was relatively more common in the minor group.</sec>ConclusionThese results suggest that the diversity of sensory issues experienced should be carefully considered when aiding persons with neurodevelopmental disorders.</sec>

Introduction: Individuals with neurodevelopmental disorders, such as autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), and specific learning disorders (SLD) have various types of sensory characteristics.Methods: This study investigated sensory issues in individuals with neurodevelopmental disorders using a web-based questionnaire for qualitative and quantitative analysis, categorized the contents of their three most distressful sensory issues, and evaluated their order of priority.Results: Auditory problems were reported as the most distressing sensory issue among the participants. In addition to auditory problems, individuals with ASD frequently reported more tactile problems, and individuals with SLD reported more visual problems. Among the individual sensory issues, in addition to aversion to sudden, strong, or specific stimuli, some participants reported confusions regarding multiple stimuli presenting concurrently. Additionally, the sensory issues related to foods (i.e., taste) was relatively more common in the minor group.Conclusion: These results suggest that the diversity of sensory issues experienced should be carefully considered when aiding persons with neurodevelopmental disorders.

Qualitative and quantitative analysis of self-reported sensory issues in individuals with neurodevelopmental disorders

Autistic individuals and individuals with a range of other neurodevelopmental conditions (NDD) often present with lower levels of daily living skills (DLS) when compared to their neurotypical peers. Importantly, lower levels of DLS have been linked to a range of negative outcomes, including lower rates of post-secondary education, lower employment rates, and higher daily support needs across autism and NDD. However, there are currently no open-source informant-reported instruments for capturing key aspects of DLS. This study describes the development, refinement, and initial psychometric evaluation of a new, relatively brief (53-item). Daily Living Skills Scale (DLSS) in a sample of 1,361 children aged 2–17 years, Confirmatory Factor Analysis demonstrated an excellent fit of unidimensional model to the data (CFI = 0.953, TLI = 0.951, RMSEA = 0.073 [95% CI: 0.071–0.074]). The single-factor CFA model showed evidence of measurement invariance of factor loadings, thresholds, and residual variance (strict invariance) across sex, age, race, and ethnicity. Model reliability and internal consistency were excellent (ω = 0.98; α = 0.97). Conditional reliability estimates indicated very good reliability (= 0.80) for the total DLS scale from very low (θ = −4.2) to high (θ = +2.4) scores. Conceptually derived self-care, homecare, and community participation subscales also showed strong reliability and internal consistency. With further replication, the EFS has excellent potential for wide adoption across research and clinical contexts.

Autistic individuals and individuals with a range of other neurodevelopmental conditions (NDD) often present with lower levels of daily living skills (DLS) when compared to their neurotypical peers. Importantly, lower levels of DLS have been linked to a range of negative outcomes, including lower rates of post-secondary education, lower employment rates, and higher daily support needs across autism and NDD. However, there are currently no open-source informant-reported instruments for capturing key aspects of DLS. This study describes the development, refinement, and initial psychometric evaluation of a new, relatively brief (53-item). Daily Living Skills Scale (DLSS) in a sample of 1,361 children aged 2&#x2013;17 years, Confirmatory Factor Analysis demonstrated an excellent fit of unidimensional model to the data (CFI = 0.953, TLI = 0.951, RMSEA = 0.073 [95% CI: 0.071&#x2013;0.074]). The single-factor CFA model showed evidence of measurement invariance of factor loadings, thresholds, and residual variance (strict invariance) across sex, age, race, and ethnicity. Model reliability and internal consistency were excellent (&#x3c9; = 0.98; &#x3b1; = 0.97). Conditional reliability estimates indicated very good reliability (= 0.80) for the total DLS scale from very low (&#x3b8; = &#x2212;4.2) to high (&#x3b8; = +2.4) scores. Conceptually derived self-care, homecare, and community participation subscales also showed strong reliability and internal consistency. With further replication, the EFS has excellent potential for wide adoption across research and clinical contexts.