ORIGINAL RESEARCH article
Front. Psychiatry
Sec. Social Psychiatry and Psychiatric Rehabilitation
This article is part of the Research TopicBridging the Gap: Integrating Performance-Based Measures and Person-Reported Outcomes in Disability EvaluationView all 7 articles
The reproducibility of structured functional assessments in a social security setting. A pre-specified explanatory analysis of the RELY-studies
Provisionally accepted
Regina Kunz1*
Stephanie Giezendanner1
David Yoh Von Allmen1
Joerg Jeger2Martin Eichhorn3Ulrike Hoffmann-Richter4Katrin Fischer5Wout De Boer1- 1University of Basel, Dept. Clinical Research, Div Clinical Epidemiology, Basel, Switzerland
- 2MEDAS Central Switzerland, Luzern, Switzerland
- 3Mental Health Practice, Basel, Switzerland
- 4Division of Insurance Medicine, Swiss National Accident Insurance, Luzern, Switzerland
- 5University of Applied Sciences and Arts Northwestern Switzerland, Institute Humans in Complex Systems, Olten, Switzerland
Select one of your emails
You have multiple emails registered with Frontiers:
Notify me on publication
Please enter your email address:
If you already have an account, please login
You don't have a Frontiers account ? You can register here
Introduction: Limitations in work capacity (WC) need to be quantified in a transparent and reproducible way when insurers of social security decide whether an individual is entitled to disability benefits and to what extent. Structured assessments of work-related physical, mental and social functioning might provide an empirical basis for judgments on residual WC (rWC) which determines entitlement to disability benefits. This study examined the functional assessments themselves, their reliability and expert agreement when applied to claimants with mental disorders, and analyzed their relationship to rWC judgments. Methods: We used RELY-data on the reproducibility of rWC judgments. 40 psychiatric experts interviewed 55 claimants for disability benefits. Interviews were videotaped and watched by three observing psychiatric experts, resulting in 280 individual ratings. All independently rated claimants' impairments in work-related mental functions and capacity limitations using the Instrument for Functional Assessment in Psychiatry (IFAP-1 mental functions, IFAP-2a/-2b functional capacities related to the last job and alternative work, scaled 0=none to 4=worst) based on the Mini-ICF-APP, and judged rWC (scaled 100%-0%) for the last job and suitable alternative work. Analysis for reliability (ICC, intraclass-correlation coefficient) included a two-way random-effects and a linear mixed-effects model. Expert agreement was estimated as standard error of measurement, SEM. Results: The mean score for mental functions (IFAP-1global) was 1.21 (SD 0.63), for functional capacities in alternative work (IFAP-2bglobal) 0.87 (SD 0.56). Reliability of IFAP ratings was low to fair (IFAP-1global: ICC=0.46; IFAP-2bglobal: ICC=0.26), similar to the low interrater reliability of rWC. Agreement showed substantial measurement error: IFAP-1global: SEM=0.47; IFAP-2bglobal: SEM=0.49. The rWC judgments for claimants with identical ratings in functional limitations (IFAP-2bglobal=1) ranged from 100% to 5%. Conclusions: Evidence indicates that Functional Assessment, if carried out well, may improve reproducibility. This explanatory analysis revealed low-to-fair interrater reproducibility in mental functions (IFAP-1), in functional capacities (IFAP-2a/b) which extends to rWC. Among various other explanations, we believe this to be mostly due to insufficient training in Functional Assessment and therefore reflects real-world variability in judgment. We recommend revising training format and intensity, monitoring adherence in practice, followed by re-evaluation of reproducibility of expert judgements. As of today, the outcome is uncertain.
Keywords: Disability Evaluation, Work Capacity Evaluation, Reproducibility of Results, Evidence-Based Medicine, Mental Disorders, International Classification of Functioning, Disability and Health, Social Security, disability insurance
Received: 08 Jun 2025; Accepted: 17 Nov 2025.
Copyright: © 2025 Kunz, Giezendanner, Von Allmen, Jeger, Eichhorn, Hoffmann-Richter, Fischer and De Boer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Regina Kunz, regina.kunz@usb.ch
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.