Depression diagnosis from patient interviews using multimodal machine learning

Jana WeberMarcel WeberJuan Miguel Lopez Alcaraz^*

• Carl von Ossietzky Universitat Oldenburg, Oldenburg, Germany

Background Depression is a major public health concern, affecting an estimated five percent of the global population. Early and accurate diagnosis is essential to initiate effective treatment, yet recognition remains challenging in many clinical con-texts. Speech, language, and behavioral cues collected during patient interviews may provide objective markers that support clinical assessment. Methods We developed a diagnostic approach that integrates features derived from patient interviews, including speech patterns, linguistic characteristics, and structured clinical information. Separate models were trained for each modality and subsequently combined through multimodal fusion to reflect the complexity of real-world psychiatric assessment. Model validity was assessed with established performance metrics, and further evaluated using calibration and decision-analytic approaches to estimate potential clinical utility. Results The multimodal model achieved superior diagnostic accuracy compared to single-modality models, with an AUROC of 0.88 and a macro F1-score of 0.75. Importantly, the fused model demonstrated good calibration and offered higher net clinical benefit compared to baseline strategies, highlighting its potential to assist clinicians in identifying patients with depression more reliably. Conclusion Multimodal analysis of patient interviews using machine learning may serve as a valuable adjunct to psychiatric evaluation. By combining speech, language, and clinical features, this approach provides a robust framework that could enhance early detection of depressive disorders and support evidence-based decision-making in mental healthcare.