AUTHOR=Liu Dongdong , Liu Bowen , Lin Tao , Liu Guangya , Yang Guoyu , Qi Dezhen , Qiu Ye , Lu Yuer , Yuan Qinmei , Shuai Stella C. , Li Xiang , Liu Ou , Tang Xiangdong , Shuai Jianwei , Cao Yuping , Lin Hai 

TITLE=Measuring depression severity based on facial expression and body movement using deep convolutional neural network

JOURNAL=Frontiers in Psychiatry

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/psychiatry/articles/10.3389/fpsyt.2022.1017064

DOI=10.3389/fpsyt.2022.1017064

ISSN=1664-0640

ABSTRACT=There is a need for real-time evaluate of the severity of depressive symptoms in major depressive disorder (MDD). This study aimed to develop a measurement of depression severity from expression and action features, and to assess its validity among the patients with MDD. Prospective and qualitative research methods were used. A multi-modal deep convolutional neural network (CNN) was used to measure the severity of depressive symptoms, which was based on the recognition of patients’ facial expression and body movement. We established the behavioral depression degree (BDD) to measure the severity of depressive symptoms of MDD patients and compared our proposed BDD with clinically validated scales to validate BDD’s effectiveness. It demonstrated an over 74% Pearson similarity between BDD and self-rating depression scale (SDS), self-rating anxiety scale (SAS), and Hamilton depression scale (HAMD). The BDD can effectively measure the current state of patients' depression and its changing trend according to the patient’s expression and action features. We can automate the measurement and real-time evaluate the psychological changes of MDD patients to guide the treatment.