Measuring depression severity based on facial expression and body movement using deep convolutional neural network - Liu, Dongdong; Liu, Bowen; Lin, Tao; Liu, Guangya; Yang, Guoyu; Qi, Dezhen; Qiu, Ye; Lu, Yuer; Yuan, Qinmei; Shuai, Stella C.; Li, Xiang; Liu, Ou; et al. (Frontiers in Psychiatry, 2022)
There is a need for real-time evaluate of the severity of depressive symptoms in major depressive disorder (MDD). This study aimed to develop a measurement of depression severity from expression and action features, and to assess its validity among the patients with MDD. Prospective and qualitative research methods were used. A multi-modal deep convolutional neural network (CNN) was used to measure the severity of depressive symptoms, which was based on the recognition of patients’ facial expression and body movement. We established the behavioral depression degree (BDD) to measure the severity of depressive symptoms of MDD patients and compared our proposed BDD with clinically validated scales to validate BDD’s effectiveness. It demonstrated an over 74% Pearson similarity between BDD and self-rating depression scale (SDS), self-rating anxiety scale (SAS), and Hamilton depression scale (HAMD). The BDD can effectively measure the current state of patients' depression and its changing trend according to the patient’s expression and action features. We can automate the measurement and real-time evaluate the psychological changes of MDD patients to guide the treatment.