AUTHOR=Liu Ying , Wang Zixuan , Yu Ge 

TITLE=The Effectiveness of Facial Expression Recognition in Detecting Emotional Responses to Sound Interventions in Older Adults With Dementia

JOURNAL=Frontiers in Psychology

VOLUME=Volume 12 - 2021

YEAR=2021

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2021.707809

DOI=10.3389/fpsyg.2021.707809

ISSN=1664-1078

ABSTRACT=This research uses facial expression recognition software (FaceReader) to explore the influence of different sound interventions on the emotions of elderly people with dementia. The field experiment was carried out in the public activity space of an elderly care facility. Three intervention sound sources were used: music, stream, and birdsong. Data collected through the Self-Assessment Manikin Scale (SAM) were compared with facial expression recognition (FER) data. FaceReader was found to be able to identify differences in the emotional responses of elderly people with dementia to different sound interventions, and able to reveal changes in facial expressions over time. Participants’ facial expressions had significantly higher valence in all three sound interventions than in the intervention without sound (p<0.01). The indexes of sadness, fear, and disgust differed significantly between the different sound interventions. For example, in the birdsong intervention, the disgust index initially increased by 0.06 from 0 s to about 20 s, before then showing a linear downward trend, with an average reduction of 0.03 per 20 s. In addition, valence and arousal were significantly lower when the sound intervention began before, rather than concurrently with, the start of the activity (p<0.01). Moreover, in the birdsong and stream interventions, there were significant differences between different intervention days (p<0.05 or p<0.01). Furthermore, facial expression valence significantly differed by age and between genders. Finally, comparison of the SAM and FER results showed that in the music intervention, the valence in the first 80 s predicts dominance (r=0.600) and acoustic comfort (r=0.545); in the stream sound intervention, the first 40 s predicts pleasure (r=0.770) and acoustic comfort (r=0.766); for the birdsong intervention, the first 20 s reflects dominance (r=0.824) and arousal (r=0.891).