AUTHOR=Ochs Magalie , Bousquet Jérémie , Pergandi Jean-Marie , Blache Philippe 

TITLE=Multimodal Behavioral Cues Analysis of the Sense of Presence and Social Presence During a Social Interaction With a Virtual Patient

JOURNAL=Frontiers in Computer Science

VOLUME=Volume 4 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2022.746804

DOI=10.3389/fcomp.2022.746804

ISSN=2624-9898

ABSTRACT=A key challenge when studying human-agent interaction, is the evaluation of user's experience. In the virtual reality domain, this question is declined through the study of the sense of presence and co-presence, generally assessed thanks to well-grounded subjective post-experience questionnaires. In this article, we aim at correlating objective multimodal cues produced by users to their subjective sense of presence and co-presence. In our study, we started from a corpus of human-agent interaction collected in task-oriented context: a virtual environment aiming at training doctors to break bad news to a patient (played by a virtual agent). Based on a corpus study, we have used machine learning approach to explore the possibility to automatically predict the sense of presence and co-presence of the user thanks to specific multimodal behavioral cues. The performance of random forests models demonstrates the capacity to automatically and accurately predict the level of presence. It also shows the relevance of a multimodal model, based on verbal and non-verbal behavioral cues as objective measures of presence.