AUTHOR=Chanda Sushovan , Fitwe Kedar , Deshpande Gauri , Schuller Björn W. , Patel Sachin 

TITLE=A Deep Audiovisual Approach for Human Confidence Classification

JOURNAL=Frontiers in Computer Science

VOLUME=Volume 3 - 2021

YEAR=2021

URL=https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2021.674533

DOI=10.3389/fcomp.2021.674533

ISSN=2624-9898

ABSTRACT=Research on self-efficacy and confidence has spread across several sub-fields of psychology and neuroscience. The role of one’s confidence is very crucial in the formation of attitudes and communication skills. The importance of differentiating the levels of confidence is quite visible in this domain. With the recent advances in extracting behavioral insight from a signal in multiple applications, detecting confidence is found to have great importance. One such prominent application is detecting confidence in interview conversations. We have collected an audio-visual dataset of interview conversations with 33 candidates. Every response (from each of the candidates) of this dataset is labeled with three levels of confidence, high, medium, and low. Further, we have also developed algorithms to efficiently compute such behavioral confidence from speech and video. A deep learning architecture is proposed for detecting confidence levels(high, medium, and low) from an audiovisual clip recorded during an interview. The achieved UnweightedAverage Recall (UAR) reaches 85.9 % on audio data, and 73.6 % on video data captured from an interview session.