The Power of Combined Modalities in Interactive Robot Learning

Helen Beierling^1,2*Robin Helmert^1,2Anna-Lisa Vollmer^1,2

• ¹Bielefeld University, Bielefeld, Germany
• ²Medical School OWL, Bielefeld University, Bielefeld, North Rhine-Westphalia, Germany

With the continuous advancement of Artificial intelligence (AI), robots as embodied intelligentsystems are increasingly becoming more present in daily life like households or in elderly care.As a result, lay users are required to interact with these systems more frequently and teach themto meet individual needs. Human-in-the-loop reinforcement learning (HIL-RL) offers an effectiveway to realize this teaching. Studies show that various feedback modalities, such as preference,guidance, or demonstration can significantly enhance learning success, though their suitabilityvaries among users expertise in robotics. Research also indicates that users apply differentscaffolding strategies when teaching a robot, such as motivating it to explore actions that promisesuccess.Thus, providing a collection of different feedback modalities allows users to choose the methodthat best suits their teaching strategy, and allows the system to individually support the userbased on their interaction behavior. However, most state-of-the-art approaches provide users withonly one feedback modality at a time. Investigating combined feedback modalities in interactiverobot learning remains an open challenge. To address this, we conducted a study that combinedcommon feedback modalities. Our research questions focused on whether these combinationsimprove learning outcomes, reveal user preferences, show differences in perceived effectiveness,and identify which modalities influence learning the most. The results show that combining thefeedback modalities improves learning, with users perceiving the effectiveness of the modalitiesvary ways, and certain modalities directly impacting learning success. The study demonstratesthat combining feedback modalities can support learning even in a simplified setting and suggeststhe potential for broader applicability, especially in robot learning scenarios with a focus onuser interaction. Thus, this paper aims to motivate the use of combined feedback modalities ininteractive imitation learning.