ORIGINAL RESEARCH article
Front. Virtual Real.
Sec. Augmented Reality
Volume 6 - 2025 | doi: 10.3389/frvir.2025.1574965
Exploring AR Hand Augmentations as Error Feedback Mechanisms for Enhancing Gesture-based Tutorials
Provisionally accepted
- 1Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States
- 2INESC-ID, Instituto Superior Técnico, University of Lisbon, Lisbon, Portugal
- 3University of Rochester, Rochester, New York, United States
- 4University of Toronto, Toronto, Ontario, Canada
Select one of your emails
You have multiple emails registered with Frontiers:
Notify me on publication
Please enter your email address:
If you already have an account, please login
You don't have a Frontiers account ? You can register here
Self-guided tutorials from videos help users learn new skills and complete tasks with varying complexity, from repairing a gadget to learning how to play an instrument. However, users may struggle to interpret 3D movements and gestures from 2D representations due to different viewpoints, occlusions, and depth perception. Augmented Reality (AR) can alleviate this challenge by enabling users to view complex instructions in their 3D space. However, most approaches only provide feedback if a live expert is present and do not consider self-guided tutorials. Our work explores virtual hand augmentations as automatic feedback mechanisms to enhance selfguided, gesture-based AR tutorials. We evaluated different error feedback designs and hand placement strategies on speed, accuracy and preference in a user study with 18 participants. Specifically, we investigate two visual feedback styles -color feedback, which changes the color of the hands' joints to signal pose correctness, and shape feedback, which exaggerates fingers length to guide correction -as well as two placement strategies: superimposed, where the feedback hand overlaps the user's own, and adjacent, where it appears beside the user's hand. Results show significantly faster replication time when users are provided with color or baseline no explicit feedback, when compared to shape manipulation feedback. Furthermore, despite users' preferences for adjacent placement for the feedback representation, superimposed placement significantly reduces replication time. We found no effects on accuracy for short-time recall, suggesting that while these factors may influence task efficiency, they may not strongly affect overall task proficiency.
Keywords: Tutorials, training, augmented reality, hand gestures, error feedback, Virtual Hand Augmentations
Received: 11 Feb 2025; Accepted: 12 May 2025.
Copyright: © 2025 Gonçalves Fidalgo, Yan, Sousa, Jorge and Lindlbauer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Catarina Gonçalves Fidalgo, Human-Computer Interaction Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, Pennsylvania, United States
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.