Motion-Based User Identification across XR and Metaverse Applications by Deep Classification and Similarity Learning

Lukas Schach^1*Christian Rack¹Ryan P. McMahan²Marc Erich Latoschik¹

• ¹Human-Computer Interaction Group, Julius-Maximilians-Universität, Würzburg, Germany
• ²Department of Computer Science, Virginia Tech, Blacksburg, United States

This paper examines the generalization capacity of two state-of-the-art classification and similarity learning models in reliably identifying users from their motion patterns across diverse eXtended Reality (XR) applications. We introduce a novel dataset comprising motion data from 49 users in five XR applications: four XR games with distinct task and action profiles, and one social XR application without predefined tasks. Using this dataset, we evaluate both models' identification performance and, in particular, their ability to generalize across applications. Our results show that while the models can accurately identify individuals within the same application, their cross-application performance remains limited. Accordingly, recent approaches to biometric motion-based verification and identification exhibit low generalization capacity. While the results suggest that current risks of unintended or privacy-critical user identification in XR and Metaverse contexts are limited, they also indicate that these risks are likely to grow rapidly as model generalization improves. To support reproducibility and encourage further research on motion-based user identification in typical Metaverse use cases, we release our cross-application XR motion dataset and accompanying code publicly.