AUTHOR=Shoa Alon , Friedman Doron 

TITLE=Milo: an LLM-based virtual human open-source platform for extended reality

JOURNAL=Frontiers in Virtual Reality

VOLUME=Volume 6 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/virtual-reality/articles/10.3389/frvir.2025.1555173

DOI=10.3389/frvir.2025.1555173

ISSN=2673-4192

ABSTRACT=Large language models (LLMs) have made dramatic advancements in recent years, allowing for a new generation of dialogue agents. This allows for new types of social experiences with virtual humans, in both virtual and augmented reality. In this paper, we introduce an open-source system specifically designed for implementing LLM-based virtual humans within extended reality (XR) environments. Our system integrates into XR platforms, providing a robust framework for the creation and management of interactive virtual agents. We detail the design and architecture of the system and showcase the system’s versatility through various scenarios. In addition to a straightforward single-agent setup, we demonstrate how an LLM-based virtual human can attend a multi-user virtual reality (VR) meeting, enhance a VR self-talk session, and take part in an augmented reality (AR) live event. We provide lessons learned, with focus on the possibilities for human intervention during live events. We provide the system as open-source, inviting collaboration and innovation within the community, paving the way for new types of social experiences.