Beyond Game Engines in Extended Reality

In the past ten years, there has been a very significant uptick in the amount of software development undertaken to build the wide variety of Extended Reality (XR) applications that are now available. Whereas previously, academic research had focussed on various community-run and niche commercial products; in the past decade tools that were originally built as game engines have come to dominate. While game engines have massively improved accessibility to application development by providing robust tools and significantly enhanced asset and code repositories to draw upon, game engines come with limitations.

These include:

• Limited Customization: Game engines often impose rigid frameworks that hinder the flexibility needed for specialized applications or novel interactions.
• Performance Trade-offs: Optimizations geared toward specific use cases, such as gaming, can introduce inefficiencies or bottlenecks in non-gaming contexts.
• Black-Box Architectures: Proprietary systems may obscure underlying mechanisms, limiting developers' ability to deeply integrate or modify features. Specifically, access to modern hardware is more complex.
• Generalization Issues: Features are often designed for broad applicability, making it challenging to meet the unique demands of cutting-edge research or highly specialized applications.
• Specialization Issues: At the same time, the display/interaction environment for game engines is usually a lot more homogeneous than academic VR requires, making it hard to support new interaction and display paradigms.
• Licensing Restrictions: Proprietary licensing models can complicate deployment in open or academic environments.
• Execution Models: Engines typically do not address, as a primary feature, the need for multi-process or distributed computing or rendering. Thus certain data-heavy applications, or rendering for high-end displays is not easily supported.

Thus while game engines are indeed good for writing games, including XR games, there are features and capabilities that the XR community either has to implement themselves or rely on 3rd party libraries to support. Indeed there are still active projects maintaining XR software for highly specialized applications such as advanced visualizations and cluster rendering.

This article collection seeks to explore XR software “beyond the game engine”. We welcome original contributions that document best practices, propose novel toolkits or architectures, or present rigorous evaluations of trade-offs of using different engines for XR development.

Topics of Interest Include (but are not limited to):

• Architectures: Support for time-critical and multi-process requirements for more complex XR software, etc.
• Abstraction Mechanisms: Entity-centered design for XR, world descriptions, or semantic modeling.
• Language and Framework Support: Class libraries, scripting, core languages, and declarative solutions for XR.
• System Considerations: Addressing operating systems, driver abstractions, portability, networking, and distribution.
• Behavior: Integration of components such as physics and AI.
• Implementation & Testing: Techniques for developing and validating XR software.
• Performance: Strategies for evaluation, latency management, and synchronization.

We invite researchers and developers to submit original articles, reviews, and case studies that align with the scope of this Research Topic. Submissions should offer substantial contributions to the understanding, design, or evaluation of XR systems and highlight their contribution beyond the state of the art of game engine technology.


Topic Editor Dirk Reiners received financial support from Lockheed Martin, Universal, and Buendea. The other Topic Editors declare no competing interests with regard to the Research Topic subject