ORIGINAL RESEARCH article

Front. Virtual Real., 05 January 2026

Sec. Virtual Reality in Medicine

Volume 6 - 2025 | https://doi.org/10.3389/frvir.2025.1666614

From screen to space: evaluating Siemens’ Cinematic Reality application for medical imaging on the Apple Vision Pro

  • 1. Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital (AöR), University of Duisburg-Essen, Essen, Germany

  • 2. Center for Virtual and Extended Reality in Medicine (ZvRM), University Hospital Essen (AöR), Essen, Germany

  • 3. Institute of Computer Graphics and Vision (ICG), Graz University of Technology, Graz, Austria

  • 4. Siemens Healthineers, Forchheim, Germany

  • 5. Department of General-, Visceral- and Transplant Surgery, Medical Center University Duisburg-Essen, Essen, Germany

  • 6. Pediatric Clinic II, University Children’s Hospital Essen, University Duisburg-Essen, Essen, Germany

  • 7. Medical Faculty, University of Duisburg-Essen, Essen, Germany

  • 8. Cancer Research Center Cologne Essen (CCCE), West German Cancer Center, University Hospital Essen (AöR), Essen, Germany

  • 9. German Cancer Consortium (DKTK), Partner site University Hospital Essen (AöR), Essen, Germany

  • 10. Department of Physics, Technical University Dortmund, Dortmund, Germany

  • 11. Faculty of Computer Science, University of Duisburg-Essen, Essen, Germany

  • 12. Department of Surgery, Maastricht University Medical Centre+, Maastricht, Netherlands

Abstract

Introduction:

As one of the first research teams with full access to Siemens’ Cinematic Reality, we evaluated its usability and clinical potential for cinematic volume rendering on the Apple Vision Pro.

Methods:

We visualized venous-phase liver computed tomography and magnetic resonance cholangiopancreatography scans from the CHAOS and MRCP_DLRecon public datasets, respectively. Fourteen medical experts assessed usability and anticipated clinical integration potential using standardized questionnaires (System Usability Scale and ISONORM 9242-110-S) and an open-ended survey. Our primary aim was not to validate direct clinical outcomes, but to evaluate the usability of Siemens’ Cinematic Reality on the Apple Vision Pro and gather expert feedback on potential use cases and missing features required for clinical adoption beyond educational purposes.

Results:

Their feedback identified feasibility, key usability strengths, and required features to catalyze the adaptation in real-world clinical workflows.

Conclusion:

The findings provide insights into the potential of immersive cinematic rendering in medical imaging and the needed features for clinical adoption as suggested by the medical experts. Siemens Cinematic Reality running on the Apple Vision Pro was deemed to have good usability, making it a promising tool.

1 Introduction

Head-mounted displays (HMDs) enable the viewing of three-dimensional (3D) medical images in a 3D environment, rather than on traditional monitors. The first HMD was developed in 1968 by Sutherland (1968). Jaron Lanier later popularized the concept, introducing the promise of virtual reality (). However, widespread adoption began only after Palmer Lucky introduced consumer-friendly HMDs in 2012, sparking rapid development (). The Apple Vision Pro (Apple Inc., Cupertino, California, United States) is currently the newest standalone high-end extended reality (XR) device available. Compared to earlier systems such as the Microsoft HoloLens 1 and 2 (Microsoft Corp., Redmond, WA) or the Meta Quest 3 (Meta Platforms Inc., Menlo Park, CA), the Apple Vision Pro (AVP) offers superior display quality, computational performance, sensors, and more precise hand-eye interactions, potentially overcoming past issues with image clarity and usability ().

In parallel with these hardware developments, there has long been interest in instantly visualizing 2D medical image slices, such as those from computed tomography (CT) or magnetic resonance imaging (MRI), as accurate 3D representations. Ideally, this process would require no manual segmentation efforts (Zhang et al., 2011). Volume rendering, a technique introduced in the late 1980s, enabled the display of 3D representations of CT and MRI data on 2D monitors (). HMDs now enable three-dimensional volume rendering (3DVR) of medical scans in immersive environments (). Ongoing advancements continue expanding its medical and educational applications (Ljung et al., 2016; Zhang et al., 2011; ; ), with research indicating that 3DVR improves radiology diagnostics, surgical decision-making, and anatomical understanding by providing superior spatial perception compared to slice-based imaging methods (; Queisner and Eisenträger, 2024).

While standard volume rendering provides crucial 3D spatial perception, photorealistic rendering techniques aim to further enhance anatomical realism and depth cues. Potentially leading to even greater clinical utility, particularly in education or complex surgical planning (; Li et al., 2019; ).

Siemens’ Cinematic Reality (Siemens Healthineers, Medical Imaging Technologies, Erlangen, Germany) () applies these cinematic rendering techniques to enhance traditional VR, offering photorealistic 3D depictions using global illumination models that simulate natural light, shadows, refraction, and occlusion (; Ropinski et al., 2010; ; ). Siemens Healthineers initially developed this Cinematic Reality (CR) application for the computer and later the Microsoft HoloLens (; ).

To contextualise these advances, it is helpful to outline the limitations of the conventional rendering methods that CR seeks to improve upon. Standard 3DVR on CT provides crucial 3D spatial perception, but typically relies on user-defined transfer functions and local lighting models. In this conventional workflow, a transfer function maps ranges of Hounsfield units to colour and opacity; when set aggressively, this can introduce harsh opacity transitions and stripe-like artefacts, and depth cues remain limited because only local shading around each voxel is considered. Isosurface rendering, commonly used on clinical workstations, further simplifies the volume into a hard surface defined by a single intensity threshold, often yielding unnaturally sharp, “plastic” boundaries between tissues. Photorealistic, global-illumination–based cinematic rendering techniques aim to overcome these limitations by simulating how light is transported, scattered, and absorbed in the volume, thereby providing smoother shading, soft occlusion, and more intuitive depth perception, which can translate into greater clinical utility, particularly for education and complex surgical planning (; Li et al., 2019; ; ). These differences across rendering approaches are illustrated in Figure 1, which shows the same CHAOS Subject 5 CT volume rendered with Siemens’ CR, with standard 3DVR using 3D Slicer (version 5.10.0; open-source medical imaging platform - https://www.slicer.org/), with Slicer’s maximum-intensity-projection–style 3D view, and with an isosurface generated using Slicer’s surface tools.

FIGURE 1

Siemens’ CR applies such cinematic rendering techniques to medical CT and MRI data. In contrast to standard 3DVR and isosurface rendering, CR uses a physically based global illumination model that generates soft shadows, ambient occlusion, and view-dependent highlights ; Ropinski et al. (2010); ; . High–dynamic-range lighting combined with tone mapping compresses the simulated radiance into the displayable range while preserving local contrast, and tissue-aware transfer function presets map relevant intensity ranges (e.g., bone, vessels, or parenchyma) to realistic colours and translucency. These tissue-specific presets reduce manual tuning while emphasising clinically relevant anatomy in a visually consistent way. Together, these differences in illumination, tone mapping, and tissue-aware transfer functions explain why CR images are often perceived as more realistic than conventional 3DVR and isosurface rendering and motivate their evaluation on emerging XR platforms such as the Apple Vision Pro.

Prior studies have demonstrated CR’s potential, such as its application on the Microsoft HoloLens for pediatric heart surgery planning (). Despite these advancements, the clinical benefits of photorealistic rendering techniques such as Siemens’ CR, particularly in routine surgical decision-making, remain unclear (; ).

Since the clinical benefits remain unclear, assessing the usability of such systems (HMDs with cinematic rendering applications) and identifying areas of clinical potential is necessary. It’s especially interesting to look beyond anatomy education. Surgical planning and intervention are an interesting example where minor improvements can still warrant the overall cost. Liver transplantation exemplifies such a domain where precise 3D visualization is paramount due to its intricate and highly variable vascular and biliary anatomy (; Yeo et al., 2018; ). Furthermore, surgical planning in this context often requires not only identifying anatomical variants but also critically assessing tumour-induced displacement or encasement of vital structures (vessels, bile ducts, parenchyma), tasks where immersive, high-fidelity photorealistic rendering could offer clinical value.

These previously mentioned uncertainties surrounding clinical effectiveness, combined with the recent advancements in XR devices and software, including the introduction of AVP with CR, provide a compelling opportunity to rigorously evaluate the usability and practical integration potential of CR into clinical workflows (; Lopes et al., 2018; ; ).

This study evaluates Siemens’ CR on the AVP, focusing on its interactive usability and its anticipated integration into clinical workflows, such as surgical planning, as assessed through expert clinician feedback. Siemens’ CR focuses on photorealistic visualization and standard interactions (Figures 2, 3), but does not yet include workflow-specific features such as measurements, segmentations, or annotations. Therefore, our study aimed not only to evaluate usability but, also to capture expert clinician feedback on which features are essential for future clinical adoption. The goal of this study was therefore twofold: to systematically evaluate the usability of Siemens’ CR on the AVP using validated questionnaires, and to capture clinician feedback identifying essential features for integration into preoperative and intraoperative planning workflows.

FIGURE 2

), and (C) a single example of subject four from the CHAOS dataset ().

FIGURE 3

Direct deployment in clinical workflows was beyond the study’s scope; the evaluation with structured usability questionnaires and domain-specific expertise provides actionable insights into the system’s readiness, features, limitations, and opportunities for clinical adoption prior to clinical trials. While participants reflected on the subjective visual quality of the photorealistic rendering, these impressions were treated as part of the overall user experience rather than an isolated outcome.

To go beyond the standard demo case included in the CR application, public datasets related to abdominal visceral surgery are used. This resembles the realistic world scenario more closely than using an optimized demo case. Both CT and MRI, specifically magnetic resonance cholangiopancreatography (MRCP), scans from the CHAOS (; ) and MRCP_DLRecon (; ) datasets are used. Fourteen medical experts evaluated the system via the System Usability Scale (SUS) (; ; ; Lewis, 2018), ISONORM 9242–110-S (Prümper, 1993; Prümper, 2006; ), and qualitative questionnaires.

This study aims to contribute to a broader understanding of how immersive 3D visualization, specifically through the CR application on the AVP, can enhance medical imaging interpretation. Moreover, the survey intends to provide a perspective on how these technologies can support clinicians, specifically liver surgeons, by identifying usability strengths, limitations, and desired features to catalyze clinical adoption.

2 Materials and methods

2.1 Study design and participants

A mixed-methods usability evaluation was conducted, incorporating both quantitative and qualitative assessments. The quantitative analysis used the System Usability Scale (SUS) score () and ISONORM 9242-110-S (Prümper, 2006), which were chosen because they are validated and used as standards within the field. Qualitative feedback was collected through an open-ended survey that also included demographic information. The study included a cohort of 11 surgeons specializing in general, visceral, vascular, liver and transplant surgery, one doctor assistant, and two medical students.

Preoperative hepatobiliary planning is variable but usually includes initial consultation with review or request of contrast enhanced CT and/or MRI (MRCP), radiology demonstration of findings, and repeated surgeon review of 2D slices to assess vascular and biliary anatomy. Surgeons often formulate a plan with alternatives, which is then confirmed or adapted intraoperatively, most commonly with ultrasound and reinspection of the 2D slices. 3DVR and CR on HMDs have the potential to reduce repeated 2D review and intraoperative plan deviations. Our evaluation therefore focused on usability of CR’s current interaction set (Figure 3) and on eliciting expert-identified features needed for integration into this workflow (Tables 1 and 2).

TABLE 1

PrincipleMean SD Median IQRQ1Q3
Suitability5.760.556.000.675.336.00
Self-descriptiveness5.211.135.171.334.676.00
Conformity5.740.716.000.835.176.00
Learnability5.500.885.671.005.006.00
Controllability5.810.776.001.005.336.33
Error tolerance5.241.025.171.504.506.00
Customizability5.191.195.001.254.756.00

ISONORM 9241-110-S results across the seven usability principles. Values are reported as mean , standard deviation , median , interquartile range (IQR), and quartiles (Q1, Q3).

TABLE 2

Positive aspectsNegative aspects
Intuitive and easy to useLack of segmentation options
Exceptional user experienceLack of annotation options
Accurate 3D visualization from all anglesLack of drawing 2D/3D lines and measurement
High-resolution 3D models and imagesIncomplete 3D reconstruction (e.g., missing pathologies). Note: also not visible in the 2D slices
Superior 2D display resolution and comfortNo fusion options or multi-scan visualization
Instant patient-specific 3D reconstructionUnclear menu for the 3D navigation
Efficient/high-quality zoom functionalityIncompatibility with personal eye-wear
Adaptive screen and model positioningMild nausea after use
Interaction via eye-trackingProlonged scene loading time (5 s)
Seamless synchronization and streamingLimited to 2D streaming; lacks multi-user support

Positive and negative aspects regarding Siemens’ Cinematic Reality application running on the Apple Vision Pro.

The entire study was conducted at the University Hospital Essen (AöR), Essen, Germany. The Ethics Committee waived the Institutional Review Board (IRB) approval requirement because the study utilized publicly available, anonymized retrospective datasets and posed no risk or harm to the participating professionals and scanned subjects.

2.2 Datasets

The visualized 3D depictions of the AVP using the CR application were based on the following medical datasets:

  • CHAOS dataset (; ): CT volumes (512512number of slices, 2 mm slice thickness) acquired during the portal venous phase after contrast agent injection, enhancing the portal veins and liver parenchyma.

  • MRCP_DLRecon dataset (; ): An MRCP scan (48038458, 1.2 mm slice thickness, 3T–3D T2w TSE sequence), selected for its detailed depiction of the biliary and pancreatic ducts, particularly relevant for hepatobiliary surgery.

From the CHAOS dataset (; ), subjects four and five were selected to ensure a representative scan of both a male and a female, and they had 94 and 95 slices, respectively. The MRCP_DLRecon dataset (; ) consists of a single volunteer scan with an unidentified gender. To prepare volumetric data for Siemens’ CR on the AVP, the so-called scenes files were generated from DICOM using Siemens’ Cinematic Playground, which requires input in DICOM series format. This process involved only file conversion and scene setup; actual rendering and interaction occur on the AVP based on the user’s viewpoint. The CHAOS dataset was already in DICOM format, while the MRCP_DLRecon dataset, originally stored in HDF5 (.h5) format, was converted using a Python script (Python Software Foundation, Wilmington, DE, United States), utilizing the h5py, numpy, and pydicom libraries. Scenes were loaded onto the AVP via iCloud, with setup times under 1 min. No manual adjustments or transfer function customization were performed; the scans were imported as-is without cropping or editing. Total time per scene preparation was thus under 5 min.

2.3 Data preparation for cinematic reality

Siemens’ CR (v2.0 build 5) for the AVP (VisionOS v2.3.1) was installed via TestFlight (v3.7.1). Unlike the freely available demo version, this version supports the loading of custom-made scenes. The scenes were prepared using Siemens’ Cinematic Playground (v0.19) on a Windows laptop (Intel Core i7-11800H, NVIDIA GeForce RTX 3050) with a required Digital Imaging and Communications in Medicine (DICOM) series. The CHAOS dataset () provided the DICOM series directly, while the MRCP_DLRecon dataset (), stored in Hierarchical Data Format 5 (HDF5), was converted to a DICOM series.

The voxel values were normalized and linearly scaled [-1000 to 2000] using Python (version 12) and the following libraries: numpy (v2.2.2), h5py (v3.12.1), nibabel (v5.3.2), pydicom (v3.0.1), and scikit-image (v0.25.1). This scaling adjustment optimized compatibility with Siemens Cinematic Playground and CR’s transfer function. Scene files were then loaded onto the AVP via iCloud.

Siemens’ CR facilitates conventional interactions, including windowing, scrolling, and zooming. The system is also capable of 3D cinematic volume rendering, as illustrated in Figure 3.

2.4 Evaluation

Quantitative data from the SUS (

) and ISONORM (

Prümper, 2006

) questionnaires were analyzed using descriptive statistics, including mean scores and standard deviations. Qualitative responses were thematically categorized based on strengths, weaknesses, current clinical applicability, feature requests, and potential future applications. The ISONORM 9241-110-S questionnaire assesses usability across seven principles derived from ISO 9241-110:

  • Suitability for the task: How well the system supports users in completing tasks.

  • Self-descriptiveness: The intuitiveness of system functionality.

  • Conformity with user expectations: Consistency with known interface standards.

  • Learnability: Ease with which new users can become proficient.

  • Controllability: The extent to which users can influence actions and outcomes.

  • Error tolerance: The system’s ability to prevent or recover from user errors.

  • Customizability: How well the system can be adapted to user needs.

These principles provide a comprehensive framework for evaluating the user-system interaction, focusing on effectiveness, efficiency, and satisfaction in a specified context of use ().

The System Usability Scale (SUS) is a widely used tool for evaluating the usability of various systems, including medical software. It consists of a 10-item questionnaire with five response options for respondents, ranging from strongly agree to strongly disagree. SUS provides a single score ranging from 0 to 100, representing a composite measure of the overall usability of the system being studied (). Notably, scores above 68 are considered above average across industries and for digital health applications, with higher scores indicating better usability (; ). Scores between 74 and 85 indicate good to excellent usability ().

2.5 Procedure and data collection

For this study, we used a research build of Siemens’ CR for the Apple Vision Pro that supports loading CTs and MRIs beyond the demo case as locally prepared scene files; this build is currently shared only with external groups via research collaborations and is planned for broader release. All sessions took place on site at our institution and were overseen solely by the local research team; no Siemens Healthineers personnel were present during data collection.

Informed Consent: All participants provided written informed consent prior to study participation.

Device Setup: Streaming from the AVP to an iPad was activated via AirPlay to facilitate observation and assistance. Participants then put on the AVP headset and completed the built-in hand–eye calibration before using the Siemens’ CR application. The researcher guided participants in this process.

Application Interaction (Task): Participants launched the Siemens CR application, which was preloaded with CR scenes from the CHAOS and MRCP_DLRecon datasets. The datasets were presented in a fixed order on the AVP: participants first explored the CT volume from the CHAOS dataset, followed by the MRCP scan. Participants thus started with a CT volume from the CHAOS dataset. From this point onward, participants were not guided; help was provided only upon request. The datasets were presented in a fixed order: participants first explored the CT volume from the CHAOS dataset, followed by the MRCP scan. This sequence enabled them to examine both vascular and biliary anatomy and assess the utility of CR in distinct clinical contexts. After viewing both datasets, participants were free to revisit scenes or explore an additional demo head CT case provided by Siemens Healthineers. Participants were tasked with the following instruction: explore key anatomical structures and evaluate the system. Specifically, they were asked to (i) explore anatomical structures relevant to hepatobiliary surgery, including portal veins, bile ducts, and liver parenchyma; (ii) examine image fidelity and photorealism using cinematic volume rendering; (iii) navigate the interface (Figure 3) using standard interaction techniques such as zooming, panning, scrolling, windowing, and rotating the volume and CT slices, as well as adapting the transfer function and switching between scenes (Siemens’ demo scene, CHAOS, MRCP_DLRecon); (iv) verbally articulate their observations, usability impressions, and any encountered challenges as part of a think-aloud protocol; and (v) describe how they would ideally integrate the system into their clinical workflow (e.g., for surgical planning or patient case discussions), including which functionalities they would use and how. These interactions and comments were observed by the researcher in real-time via AirPlay streaming to an iPad. If assistance was required, the researcher provided guidance, utilizing the AirPlay stream to facilitate real-time support.

While no rigid task list or time limit was imposed, the average interaction time during the application interaction was approximately 15–20 min, while hand–eye calibration took less than 5 min. Researchers provided minimal assistance, intervening only when participants requested support.

Data Collection: Following the interaction session, participants completed the System Usability Scale (SUS), the ISONORM 9241-110-S questionnaire, and an open-ended survey capturing qualitative feedback. All responses were collected via Google Forms.

3 Results

3.1 Participant demographics

Fourteen subjects participated in the study, including two medical students, a doctor/surgeon’s assistant, and 11 surgeons, with a gender distribution of eight females and six males. The ages of the female subjects ranged from 22 to 43 years, with a mean of years, standard deviation , and a median of 32 years . In contrast, males, ranging in age from 35 to 68 years, exhibited a mean age of years, standard deviation , and a median of 46 years .

The AVP does not permit the use of spectacles within the HMD; rather, it necessitates the acquisition of its proprietary insert lenses. Consequently, the diopter of the subjects is an essential factor. Among the ten participants who wear spectacles, one subject was classified as farsighted , while the remaining subjects were nearsighted , , minimum , maximum . Of the ten participants who normally wear spectacles, eight used the AVP with compatible insert lenses. The remaining two completed the study without visual correction. While both completed all tasks, they reported reduced clarity when inspecting fine anatomical structures. Although this did not prevent participation, it may have influenced their subjective usability ratings, exemplifying the need for vision correction support in clinical extended reality systems.

Notably, only two subjects had prior experience with HMDs, which was limited to one and 8 h, respectively. The HMDs were the HoloLens 2 and/or Meta Quest 3 (Meta Platforms, Inc., Menlo Park, CA). All subjects were unfamiliar with 3DVR; their familiarity with 3D rendering was limited to segmentation-based 3D rendering.

3.2 System Usability Scale

The SUS is technology-agnostic and combines effectiveness, efficiency and satisfaction with high reliability (Cronbach alpha = 0.91) in a single score (0–100) (; ; Vlachogianni and Tselios, 2022; ). Based on over 10 years of empirical evidence, the score can be divided into seven categories, from worst to best imaginable (; Vlachogianni and Tselios, 2022).

The mean score was σ = 77.68 (σ = 15.01, IQR = 27.50; Q1 = 63.75, Q3 = 91.25), indicating a score between good and excellent. Surgeons and assistant doctors rated the system higher than students. There were no noticeable differences between age or gender groups. An overview of the results for each group is shown in Figure 4, including students, residents, and surgeons.

FIGURE 4

3.3 ISONORM 9242-110-S

ISONORM results across the seven usability principles are summarized in Table 1. Suitability, conformity, and controllability were rated highest, while self-descriptiveness and customizability showed the most room for improvement. Figures 5, 6 provide a visual representation of these results as a heatmap and a stacked bar plot, respectively.

FIGURE 5

FIGURE 6

3.4 Qualitative feedback

The open-ended questionnaire delved into the advantages and disadvantages of the CR application on the AVP (Table 2), its potential applications in a medical settings, and its current clinical usability (Table 3), as well as specific complex use cases and desired improvements for both software.

TABLE 3

Medical applicationPotential medical use casesFeasibility in clinical daily routine
Education/TrainingMedical education (e.g., ultrasound–anatomy correlation); patient information; student learningUseful for student education and patient information
Preoperative planningSurgical planning; anatomical assessment (e.g., tumor/vessel measurement; anatomical variants)Feasible for complex surgical planning and vascular anatomy assessment
Intraoperative guidanceOrgan overlay; adapting steps to anatomical variationsEnhances visualization of complex anatomy for improved surgical steps
Diagnostic imaging supportPathology localization; radiology reviewCan supplement CT for spatial understanding
Interventional proceduresPuncture guidance; intervention planningHard to assess without an OR study
Surgical discussionsPlanning discussions; case presentationsSuited for case discussions and general preparation
OperationsDirect intraoperative use (overlay, quick consult, entry path, resection planes)Unclear feasibility; some users see potential, others do not

Potential medical use cases and feasibility in daily clinical routine for the Apple Vision Pro.

Surgeons identified key areas for enhancing the Siemens’ CR application on AVP to optimize its clinical utility. Recommended improvements include integration with electronic patient records, advanced measurement tools for tumor and vessel size assessment, and the ability to insert 3D models of surgical instruments. Enhanced segmentation functionality enabling seamless switching between anatomical structures (e.g., arteries, portal veins, bile ducts), while gesture-based removal of irrelevant areas should provide more focused visualizations. Additionally, multi-user and multi-platform compatibility to facilitate collaborative planning and real-time surgical discussions. Finally, incorporating a voice activation function was suggested to enhance operability in sterile settings or when both hands are occupied with other tasks.

The setup process for the AVP was typically less than 5 min and considered straightforward. This duration includes donning the headset, performing the initial eye and hand calibration, and launching the CR application. Such a setup time is likely more acceptable for preoperative planning sessions, where time constraints are less critical. However, this setup duration may be undesirable for intraoperative applications, but the participants did not see this as a problem. In practice, once the device is calibrated and the CR application is running, re-donning the AVP and waking it from standby can be performed considerably faster than the full initial setup.

In addition to qualitative comments on usability, participants also described their real-time experience with Siemens’ CR on the Apple Vision Pro. Although no performance measurements were recorded, all users who were able to wear the headset reported smooth interaction and perceived the application as running in real-time, consistent with AVP’s reported refresh rate of 90 Hertz. No lag was noted during rotation, clipping or window adjustments. Eye-gaze selection and pinch-to-click gestures remained accurate throughout each session following the brief calibration, and no tracking interruptions occurred. Most participants used the system for about 15–20 min without discomfort, except for one individual who experienced mild nausea near the end. One participant was unable to use the device due to incompatible glasses, and a few noted minor light leakage from imperfect face-sealing, though this did not affect comfort. Some participants needed a short moment to adjust to the hand-eye coordination for certain gestures, particularly wrist-based rotation, but this reflected user familiarity rather than system issues. No crashes or freezes occurred. Overall, these observations indicate that under the evaluated conditions, the AVP provided stable and comfortable performance for real-time exploration of cinematic rendering.

4 Conclusion and discussion

This study presents one of the first empirical evaluations of Siemens’ Cinematic Reality on the Apple Vision Pro for hepatobiliary CT and MR imaging. Clinicians and trainees reported good to excellent usability on standardized instruments (SUS and ISONORM) and provided convergent qualitative feedback about strengths and missing functionality. Taken together, these results indicate that the current AVP–CR implementation is usable in a controlled setting and delineate the feature gaps that must be addressed. Still, they do not yet demonstrate effectiveness or impact on clinical or educational outcomes.

4.1 Contextualizing the findings

In contrast with the current surgical planning, (immersive) 3D visualization may add value by improving spatial understanding of tumor-vessel-bile duct relationships and by enabling rapid exploration of alternative resection planes directly in 3D rather than on 2D slices before intraoperative confirmation. This may reduce intraoperative plan deviations, shorten planning time, and improve surgeon confidence. As highlighted in Tables 1, 2, our experts already emphasized these use cases. Importantly, our study also identifies which functionalities are prerequisites for unlocking this potential in clinical planning: measurements, annotations, segmentation, and multi-modal fusion. Without these, the technology remains mainly useful for education and demonstration; with them, the step toward clinical decision support becomes more feasible.

While the focus of this work is on Siemens’ CR on the AVP, there is a long-standing use of non- and semi-immersive visualization systems for 3D visualization in medical imaging, such as cave automatic virtual environments (CAVEs), multi-projector setups for holographic visualization, and stereoscopic monitors.

pioneered surround-screen VR such as the CAVE which is promising for medical teaching (). Yet, the physical space required and the financial cost make it less accessible than current cheaper HMDs (), which might be a reason its adoption in, for example, anatomy education is rare (). An advantage of CAVEs is the direct option for multi-user interaction, which Siemens’ CR on the AVP or HoloLens 2 does not have (Table 2). Additionally, no extra (heavy) head-mounted hardware is needed, allowing users more comfort and the ability to wear their own glasses, unlike in the AVP. The same can be said for holographic projections showing promise, but likely not enough to warrant the investment for clinical purposes (; ). Not to mention that when moving from surgical planning to in-surgery usages, it requires an entire specialized operating room, therefore we postulate it’s more effective to explore HMDs until the above technology matures more and costs are lowered.

Stereoscopic monitors remain a practical alternative in many operating rooms. They’re easier to deploy than projection setups, can be made glasses-free with eye-tracking and polarization technology, and allow clinicians to maintain sterility (). Studies suggest that stereoscopic displays can improve anatomical understanding, aid novice surgeons in spatial tasks, and enhance diagnostic performance (; van Beurden et al., 2012). Recent innovations in autostereoscopic displays that integrate eye tracking show promise; for instance, developed a glasses-free 3D autostereoscopic system using eye tracking for cardiac CT imaging, which expert readers identified key arterial structures faster and without discomfort. Still, these monitors lack the immersive, hands-free 360 spatial interaction of HMDs. While outside the scope of this study, future studies or meta-analyses should compare these approaches when evaluating cost, usability, and above all, surgical outcomes.

While semi-immersive setups deliver valuable stereoscopic perception, they do not provide the fully immersive, high-fidelity interaction achieved by contemporary head-mounted displays. The AVP represents a leap in HMD capabilities (e.g., resolution, interaction fidelity) compared to predecessors like the HoloLens 2 and Meta Quest 3 (). When paired with CR, which enhances traditional volume rendering through photorealistic lighting, this hardware promises clinically actionable 3D visualization. However, prior CR evaluations lacked usability assessment in clinical contexts, particularly on cutting-edge platforms like the AVP.

The applications of cinematic rendering techniques go beyond direct visualization. For example, it can generate photorealistic synthetic data used to train deep learning algorithms for medical image interpretation (Mahmood et al., 2018) or be used in various clinical and educational scenarios (; Li et al., 2019). Cinematic rendering has therefore become an enticing alternative to volume rendering (). More user-centered studies have applied CR for patient education (Pachowsky et al., 2022) and student learning (; ; ). These studies reported improved patient understanding and student recall compared to traditional CT scans. However, these evaluations were conducted on conventional 2D monitors and did not directly assess software usability.

Combining advanced HMD hardware and cinematic rendering software has led to the development of interactive 3DVR. 3DVR allows medical scans to be displayed and interacted with in true 3D using an HMD (). Although 3DVR and likewise cinematic rendering technologies hold promise, particularly for complex patient-specific anatomy, the exact clinical benefit remains uncertain (; Queisner and Eisenträger, 2024; ).

Siemens Healthineers previously explored this technology by adapting CR for the HoloLens. One study showed improved surgical decision-making compared to traditional 2D CT images or 3D printed models in cardiac surgery (). Nevertheless, despite the HoloLens’ popularity in medical research, few clinicians used the commercial CR software ().

While open-source CR alternatives exist, they lack the refined transfer functions and realism of Siemens’ commercial solution (); clinicians prefer Siemens’ CR application over commercial alternatives. The performance gap likely arises from Siemens Healthineers’ dedicated team working extensively to perfect the transfer functions for each specific use case, which is a highly specialized and time-intensive task. This illustrates a trade-off between open-source and commercial systems: open-source tools allow rapid prototyping of workflow-specific features such as measurement or annotation, whereas Siemens’ CR prioritizes photorealism and refined transfer functions but is currently limited in functionality. This hurdle towards clinical integration and feature desires was consistently noted by expert participants in our study (Tables 1, 2).

Most existing studies on CR focused either on educational uses displayed on monitors or on synthetic data generation. Few studies have explored actual clinical scenarios to demonstrate tangible benefits. This leaves a significant gap in understanding usability and the specific features clinicians require for clinical adoption.

To address this gap, our study specifically examines the usability of the CR software on the AVP, gathering direct clinician feedback and ideas for clinical integration. We selected the latest hardware (AVP) and software (CR application) to ensure the best possible fidelity, realism, and user interaction–factors crucial for future clinical acceptance. This approach also provides insight into the future potential as these technologies become increasingly accessible.

4.2 Key contributions and strengths

This study is the first to evaluate the usability of Siemens’ CR application on the AVP within a clinical context. It provides detailed insights into usability (Figures 4, 6), highlights specific positive and negative aspects (Table 2), and presents potential clinical applications identified by medical experts (Table 3). Using publicly available imaging data instead of Siemens-specific datasets ensured better alignment with clinicians’ expertise and more realistic testing conditions. Additionally, the public datasets increase the reproducibility of this research.

Our results show that CR on the AVP has good-to-excellent usability, even compared with other digital health applications, with an average SUS score of 77.68 (; ; ; ). Usability ratings were positive across participants, ranging from medical students to experienced surgeons.

Medical students rated usability notably lower than medical professionals (Figure 4). While this difference may partly result from the small sample size, several factors likely contribute. The CR application uses features and icons similar to those found in standard radiological software, such as windowing, slice scrolling, zooming, plane selection, and visualization methods like maximum intensity projection (Figure 3). Surgeons, who frequently use these tools, likely felt more comfortable from the start. This interpretation is further supported by the lower ISONORM scores for self-descriptiveness (Figures 5, 6).

Medical professionals may also have built higher frustration tolerance and better learning strategies over time due to exposure to various technologies throughout their careers. Their practical experience likely helped them recognize clinical use cases more readily, possibly influencing their higher appreciation.

The ISONORM results (Figure 6) also suggest improvements are needed in learnability and error tolerance. Importantly, the study had no genuine technical failures (software or hardware malfunctions) observed during the live video stream. Rather, participants occasionally experienced unexpected interactions with the 3D anatomy, likely due to unfamiliarity with the specific hand-eye interactions, interface elements, or icons. Despite these minor issues, users rated the controllability, conformity, and suitability highly, reflecting the system’s consistent performance.

Lastly, customizability received comparatively lower scores. The qualitative feedback (Table 2) confirms that negative aspects mainly consisted of feature requests rather than fundamental problems. The positive feedback aligns well with the overall SUS and ISONORM scores. Thus, exploring requested features and incorporating them without negatively affecting usability, learnability, or self-descriptiveness will be an essential next step.

These findings directly reflect the primary aim of our study: not to demonstrate clinical outcomes, but to assess usability and collect expert feedback on which functionalities must be added for CR on the AVP to transition from promising usability toward practical workflow integration.

4.3 Limitations and challenges

Despite promising results, several limitations must be acknowledged. The current version of the CR application lacks key clinical features such as segmentation, measurement tools, annotations, and multi-modal scan fusion. These features are routinely used in surgical planning and diagnostics. Nevertheless, participating surgeons, who are often familiar with automated 3D reconstructions from surgical navigation software, rated the system’s usability highly. This indicates that the core interaction design is valuable even without these advanced features.

Hardware constraints also posed minor challenges and affect how easily the device can be shared. The AVP requires proprietary vision correction inserts rather than spectacles, which makes each unit effectively personalized. One participant withdrew because their glasses could not be accommodated; most other participants adapted comfortably using contact lenses. Despite two face-interface sizes, slight ambient light leakage was observed when the seal was not perfect, which may subtly influence perceived image quality, although no formal complaints were reported. Participants did not report problems with headset weight during the short 15–20 min sessions, even though the AVP headset itself weighs roughly 750–800 g and the external battery about 350 g, connected via a tethered cable that can restrict movement and compromise sterility if not routed carefully. Only one participant reported mild nausea. While HMD technology has improved substantially, cybersickness remains an important consideration, particularly during prolonged clinical use (Tian et al., 2022).

In practical terms, the AVP therefore behaves more like a personal device that is best suited to single-user preoperative planning or focused intraoperative use, unless streaming to an external display is used for observers. The high purchase cost is less problematic in complex, high-value surgical cases where even small gains in decision-making may be justified. Still, it is a more substantial barrier for broad deployment in education and routine teaching, where many simultaneous viewers would be needed. By contrast, the external battery and cable are more limiting in sterile intraoperative workflows than in educational or non-operative contexts, where restricted movement and cable routing are easier to accommodate.

The sample size of the study was relatively small, yet it included highly experienced surgeons, lending credibility to the findings. Although no direct comparison was made with traditional 2D displays or open-source alternatives, validated usability measures (SUS and ISONORM) allowed for usability assessments. In addition, all participants first viewed the CT dataset and then the MRI dataset on the AVP, a fixed order that may introduce learning effects; future studies will use a counterbalanced design with randomized modality order and explicit baseline comparators, for example, AVP-CR versus standard 2D workstation review in real clinical cases. Thus, while participants identified relevant potential clinical uses ranging from surgical planning to education, these remain speculative until confirmed through real-world clinical studies.

Finally, note that at present, access to the AVP build of Siemens’ CR used in this study requires research collaboration with the vendor. However, broader release is planned, so this is a temporary practical constraint on replication rather than a methodological limitation.

4.4 Integration into clinical practice

The AVP cannot easily be shared among multiple users because prescription glasses and face-fit are personalized for each device owner. This limitation presents a practical and financial hurdle, especially for educational scenarios such as patient education (). Although multiple face-fits can be purchased or users can wear contact lenses, these solutions remain suboptimal for widespread clinical adoption. Ideally, Apple would develop a face-fit option that allows regular prescription glasses to be comfortably worn. Alternatively, the Siemens CR software or open-source alternatives could be adapted for other available or upcoming HMD models. Combining this adaptation with real-time streaming to an iPad would immediately enable effective patient education. Optimally, a multi-user option would also be integrated. In an ideal educational scenario, this would support multiple users simultaneously, both co-located and remotely located (Perz et al., 2024).

Cost considerations are likely less critical for surgical or treatment-related applications. In these scenarios, participants particularly valued the photorealistic rendering, intuitive eye-based navigation, and rapid setup, all beneficial for preoperative planning. Qualitative feedback further supports the view that immersive visualization clarifies anatomical variations and spatial relationships. Such clarity may enable more confident, personalized surgical decisions. Some surgeons even suggested intraoperative applications, like quickly reassessing puncture or biopsy entry points during procedures. While promising, targeted clinical studies are needed to verify this benefit. Such studies could feasibly be executed within relatively short timelines. Studies should focus on measurable clinical impact and integration, prioritizing interventions where the most impact on patient outcome is expected. Even small studies could eventually be combined into a larger meta-analysis.

However, to fully realize the potential of CR on the AVP for preoperative planning, additional features are needed. Examples include drawing lines to visualize entry points or perform precise measurements. Viewing multiple scans simultaneously, or fusing multi-modal scans, would ensure that all relevant imaging data is visible in one cohesive overview. Other desired functionalities include incorporating artificial intelligence-driven segmentations, adding annotations, and removing anatomical structures via hand gestures. In a future scenario, segmentation could even be pre-generated automatically, allowing voice commands to highlight or render specific anatomical structures transparent.

Integrating these advanced features, particularly combined with real-time streaming to an iPad and multi-user capabilities, would also greatly enhance multidisciplinary case discussions. Additionally, incorporating automatic overlays with error indications could make the system suitable for intraoperative guidance. Such functionality might also be highly beneficial for ultrasound training. For instance, clearly indicating ultrasound imaging planes and corresponding anatomical structures could significantly enhance learning and spatial comprehension.

A crucial final feature, requiring close cooperation between hospitals and software developers, is integration with hospital electronic patient record systems. Currently, creating and transferring visualization scenes to the AVP still involves manual steps. Automating these processes would significantly reduce staff workload and associated costs, an important factor considering the current healthcare labor shortages.

From a commercial perspective, especially in insurance-driven healthcare markets, clearly defining a clinical use case with sufficient benefits to justify these additional features is essential. For clinical researchers, commercially superior products like Siemens’ CR offer robust testing environments. However, open-source alternatives might become attractive due to their flexibility for rapidly developing research-specific functionalities ().

Ultimately, we anticipate that cinematic 3DVR will become a standard feature integrated directly into advanced medical imaging equipment, such as state-of-the-art CT and MRI systems.

4.5 Conclusion and future directions

This study suggests the potential of immersive cinematic 3D volume rendering using Siemens’ CR application on the AVP to enhance medical image interpretation. Participating surgeons and trainees gave high usability ratings, indicating that the current implementation is usable in this pilot setting and may support future clinical adoption if subsequent studies demonstrate effectiveness. The intuitive interface facilitates detailed, photorealistic visualization of CT and MRI data. It was perceived as particularly promising for educational purposes, surgical planning, and intraoperative visualization of complex anatomy and anatomical variations.

The ISONORM profile, which showed high suitability, consistency, and controllability but weaker self-descriptiveness and customizability, was consistent with the open-ended feedback. Clinicians repeatedly requested advanced segmentation capabilities, more robust measurement and annotation tools, multimodal fusion, streaming or multi-user options, voice control, and smoother integration with clinical data systems as prerequisites for routine clinical use. Addressing these feature requests is likely to further enhance both the usability and the perceived clinical relevance of this technology.

Future research should focus on longitudinal, real-world evaluations of CR-assisted clinical workflows. Controlled studies are needed to determine measurable benefits, such as improved surgical accuracy, reduced operative times, or enhanced educational outcomes. Moreover, technical enhancements should emphasize collaborative functionalities, including multi-user capabilities, annotations (lines and notes), remote consultations, and voice-controlled interactions. Implementing these features would likely accelerate clinical acceptance, as suggested by our participants’ feedback.

In summary, while the core usability of CR on the AVP was rated good to excellent, our participants identified key barriers to clinical adoption, including the absence of workflow-specific features, seamless integration with hospital systems, and operability in sterile environments for intraoperative usage.

Finally, while our study focused specifically on hepatobiliary surgery, future investigations should expand into other medical fields, such as orthopedics, cardiology, and interventional radiology. Broadening the scope of evaluation will be necessary to assess the generalizability and potential clinical impact of immersive 3D visualization across modern medicine.

5 Resource identification initiative

At the time of writing, Siemens’ Cinematic Reality and Cinematic Playground, as well as the Apple Vision Pro, are not registered with RRIDs (Research Resource Identifiers). As such, no RRIDs can be provided for the resources used in this study.

6 Life science identifiers

Life Science Identifiers are not applicable to this study, as it does not involve zoological nomenclature or related taxonomic acts.

Statements

Data availability statement

The datasets used in this study are publicly available in a publicly accessible repository. This data can be found here: CHAOS (https://doi.org/10.5281/zenodo.3431873, v1.03) and MRCP DLRecon (https://doi.org/10.5281/zenodo.13912092, v3). The questionnaires used in the user study and the scripts used to prepare the public imaging data into formats suitable for the Cinematic Reality application will be made available by the authors upon request, without undue reservation.

Ethics statement

The requirement of ethical approval was waived by Ethics committee of the medical faculty of the university of Duisburg -Essen and university hospital Essen for the studies involving humans because the study involved no significant physical or psychological risk to participants or patients. All participants were medical professionals or students who voluntarily interacted with a non-invasive visualization system (Apple Vision Pro headset).

Author contributions

GL: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review and editing. LFdP: Validation, Visualization, Writing – original draft, Writing – review and editing. SK: Methodology, Software, Writing – review and editing, Resources. AB: Methodology, Resources, Software, Supervision, Writing – review and editing. LM: Data curation, Investigation, Project administration, Writing – review and editing. AS: Validation, Visualization, Writing – review and editing. PH: Investigation, Methodology, Writing – review and editing. JK: Conceptualization, Methodology, Resources, Writing – review and editing. SS: Conceptualization, Investigation, Methodology, Resources, Supervision, Writing – review and editing. UN: Conceptualization, Methodology, Resources, Writing – review and editing. JE: Conceptualization, Methodology, Resources, Supervision, Writing – original draft, Writing – review and editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. This work was made possible through a collaboration with Siemens Healthineers, who provided early access to their Cinematic Reality Application on the Apple Vision Pro via TestFlight. We also benefited from access to the Siemens Cinematic Playground, which enabled us to visualize multiple publicly available scans (CHAOS dataset (Kavur et al., 2021; Kavur et al., 2019) and the MRCP_DLRecon dataset (Kim et al., 2025; Kim et al., 2024)) in 3D Cinematic Reality. We thank Siemens Healthineers for their support and commitment to this project. This research was partially supported by the REACT-EU project KITE (grant number EFRE-2920801977, Plattform für KI-Translation Essen, https://kite.ikim.nrw/). We further acknowledge funding from the European Union under Grant Agreement No. 101168715 (INSIDE: INSIGHT, https://inside-insight.eu/). The views and opinions expressed are those of the authors and do not necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them. Finally, we acknowledge support by the Open Access Publication Fund of the University of Duisburg-Essen.

Acknowledgments

We thank the Department of General-, Visceral- and Transplant Surgery, Medical Center University Duisburg-Essen, Essen, Germany, for their support and for enabling this research within their clinical environment. We are grateful to the open-source community for enabling reproducible research in medical imaging. This study made use of publicly available datasets, including the CHAOS dataset (; ) and the MRCP_DLRecon dataset (; ). We also acknowledge Siemens Healthineers for providing early access to the Cinematic Playground and Cinematic Reality software. A preprint version of this manuscript is available on arXiv Luijten et al. (2025). Finally, we acknowledge support by the Open Access Publication Fund of the University of Duisburg-Essen.

Conflict of interest

Authors SK and AB were employed by Siemens Healthineers.

The remaining author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. The authors disclose the intermittent use of generative AI tools between February and July 2025 to assist in refining grammar, phrasing, tone, and writing style. The following tools were used: DeepL Write (free version); DeepL SE, Cologne, Germany, https://www.deepl.com/write. Grammarly: AI Writing and Grammar Checker (paid version, Chrome extension v14.1242.0); Grammarly Inc., San Francisco, CA, United States, https://www.grammarly.com. ChatGPT-4 (via ChatGPT Plus subscription, model GPT-4-turbo); OpenAI, San Francisco, CA, United States, https://chat.openai.com. All content was critically reviewed and edited by the authors to ensure accuracy and adherence to scientific standards.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2025.1666614/full#supplementary-material

References

Summary

Keywords

extended reality, augmented reality, apple vision pro, 3D medical imaging, volume rendering, clinical integration, usability assessment, cinematic rendering

Citation

Luijten G, Faray de Paiva L, Krueger S, Brost A, Mazilescu L, Santos AS, Hoyer P, Kleesiek J, Schmitz SM-T, Neumann UP and Egger J (2026) From screen to space: evaluating Siemens’ Cinematic Reality application for medical imaging on the Apple Vision Pro. Front. Virtual Real. 6:1666614. doi: 10.3389/frvir.2025.1666614

Received

15 July 2025

Revised

22 November 2025

Accepted

28 November 2025

Published

05 January 2026

Volume

6 - 2025

Edited by

Caitlin R. Rawlins, Department of Veterans Affairs, United States

Reviewed by

Eleftherios Garyfallidis, Indiana University Bloomington, United States

Shreeraj Jadhav, Kitware, United States

Updates

Copyright

*Correspondence: Gijs Luijten, ; Jan Egger,

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics