HIPS: evaluating a virtual reality training simulation for total hip arthroplasty (THA) and insights into qualitative assessment of surgical training simulations

Abstract

Surgery is a craft that has been handed down in a master–apprentice manner via supervised surgeries that demonstrates the need for high-quality training simulators. Despite advancements in this area, little progress has been made for one of the most common surgeries: total hip arthroplasty (THA). In this work, we describe a multiuser virtual reality training simulator for THA called HIPS that covers the surgical steps from cutting off the femoral head to insertion of the stem. This encompasses the simulation environment, surgical errors detected, multiuser capabilities, as well as user guidance and user feedback methods. In a mixed-method evaluation (N = 12), HIPS was deemed to be realistic, enjoyable, and needed in resident training. The qualitative evaluation methods used herein reveal valuable feedback for further improvements and important learnings for general assessment of surgical training simulators. The variations in surgical techniques need to be evaluated further at different hospitals in different countries along with high-quality simulations of all sensory cues.

1 Introduction

Surgery is considered a craft that requires gaining a feel through experience in addition to having knowledge of how to perform the procedures. This is why surgical skills are often passed down in a master–apprentice manner, where specialist surgeons supervise residents during surgical procedures. Numerous training methods have been developed to prepare residents for their first surgeries to increase patient safety. These methods extend from cadaver training to the utilization of physical models and virtual reality (VR)- or augmented reality (AR)-based training simulators (Gilbody et al., 2011; Huri et al., 2021; LeBlanc et al., 2010; Melcher et al., 2023). In particular, VR-based training systems are very promising as they inherently have fewer ethical implications than cadaver training, along with greater variability and repeatability. Furthermore, they allow the implementation of sophisticated training curricula to a significantly higher degree than physical or cadaver training methods without consuming valuable resources, i.e., physical models and cadavers. Although traditional cadaver training remains an important method for learning surgical skills owing to its high realism, it can be gradually substituted with the increasing realism achieved through VR training simulations. However, this transition requires multimodal VR training environments with high degrees of simulation realism for every sensory cue as surgeons frequently navigate procedures using visual, haptic, tactile, and auditive cues (Lorenz et al., 2025). Until VR training simulations can provide such multimodal realism, traditional training methods will continue to prevail, even though VR simulations are superior in terms of repeatability, training scenario variations, stress scenario training, and self-guided learning using advanced feedback mechanisms. This is one of the primary reasons why VR training simulations are increasingly used in surgical training to enable surgeons to acquire real surgical techniques before their first interventions with live patients (McKnight et al., 2020). Although such systems are effectively designed, VR training systems that can ensure skill acquisition are still under investigation, and there are methodological concepts available presently that have helped the developers of surgical VR training simulators (Gupta et al., 2019).

VR training simulators ensure that the users are completely immersed in a virtual world (Skarbez et al., 2021). There are numerous VR applications in the medical field (Tene et al., 2024; Barré et al., 2019; Zhang et al., 2021; Verhey et al., 2020); some illustrative examples of these include training for laparoscopic surgery in VR (Toale et al., 2023; Burden et al., 2011) or performing dental implants in VR (Chen et al., 2018). Since the 1990s, VR has been steadily used to develop and deliver surgical training simulators that are now being incorporated in curricula in fields like arthroplasty (Syamlan et al., 2022) and laparoscopy (Dawe et al., 2014). Despite these remarkable accomplishments, there are still many surgical fields where realistic virtual training systems are not available. One such field is total hip arthroplasty (THA), where the major challenge is to provide realistic simulations of the very high forces occurring during the procedure (Pelliccia et al., 2020). THA is performed several hundred thousand times a year worldwide (Ferguson et al., 2018), so it is especially important to integrate simulators into the training of orthopedic surgeons. However, there is very limited literature on VR training systems in this field that is also largely lacking implementation details (see Section 2.1).

2 State of the art

2.1 VR training simulators in orthopedic surgery

In orthopedic surgery, most VR applications revolve around operation planning, fracture treatment, and arthroscopy. Only a small set of applications focus on arthroplasty, while even fewer applications are based on THA (Cate et al., 2023; Clarke, 2021; Su et al., 2023; Syamlan et al., 2022; Verhey et al., 2020). A recent review on orthopedic VR training simulators used in resident training reveals that only two studies from 2011 to 2021 were related to hip arthroplasty (Cate et al., 2023) and that both studies did not incorporate haptic feedback (Hooper et al., 2019; Logishetty et al., 2020). Furthermore, the review by Sun et al. (2023) reported that most of the implant positioning applications were related to the cup and stem for THA. In a specific review regarding the inclination accuracy, anteversion, and surgical duration, Su et al. (2023) identified three studies that confirmed the benefits of THA training using VR. Earlier reviews are strongly focused on arthroscopic procedures and almost neglect arthroplasty (Aïm et al., 2016; Morgan et al., 2017). In a recent review by Syamlan et al. (2022), only one VR application was listed for training the acetabular reaming step (Kaluschke et al., 2018a; Panariello et al., 2019). Wu et al. (2025) recently presented a THA simulator that allows training on the correct angles for sawing off the femoral head, reaming the acetabulum, and inserting the stem; although this setup is based on AR using artificial bones for passive haptic feedback, it represents an important effort in the sparse literature available on virtual training systems for THA.

Multiuser VR training systems are very rare in the surgical field. One such example is provided in Edwards et al. (2021), Edwards et al. (2023), where novice surgeons and novice scrub nurses were jointly trained in VR with a focus on the communication between them. Wiese et al. (2021) provide a rare example of a THA training simulation in VR for the surgical approach (not covered in this study), starting from incision to joint dislocation; using a 3D model recreated from the patients’ image data, the authors implemented a self-guided learning simulator that leads the trainees through the procedure while providing detailed feedback regarding the correct execution of the approach. Hooper et al. (2019) evaluated a THA VR training simulation based on the ORamaVR (Heraklion, Crete, Greece) software platform; however, they do not provide further details on the implementation and user feedback. Another THA VR training simulation was developed by Younis and Al-Hemiary (2021) that enabled trainees to perform THA from incision to joint relocating; herein, the authors created 3D models of the operating theater, including all surgical instruments and the hip anatomy, and the simulator provides the opportunity to either train on a skeleton model or a model with soft tissue. However, the authors do not clarify the basis for modeling the anatomy. Although this simulator provides some level of instruction to the trainees about the procedural steps and also measures some performance indicators, the exact design of the simulator, including the user feedback mechanism, is not detailed. Logishetty et al. (2019b) presented an evaluation of the THA VR Simulation v1.1 from Pixelmolkerei AG, Chur, Switzerland; however, they do not indicate the THA steps included in the simulator or the basis for the anatomical 3D model. Although information about the user feedback mechanism is provided, the THA VR Simulation v1.1 can be operated in both training and assessment modes; here, guidance is provided on the next surgical steps, instrument positions, labeled anatomical regions, and implant orientation in the training mode. In both modes, the trainees are provided “[…] data on their errors in instrument selection and position, the number of prompts given by the simulator (triggered when the surgeon does not make stepwise progress for more than 30 seconds), hand path lengths, final component orientation, and the time taken to perform THA” (Logishetty et al., 2019b). Another notable effort was reported by Kaluschke et al. (2018a), 2Kaluschke et al. (2018b) and Knopp et al. (2018), who presented a training simulator for acetabular reaming using the KUKA iiwa robot for force feedback; this device is from a previous developmental stage of the HIPS training simulator presented in this study.

2.2 Evaluation methods for surgical VR training simulators

Systematic reviews indicate that the evaluations of surgical training simulators based on VR frequently entail quantitative approaches to determine the validity of these devices (Cate et al., 2023; Iskander et al., 2021; Lungu et al., 2021; Mao et al., 2021; Pfandler et al., 2017; Zhang et al., 2021). Many researchers choose to primarily assess the facial validity, performance times, error rates, and precision/accuracy (Cate et al., 2023; Iskander et al., 2021; Mao et al., 2021; Pfandler et al., 2017; Zhang et al., 2021); nevertheless, considerations such as cognitive load (Barré et al., 2019) and user experience (Färber et al., 2009) are sometimes included in these evaluations. To a much lesser extent, qualitative methods like interviews have been used to assess the general benefits and limitations of using VR for surgical training (Kuhn et al., 2021, 2024) or to assess specific training simulators (Koch et al., 2019). Notably, feedback on the qualitative nature of the quality of simulation and enhancements is scarce in existing literature. Koch et al. (2019) evaluated a vertebroplasty simulator with 13 specialist participants. Similarly, Allgaier et al. (2022) evaluated a simulator intended for clipping intracranial aneurysms with only two specialist participants. Furthermore, Frendø et al. (2019) examined a mastoidectomy training simulator with 20 resident participants.

3 Materials and methods

3.1 HIPS training simulator

The HIPS training simulator is a multiuser VR application for learning the procedures involved during a hip implant (i.e., THA) with realistic haptic feedback. HIPS can also be used as a visual VR-only application without haptic feedback. The multiuser capabilities allow a trainee to perform a virtual surgery while receiving guidance from a supervising specialist through the use of vocal instructions, pointing, and highlighting in VR. In addition, multiple spectators can observe the training in VR. The main mode of HIPS utilization is the self-guided single-user mode where the trainee is guided through the five main steps of the procedure using text, images, and visual guidance at the virtual patient’s situs. There are three levels of support that the user can choose from, namely, a beginner level with complete guidance through the procedure, an intermediate level with only crucial supportive information, and an expert level without any support. The five steps included in HIPS are cutting off the femoral head, reaming the acetabulum, inserting the acetabular implant, reaming the femur, and inserting the stem implant. These five steps are organized into five distinct modules that can be executed individually or consecutively. The status at the end of each module is not carried over to the next one; instead, each module starts at an ideal state. The reason for this is that it would not be realistic to proceed to the next step in a procedure when an error has occurred in the previous one. During training, any errors in the execution of a procedure are detected and directly fed back to the user. At the end of each module and at the end of training, a detailed assessment of the trainee’s performance is displayed. In the following subsections, the multiuser functionality, training modules, and feedback system are described in greater detail. Highly experienced surgeons from the Zeisigwaldkliniken Bethanien Chemnitz, Germany, who have completed more than 1,000 hip prosthesis implants were consulted for the development of the HIPS training simulator.

3.1.1 Training modules and user interactions

The training in HIPS is conducted in the virtual operating theater depicted in Figure 1, where the virtual patient is supine in the center of the room. Located behind the virtual patient is a big interactable information screen that guides the trainee through the procedures (see Section 3.1.2 for details). On the right side of the patient is a table on which the most important instruments for each module are placed. Furthermore, the height of the bed can be adjusted using the arrows near the virtual patient’s head. All interactions with the information screen, adjustment of the bed height, and selection of instruments in each module are achieved via look-and-wait. The trainee can view a small purple dot fixed at the center of their display that changes to a small ring when hovered over an interactable field; the ring then fills with purple color within 1 s and triggers the interaction event. The surgical instruments for each training module can be controlled using haptic devices in the force-feedback mode or using VR controllers in the VR-only mode of HIPS. However, as the aim of this study is to solely obtain feedback regarding the user experience and validity of HIPS in the VR-only mode, the interactions using VR controllers are explained in further detail.

FIGURE 1

The situses of the five modules are shown in Figures 2, 4. In the first module, the trainee operates an oscillating saw for cutting off the femoral head; the saw is moved using a VR controller and switched on by pressing the trigger button. Similarly, a surgical acetabular reamer is used in module 2 for preparing the acetabulum. Acetabular implant insertion, femoral reaming, and stem implantation are performed in modules 3 to 5, where the trainee operates a virtual surgical hammer attached to the VR controller. The virtual hammer has to strike the virtual implantation and corresponding reaming instruments during these steps. In the case of femoral reaming in module 4, three reaming rasps are used consecutively that have to be extracted by hitting the underside of the striking plate. Currently, only one patient anatomy, one hip side, one set of implants, and one surgical approach (lateral, transgluteal approach according to Bauer et al. (1979)) are available for training. The patient anatomical model was created using anatomy books, guidelines for THA, and feedback from consulting specialist surgeons.

FIGURE 2

3.1.2 Surgical errors and user feedback

For efficient training with HIPS, it is important to detect surgical errors during training. Hence, the most severe surgical errors for each module were identified in collaboration with the consulting surgeons (Table 1). Each of these errors would or could lead to severe and irreversible damage to the patient’s health. The detection of these errors during the virtual surgery has been incorporated into HIPS, although the errors concerning the wrong application of forces are only detectable using haptic devices and not in the VR-only mode of HIPS. In the event that a trainee makes a surgical error, the specifics are listed on the evaluation screen of the module (Figure 3, bottom left). Furthermore, a red X is displayed next to the situs to inform the trainee that an error was detected but whose details are not exactly known.

FIGURE 3

Figure 3 depicts the information screen slides that guide the trainee through training and inform them about their performance. At the beginning of each module, the trainee is provided information on how this surgical step is carried out and the important points to watch out for during surgery (Figure 3, top row). To emphasize these important points, four pictograms are displayed that provide in-depth information below when selected. After starting a module, the information screen lists the steps that need to be performed and their progress (Figure 3, second row from top). When a training module is finished, the trainee is first asked to assess their own performance before the system informs them of their performance objectively (Figure 3, third row from top). The same color-coded horizontal bars are used for both assessments; the time and error points are listed below these (more points means worse), where the surgical errors listed in Table 1 are assigned point values. The most important information is then provided in the lower half of the information screen, where the surgical errors as well as correctly carried out steps are listed. After the last module is completed, the assessment of the entire training is displayed on the information screen (Figure 3, bottom row). For each module, the error points, time, and color-coded assessment bar are displayed concisely. Further, a reference to the previous training performances of the trainee is shown along with their high scores.

During the training for cutting off the femoral head in module 1 (Figure 4), the trainees are guided to find the correct position and angle. A change in color from green to yellow and red indicates how well they perform. A similar guide is also offered in module 2 for acetabular reaming. Further, some typical sounds of the surgical instruments and equipment are included to enhance immersion.

FIGURE 4

3.1.3 Multiuser and recording functionalities

The multiuser mode enables HIPS users to observe and interact with their training through a network connection. Here, the specialist and trainee are physically separated but virtually united in the same operating room. In one-to-one supervision, the specialist can guide the trainee and provide direct feedback. Any number of viewers can watch a trainee at the same time by connecting to the simulator user through a personal computer (PC) with or without a VR system. To enable this functionality, a system was developed to distinguish between different user roles. HIPS also offers the functionality to record a training session along with its associated audio comments; this allows experienced surgeons to record a VR training session that trainees can watch and reenact to learn the exact handling of the surgical instruments. Furthermore, trainees can also record their training to assess and learn from their mistakes.

3.1.3.1 Trainee

This person carries out the training, and there can only be one user claiming this role. Here, HIPS is always implemented in expert mode, and the trainee is guided by the supervising specialist and not by the system. The screen in the operating theatre displays the module overview as well as the number and names of the other users present. Furthermore, only the trainee can switch between modules.

3.1.3.2 Specialist

This person is an experienced surgeon who guides the trainee through the surgery, and there can only be one user claiming this role. The specialist can place 3D markers on the virtual patient to guide the trainee, talk to the trainee, and use a pointing ray. This role can also be implemented in a non-VR setting using only the mouse and keyboard of a desktop PC.

3.1.3.3 Spectators

There can be an unlimited number of persons claiming this role. These users are only able to passively observe the scene from any perspective they want or choose to watch from the trainee’s perspective. This role can also be implemented in a non-VR setting using only the mouse and keyboard of a desktop PC.

3.2 Experimental setup

The HIPS training simulator was evaluated at the Clinic for Orthopedics, Trauma Surgery, and Sports Traumatology at the Zeisigwaldklinik Bethaninen Hospital in Chemnitz, Germany. Almost all members of the clinic participated in the evaluations. Their experience levels ranged from students to residents and even up to very experienced surgeons. Owing to this range in experience, it was possible to gather the opinions of actual end-users, namely, the students, residents, and specialists. The evaluations were conducted using the HP Reverb G2 head-mounted display (HMD, Figure 5). Initially, the participants were informed about the aims of the study and processing of their data. Thereafter, the participants provided their written informed consent. During the pre-assessments, the participants provided demographic data, including age, gender, year of residency or number of years since completion of residency, as well as the number of hip prosthesis implantation procedures that they are actively or passively participating in annually. Then, the participants were allowed to familiarize themselves with the VR system, and HIPS training simulation was started in the beginner mode with maximum guidance to gather feedback on trainee guidance and the feedback elements. The participants completed the five training modules consecutively; then, they were encouraged to report their impressions while experiencing the training simulator. The study investigators made further inquiries in the event of interesting or unclear comments. After the participants finished providing their comments on a module, the experimenter confronted them with selected comments from previous participants to better assess the severity and validity of the comments.

FIGURE 5

Lastly, the participants could provide additional comments and summarize their experiences.

3.3 Data analysis

The descriptive statistics were calculated using SPSS 28.0 from IBM. Regarding the qualitative analysis, each new comment by a participant was noted in a list by the experimenter. When another participant provided the same comment, this was noted so that the significance of a particular comment can be determined by the number of mentions.

3.4 Demographics

A total of N = 12 participants volunteered for the study, and their demographic data are presented in Table 3. Of these participants who had already experienced medical VR applications, two persons had experienced a previous version of the HIPS training simulator, one person had experienced a spine simulation, one person had experienced a knee implant simulation, and one person had experienced an application for navigation.

4 Results

4.1 User acceptance, intention of use, and presence

The results of the user acceptance and intention of use questionnaire reveal a positive assessment of the HIPS training simulator overall (Figures 6–8). The participants showed high levels of agreement with the user experience and usability that are also reflected in the median value of 4.00 for the presence, usefulness, playfulness, and ease of use scales; median value of 4.38 for the attitude scale; and median score of 4.13 for the intention of use scale. In detail, the results show that the situs, surgical instruments, and operating theater were assessed as realistic by a majority of at least 66% (questions 1–3). It is further evident that HIPS received very positive ratings in terms of enjoyment, ease of use, and comfort, leading to a positive attitude with at least 75% agreement (questions 5–9, 13). The general idea of training for THA in VR, its usefulness, and the motivation to continue practicing in VR also received positive responses with at least 66% agreement (questions 4, 10–12, 14, and 15). The usefulness of HIPS for medical students and residents was seen as very high with at least 75% agreement (questions 16, 17, and 19). For specialist surgeons, the benefits provided by HIPS were not seen as useful (25% agreeing with question 18) despite its perceived desirability in specialist training (53% agreeing to question 20). Given the 91% agreement to question 21, a large majority of the participants wished for the opportunity of patient-specific training with HIPS in preparation for surgery. The question by Bouchard et al. (2004) regarding the participants’ feeling of “being there” in the virtual environment received an average rating of M = 6.25 with a standard deviation of SD = 1.6.

FIGURE 6

FIGURE 7

FIGURE 8

4.2 Qualitative feedback

In the following report regarding the participant comments, we focus on the negative aspects and important improvements since the positive comments only serve to underscore the positive feedback in the questionnaire without further substantial findings. Although unusual, we also report the experimenter’s observations as these might prove useful for other researchers planning user studies with medical staff and help them to interpret their results. In total, 66 statements were noted, of which 17 were at least mentioned by three participants each. Only four out of the 66 statements were mentioned by at least seven participants each, which represents a majority vote.

One of the interesting findings was that five participants commented that the positions of the hooks for cutting the femoral head in module 1 were wrong, even though their positions were agreed upon beforehand with the consulting specialists. The other participants did not comment on these wrong positions. The experimenter asked the specialist participants about this observation, and the specialists responded that this could probably be attributed to the manner in which surgery is taught. The hook settings are not standardized; instead, each specialist favors a certain configuration that they then impart to the residents, who perceive this as the only correct method of positioning. The specialist participants noted that such variations in the operating techniques were usually only perceived when switching hospitals as the specialists there have their own unique styles. Furthermore, the participants gave diverse feedback about the details of the operating theatre like the positioning of the lamps and vital monitors. There was conflicting feedback regarding the visibility of the patient; while one specialist stated that the patient’s head was not visible during surgery owing to a curtain, another specialist mentioned that this diversion from a realistic setup would be helpful for training students and residents to better immerse into the scenario and identify with the patient. Six participants noted that the look-and-wait interactions were not direct or clear and needed more detailed instructions by the experimenter. An important error that was detected by eight participants was that the angle indicated for acetabular reaming in module 2 and hence the implantation of the acetabular prosthesis in module 3 were wrong. Furthermore, five participants mentioned that they missed the visual changes to the acetabulum during reaming as this is an important indicator for them to determine when the reaming is complete. As the other specialist participants did not comment on this inadequacy, the experimenter specifically asked these specialists if it disturbed them that the acetabulum did not change visually during reaming; given this prompt, the response was that even though this was unusual, the users preferred to touch the acetabular surface and feel if sufficient material was reamed off. Although it was clarified that HIPS was evaluated without haptic feedback and only with the VR-HMD and controllers, 10 participants commented that they would have liked to use a second hand for controlling the instruments.

Thus far, only the most relevant comments made by the participants have been reported. Although the participants provided plenty of detailed feedback regarding further improvements to HIPS, these do not contribute to the general insights. However, the experimenter reported that the participants’ feedback points were very broad and contradictory in parts based on different aspects and the experience levels. The specialist participants tended to provide more feedback on the exact procedures of the surgery and focused on simulation quality of the cues that they used to orient themselves during surgery, be it visual, auditive, or haptic. The inexperienced participants often commented on deviations from the operating theatre and instruments familiar to them, e.g., the monitor or saw looks a bit different. Furthermore, the participants placed different levels of importance to the same aspects depending on their expectations of the simulation quality. When confronted with comments from other participants, e.g., the sound of the saw does not change when sawing bone, they mentioned that this was true but felt that this would be technically impossible to simulate in VR and would not have expected it to be accurate anyway.

5 Discussion

5.1 Positive assessments of the HIPS training simulator

In this study, we present HIPS as a VR training simulator for THA (excluding the surgical approach and closure) and evaluate its user experience and validity with 12 participants ranging from medical students to residents and experienced specialists. To the best of our knowledge, we are the first research group to describe the implementation and user feedback of such a THA simulator in detail as well as present a list of the critical errors for THA in the context of a VR training system covering the steps from cutting the femoral head to implanting the stem. Unfortunately, the few other THA VR training simulators reported previously (Hooper et al., 2019; Logishetty et al., 2019a; Logishetty et al., 2020; Younis and Al-Hemiary, 2021) do not provide details about the capabilities and user feedback mechanisms of their devices, allowing only a superficial comparison with our work. Younis and Al-Hemiary, (2021) have covered all the surgical steps that we have presented herein and additionally the incision. In the absence of implementation details, we can only state that our anatomical models are not based on real data of patient images, in contrast to Hooper et al. (2019). Our anatomical model was created using anatomy books, guidelines for THA, and feedback from consulting specialist surgeons. We consider both approaches to be equal in terms of quality and accuracy. The THA VR Simulation v1.1 from Pixelmolkerei AG, Chur, Switzerland used by Logishetty et al. (2019b) also provides distinct training and assessment modes comparable to ours; further, it provides guidance on surgical instruments and implant orientations, along with tracking of certain types of errors during the operation. Given the lack of further information about the simulator, a more detailed comparison is unfortunately impossible. The study by Logishetty et al. (2019b) was focused on the training success of residents and did not assess the training simulator itself, so we cannot compare the study results directly. Furthermore, Wiese et al. (2021) reported the use of a similar concept as Logishetty et al. (2019b) and ours by providing detailed instructions and error detection in their simulator for training the approach for THA, which we do not cover with HIPS. Wu et al. (2025) compared three types of AR visualization overlays over a physical artificial bone, air-floating virtual model, as well as 2D perspectives for sawing the femoral head, reaming the acetabulum, and inserting the stem; these steps correspond to the HIPS modules 1, 2, and 5. It is worth noting that Wu et al. (2025) only trained the correct angles for these three steps using real tools and instruments; however, the procedures were not actually performed, i.e., the saw was placed on the femoral head in the correct position but the bone was not actually cut.

The feedback from the questionnaire as well as the comments of the participants were generally very positive. Nonetheless, many important details for improvements and a few errors regarding the angle of the acetabular reaming and implantation were reported. The correction of these errors and incorporation of the other feedback points will likely lead to better reception of HIPS in the subsequent evaluations. The most notable and interesting feedback was provided by the specialists, who revealed different methods used by experienced surgeons to perform the individual surgical steps. As an example, we note the reaming of the acetabulum, wherein some surgeons predominantly rely on the sound of the reaming process to determine the progress while others touch the surface of the acetabula or consider visual changes to the surface. This underscores the importance of simulating multiple cues to provide a training system that is usable by all. This effect is likely true for other surgical training methods as well, regardless of the use of VR, physical models, or cadavers. This curricular aspect for developing a surgical training simulation has not been described in literature yet. In our follow-up study including haptic feedback, these findings were confirmed by surgeons who expressed the need for simulating more cues (Lorenz et al., 2025).

The second very important insight for developing a surgical training simulator was obtained from the participant comments on seemingly wrongly placed hooks, which were in fact only placed differently than the common configuration used at the hospital where the participants worked. Although the positioning of the hooks was not wrong, it is crucial to provide the participants with the possibility of rearranging the hooks to their preference to improve the simulation and achieve greater acceptance. The low level of acceptance shown toward the arrangement of the hooks that contradicts their personal experiences could result in the surgeons doubting the medical competence of the training system developers. Alternatively, such doubts could be mitigated by naming a trustworthy medical author at the beginning of the simulation according to whose suggestion the hooks are placed. This insight on the importance of providing variations of the situs or other trust-increasing measures is generally useful for the developers of surgical training simulators and should always be considered. In our follow-up study, we built on these insights and reassured the participants that the hook positions were validated by a trusted expert so that the validity could not be doubted as in the present study (Lorenz et al., 2025).

The relatively low rating for presence may be explained by the lack of haptic feedback and the small number of errors detected during surgery simulation. This could have motivated the participants to provide rather low ratings for presence as it was plausible that the training simulation was affected. Indeed, we noted higher presence ratings (M = 7.20, SD = 1.90) in our follow-up study that provided haptic feedback (Lorenz et al., 2025). Although the multiuser capabilities of the HIPS training simulator were not assessed in the present study, this is the first time that a multiuser VR training program for THA is described in a scientific paper. Aside from THA, multiuser VR training is rarely implemented but shows promising benefits for improving medical education (Edwards et al., 2021; Edwards et al., 2023).

5.2 Lessons learned for evaluating surgical training simulators

Our approach on including thinking aloud as a qualitative measure for assessing a VR surgical training system is a relatively new concept that has sparse literature; Koch et al. (2019) claim to be the pioneers on using this method with 13 specialist vertebroplasty surgeons. In contrast to our findings, Koch et al. (2019), Allgaier et al. (2022), and Frendø et al. (2019) do not report such a variety of partly contradicting comments. This can have several reasons: the authors may simply not have reported them, no contradictions emerged as the investigators did not confront their participants with the comments of other participants, or their samples were more homogeneous in terms of experience; however, a mix of these reasons is also possible. Given our interrogatory style of qualitative evaluation, we moved away from the common method used to assess surgical training simulators. Thus, we not only gained important information for improving the HIPS simulator but also demonstrated the value of using this evaluation method to assess other surgical simulators. Even though there are guidelines available for surgical standard procedures, individual surgical operation techniques may vary greatly in detail. In combination with the fact that surgery is still taught in a master–apprentice manner like other traditional crafts, this creates a challenge for developing surgical training simulators that has been neglected thus far. Our study demonstrates the importance of performing evaluations in different hospitals and countries to uncover a larger number of techniques used. As demonstrated by the example with hook placement, even minor variations that deviate from known procedures may lead to reduced acceptance of a surgical training simulation. To ensure the success of a surgical training simulator, it is of utmost importance to discover these variations in surgical techniques and implement capabilities to adapt the training scenario so that the trainees feel comfortable and accept the training as plausible. Another learning here is the importance of taking into account the expectations of the participants; it is not possible to assume the absence of errors just because the users do not mention them. As demonstrated by the comment regarding the sound of the bone saw, users may also take for granted that it is impossible to simulate a change in the sawing sound and therefore accept this reduction in realism without mentioning it negatively.

5.3 Limitations

Despite the good coverage of the targeted user groups in this study, the results must be considered carefully owing to the small sample size given the difficulty of recruiting orthopedic surgeons as study participants. Further substantiation of these results with a greater number of participants from different hospitals and different countries is highly desirable. Currently, our training simulator offers only one patient anatomy, one hip side, one set of implants, and one surgical approach. Variations to the anatomy and approaches are desirable for increasing the bandwidth of situations experienced by the trainees. The authors are aware that Osso VR, Inc. (San Francisco, CA, United States) and Fundamental Surgery or Fundamental VR (FVRVS Ltd., London, United Kingdom) provide THA VR training simulators with multiuser capabilities that appear to be comparable to HIPS. Unfortunately, as these capabilities are only described in gray literature (company videos and websites), thorough scientific comparisons are not possible. Therefore, it would be highly desirable to describe and assess these commercial simulators in peer-reviewed papers to enable objective assessments and comparisons.

6 Conclusion

In this work, we present and evaluate HIPS as a VR training simulator for THA. HIPS allows trainee surgeons to gain experience with cutting off the femoral head, reaming the acetabulum and femur, as well as implanting the cup and stem in THA. We also present detailed descriptions of detectable surgical errors, multiuser functionalities, user feedback, and user guidance mechanisms. In a user study with 12 participants ranging from medical students to residents and specialist surgeons, we confirmed the correctness of HIPS along with high user experience and the need for such a VR training simulator for THA. Aside from common qualitative methods, the assessment of HIPS included thinking-aloud measures and interviews as additional qualitative features that are rarely used to evaluate surgical training simulators. The qualitative assessment allowed very detailed and insightful evaluation of HIPS as well as further revealed important insights for the general assessment of surgical training simulators. The master–apprentice manner of teaching surgical skills and broad variations in surgical techniques require flexible configurability of the surgical training simulation to achieve high plausibility and to enable evaluations at different hospitals in different countries. Further, it is important to incorporate multiple cues in the simulation to reflect different methods by which surgeons evaluate their progress during surgery. Aside from further studies to substantiate our findings, it is important to have at least similarly detailed descriptions of other THA training simulators from laboratories and commercial providers to allow meaningful comparisons and accelerate progress in the field.

References

• 1

  AïmF.LonjonG.HannoucheD.NizardR. (2016). Effectiveness of virtual reality training in orthopaedic surgery. Arthroscopy32, 224–232. 10.1016/j.arthro.2015.07.023

  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  AllgaierM.NeyaziB.Erol SandalciogluI.PreimB.SaalfeldS. (2022). Immersive VR training system for clipping intracranial aneurysms. Curr. Dir. Biomed. Eng.8, 9–12. 10.1515/cdbme-2022-0003

  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  BarréJ.MicheletD.TruchotJ.JolivetE.RecanzoneT.StitiS.et al (2019). Virtual reality single-port sleeve gastrectomy training decreases physical and mental workload in novice surgeons: an exploratory study. Obes. Surg.29, 1309–1316. 10.1007/s11695-018-03680-9

  • CrossRef
  • Google Scholar
  • View reference in article

• 4

  BauerR.KerschbaumerF.PoiselS.OberthalerW. (1979). The transgluteal approach to the hip joint. Arch. Orthop. Trauma Surg.95 (95), 47–49. 10.1007/BF00379169

  • CrossRef
  • Google Scholar
  • View reference in article

• 5

  BouchardS.RobillardG.St-JacquesJ.DumoulinS.PatryM. J.RenaudP. (2004). “Reliability and validity of a single-item measure of presence in VR,” in Proceedings of the Second International Conference on Creating, Connecting and Collaborating through Computing (IEEE), 59–61.

  • Google Scholar
  • View reference in article

• 6

  BurdenC.OestergaardJ.LarsenC. R. (2011). Integration of laparoscopic virtual-reality simulation into gynaecology training. BJOG Int. J. Obstetrics Gynaecol.118 (Suppl. 3), 5–10. 10.1111/j.1471-0528.2011.03174.x

  • CrossRef
  • Google Scholar
  • View reference in article

• 7

  CateG.BarnesJ.CherneyS.StamboughJ.BumpassD.BarnesC. L.et al (2023). Current status of virtual reality simulation education for orthopedic residents: the need for a change in focus. Glob. Surg. Educ.2, 46. 10.1007/s44186-023-00120-w

  • CrossRef
  • Google Scholar
  • View reference in article

• 8

  ChenX.SunP.LiaoD. (2018). A patient-specific haptic drilling simulator based on virtual reality for dental implant surgery. Int. J. Comput. Assist. Radiol. Surg.13, 1861–1870. 10.1007/s11548-018-1845-0

  • CrossRef
  • Google Scholar
  • View reference in article

• 9

  ClarkeE. (2021). Virtual reality simulation-the future of orthopaedic training? A systematic review and narrative analysis. Adv. Simul. (Lond)6, 2. 10.1186/s41077-020-00153-x

  • CrossRef
  • Google Scholar
  • View reference in article

• 10

  DavisF. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information Technology. MIS Q.13, 319. 10.2307/249008

  • CrossRef
  • Google Scholar
  • View reference in article

• 11

  DaweS. R.WindsorJ. A.BroedersJAJLCreganP. C.HewettP. J.MaddernG. J. (2014). A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann. Surg.259, 236–248. 10.1097/SLA.0000000000000245

  • CrossRef
  • Google Scholar
  • View reference in article

• 12

  EdwardsT.GuptaS.SoussiD.PatelA.KhanS.LiddleA.et al (2023). Surgical teams who train together in virtual reality outperform those who train individually: a randomized controlled trial for learning anterior approach total hip arthroplasty. Bone Joint105 (B), 13. 10.1302/1358-992X.2023.8.013

  • CrossRef
  • Google Scholar
  • View reference in article

• 13

  EdwardsT.SoussiD.GuptaS.PatelA.LiddleA.KhanS.et al (2021). A randomized controlled trial comparing multiplayer and single-player immersive virtual reality training for anterior approach total hip arthroplasty surgery. Orthop. Proc.103, 9.

  • Google Scholar
  • View reference in article

• 14

  FangT.-Y.WangP.-C.LiuC.-H.SuM.-C.YehS.-C. (2014). Evaluation of a haptics-based virtual reality temporal bone simulator for anatomy and surgery training. Comput. Methods Programs Biomed.113, 674–681. 10.1016/j.cmpb.2013.11.005

  • CrossRef
  • Google Scholar
  • View reference in article

• 15

  FärberM.HummelF.GerloffC.HandelsH. (2009). Virtual reality simulator for the training of lumbar punctures. Methods Inf. Med.48, 493–501. 10.3414/ME0566

  • CrossRef
  • Google Scholar
  • View reference in article

• 16

  FergusonR. J.PalmerA.JRTaylorA.PorterM. L.MalchauH.Glyn-JonesS. (2018). Hip replacement. Lancet392, 1662–1671. 10.1016/S0140-6736(18)31777-X

  • CrossRef
  • Google Scholar
  • View reference in article

• 17

  FrendøM.ThingaardE.KongeL.SørensenM. S.AndersenS. A. W. (2019). Decentralized virtual reality mastoidectomy simulation training: a prospective, mixed-methods study. Eur. Arch. Otorhinolaryngol.276, 2783–2789. 10.1007/s00405-019-05572-9

  • CrossRef
  • Google Scholar
  • View reference in article

• 18

  GilbodyJ.PrasthoferA. W.HoK.CostaM. L. (2011). The use and effectiveness of cadaveric workshops in higher surgical training: a systematic review. Ann. R. Coll. Surg. Engl.93, 347–352. 10.1308/147870811X582954

  • CrossRef
  • Google Scholar
  • View reference in article

• 19

  GuptaA.CecilJ.Pirela-CruzM.RamanathanP. (2019). A virtual reality enhanced cyber-human framework for orthopedic surgical training. IEEE Syst. J.13, 3501–3512. 10.1109/JSYST.2019.2896061

  • CrossRef
  • Google Scholar
  • View reference in article

• 20

  HooperJ.TsiridisE.FengJ. E.SchwarzkopfR.WarenD.LongW. J.et al (2019). Virtual reality simulation facilitates resident training in total hip arthroplasty: a randomized controlled trial. J. Arthroplasty34, 2278–2283. 10.1016/j.arth.2019.04.002

  • CrossRef
  • Google Scholar
  • View reference in article

• 21

  HuriG.GülşenM. R.KarmışE. B.KaragüvenD. (2021). Cadaver versus simulator based arthroscopic training in shoulder surgery. Turk. J. Med. Sci.51, 1179–1190. 10.3906/sag-2011-71

  • CrossRef
  • Google Scholar
  • View reference in article

• 22

  IskanderM.OgunsolaT.RamachandranR.McGowanR.Al-AswadL. A. (2021). Virtual reality and augmented reality in ophthalmology: a contemporary prospective. Asia Pac. J. Ophthalmol. (Phila)10, 244–252. 10.1097/APO.0000000000000409

  • CrossRef
  • Google Scholar
  • View reference in article

• 23

  KaluschkeM.WellerR.ZachmannG.PellicciaL.LorenzM.KlimantP.et al (2018a). “A virtual hip replacement surgery simulator with realistic haptic feedback,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (IEEE), 759–760.

  • Google Scholar
  • View reference in article

• 24

  KaluschkeM.WellerR.ZachmannG.PellicciaL.LorenzM.KlimantP.et al (2018b). “HIPS - a virtual reality hip prosthesis implantation simulator,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (IEEE), 591–592.

  • Google Scholar
  • View reference in article

• 25

  KnoppS.LorenzM.PellicciaL.KlimantP. (2018). “Using industrial robots as haptic devices for VR-training,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (IEEE), 607–608.

  • Google Scholar
  • View reference in article

• 26

  KochA.PfandlerM.StefanP.WuchererP.LazaroviciM.NavabN.et al (2019). Say, what is on your mind? Surgeons' evaluations of realism and usability of a virtual reality vertebroplasty simulator. Surg. Innov.26, 234–243. 10.1177/1553350618822869

  • CrossRef
  • Google Scholar
  • View reference in article

• 27

  KuhnA. W.YuJ. K.GerullK. M.SilvermanR. M.AleemA. W. (2024). Virtual reality and surgical simulation training for orthopaedic surgery residents: a qualitative assessment of trainee perspectives. JB JS Open Access9, e23.00142. 10.2106/JBJS.OA.23.00142

  • CrossRef
  • Google Scholar
  • View reference in article

• 28

  KuhnS.HuettlF.DeutschK.KirchgässnerE.HuberT.KneistW. (2021). Chirurgische Ausbildung im digitalen Zeitalter – Virtual Reality, Augmented Reality und Robotik im Medizinstudium (Surgical Education in the Digital Age - Virtual Reality, Augmented Reality and Robotics in the Medical School). Zentralbl. Chir.146, 37–43. 10.1055/a-1265-7259

  • CrossRef
  • Google Scholar
  • View reference in article

• 29

  LeBlancF.ChampagneB. J.AugestadK. M.NearyP. C.SenagoreA. J.EllisC. N.et al (2010). A comparison of human cadaver and augmented reality simulator models for straight laparoscopic colorectal skills acquisition training. J. Am. Coll. Surg.211, 250–255. 10.1016/j.jamcollsurg.2010.04.002

  • CrossRef
  • Google Scholar
  • View reference in article

• 30

  LogishettyK.GoftonW. T.RudranB.BeauléP. E.CobbJ. P. (2020). Fully immersive virtual reality for total hip arthroplasty: objective measurement of skills and transfer of visuospatial performance after a competency-based simulation curriculum. J. Bone Joint Surg. Am.102, e27. 10.2106/JBJS.19.00629

  • CrossRef
  • Google Scholar
  • View reference in article

• 31

  LogishettyK.RudranB.CobbJ. P. (2019b). Virtual reality training improves trainee performance in total hip arthroplasty: a randomized controlled trial. Bone Joint101-B, 1585–1592. 10.1302/0301-620x.101b12.bjj-2019-0643.r1

  • CrossRef
  • Google Scholar
  • View reference in article

• 32

  LogishettyK.RudranB.GoftonW.BeauleP.CobbJ. P. (2019a). Total hip arthroplasty training in virtual reality transfers to the real world: a randomized blinded study. Orthop. Proc.101, 27. 10.1302/1358-992X.2019.6.027

  • CrossRef
  • Google Scholar

• 33

  LorenzM.KaluschkeM.MelzerA.PillenN.SanrowM.HoffmannA.et al (2025). HIPS - a surgical virtual reality training system for total hip arthroplasty (THA) with realistic force feedback. IEEE Trans. Vis. Comput. Graph.PP, 3418–3428. 10.1109/TVCG.2025.3549896

  • CrossRef
  • Google Scholar
  • View reference in article

• 34

  LunguA. J.SwinkelsW.ClaesenL.TuP.EggerJ.ChenX. (2021). A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev. Med. Devices18, 47–62. 10.1080/17434440.2021.1860750

  • CrossRef
  • Google Scholar

• 35

  MaoR. Q.LanL.KayJ.LohreR.AyeniO. R.GoelD. P.et al (2021). Immersive virtual reality for surgical training: a systematic review. J. Surg. Res.268, 40–58. 10.1016/j.jss.2021.06.045

  • CrossRef
  • Google Scholar

• 36

  McKnightR. R.PeanC. A.BuckJ. S.HwangJ. S.HsuJ. R.PierrieS. N. (2020). Virtual reality and augmented reality-translating surgical training into surgical technique. Curr. Rev. Musculoskelet. Med.13, 663–674. 10.1007/s12178-020-09667-3

  • CrossRef
  • Google Scholar

• 37

  MelcherC.HussainI.KirnazS.GoldbergJ. L.SommerF.Navarro-RamirezR.et al (2023). Use of a high-fidelity training simulator for minimally invasive lumbar decompression increases working knowledge and technical skills among orthopedic and neurosurgical trainees. Glob. Spine J.13, 2182–2192. 10.1177/21925682221076044

  • CrossRef
  • Google Scholar

• 38

  MorganM.AydinA.SalihA.RobatiS.AhmedK. (2017). Current status of simulation-based training tools in orthopedic surgery: a systematic review. J. Surg. Educ.74, 698–716. 10.1016/j.jsurg.2017.01.005

  • CrossRef
  • Google Scholar

• 39

  PanarielloD.CaporasoT.GraziosoS.Di GironimoG.LanzottiA.KnoppS.et al (2019). “Using the kuka lbr iiwa robot as haptic device for virtual reality training of hip replacement surgery,” in 2019 Third IEEE International Conference on Robotic Computing (IRC), 449–450. 10.1109/irc.2019.00094

  • CrossRef
  • Google Scholar

• 40

  PellicciaL.LorenzM.HeydeC.-E.KaluschkeM.KlimantP.KnoppS.et al (2020). A cadaver-based biomechanical model of acetabulum reaming for surgical virtual reality training simulators. Sci. Rep.10, 14545. 10.1038/s41598-020-71499-5

  • CrossRef
  • Google Scholar

• 41

  PfandlerM.LazaroviciM.StefanP.WuchererP.WeiglM. (2017). Virtual reality-based simulators for spine surgery: a systematic review. Spine J.17, 1352–1363. 10.1016/j.spinee.2017.05.016

  • CrossRef
  • Google Scholar

• 42

  SkarbezR.SmithM.WhittonM. C. (2021). Revisiting Milgram and Kishino's reality-virtuality continuum. Front. Virtual Real.2. 10.3389/frvir.2021.647997

  • CrossRef
  • Google Scholar

• 43

  SuS.WangR.ZhouR.ChenZ.ZhouF. (2023). The effectiveness of virtual reality, augmented reality, and mixed reality training in total hip arthroplasty: a systematic review and meta-analysis. J. Orthop. Surg. Res.18, 121. 10.1186/s13018-023-03604-z

  • CrossRef
  • Google Scholar

• 44

  SunP.ZhaoY.MenJ.MaZ.-R.JiangH.-Z.LiuC.-Y.et al (2023). Application of virtual and augmented reality Technology in hip surgery: systematic review. J. Med. Internet Res.25, e37599. 10.2196/37599

  • CrossRef
  • Google Scholar

• 45

  SyamlanA.FathurachmanD. K.Vander PoortenE.PramujatiB.TjahjowidodoT. (2022). Haptic/virtual reality orthopedic surgical simulators: a literature review. Virtual Real.26, 1795–1825. 10.1007/s10055-022-00666-y

  • CrossRef
  • Google Scholar

• 46

  TeneT.Vique LópezD. F.Valverde AguirreP. E.Orna PuenteL. M.Vacacela GomezC. (2024). Virtual reality and augmented reality in medical education: an umbrella review. Front. Dig. Health6, 1365345. 10.3389/fdgth.2024.1365345

  • CrossRef
  • Google Scholar

• 47

  ToaleC.MorrisM.KavanaghD. O. (2023). Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence. Surg. Endosc.37, 1658–1671. 10.1007/s00464-022-09593-0

  • CrossRef
  • Google Scholar

• 48

  VerheyJ. T.HaglinJ. M.VerheyE. M.HartiganD. E. (2020). Virtual, augmented, and mixed reality applications in orthopedic surgery. Int. J. Med. Robot.16, e2067. 10.1002/rcs.2067

  • CrossRef
  • Google Scholar

• 49

  WeikD.LorenzM.KnoppS.PellicciaL.FeierabendS.RotschC.et al (2019). “Integrating tactile feedback in an acetabular reamer for surgical VR-training,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (IEEE), 1227–1228.

  • Google Scholar

• 50

  WieseA.WilliamsG.LecakesG.MorleyM.KimT. W.AlmonA.et al (2021). “Virtual medical instruments for orthopedic surgery training: a hip arthroplasty application,” in 2021 IEEE Sensors Applications Symposium (SAS) (IEEE), 1–6.

  • Google Scholar

• 51

  WuL.SeiboldM.CavalcantiN. A.HeinJ.GerthT.LekarR.et al (2025). A novel augmented reality-based simulator for enhancing orthopedic surgical training. Comput. Biol. Med.185, 109536. 10.1016/j.compbiomed.2024.109536

  • CrossRef
  • Google Scholar

• 52

  YounisS. B.Al-HemiaryE. H. (2021). “Immersive virtual reality application for total hip replacement surgical training,” in 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART) (IEEE), 1–5.

  • Google Scholar

• 53

  ZhangW.LiuX.ZhengB. (2021). Virtual reality simulation in training endoscopic skills: a systematic review. Laparosc. Endosc. Robot. Surg.4, 97–104. 10.1016/j.lers.2021.09.002

  • CrossRef
  • Google Scholar