Assessing the Value of a Novel Augmented Reality Application for Presurgical Planning in Adolescent Elbow Fractures

Objectives: We retrospectively assess the potential impact of a novel, investigational Augmented Reality (AR) software application, Radiology with Holographic Augmentation (RadHA), on pediatric orthopedic surgeon's confidence in surgical planning, hardware selection, hardware fit, and estimated potential intraoperative time savings in the setting of complex adolescent elbow fractures. Methods: After study selection, 12 individual cases of complex elbow fractures in adolescent pediatric patients were identified for review. AR models were generated for each case derived from the patient's CT. Five fellowship-trained pediatric orthopedic surgeons reviewed each case for a total of 60 separate observations. Surgeons reviewed clinical data, radiologic imaging, and AR models and then answered Likert Scale questions on measures of confidence in presurgical planning and projected potential time savings. These data were reviewed and analyzed using various statistical tools. Results: Surgeons reported high confidence in the quality of the AR models created. Additionally, surgeons reported increased confidence in their surgical plan, increased confidence in hardware selection, and increased confidence in hardware fit. Within the sub-analysis of complex (comminuted) fractures, surgeons reported greater expected increases in confidence of their surgical plan and hardware fit. Overall, surgeons estimated potential intraoperative time savings, averaging 17.3 min for all fracture types and 17.6 min for complex fractures. Conclusions: Preoperative planning using AR-based models can increase surgeon confidence in preoperative planning, hardware selection, and confidence in hardware fit.


HIGHLIGHTS
-Evaluated the utility of AR holograms in pre-operative planning within the use of case of pediatric elbow fractures. -Characterized the potential impact of AR on key measures, such as surgeon confidence and potential operative time savings.
FIGURE 1 | Consort diagram detailing specific search criteria for obtaining our representative cases.

INTRODUCTION
Repair of complex fractures in the pediatric orthopedic patient can be challenging, as often the exact type and approach of repair is dependent on the degree, pattern, and relationships of osseous fragmentation (Houshian et al., 2001;Abzug and Herman, 2012). Distal humeral fracture patterns, in particular, differ in adolescent patients, who often demonstrate a more complex "adult-type" fracture, compared to juveniles, who are more likely to have a straightforward supracondylar, lateral condyle, or medial epicondyle fracture pattern, all of which have standard treatment algorithms. These "adult-type" adolescent distal humerus fractures may require open reduction and internal fixation with plating, possible olecranon osteotomy; more typical juvenile patterns often require no more than closed reduction and pin fixation (Handelsman et al., 2006;Howard et al., 2012). For complex intra-articular cases, cross-sectional imaging plays an integral role in preoperative workup, including evaluating fracture patterns, determining a specific surgical approach, and selecting appropriate hardware (Koh et al., 2010;Park et al., 2017). The addition of three-dimensional (3D) imaging has been shown to be of potential benefit in other areas of orthopedic surgical planning (Wicky et al., 2000;Schepers et al., 2016;Fotouhi et al., 2018). However, conventional 3D reconstructions displayed on a two-dimensional flat screen may not always provide sufficient demonstration of complex, comminuted fractures and the relationships of various fracture fragments (Atesok and Schemitsch, 2010). These reformations are typically generated by the CT technologist and are limited to arbitrary planes of rotation, often not demonstrating the relevant plane the orthopedic surgeon requires for operative planning. Furthermore, these reconstructions are typically performed at the time of acquisition, and generating post-hoc reconstructions can be challenging and cumbersome.
Interventions that can help reduce operative time and increase surgeon confidence are of utmost importance in the pediatric patient. Within healthcare, a variety of different technologies have been developed to help physicians leverage patient-specific imaging (Wicky et al., 2000;Schepers et al., 2016). These include the utilization of 3D reconstructions that are usually provided at the time of image acquisition, 3D printing, and digital technologies such as Virtual Reality (VR), Augmented Reality (AR), and other types of Mixed Reality (MR) visualizations, where digital information is displayed on varying levels of digital backgrounds. Augmented Reality has been utilized in a variety of different applications, including military, industry, and increasingly, within healthcare (Fotouhi et al., 2018;Wake et al., 2020). Many of these applications leverage AR's ability to superimpose digital images on the physical world and use this superimposition to help users achieve specific tasks. AR applications have the potential to enhance visualization of complex medical imaging and add higher degrees of interactivity than conventional 3D reformations on two-dimensional flat screens (Vávra et al., 2017). Improved visualization of complex imaging can potentially be beneficial in reducing operating room times and complication FIGURE 3 | 3D image representation of the elbow fracture, optimized for visualization using Augmented Reality (AR). FIGURE 4 | Screen capture placeholders for video clip of the same complex elbow fracture, (A) as seen through an Augmented Reality (AR) head mounted display and (B) highlighting a small fracture fragment (yellow circle), which was reported to be better appreciated on the AR model by reviewers.  Mark only one oval.
Negatively impact my plan (5) Minor negative deviation in my plan (4) No impact on my surgical plan (3) Minor improvement in my surgical plan (2) Major positive change in my surgical plan (1) 3. Would review of this model before surgery improve your confidence in your surgical plan?
Mark only one oval.
Negatively affect my confidence (5) Minor negative affect in my confidence (4) No impact on my confidence in my surgical plan (3) Minor improvement in confidence in my surgical plan (2) Major improvement in confidence in my surgical plan (1) 4. Would review of this model before surgery improve your confidence in your hardware selection?
Mark only one oval.
Negatively affect my confidence (5) Minor negative affect in my confidence (4) No impact on my confidence in my hardware selection (3) Minor improvement in confidence in my hardware selection (2) Major improvement in confidence in my hardware selection (1) 5. Would review of this model before surgery improve your confidence in your hardware fit?
Mark only one oval.
Negatively affect my confidence (5) Minor negative affect in my confidence (4) No impact on my confidence in hardware fit (3) Minor improvement in confidence in hardware fit (2) Major improvement in confidence in hardware fit (1) 6. Is review of A.R. models useful for presurgical planning?
Mark only one oval. Frontiers in Virtual Reality | www.frontiersin.org rates and increasing physician confidence prior to performing an operation, specifically by improving user understanding and awareness of complex anatomy that can sometimes be lost in translation with current visualization standards (Oishi et al., 2013;Hassani et al., 2014;Schepers et al., 2015). Cost savings in terms of more-efficient use of hardware has also been demonstrated using digital templating software, where preoperative planning can help in anticipating appropriate hardware needs specific to a patient, and prior to the intervention, preventing the need to open and potentially waste numerous mal-sized hardware during a procedure (Hsu et al., 2012). These improvements in visualization can become even more important in the pediatric patient, where reductions in OR time, length of anesthesia, and long-term sequelae of repair can have significant consequences (Mehlman et al., 2001;Ramachandran et al., 2008;Howard et al., 2012). It is critical to utilize available tools to try and limit time under anesthesia, particularly in pediatric patients, who may still be undergoing cognitive development. While time for precise characterization and surgical planning is available in non-emergent surgical cases, this time is finite and can be shortened if users are provided with the appropriate tools and imaging to best plan for a procedure (Leet et al., 2002). In many instances, given the precise nature of complex repairs, having an accurate 3D representation of the underlying anatomy is critical.
Previous work with quality control phantoms have demonstrated a high degree of accuracy and precision of AR models in comparison with gold standard measurements (Oral, Scientific, andEducational Abstracts 2018, 2019). Early preliminary work has demonstrated preplanning with 3D models has the potential to decrease operating room times (Jacobs et al., 2008;Zhang et al., 2011;Lethaus et al., 2012) and improve accuracy in hardware placement (Teber et al., 2009;Elmi-Terander et al., 2016;Zhang et al., 2017).
In this study, we retrospectively assess the potential impact of a novel AR software application, Radiology with Holographic Augmentation (RadHA), on pediatric orthopedic surgeons' confidence in surgical planning, hardware selection, hardware fit, and estimated potential intraoperative time savings in the setting of complex adolescent elbow fractures.

MATERIALS/METHODS
Approval for the study was obtained through our institutional review board. A total of 12 cases of intra-articular elbow fractures of varying levels of complexity in pediatric patients 10-18 years of age were identified through retrospective review using specified search criteria (Figure 1). Search criteria included patients <18 years of age, undergoing CT, with reports including  (29) FIGURE 5 | Times savings averages by reviewers for all and comminuted fracture types.
Frontiers in Virtual Reality | www.frontiersin.org "comminuted, " "intra-articular, " and "elbow." Digital Imaging and Communications in Medicine (DICOM) images (Figure 2) were converted to 3D.stl mesh files using OSIRIX MD (Pixmeo SARL, Bernex Switzerland) and optimized for AR using Blender (Stichting Blender Foundation, Amsterdam, Netherlands) (Figure 3). The 3D images were adapted (Figures 4A,B) and loaded onto the Microsoft HoloLens (Microsoft Corporation, Redmond, WA, USA) platform using a novel, investigational software application written in C# programming language-based code on the Unity Platform (Unity Technologies, San Francisco, CA, USA), which was developed at our institution. Abstracted pertinent clinical information ( Table 1) and conventional CT DICOM data of the 12 models were reviewed by five separate fellowship-trained attending pediatric orthopedic surgeons (averaging 4.6 years post-fellowship completion, ranging between 2 and 8 years). Subsequently, the same surgeons evaluated an AR model derived from the same CT DICOM data. Surgeons then completed a survey with multiple Likert scale questions regarding use of AR models for planning ( Table 2), and included their own subjective estimation of time savings based on use of these AR models (Figure 5 and Table 3). A one sample, two-tailed t test was performed to compare the mean Likert responses to a null hypothesis of no significant change upon review of AR models (Likert = 3) among all and comminuted fracture types (

RESULTS
Of the 25 total reports found using the specified search criteria, 13 unique patients were present. A single study was excluded on the basis of absence of acute imaging. A total of 12 patients were therefore included in the study [average age, 13.8 (range, 11-17),  Table 1. Summary of responses to survey questions (Figures 6A,B) and summary of responses separated by reviewer (Figures 7A,B) are provided. Surgeons rated models "highly accurate" (median = 1, mode = 1, interquartile range = 0). Raters noted no overall change in surgical plan (median = 3, mode = 3, interquartile range = 1). Raters noted improved confidence in surgical plan (median = 2, mode = 1, interquartile range = 1). Confidence scores for hardware selection were on average mildly increased (median = 2, mode = 2, interquartile range = 2), as were confidence scores in hardware fit (median = 2, mode = 2, interquartile range = 1), and utility for planning for specific fracture type (median = 2, mode = 2, interquartile range = 1). Four of the five orthopedic surgeons felt comfortable providing time estimations (the surgeon who most recently completed training abstained, Reviewer 3). Overall average estimated intraoperative time savings for all fracture types =  Table 3). When asked to clarify/expand on how AR models would impact case or affect time estimates, surgeons responded with "better understanding of alignment/fracture read, " "better understanding of surgical approach, " "can see extent better, " "retractor positioning, " and "better understanding of relationships of fragments" ( Table 6). A Kruskal-Wallis test was performed to evaluate if there was any significant difference among between different raters and their survey responses. Overall, we did not find a statistical difference between respondents, both in the case of all fractures and in the case of comminuted fracture types ( Table 5). This is also graphically depicted (Figures 7A,B), where mean Likert responses and 95% confidence intervals are displayed between reviewers, visually confirming the Kruskal-Wallis test results. Additionally, a t-test ( Table 4) allowed us to reject our null hypothesis that the review of AR models would result in no difference among surgeons preparing for pediatric elbow fracture repair.

DISCUSSION
In this study, we evaluated orthopedic surgeons' assessment of 3D AR models for presurgical planning in complex elbow fractures. Surgeons reported high confidence in the quality of the AR models created. Additionally, surgeons reported increased confidence in their surgical plan, increased confidence in hardware selection, and increased confidence in hardware fit. Within the subanalysis of comminuted fractures, there was a greater overall increase in surgeon confidence in surgical plan. Overall, there was a projected intraoperative time savings (averaging 17.3 min for all fracture types and 17.6 min for comminuted fractures). The most experienced of the surgeons interestingly estimated that the use of this technology could have up to a 30-min time reduction in the cases of complex fractures.
The improved 3D visual spatial awareness of the fracture fragments and fracture patterns, as cited by the surveyed surgeons, is similar with previously reported studies of 3D imaging use in orthopedics (Sasso and Garrido, 2007;Citak et al., 2008;Yu et al., 2015). This includes a study of 3D modeling for pelvic acetabular fractures, which allowed for improved hardware placement and reduced soft tissue trauma (Yu et al., 2015). Notably, reviewers in this study found that AR models provided additional value over standard 3D reformations through improved ability to interact with and freely manipulate TABLE 6 | Responses to "any other comments or clarification on how these AR models change your management or potentially help reduce operative time?" -Can see the comminution better and where the fragments are located better than on CT -Help with decision for tension band vs. screw; helps you decide the size and the obliquity for that decision -Useful for hardware fit; would use a plate -Would go both medial and lateral; would have done just lateral, but can see there is another fragment on the model -Would have changed from pinning to open; it makes the fracture clearer -Would change her "read" of the fracture; would change from anterior to superior approach; might lengthen surgery, but would improve overall outcome -Would change the how I'd approach; looking at this makes it more likely I'd have recognized the need for bone grafting; allograft, decide how to approach first -Shows where you will look for your fracture reduction read; looking for the apex of a triangle, shows that nicely.
-Helpful to envision where your fracture reads are going to be, what you will look for to line up; where your plates will fit -Complex intra-articular with a bad CT, gives you more info on how it will fit together, relationships of fragments and where you will get your fracture "read" Furthermore, the ability to interact with real world objects with the AR models was reported as an advantage, providing the potential opportunity for hardware sizing.
Time reduction of ∼18 min per case in cases of complex fractures in our study, while an estimation, is in keeping with other studies of time reduction using 3D visualization (Sasso and Garrido, 2007;Citak et al., 2008). Physicians reported a variety of mechanisms for time reduction (Table 6). These included enhanced visualization of relevant anatomy, reducing total preoperative review time; prevention of unnecessary procedures during a case, such as an arthrogram or fluoroscopy to understand anatomy; precise hardware selection, reducing time spent deciding between hardware types; aid in appropriate surgical technique, either in approach or specific technique, potentially reducing intraprocedural time; and aids operator "read" of fracture pattern, possibly reducing time spent reviewing CT prior to surgery. Reported operating times for repair of complex distal humeral fractures in a study in the adult population averaged 2.5 h (Frankle et al., 2003). Furthermore, reported costs per minute of operating room time range from $36.50/min to $133.12, averaging $66/min (Shippert, 2005;Macario, 2010;Childers and Maggard-Gibbons, 2018). While an estimation, using the provided estimates of our study, this could represent a potential 12% overall decrease in time per case and a reduction in cost of $1,122 per case.
The time to segment, process, and optimize orthopedic models for this AR application is <5 min, as the processing is done electronically. This increases the potential for such applications to be used more broadly than alternative options such as 3D printing that require more resources and infrastructure. Further, the opportunity for interaction between physical orthopedic hardware and scale 3D holographic models provides a degree of interactivity that is beyond conventional 3D-rendered reformats displayed on a 2D monitor.
Our study was not without limitations. In addition to the retrospective nature of the study, our study was limited by the subjective assessment of projected time savings and by the differences in training and degree of experience by the surgeons reviewing models. Further, while the models available for review had limitations in degree of interactivity in that the fracture fragments on the tested version could not be manipulated or repositioned, this was a feature that reviewing surgeons stated would have been useful in planning. Within these limitations, we do believe that our study not only provides critical information on the improvement in surgeon confidence in plan and hardware choice but also provides an early preliminary window into the potential economic impact that this technology can provide, which may influence further adoption.
Preoperative planning using AR-based models statistically increases surgeon confidence in preoperative planning, hardware selection, and confidence in hardware fit. This study lays a foundation for further, larger-scale prospective trials assessing AR technology for planning in a wider variety of orthopedic fracture types. Additionally, further testing should be focused on the safety and barriers of using intraoperative AR systems for surgical guidance and reference.

DATA AVAILABILITY STATEMENT
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
This work was conducted with approval of our University's Institutional Review Board.

AUTHOR CONTRIBUTIONS
BL: manuscript preparation, statistical analysis, and figure legends. KL: study design, survey design input, performed measurements, and manuscript preparation. RB, JJ, NP, and CS: manuscript preparation and performed measurements. JC: study design, manuscript preparation, figure annotation, video editing, statistical analysis, and project oversight. All authors contributed to the article and approved the submitted version.

ACKNOWLEDGMENTS
Preliminary findings of this work were presented at the 2018 Annual Meeting of the American Roentgen Ray Society, Washington, DC. Microsoft HoloLens v1 was used with permission from Microsoft (this equipment was purchased and owned by the BL). We thank Dr. Ivor Asztalos for statistical guidance.