Your new experience awaits. Try the new design now and help us make it even better

ORIGINAL RESEARCH article

Front. Surg., 23 October 2025

Sec. Neurosurgery

Volume 12 - 2025 | https://doi.org/10.3389/fsurg.2025.1679617

This article is part of the Research TopicDoing More with Less: Neurosurgery Strategies and Tricks of the Trade in the Technological EraView all 12 articles

Inter-observer agreement in interpreting intraoperative ultrasonography during brain tumour surgery


Aimun A. B. Jamjoom,
Aimun A. B. Jamjoom1,2*Olivier J. J. SluijtersOlivier J. J. Sluijters3Jack WildmanJack Wildman3Davide Giampiccolo
Davide Giampiccolo4*Constantinos CharalambidesConstantinos Charalambides3Neil U. Barua
Neil U. Barua3
  • 1Department of Neurosurgery, Queens Hospital, Romford, United Kingdom
  • 2The Blizard Institute, Queen Mary University of London, London, United Kingdom
  • 3Department of Neurosurgery, Southmead Hospital, Bristol, United Kingdom
  • 4Department of Clinical and Experimental Epilepsy, Institute of Neurology, University College London, London, United Kingdom

Background: Intraoperative ultrasonography (iUS) is a powerful technology that is being increasingly utilized for brain tumour surgery. However, ultrasonography has been documented to be observer dependent in a range of healthcare settings. Here we objectively assess the degree of inter-observer variability in iUS for brain tumour surgery.

Methods: Nine images taken from routinely collected iUS videos from brain tumour surgery were presented to 18 neurosurgeons (5 consultants, 7 senior fellows, 6 residents). This included three tumour types [metastasis, high-grade (HGG) and low-grade glioma (LGG)] at three operative stages (before, during and near resection completion). Using 3D Slicer, observers segmented what they deemed to be tumour. Digital Image Correlation Engine Similarity Coefficients (DSC) were calculated to examine inter-observer variability.

Results: A total of 1,377 DSCs were calculated between 18 observers across 9 images. Metastasis had the highest DSC (0.72 ± 0.32), followed by HGG (0.64 ± 0.33) and LGG (0.58 ± 0.25; p < 0.00001). As the resection progressed, the degree of inter-observer agreement broke down. Before resection the DSC was 0.87 ± 0.11; during resection (0.74 ± 0.17) and at completion (0.32 ± 0.27; p < 0.00001). The trend of decreasing agreement as the resection progressed held across tumour types. Observers reported increasing difficulty with iUS interpretation as the resection proceeded and there was statistically significant (p = 0.014) negative correlation (−0.775) between DSC and difficulty rating of the segmentation.

Conclusion: Here we demonstrate significant inter-observer variability in iUS for brain tumour surgery. The degree of variability is tumour-type and operative stage dependent. This work adds weight to the value of building machine learning augmented iUS for brain tumour surgery.

Introduction

Intraoperative ultrasonography (iUS) is a powerful technology that is increasingly being utilized to support surgical resection of brain tumours. iUS has several benefits compared to other intra-operative imaging modalities such as intraoperative MRI including: real-time visualisation of the tumour, more readily available and cost efficient to implement. Importantly, it is also easily integrated into surgical work-flows and does not increase operative time as significantly as intra-operative MRI (1). Synthesis of the literature points towards iUS enhancing glioma extent of resection which is an important determinant of improved survival (2, 3). Though point-of-care ultrasonography (performed and interpreted by the treating clinician at bedside) is user-friendly, there are learning curves for the psychomotor skills required and image interpretation (4).

There is evidence of variable inter-observer agreement in interpreting ultrasound images from different organs including thyroid, pelvic, musculoskeletal and lung examinations (59). Interpreting iUS images in brain tumour surgery is complicated by posterior enhancement of the floor of the resection cavity (10). A study examining agreement across 30 brain tumour iUS images found moderate levels of inter-observer variance (11). Importantly, this study did not examine the impact of operative stage on segmenting iUS images. This is a limitation as the most difficult and important point to interpret iUS images and near the end of tumour resection when the surgeon is using ultrasound to help assist identifying residual tumour. Our study aims to evaluate the degree of inter-observer agreement between a group of neurosurgeons in interpreting iUS images from three different brain tumours and across three operative stages.

Methods

A total of nine anonymized representative iUS images were captured from three procedures (Figure 1). The GE healthcare bk5000 ultrasound device was utilized. The iUS images were optimised by the operating surgeon (NB) across 3 parameters: frequency (5–13 MHz), depth (10–80 mm) and gain. The images included a histologically confirmed high grade glioma (HGG), low grade glioma (LGG) and brain metastasis. We chose the commonest intra-axial tumours treated surgically in our department with a known span of margin ambiguity. During each procedure, images were captured at three stages: before tumour resection, during resection and near completion of resection. A total of 18 neurosurgeons participated in the study. First, participants completed a survey detailing their grade (attending, fellow or resident) and their experience with using intra-operative ultrasonography for brain tumour surgery (<10 cases; 10–25 cases; 25–50 cases or >50 cases). Using 3D Slicer, participants were invited to draw around (segment) what they believed was tumour in the nine images. Participants were supported with the use of 3D Slicer by members of the research team (AJ, JW, OS). Participants were free to decide how many areas to segment, including none. For each image, the participant also rated how difficult they found the segmentation on a Likert scale from 1 (very easy) to 10 (very hard). Ethical approval was waived for this study as it used anonymised images captured as part of routine practice and involved voluntary participation by neurosurgeons.

Figure 1
Ultrasound images showing three rows labeled Metastasis, HGG, and LGG, with three columns labeled Before, During, and After. Each cell displays an ultrasound image illustrating the progression or treatment at each stage for the conditions Metastasis, High-Grade Glioma (HGG), and Low-Grade Glioma (LGG).

Figure 1. Intra-operative B-mode ultrasound images from three tumour types across three stages of surgery.

Statistical analysis

Using Python through the ChatGPT Data Analyst interface, segmentations were converted into binary masks and then analysed using the Digital Image Correlation Engine Similarity Coefficient (DSC) which measures the similarity between two images by comparing pixel patterns (12). DSC range from 0 to 1 with segmentations that are perfectly aligned getting a score of 1 and segmentations that are not aligned at all getting a score of 0. For each image, every observer was compared against the other, producing a DSC for each comparison. In this study, a total of 153 DSCs were calculated for each image across the 18 participants. Heat maps were created for each image by converting the binary masks into arrays and allocating each pixel an aggregate score based upon whether it was incorporated within the segmented tumour. Each pixel could have a score ranging from 0 (all participants deemed this pixel not to be tumour) to 18 (all participants deemed this pixel to be tumour). The following colour code was then allocated to the pixels based on their aggregate score: green (15–18), red (4–14) and white (0–4). This heatmap was made transparent and then overlaid onto the original iUS image. Comparisons between the stages of surgery and tumour type were performed with the Kruskal–Wallis test with Dunn's multiple comparisons test. Correlation between the average difficulty ratings and DSCs for the 9 iUS images was performed using Spearman's correlation. Comparisons between high and low experience surgeons across tumour type and operation stage was performed using two-way ANOVA with Tukey multiple comparisons test. Statistical analysis and graphical visualisation were conducted using R through the BioRender interface.

Results

A total of 18 neurosurgeons participated in the study including 5 attendings, 7 fellows and 6 residents. From this group, a total of 1,377 DSCs were calculated across the 9 iUS images. Metastasis had the highest DSC (0.72 ± 0.32), followed by HGG (0.64 ± 0.33) and then LGG (0.58 ± 0.25; p < 0.00001) (Figure 2A). As the resection progressed, the degree of inter-observer agreement broke down. Before resection the DSC was 0.871 ± 0.105; during resection (0.74 ± 0.174) and at completion (0.32 ± 0.270; p < 0.00001) (Figure 2B). Observers reported increasing difficulty with iUS interpretation as the resection proceeded and there was statistically significant (p = 0.014) negative correlation (−0.775) between DSC and difficulty rating of the segmentation (Figure 2C). The trend of decreasing agreement as the resection progressed held across tumour types. It was most pronounced in HGG where there was a near complete breakdown of agreement in the resection cavity (0.26 ± 0.28) (Figure 3). Heats maps for the iUS demonstrated the spatial variation in observer agreement across the tumour types and stages of surgery (Figure 4). It shows decreasing agreement as the resection proceeds particularly within the margins of the resection cavity near the end of the resection. From the 18 participants, there were 5 observes who were highly experienced with iUS (>50 cases) and 6 observers with low experience (<10 cases). There was no significant difference in DSCs between the high and low experience observers across the tumour types and stages of surgery.

Figure 2
Three-panel image showing data visualizations. Panel A: Box plot comparing DSC values for Metastasis, HGG, and LGG, all significant at four asterisks. Panel B: Box plot comparing DSC values before, during, and after treatment, with significant differences. Panel C: Scatter plot showing a negative correlation between difficulty ratings and DSC values, with a trend line and shaded confidence interval.

Figure 2. Comparison of DSC between the three tumour types (A) between the three stages of surgery (B) and correlation between DSC and difficulty rating (C).

Figure 3
Three box plots show DSC values for Met, HGG, and LGG across three conditions: Before, During, and After. The Met plot shows a decrease in values from Blue to Red to Green. The HGG plot follows a similar trend, with significant changes indicated. The LGG plot also shows a decrease with significance levels noted. Each plot indicates statistical significance with asterisks.

Figure 3. Tumour-specific comparison of DSC across three stages of surgery.

Figure 4
Ultrasound images of brain tumors shown in three stages: before, during, and after treatment. The top row depicts metastasis, the middle row shows high-grade glioma (HGG), and the bottom row represents low-grade glioma (LGG). Tumor areas are highlighted in green indicated greater than 90% agreement and the areas in red indicate 20%-79% agreement.

Figure 4. Spatial heat maps of observer agreement across 9 iUS images (green >80% agreement; red 20%–79% agreement).

Discussion

In this study, we report the an assessment of inter-observer agreement in interpreting iUS images from brain tumour surgery. We found that inter-observer agreement was both tumour-type and operation-stage dependent. Agreement was strongest for brain metastases and then diminished for diffuse primary brain tumours. Also, agreement started strong before the surgical resection commenced but broke down as the surgery proceeded. When we conducted 2D spatial analysis, we found that the breakdown in agreement was most marked at the base of the resection cavity. Extent of resection is an important factor in improving survival in patients with brain tumours (17, 18). A range of technologies have been introduced to support neurosurgeons to enhance the extent of resection during brain tumour surgery including fluorescence-guided surgery, intra-operative MRI and optical neuro-navigation (13). Ultrasonography is part of this armamentarium and there is evidence that its use can improve extent of resection (2, 3). However, iUS is user dependent as it requires real-time subjective interpretation of images. This is particularly relevant in brain tumour surgery, as interpretation of the surgical cavity margins is important in supporting intra-operative decision-making to extend the resection or not.

Inter-observer agreement in ultrasonography

Broadly, there is good inter-observer agreement in the use of ultrasonography in medicine (5, 14, 15). However, this agreement can be variable and starts to breakdown for certain measurements and assessments. A study examining interpretation of thyroid nodule ultrasound images found that there was strong agreement on determining nodule calcification and vascularity but moderate agreement on the nodule shapes and margins (5). This variability in agreement based on the assessed parameters is important, particularly in the context of iUS for brain tumour surgery. Given the importance of extent of resection and the interpretation of tumour margins, our study was specifically designed to probe the question of agreement at the resection cavity margins. Interpreting these images is notoriously difficult due to the phenomenon known as posterior wall acoustic enhancement (PAE) (10). PAE often appears beneath fluid-filled structures like cysts as fluid attenuates ultrasound less than solid tissue, creating stronger echoes and higher echogenicity. This can complicate differentiation between residual tumour and PAE at the base of resection cavities. Chacko and colleagues took samples from resection cavities and found the sensitivity of iUS in detecting tumour at the tumour-brain interface was 97.1%, however its specificity was lower at 53.6% (16). The high false positive rate is likely related to PAE and hyperechoic clot. A novel approach to minimise PAE is to use a coupling fluid designed to have an attenuation coefficient similar to brain tissue which helps reduce the enhancement artifacts caused by saline solutions typically used in resection cavities (19). The difference in DSC between tumour types is interesting and relates to differing echogenic signal of each tumour. Metastases and HGG usually appear more hyperechoic, irregular, and well-circumscribed on iUS, often with hypoechoic necrotic areas and hyperechoic oedema, whereas LGG are typically iso- to mildly hyperechoic with less distinct margin (10). This distinction makes it easier to segment metastases and HGG compared to LGG which is supported by our study.

Role of experience in inter-observer agreement

The experience of observers has been shown to improve inter-observer agreement in MRI (20). The theory behind this is that experienced observers coalesce around an accurate diagnosis compared to less experienced observers where is a higher chance of random or incorrect interpretations. Our study found that there was no significant difference between the observers based on their experience. The ultrasound literature has shown conflicting findings on the impact of experience on inter-observer agreement. A study examining transvaginal ultrasound for local staging of cervical cancer found that experience improved observer agreement for only one of three measured metrics (21). Conversely, the Swiss Sonography in Arthritis and Rheumatism (SONAR) group developed a semi-quantitative score for synovitis and erosion in Rheumatoid Arthritis and found that the experience of the sonographer substantially improved agreement (22). The explanation for the variability in these findings may be related to the highly user dependent nature of ultrasound. Given raters would need to position the ultrasound probe to generate the image for interpretation, this adds a further layer of variability which may impact the degree of agreement. This may provide an explanation for why we did not identify differences between low and high experience observers. Experienced observers would manipulate the ultrasound probe in a way that helps them identify residual tumour and this important facet of experience is lost when using still images which are more open to random interpretation.

Future directions

Artificial intelligence is playing an increasing large role in neurosurgery. This spans improving surgical workflows, real-time monitoring and diagnosis, outcome prediction, volumetric assessment, and neurosurgical education (2325). One approach to address this agreement gap in iUS, is to take a quantitative approach to interpreting the ultrasound images. Basic image analysis techniques for iUS for brain tumour surgery showed that pixel brightness correlated with histological features (26). More recently, machine learning approaches have been utilised for both real-time tumour differentiation and histological and molecular diagnosis (2729). Cepeda and colleagues characterized quantitative texture analysis in B-mode and elastography which was found to be significantly associated with overall survival (30). More recently, a multicentre study using the brain tumour intraoperative ultrasound database (BraTioUS) demonstrated the feasibility of a convolutional neural network (CNN) model for glioma segmentation (31).

Limitations

The study had several limitations. The main limitation was that we did not have a ground truth to compare the observers' segmentations to. This could have included histological biopsies that were correlated to the iUS images to help determine what was truly tumour and what was not in the image. Secondly, the experimental design differed from routine practice with static images presented to the observer to interpret. This is different to real life situations where surgeons can move the probe to get real-time feedback which can help with interpreting the iUS images. Finally, our study included 9 images which is less compared to other studies in the literature (11). We optimised the study to maximise the number of observers and collected responses from 18 neurosurgeons which is a major strength of the study.

Conclusions

This study demonstrates that inter-observer agreement in interpreting iUS images during brain tumour surgery varies significantly by tumour type and resection stage. The findings underscore the challenges of accurately interpreting tumour margins, particularly in the resection cavity. While experience did not significantly impact agreement, the study highlights the need for quantitative approaches using machine learning to improve consistency and accuracy in iUS interpretation for brain tumour surgery.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

AJ: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. OS: Data curation, Project administration, Writing – original draft, Writing – review & editing. JW: Data curation, Project administration, Writing – original draft, Writing – review & editing. DG: Data curation, Methodology, Project administration, Resources, Writing – original draft, Writing – review & editing. CC: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing. NB: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research and/or publication of this article.

Acknowledgements

We would like to extend our appreciation to the neurosurgeons who took time out of their busy schedule to contribute to study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Mosteiro A, Di Somma A, Ramos PR, Ferrés A, De Rosa A, González-Ortiz S, et al. Is intraoperative ultrasound more efficient than magnetic resonance in neurosurgical oncology? An exploratory cost-effectiveness analysis. Front Oncol. (2022) 12:1016264. doi: 10.3389/fonc.2022.1016264

PubMed Abstract | Crossref Full Text | Google Scholar

2. Mahboob S, McPhillips R, Qiu Z, Jiang Y, Meggs C, Schiavone G, et al. Intraoperative ultrasound-guided resection of gliomas: a meta-analysis and review of the literature. World Neurosurg. (2016) 92:255–63. doi: 10.1016/j.wneu.2016.05.007

PubMed Abstract | Crossref Full Text | Google Scholar

3. Palavani LB, Ferreira MY, Borges PGLB, Bandeira L, da Silva Semione G, Almeida MV, et al. Ultrasound-guided resection of high-grade gliomas: a single-arm meta-analysis. World Neurosurg. (2024) 186:17–26. doi: 10.1016/j.wneu.2024.03.033

PubMed Abstract | Crossref Full Text | Google Scholar

4. Breunig M, Hanson A, Huckabee M. Learning curves for point-of-care ultrasound image acquisition for novice learners in a longitudinal curriculum. Ultrasound J. (2023) 15(1):31. doi: 10.1186/s13089-023-00329-2

PubMed Abstract | Crossref Full Text | Google Scholar

5. Alyami J, Almutairi FF, Aldoassary S, Albeshry A, Almontashri A, Abounassif M, et al. Interobserver variability in ultrasound assessment of thyroid nodules. Medicine (United States). (2022) 101(41):E31106. doi: 10.1097/MD.0000000000031106

PubMed Abstract | Crossref Full Text | Google Scholar

6. Choi SH, Kim EK, Kwak JY, Kim MJ, Son EJ. Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid®. (2009) 20(2):167–72. doi: 10.1089/thy.2008.0354

PubMed Abstract | Crossref Full Text | Google Scholar

7. Bean E, Chaggar P, Thanatsis N, Dooley W, Bottomley C, Jurkovic D. Intra- and interobserver reproducibility of pelvic ultrasound for the detection and measurement of endometriotic lesions. Hum Reprod Open. (2020) 2020(2):hoaa001. doi: 10.1093/hropen/hoaa001

PubMed Abstract | Crossref Full Text | Google Scholar

8. Micu MC, Serra S, Fodor D, Crespo M, Naredo E. Inter-observer reliability of ultrasound detection of tendon abnormalities at the wrist and ankle in patients with rheumatoid arthritis. Rheumatology. (2011) 50(6):1120–4. doi: 10.1093/rheumatology/keq441

PubMed Abstract | Crossref Full Text | Google Scholar

9. Lerchbaumer MH, Lauryn JH, Bachmann U, Enghard P, Fischer T, Grune J, et al. Point-of-care lung ultrasound in COVID-19 patients: inter- and intra-observer agreement in a prospective observational study. Sci Rep. (2021) 11(1):10678. doi: 10.1038/s41598-021-90153-2

PubMed Abstract | Crossref Full Text | Google Scholar

10. Dixon L, Lim A, Grech-Sollars M, Nandi D, Camp S. Intraoperative ultrasound in brain tumor surgery: a review and implementation guide. Neurosurg Rev. (2022) 45:2503–15. Springer Science and Business Media Deutschland GmbH. doi: 10.1007/s10143-022-01778-4

PubMed Abstract | Crossref Full Text | Google Scholar

11. Weld A, Dixon L, Anichini G, Patel N, Nimer A, Dyck M, et al. Challenges with segmenting intraoperative ultrasound for brain tumours. Acta Neurochir (Wien). (2024) 166(1):317. doi: 10.1007/s00701-024-06179-8

PubMed Abstract | Crossref Full Text | Google Scholar

12. Van Rossum G, Drake FL. Python 3 Reference Manual. (2009).

Google Scholar

13. Fountain DM, Bryant A, Barone DG, Waqar M, Hart MG, Bulbeck H, et al. Intraoperative imaging technology to maximise extent of resection for glioma: a network meta-analysis. Cochrane Database Syst Rev. (2021) 1:CD013630. doi: 10.1002/14651858.CD013630.pub2

PubMed Abstract | Crossref Full Text | Google Scholar

14. Liu H, Ma AL, Zhou YS, Yang DH, Ruan JL, Liu XD, et al. Variability in the interpretation of grey-scale ultrasound features in assessing thyroid nodules: a systematic review and meta-analysis. Eur J Radiol. (2020) 129:109050. doi: 10.1016/j.ejrad.2020.109050

PubMed Abstract | Crossref Full Text | Google Scholar

15. Li J, Chen M, Wang ZJ, Li SG, Jiang M, Shi L, et al. Interobserver agreement for contrast-enhanced ultrasound of liver imaging reporting and data system: a systematic review and metaanalysis. World J Clin Cases. (2020) 8(22):5589–602. doi: 10.12998/wjcc.v8.i22.5589

PubMed Abstract | Crossref Full Text | Google Scholar

16. Chacko AG, Kumar NKS, Chacko G, Athyal R, Rajshekhar V. Intraoperative ultrasound in determining the extent of resection of parenchymal brain tumours—a comparative study with computed tomography and histopathology. Acta Neurochir (Wien). (2003) 145(9):743–8. doi: 10.1007/s00701-003-0009-2

PubMed Abstract | Crossref Full Text | Google Scholar

17. Karschnia P, Gerritsen JKW, Teske N, Cahill DP, Jakola AS, van den Bent M, et al. The oncological role of resection in newly diagnosed diffuse adult-type glioma defined by the WHO 2021 classification: a review by the RANO resect group. Lancet Oncol. (2024) 25(9):e404–19. doi: 10.1016/S1470-2045(24)00130-X

PubMed Abstract | Crossref Full Text | Google Scholar

18. Winther RR, Hjermstad MJ, Skovlund E, Aass N, Helseth E, Kaasa S, et al. Surgery for brain metastases—impact of the extent of resection. Acta Neurochir (Wien). (2022) 164(10):2773–80. doi: 10.1007/s00701-021-05104-7

PubMed Abstract | Crossref Full Text | Google Scholar

19. Selbekk T, Jakola AS, Solheim O, Johansen TF, Lindseth F, Reinertsen I, et al. Ultrasound imaging in neurosurgery: approaches to minimize surgically induced image artefacts for improved resection control. Acta Neurochir. (2013) 155:973–80. doi: 10.1007/s00701-013-1647-7

PubMed Abstract | Crossref Full Text | Google Scholar

20. Tsurusaki M, Numoto I, Oda T, Wakana M, Suzuki A, Yagyu Y, et al. Assessment of liver metastases using CT and MRI scans in patients with pancreatic ductal adenocarcinoma: effects of observer experience on diagnostic accuracy. Cancers (Basel). (2020) 12(6):1–9. doi: 10.3390/cancers12061455

Crossref Full Text | Google Scholar

21. Pálsdóttir K, Fridsten S, Blomqvist L, Alagic Z, Fischerova D, Gaurilcikas A, et al. Interobserver agreement of transvaginal ultrasound and magnetic resonance imaging in local staging of cervical cancer. Ultrasound Obstet Gynecol. (2021) 58(5):773–9. doi: 10.1002/uog.23662

Crossref Full Text | Google Scholar

22. Brulhart L, Ziswiler HR, Tamborrini G, Zufferey P, SONAR/SCQM Programmes. The importance of sonographer experience and machine quality with regards to the role of musculoskeletal ultrasound in routine care of rheumatoid arthritis patients. Clin Exp Rheumatol. (2015) 33(1):98–101.25603279

PubMed Abstract | Google Scholar

23. Nohman AI, Ivren M, Alhalabi OT, Sahm F, Dao Trong P, Krieg SM, et al. Intraoperative label-free tissue diagnostics using a stimulated Raman histology imaging system with artificial intelligence: an initial experience. Clin Neurol Neurosurg. (2024) 247:108646. doi: 10.1016/j.clineuro.2024.108646

PubMed Abstract | Crossref Full Text | Google Scholar

24. Yangi K, On TJ, Xu Y, Gholami AS, Hong J, Reed AG, et al. Artificial intelligence integration in surgery through hand and instrument tracking: a systematic literature review. Front Surg. (2025) 12:1528362, Frontiers Media SA. doi: 10.3389/fsurg.2025.1528362

PubMed Abstract | Crossref Full Text | Google Scholar

25. Yangi K, Hong J, Gholami AS, On TJ, Reed AG, Puppalla P, et al. Deep learning in neurosurgery: a systematic literature review with a structured analysis of applications across subspecialties. Front Neurol. (2025) 16:1532398. Frontiers Media SA. doi: 10.3389/fneur.2025.1532398

PubMed Abstract | Crossref Full Text | Google Scholar

26. Camp SJ, Apostolopoulos V, Raptopoulos V, Mehta A, O’Neill K, Awad M, et al. Objective image analysis of real-time three-dimensional intraoperative ultrasound for intrinsic brain tumour surgery. J Ther Ultrasound. (2017) 5(1):2. doi: 10.1186/s40349-017-0084-0

PubMed Abstract | Crossref Full Text | Google Scholar

27. Xie X, Shen C, Zhang X, Wu G, Yang B, Qi Z, et al. Rapid intraoperative multi-molecular diagnosis of glioma with ultrasound radio frequency signals and deep learning. EBioMedicine. (2023) 98:104899. doi: 10.1016/j.ebiom.2023.104899

PubMed Abstract | Crossref Full Text | Google Scholar

28. Thomson H, Yang S, Cochran S. Machine learning-enabled quantitative ultrasound techniques for tissue differentiation. J Med Ultrason. (2022) 49(4):517–28. doi: 10.1007/s10396-022-01230-6

PubMed Abstract | Crossref Full Text | Google Scholar

29. Cepeda S, Esteban-Sinovas O, Romero R, Singh V, Shett P, Moiyadi A, et al. Real-time brain tumor detection in intraoperative ultrasound: from model training to deployment in the operating room. Comput Biol Med. (2025) 193:110481. doi: 10.1016/j.compbiomed.2025.110481

PubMed Abstract | Crossref Full Text | Google Scholar

30. Cepeda S, García-García S, Arrese I, Velasco-Casares M, Sarabia R. Relationship between the overall survival in glioblastomas and the radiomic features of intraoperative ultrasound: a feasibility study. J Ultrasound. (2022) 25(1):121–8. doi: 10.1007/s40477-021-00569-9

PubMed Abstract | Crossref Full Text | Google Scholar

31. Cepeda S, Esteban-Sinovas O, Singh V, Shetty P, Moiyadi A, Dixon L, et al. Deep learning-based glioma segmentation of 2D intraoperative ultrasound images: a multicenter study using the brain tumor intraoperative ultrasound database (BraTioUS). Cancers (Basel). (2025) 17(2):315. doi: 10.3390/cancers17020315

PubMed Abstract | Crossref Full Text | Google Scholar

Keywords: intra-operative ultrasonography, brain tumour surgery, observer agreement, machine learning, glioma

Citation: Jamjoom AAB, Sluijters OJJ, Wildman J, Giampiccolo D, Charalambides C and Barua NU (2025) Inter-observer agreement in interpreting intraoperative ultrasonography during brain tumour surgery. Front. Surg. 12:1679617. doi: 10.3389/fsurg.2025.1679617

Received: 4 August 2025; Accepted: 29 September 2025;
Published: 23 October 2025.

Edited by:

Roberto Colasanti, Maurizio Bufalini Hospital, Italy

Reviewed by:

Santiago Cepeda, Hospital Universitario Río Hortega, Spain
Alejandra Mosteiro, Hospital Clinic of Barcelona, Spain
Kivanc Yangi, Barrow Neurological Institute (BNI), United States

Copyright: © 2025 Jamjoom, Sluijters, Wildman, Giampiccolo, Charalambides and Barua. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Aimun A. B. Jamjoom, YWltdW4uamFtam9vbUBuaHMubmV0; Davide Giampiccolo, ZC5naWFtcGljY29sb0B1Y2wuYWMudWs=

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.