Skip to main content

ORIGINAL RESEARCH article

Front. Neuroergonomics, 09 January 2023
Sec. Clinical Neuroergonomics
Volume 3 - 2022 | https://doi.org/10.3389/fnrgo.2022.1052411

The neurophysiology of intraoperative error: An EEG study of trainee surgeons during robotic-assisted surgery simulations

Christopher D'Ambrosia1,2* Eliah Aronoff-Spencer3 Estella Y. Huang4 Nicole H. Goldhaber4 Henrik I. Christensen2 Ryan C. Broderick4 Lawrence G. Appelbaum5
  • 1College of Physicians and Surgeons, Columbia University, New York, NY, United States
  • 2Cognitive Robotics Laboratory, Department of Computer Science and Engineering, Contextual Robotics Institute, University of California, San Diego, La Jolla, CA, United States
  • 3Department of Medicine, University of California, San Diego, La Jolla, CA, United States
  • 4Division of Minimally Invasive Surgery, Department of Surgery, University of California, San Diego, La Jolla, CA, United States
  • 5Department of Psychiatry, University of California, San Diego, La Jolla, CA, United States

Surgeons operate in mentally and physically demanding workspaces where the impact of error is highly consequential. Accurately characterizing the neurophysiology of surgeons during intraoperative error will help guide more accurate performance assessment and precision training for surgeons and other teleoperators. To better understand the neurophysiology of intraoperative error, we build and deploy a system for intraoperative error detection and electroencephalography (EEG) signal synchronization during robot-assisted surgery (RAS). We then examine the association between EEG data and detected errors. Our results suggest that there are significant EEG changes during intraoperative error that are detectable irrespective of surgical experience level.

1. Introduction

Surgeons operate in mentally and physically demanding workspaces where the impact of error is highly consequential (Christian et al., 2006; Modi et al., 2018). The cognitive and affective states of surgeons in the operating room (OR) are primary determinants of surgical performance (Schuetz et al., 2008; Yurko et al., 2010; Haji et al., 2015). Measuring changes in these states contributes to insightful performance analysis, training development and skills improvement (Eversbusch and Grantcharov, 2004; Carswell et al., 2005). However, these cognitive and affective state measurements, which can be derived from neurophysiological metrics, must be coupled with measures of surgical task success or failure to enable any conclusions about the correspondences between psychological indicators and surgical performance (Modi et al., 2017). Unfortunately, the majority of existing tools to measure surgical task performance are retrospective, reliant on subjective, manual expert annotation and often focused on entire procedures rather than on intraoperative performance (Maruthappu et al., 2014).

Examining the associations between neurophysiology and intraoperative performance requires detecting markers of intraoperative success or failure at millisecond to second resolutions and synchronizing those markers with neurophysiological data. In this study, we build and deploy an automated, objective error detection system for robotic-assisted surgery (RAS) simulations that is synchronized with an electroencephalogram (EEG) headset worn by trainee surgeons at an RAS operating console. We use this system to detect intraoperative errors and analyze the accompanying EEG changes during those errors.

Prior work examining EEG changes during RAS has primarily focused on correlating EEG measurements with retrospective, subjective surveys of perceived mental workload (Guru et al., 2015b; Yu et al., 2017; Zander et al., 2017; Zhou et al., 2018, 2020). While mental workload is a significant contributor to surgical performance (Sarker and Vincent, 2005; Healey et al., 2006; Balch et al., 2009), it is not a proxy for actual performance. Understanding the relationship between operator neurophysiology and actual surgical performance requires objective, high temporal-resolution measurements of intraoperative performance that can be synchronized with changes in neurophysiological indicators. Automated skill and error assessment algorithms for RAS platforms do exist (Chen et al., 2019) but are rarely used in conjunction with biometric data capture. Integrating intraoperative performance data with intraoperative physiological data enables investigation of the potential cognitive and affective features that influence surgical success or failure.

For neurophysiological assessment, we choose to focus on the EEG indicators of psychological features that have been shown to impact task performance in mentally and physically demanding workspaces as well as human-machine teams. “Focused attention” or “vigilance” is the ability of an individual to maintain sustained attention on a task (Oken et al., 2006). Decreases in focused attention increase error rates in human-machine teamwork (Gill, 2012). Fluctuations in levels of attention can be measured through changes in EEG spectral power analyses (Borghini et al., 2014). For this study, we examine global β: (α+θ) power ratio (Pope et al., 1995), occipital β power (Gola et al., 2013), occipital δ power (Harmony, 2013), occipital θ power (Putman et al., 2014), frontal θ power (Strijkstra et al., 2003), and global γ power (Kaiser and Lutzenberger, 2005) as quantitative indicators of attention.

Cognitive load and fatigue are similarly relevant to human-robot teaming. High levels of cognitive load lead to task errors (Sweller, 2011). Cognitive fatigue, a decrease in cognitive performance after sustained cognitive load (Ackerman, 2011), is of particular importance in safety-critical aviation, transportation, aerospace, military, medicine, and industrial settings where fatigued individuals routinely operate complex, automated systems (Zhang et al., 2009). EEG spectral power ratios (Ryu and Myung, 2005) can also be used as measures of cognitive load or fatigue. We focus on frontal θ power (Shou and Ding, 2013; So et al., 2017; Raufi and Longo, 2022), frontal α power (Palva and Palva, 2007) and parietal α power (Frey et al., 2015) as indicators of cognitive load. For cognitive fatigue, we examine global α power (Santamaria and Chiappa, 1987), occipital α power (Thut et al., 2006; Zumer et al., 2014), frontal θ power (Strijkstra et al., 2003), and θ:α power ratio at Fz (Berka et al., 2007).

Affective characteristics such as valence, arousal, and dominance (VAD) may also influence human-robot system performance. Valence, the positivity or negativity of emotion, modulates attention and has shown an inverted u-shaped relationship with human performance in various tasks (Cai and Lin, 2011). Arousal, often referred to as “stress,” has a similar inverted u-shaped relationship with cognitive load (Teigen, 1994; Staal, 2004). Dominance, feelings of being “in control” of a situation, has a positive relationship with performance during sustained tasks (Cohen et al., 2016). EEG correlates of valence include α power asymmetry (Ohme et al., 2010; Gordon et al., 2018), global β power (Liu and Sourina, 2013), global γ power (Oathes et al., 2008) and global α power (Ahern and Schwartz, 1985). Indicators of arousal include global θ power, global β power and global γ power (Liu and Sourina, 2013). EEG correlates of dominance include global α:β power ratio and parietal β power (Verma and Tiwary, 2017).

Finally, we examine the association of proposed error recognition-related EEG metrics with intraoperative error. We include global θ power (Cavanagh and Frank, 2014), and global β power (Ray and Cole, 1985) as quantitative EEG indicators of error recognition.

To correlate these EEG metrics with markers of intraoperative error, we require high temporal-resolution analysis of surgical performance data. Prior work in automated surgical performance analysis has primarily focused on kinematic, system event, and haptic data obtained from RAS platforms. This data is then used to find associations between features such as instrument traveling distance (kinematics), camera clutch engagement (system event), and grip force (haptics) and self-reported level of surgical experience (Chen et al., 2019). Other research has used surgical video analysis for competency assessment (Reiley and Hager, 2009; Tao et al., 2012; Wang and Majewicz Fey, 2018). While kinematic, system, haptic, and image data can be correlated with surgical experience and used for skills assessments, in order to examine the EEG correlates of actual task errors, automated surgical error detection is required. Prior work in intraoperative error detection largely relies on retrospective, manual annotations by expert surgeons (Chen et al., 2019). These manual annotations provide sub-task level resolution for error detection but sub-tasks within surgical procedures can take seconds to minutes. A sub-task that is labeled with an error then implies an error marker of seconds to minutes in length. The EEG signal associated with a long-duration error marker would include more than the neurophysiological response to error.

To more precisely examine the neurophysiology of intraoperative error, we use a video analysis pipeline to automatically label each video frame from all surgical simulation videos for the presence or absence of an error. Because RAS operating console video is captured at a frame rate of 30 frames-per-second (fps) and each frame has an error label, we achieve error detection resolution of 30 Hertz (Hz). Each of these error labels is associated with a timestamp that is then synchronized with raw EEG data. By creating non-overlapping EEG data windows that align with each video frame, we create a more accurate measure of the neurophysiology of intraoperative error.

Parameterizing the EEG-based neurophysiology of intraoperative error creates the possibility of designing training interventions to reduce or prevent these errors. Surgeons who exhibit high correlations between EEG indices that may be related to attention and error, cognitive load and error, or emotional valence and error, for example, may benefit from EEG neurofeedback training to improve attention (Egner and Gruzelier, 2001, 2004), cognitive load management (Gruzelier et al., 2006), memory (Vernon et al., 2003), or emotional regulation (Raymond et al., 2005). Other surgeons who have high correlations between EEG indicators of arousal or stress and error, or sensorimotor desynchronization and error, for example, may benefit from focused neurostimulation for stress reduction (Modi et al., 2019) and fine motor skill augmentation (Cox et al., 2020).

In this paper, we aim to characterize the neurophysiological response to intraoperative error during RAS. To achieve this, we build and deploy a system for intraoperative error detection and EEG signal synchronization. We then use this system in a series of experiments with trainee surgeons and analyze the results. Our motivation is a better understanding of the neurophysiology of intraoperative error to help guide more accurate human performance assessments and training for surgeons and other teleoperators.

2. Materials and methods

2.1. Materials

We use a daVinci Xi operating console (Intuitive Surgical, Inc.) for image-based surgical data. The Xi operating console provides video of the surgeon's field-of-view (FOV) at 30 fps. Video analysis of the operating console video (surgeon's FOV) is used for intraoperative error detection. For neurophysiological data capture, we use a Cognionics Quick-20 EEG headset (Cognionics, Inc.) with a sampling rate of 500 Hz.

2.2. Methods

2.2.1. Experimental protocol

Twenty participants were recruited for this study under institutional IRB approval. Of these participants, 5 were non-medical graduate students and the remainder were general surgery residents at various levels of training. All participants were naïve to the hypotheses of the study prior to participation.

Each participant completed three simulation tasks during a single sitting without breaks. These tasks are standard exercises used in robotic surgical training to teach surgeons operational procedures for robot-assisted surgery. They are part of a larger standard learning curriculum that is widely used in surgical education curricula that are designed, tested, and deployed to improve robot assisted surgery skills. These tasks were completed in the same order for all participants. The tasks utilized were “Ring Rollercoaster 1,” “Ring Rollercoaster 3,” and “Wrist Articulation 1” in this order for all participants (Figure 1).

FIGURE 1
www.frontiersin.org

Figure 1. Surgical simulation tasks.

An EEG headset was fitted on each participant prior to beginning the first task and checked for appropriate positioning and data transfer. We use the international 10–20 EEG system to ensure proper electrode placement. An example of a deployed headset is shown in Figure 2. Following appropriate electrode placement, we confirm adequate signal through the Cognionics Data Aquisition software (Cognionics, Inc.) which provides visual cues to monitor signal quality at individual EEG electrodes. Electrodes with low signal quality are re-adjusted until signal quality meets the data acquisition software criteria. This signal quality is monitored throughout the experiment, and the EEG headset is re-adjusted between simulation tasks if necessary.

FIGURE 2
www.frontiersin.org

Figure 2. EEG headset deployed in RAS-equipped OR.

Four minutes of baseline EEG readings were then recorded as each participant sat still at the operating console. Following baseline recording, each participant was directed to start the first surgical simulation task and given no other instruction. After completion of the first task, each participant was directed to start the second simulation, also without additional instruction or feedback. After completion of the second simulation, each participant was directed to start the third and final simulation, also without additional instruction or feedback. Following completion of the final simulation, each participant was instructed to complete a demographics survey after which the EEG headset was removed and the system was reset for future participants.

2.2.2. Video analysis

To generate error markers from operating console (i.e., surgeon FOV) video, we use each video frame as input for a classical computer vision feature extraction algorithm. Our algorithm uses pixel-based frequency filters, intensity filters and Hough transforms to generate a feature representation for each video frame. This feature representation is used to label each video frame for the presence or absence of intraoperative error. The types of error we consider are: contact between the manipulated object and an obstacle, excessive grip force, dropping the manipulated object, contact between end effectors, end effectors going out of view during camera clutch movement, contact between end effectors and obstacles.

To validate this video analysis pipeline, a single expert human adjudicator manually annotated a validation set of 18,887 randomly chosen operating console video frames for the presence of intraoperative error. We compared these manual annotations to the annotations generated by our computer vision algorithm on the same validation set of operating console video frames. Our algorithm's overall error detection accuracy on the validation set of 18,887 operating console video frames is 98.0%, precision is 90.4%, and recall is 93.7%. We then use this pipeline for frame-by-frame inference on the video data. Given the operating console video frame rate of 30 fps, frame-by-frame error detection provides intraoperative error markers at 30 Hz.

2.3. EEG analysis

To pre-process the raw EEG data, we remove the DC offset and bandpass filter the resulting signal with a passband of 0.01–50 Hz. We eliminate EEG noise using artifact subspace reconstruction (Chang et al., 2018). This pre-processed data is then filtered in parallel into five spectral power bands: delta (1–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–30 Hz), and gamma (30–50 Hz). This filtering is done on the entire EEG data stream i.e., from the start of the baseline or start of the simulation period to the end of the baseline or end of the simulation period. After this filtering is complete, the EEG data for each of the spectral power bands is then windowed.

To accurately associate each operating console video frame with a non-overlapping window of EEG data, we align each video frame timestamp with the closest EEG signal timestamp. This timestamp is the “start window” timestamp. Each EEG window includes all samples received between a “start window” timestamp and the “start window” timestamp corresponding to the next video frame. The operating console video has a frame rate of 30 fps which implies that each video frame has a duration of ~33 ms. Each EEG window therefore has a size of ~33 ms.

Using these non-overlapping ~33 ms EEG windows, we compute the root-mean-square amplitude of each band for every EEG channel. Channel-specific spectral band powers for each window (i.e., video frame) are then aggregated into features that correspond to each operating console video frame. These features are compared to the average feature value for the baseline period.

2.4. Statistical analysis

For analysis of all EEG channels and all spectral power bands, we use separate linear mixed effects models for each channel and each band. Given 19 EEG channels and 5 spectral power bands, this results in 95 mixed effects models. In each of these 95 models, the dependent variable (channel-band power) is standardized, intraoperative error is modeled as a fixed effect and individual participant variance is modeled as a random effect.

For EEG feature analysis, we examine a total of 15 distinct EEG features. We select these 15 features (see Table 1 for all features) based on previously published work from other contexts that associates these EEG features with cognitive and affective attributes that could be important during RAS. We do not attempt to reproduce earlier work from other contexts by testing whether these EEG features are appropriate indicators of cognition or affect in an RAS simulation. Instead, we test whether these 15 EEG features are associated with intraoperative error. We use a separate linear mixed effects model for each EEG feature. The dependent variable (EEG feature) is standardized, intraoperative error is modeled as a fixed effect and individual participant variance is modeled as a random effect.

TABLE 1
www.frontiersin.org

Table 1. EEG correlates of selected cognitive and affective features from previously published research.

Given the independent testing of 95 channel-band combinations as well as 15 EEG features in this experiment, we correct our baseline significance level of P = 0.01 by a factor of 1110. Our multiple comparisons corrected significance level is P = 0.00009.

We conducted a simulation-based power analysis (Green and MacLeod, 2016) of the EEG feature correlates using simulated effect sizes approximately equal to those reported in our results (Brysbaert and Stevens, 2018) and significance levels of P = 0.00009. With a simulated effect size of 0.05, our statistical power to detect error-related changes in occipital delta band power has a 95% confidence interval (C.I.) of 99.6–100.0%. With a simulated effect size of –0.03, the 95% statistical power C.I. for parietal alpha band power is 99.6 to 100.0%. With a simulated effect size of 0.03, the 95% statistical power C.I. for frontal theta band power is 99.6–100.0%. With a simulated effect size of –0.05, the 95% statistical power C.I. for left-to-right alpha band power asymmetry is 99.6–100.0%. With a simulated effect size of –0.02, the 95% statistical power C.I. for global beta band power is 99.3–99.9%. With a simulated effect size of –0.04, the 95% statistical power C.I. for parietal beta band power is 99.6–100.0%.

3. Results

3.1. Video analysis

The sample size of detected errors using was smaller than that of the non-errors. There were a total of 62,809 detected error frames (17.1%) as compared to 305,234 non-error frames (82.9%). See Figure 3 for example error detection frames from each simulation task.

FIGURE 3
www.frontiersin.org

Figure 3. Error detection example for simulation tasks.

3.2. All channels all bands analysis

Seventy-nine of 95 channel-band power combinations differ significantly between error and non-error. The five largest magnitude standardized effect sizes correspond to δ band power relative to baseline at channels Fp2 (estimate: 0.106, standard error: 0.003, P < 2E-16), F4 (estimate: 0.143, standard error: 0.003, P < 2E-16), Pz (estimate: 0.175, standard error: 0.004, P < 2E-16), and P4 (estimate: 0.149, standard error: 0.004, P < 2E-16), and β band power relative to baseline at channel P4 (estimate: –0.119, standard error: 0.003, P < 2E-16). Channel-band combinations that show no significant difference between error and non-error are: δ at Fp1, F8, P3, P7; θ at Cz, O1, C4; α at F4, Cz, P8, P4, O1; β at P7, T4; and γ at P7, T3 (Figure 4).

FIGURE 4
www.frontiersin.org

Figure 4. Heat map of estimated fixed effects for 95 linear mixed effects models representing all channel-band combinations of 19 EEG channels and 5 spectral power bands. Fixed effect estimate labeled numerically and by color intensity in each cell. Only significant channel-band combinations are included in heat map. Blank cells did not meet corrected significance threshold of P = 0.00009.

3.3. All features analysis

Twelve of 15 EEG features differ significantly between error and non-error. For attention, δ power at the occipital electrodes shows the largest magnitude difference between error and non-error (estimate: 0.05, standard error: 0.004, P < 2E-16). For cognitive load, α power at the parietal electrodes has the largest magnitude difference (estimate: –0.037, standard error: 0.003, P < 2E-16). The most informative feature for cognitive fatigue is θ power at the frontal electrodes (estimate: 0.034, standard error: 0.003, P < 2E-16). For valence, the most informative feature is α power asymmetry between the left and right electrodes (estimate: –0.059, standard error: 0.004, P < 2E-16). For arousal, β power globally has the largest magnitude difference (estimate: –0.027, standard error: 0.003, P < 2E-16). For dominance, global α:β power ratio (estimate: –0.040, standard error: 0.004, P < 2E-16) and β power at the parietal electrodes (estimate: –0.040, standard error: 0.003, P < 2E-16) have similar effect sizes. The most informative feature for error recognition is global β power (estimate: –0.027, standard error: 0.003, P < 2E-16) (Figure 5).

FIGURE 5
www.frontiersin.org

Figure 5. Standardized fixed effect estimates of intraoperative error on previously studied EEG correlates of cognitive and affective features including Attention, Cognitive Load, Cognitive Fatigue, Valence, Arousal, Dominance, and Error Recognition. Electrode regions include O, occipital; F, frontal; G, global; P, parietal; L, left hemisphere; R, right hemisphere. *: Fixed effect estimate meets significance threshold of P = 0.00009. All fixed effect estimates are standardized.

To validate our EEG feature results against potential signals due primarily to EEG pre-processing methods, we compare our full model (a fixed effect of intraoperative error and a random effect of individual variance) with a model that only incorporates the random effect of individual variance in post-processed EEG data. Using a simulation based approach, the likelihood ratio lower bounds for the 95% statistical power C.I.s to detect intraoperative error effects in the full EEG feature models as compared to the simple random effect models ranged from 98.84 to 99.63%. To validate our EEG feature results against potential signals in EEG noise, we compare our full model using post-processed EEG data with our full model using raw EEG data. The likelihood ratio lower bounds for the 95% statistical power C.I.s to detect intraoperative error effects using post-processed EEG data as compared to raw EEG data ranged from 99.3 to 99.6%.

4. Discussion

Potential cognitive and affective determinants of surgical performance, and their EEG correlates, have been explored in prior research (Gruzelier, 2014; Guru et al., 2015a; Ndaro and Wang, 2018). This work, however, does not couple EEG analysis with automated surgical error detection to capture high temporal-resolution indicators of intraoperative error. By integrating intraoperative error detection with synchronized EEG analysis, we aimed to characterize the neurophysiology of surgeons during robot-assisted surgery (RAS), particularly during intraoperative error.

To achieve this, we built and deployed a novel RAS data capture and analysis platform to examine the relationships among EEG spectral band powers at various channels, previously proposed EEG correlates of various cognitive and affective features in other contexts, and intraoperative error. Our primary statistical tools, linear mixed effects models (LMMs), are widely used in psychophysiological research particularly because of the non-independence of the data in hierarchical structures (Bagiella et al., 2000; Koerner and Zhang, 2017; Frömer et al., 2018). While sequential 30ms EEG samples may be correlated, we are not examining a temporal effect in our model, so the impact of potential residual error autocorrelation on the error effect is not meaningful for our hypotheses. For this reason, and to our knowledge, the standard LMMs in neurophysiological research do not use autoregressive residual covariance matrices (West et al., 2006). Additionally, when we randomly shuffle our datasets to remove any potential residual autocorrelation that may impact our results, our effect sizes and significance tests remain unchanged.

Our results suggest that relative band powers at most channels demonstrate significant changes during errors as compared to when errors are not being committed. We observe that EEG features which are proposed correlates of attention, cognitive load, cognitive fatigue, valence, arousal, dominance, and error recognition in other contexts (Klimesch, 1999; Strijkstra et al., 2003; Harmony, 2013; Liu and Sourina, 2013; Verma and Tiwary, 2017; Tempel et al., 2020) demonstrate significant changes during intraoperative error and may underlie the error-related signals detected in this study.

The direction of change in EEG features largely aligns with proposed associations between EEG statistics and cognitive or affective features in other contexts. Increases in occipital δ power have been associated with increases in attention (Harmony, 2013), and in our study, these increases are also observed during intraoperative error. Decreases in parietal α power have been associated with decreases in cognitive load (Klimesch, 1999). In our study, decreases in parietal α power were also observed during intraoperative error. Increases in frontal θ power have been associated with increases in cognitive fatigue (Strijkstra et al., 2003), and we observed an increase in frontal θ power during error. Left and right hemisphere α power asymmetry has been proposed as an indicator of emotional valence (Strijkstra et al., 2003) with higher asymmetry corresponding to more positive valence. We observed a decrease in α power asymmetry during error. Decreases in global β power have been associated with increased arousal (Liu and Sourina, 2013), and we observed a decrease in global β power during error. Increases in parietal β power have been associated with increases in dominance (Verma and Tiwary, 2017), and we observed decreases in parietal β power during error. For error recognition, increases in global β power have been associated with increased response inhibition during voluntary motor tasks (Tempel et al., 2020). In our study, global β power decreased during intraoperative error.

While this study attempts to characterize the physiological correlates of intraoperative error, this study is not designed to test the hypothesis that the cognitive and affective states correlate with physiological metrics that are causative of error. The EEG activations described in these results are exploratory and intended to generate hypotheses for future studies that can formally test hypotheses generated from the current findings. Moreover, correlations between physiological signals are not indicative of causation and therefore future studies may attempt to implement causal interventions to test the directional relationship between errors and physiological states.

There are several limitations of this study. First, the number of participants was neither large nor evenly distributed across skill level, potentially skewing cohort analysis. Second, using bandpass filters and RMS power calculations for frame-by-frame channel-band and EEG feature power measurements are accompanied by a tradeoff in the settling time of the filter. Third, the error detection video pipeline we use is based on color and text-based error indicators that appear in the operating console during surgical simulations. These error indicators are part of the built-in simulation software and cannot be turned off or altered during simulation tasks. This presents the possibility that the neurophysiological changes that accompany error may be confounded by a response to the specific error indicator in the stimulator. That said, we observed significant changes in parietal channels that would seem to be less likely if the neurophysiological changes were due to vision-based stimuli alone. Finally, these results may be affected by other variables such as movement at the console and time at the console. Based on our observations, low performers tended to move and shift body positions more frequently than high performers, potentially indicative of discomfort at the console. Low performers also required more time to complete the simulation tasks.

Despite these limitations, this study shows that a novel RAS data capture and analysis platform can enable the detection of distinct neurophysiological changes associated with intraoperative error. These changes are largely consistent with known relationships between EEG metrics and cognitive or affective factors that impact performance. This data highlights the possibility of using objective biometric data capture and error detection to better understand the neurophysiology of intraoperative error and help guide more accurate human performance assessments and precision training for surgeons and other teleoperators.

Data availability statement

The datasets presented in this article are not readily available because HIPAA. Requests to access the datasets should be directed to CD'A.

Ethics statement

The studies involving human participants were reviewed and approved by University of California, San Diego School of Medicine Institutional Review Board IRB#: 800026. The patients/participants provided their written informed consent to participate in this study.

Author contributions

CD'A: conceptualization, software, formal analysis, and writing—original draft. EA-S: supervision and funding acquisition. EH: investigation and project administration. NG: investigation. HC and LA: writing—review and editing, supervision, and funding acquisition. RB: writing—review and editing, project administration, supervision, and funding acquisition. All authors contributed to the article and approved the submitted version.

Funding

LA was supported by the Army Research Office, Grant#: W911NF-15-1-0390.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Ackerman, P. L. (2011). Cognitive Fatigue: Multidisciplinary Perspectives on Current Research and Future Applications. Washington, DC: American Psychological Association.

Google Scholar

Ahern, G. L., and Schwartz, G. E. (1985). Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23, 745–755. doi: 10.1016/0028-3932(85)90081-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Bagiella, E., Sloan, R. P., and Heitjan, D. F. (2000). Mixed-effects models in psychophysiology. Psychophysiology 37, 13–20. doi: 10.1111/1469-8986.3710013

CrossRef Full Text | Google Scholar

Balch, C. M., Freischlag, J. A., and Shanafelt, T. D. (2009). Stress and burnout among surgeons: understanding and managing the syndrome and avoiding the adverse consequences. Arch. Surg. 144, 371–376. doi: 10.1001/archsurg.2008.575

PubMed Abstract | CrossRef Full Text | Google Scholar

Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., et al. (2007). EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231-B244.

PubMed Abstract | Google Scholar

Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Brysbaert, M., and Stevens, M. (2018). Power analysis and effect size in mixed effects models: a tutorial. J. Cogn. 1, 9. doi: 10.5334/joc.10

PubMed Abstract | CrossRef Full Text | Google Scholar

Cai, H., and Lin, Y. (2011). Modeling of operators' emotion and task performance in a virtual driving environment. Int. J. Hum. Comput. Stud. 69, 571–586. doi: 10.1016/j.ijhcs.2011.05.003

CrossRef Full Text | Google Scholar

Carswell, C. M., Clarke, D., and Seales, W. B. (2005). Assessing mental workload during laparoscopic surgery. Surg Innov. 12, 80–90. doi: 10.1177/155335060501200112

PubMed Abstract | CrossRef Full Text | Google Scholar

Cavanagh, J. F., and Frank, M. J. (2014). Frontal theta as a mechanism for cognitive control. Trends Cogn. Sci. 18, 414–421. doi: 10.1016/j.tics.2014.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, C.-Y., Hsu, S.-H., Pion-Tonachini, L., and Jung, T.-P. (2018). “Evaluation of artifact subspace reconstruction for automatic eeg artifact removal,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Honolulu, HI: IEEE), 1242–1245.

PubMed Abstract | Google Scholar

Chen, J., Cheng, N., Cacciamani, G., Oh, P., Lin-Brande, M., Remulla, D., et al. (2019). Objective assessment of robotic surgical technical skill: a systematic review. J. Urol. 201, 461–469. doi: 10.1016/j.juro.2018.06.078

PubMed Abstract | CrossRef Full Text | Google Scholar

Christian, C. K., Gustafson, M. L., Roth, E. M., Sheridan, T. B., Gandhi, T. K., Dwyer, K., et al. (2006). A prospective study of patient safety in the operating room. Surgery 139, 159–173. doi: 10.1016/j.surg.2005.07.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, I., den Braber, N., Smets, N. J., van Diggelen, J., Brinkman, W.-P., and Neerincx, M. A. (2016). Work content influences on cognitive task load, emotional state and performance during a simulated 520-days' mars mission. Comput. Hum. Behav. 55, 642–652. doi: 10.1016/j.chb.2015.10.011

CrossRef Full Text | Google Scholar

Cox, M. L., Deng, Z.-D., Palmer, H., Watts, A., Beynel, L., Young, J. R., et al. (2020). Utilizing transcranial direct current stimulation to enhance laparoscopic technical skills training: a randomized controlled trial. Brain Stimul. 13, 863–872. doi: 10.1016/j.brs.2020.03.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Egner, T., and Gruzelier, J. H. (2001). Learned self-regulation of EEG frequency components affects attention and event-related brain potentials in humans. Neuroreport 12, 4155–4159. doi: 10.1097/00001756-200112210-00058

PubMed Abstract | CrossRef Full Text | Google Scholar

Egner, T., and Gruzelier, J. H. (2004). EEG biofeedback of low beta band components: frequency-specific effects on variables of attention and event-related brain potentials. Clin. Neurophysiol. 115, 131–139. doi: 10.1016/S1388-2457(03)00353-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Eversbusch, A., and Grantcharov, T. (2004). Learning curves and impact of psychomotor training on performance in simulated colonoscopy: a randomized trial using a virtual reality endoscopy trainer. Surg. Endoscopy Other Intervent. Techn. 18, 1514–1518. doi: 10.1007/s00464-003-9264-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Frey, J. N., Ruhnau, P., and Weisz, N. (2015). Not so different after all: the same oscillatory processes support different types of attention. Brain Res. 1626, 183–197. doi: 10.1016/j.brainres.2015.02.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Frömer, R., Maier, M., and Abdel Rahman, R. (2018). Group-level eeg-processing pipeline for flexible single trial-based analyses including linear mixed models. Front. Neurosci. 12, 48. doi: 10.3389/fnins.2018.00048

PubMed Abstract | CrossRef Full Text | Google Scholar

Gill, K. S. (2012). Human Machine Symbiosis: The Foundations of Human-Centred Systems Design. Springer Science & Business Media.

Google Scholar

Gola, M., Magnuski, M., Szumska, I., and Wróbel, A. (2013). EEG beta band activity is related to attention and attentional deficits in the visual performance of elderly subjects. Int. J. Psychophysiol. 89, 334–341. doi: 10.1016/j.ijpsycho.2013.05.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Gordon, R., Ciorciari, J., and van Laer, T. (2018). Using eeg to examine the role of attention, working memory, emotion, and imagination in narrative transportation. Eur. J. Mark. 52, 92–117. doi: 10.1108/EJM-12-2016-0881

CrossRef Full Text | Google Scholar

Green, P., and MacLeod, C. J. (2016). Simr: an r package for power analysis of generalized linear mixed models by simulation. Methods Ecol. Evolut. 7, 493–498. doi: 10.1111/2041-210X.12504

CrossRef Full Text | Google Scholar

Gruzelier, J., Egner, T., and Vernon, D. (2006). Validating the efficacy of neurofeedback for optimising performance. Prog. Brain Res. 159, 421–431. doi: 10.1016/S0079-6123(06)59027-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Gruzelier, J. H. (2014). Eeg-neurofeedback for optimising performance. i: a review of cognitive and affective outcome in healthy participants. Neurosci. Biobehav. Rev. 44, 124–141. doi: 10.1016/j.neubiorev.2013.09.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Guru, K. A., Esfahani, E. T., Raza, S. J., Bhat, R., Wang, K., Hammond, Y., et al. (2015a). Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff. BJU Int. 115, 166–174. doi: 10.1111/bju.12657

PubMed Abstract | CrossRef Full Text | Google Scholar

Guru, K. A., Shafiei, S. B., Khan, A., Hussein, A. A., Sharif, M., and Esfahani, E. T. (2015b). Understanding cognitive performance during robot-assisted surgery. Urology 86, 751–757. doi: 10.1016/j.urology.2015.07.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Haji, F. A., Rojas, D., Childs, R., de Ribaupierre, S., and Dubrowski, A. (2015). Measuring cognitive load: performance, mental effort and simulation task complexity. Med. Educ. 49, 815–827. doi: 10.1111/medu.12773

PubMed Abstract | CrossRef Full Text | Google Scholar

Harmony, T. (2013). The functional significance of delta oscillations in cognitive processing. Front. Integr. Neurosci. 7, 83. doi: 10.3389/fnint.2013.00083

PubMed Abstract | CrossRef Full Text | Google Scholar

Healey, A., Sevdalis, N., and Vincent, C. (2006). Measuring intra-operative interference from distraction and interruption observedin the operating theatre. Ergonomics 49, 589–604. doi: 10.1080/00140130600568899

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaiser, J., and Lutzenberger, W. (2005). Human gamma-band activity: a window to cognitive processing. Neuroreport 16, 207–211. doi: 10.1097/00001756-200502280-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev. 29, 169–195. doi: 10.1016/S0165-0173(98)00056-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Koerner, T. K., and Zhang, Y. (2017). Application of linear mixed-effects models in human neuroscience research: a comparison with pearson correlation in two auditory electrophysiology studies. Brain Sci. 7, 26. doi: 10.3390/brainsci7030026

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Y., and Sourina, O. (2013). “EEG databases for emotion recognition,” in 2013 International Conference on Cyberworlds (Yokohama: IEEE), 302–309.

Google Scholar

Maruthappu, M., El-Harasis, M., Nagendran, M., Orgill, D., McCulloch, P., Duclos, A., et al. (2014). Systematic review of methodological quality of individual performance measurement in surgery. J. Br. Surg. 101, 1491–1498. doi: 10.1002/bjs.9642

PubMed Abstract | CrossRef Full Text | Google Scholar

Modi, H. N., Singh, H., Fiorentino, F., Orihuela-Espina, F., Athanasiou, T., Yang, G.-Z., et al. (2019). Association of residents' neural signatures with stress resilience during surgery. JAMA Surg. 154, e192552-e192552. doi: 10.1001/jamasurg.2019.2552

PubMed Abstract | CrossRef Full Text | Google Scholar

Modi, H. N., Singh, H., Orihuela-Espina, F., Athanasiou, T., Fiorentino, F., Yang, G.-Z., et al. (2018). Temporal stress in the operating room: brain engagement promotes “coping” and disengagement prompts “choking”. Ann. Surg. 267, 683–691. doi: 10.1097/SLA.0000000000002289

PubMed Abstract | CrossRef Full Text | Google Scholar

Modi, H. N., Singh, H., Yang, G.-Z., Darzi, A., and Leff, D. R. (2017). A decade of imaging surgeons' brain function (part i): terminology, techniques, and clinical translation. Surgery 162, 1121–1130. doi: 10.1016/j.surg.2017.05.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Ndaro, N. Z., and Wang, S.-Y. (2018). Effects of fatigue based on electroencephalography signal during laparoscopic surgical simulation. Minim. Invasive Surg. 2018, 2389158. doi: 10.1155/2018/2389158

PubMed Abstract | CrossRef Full Text | Google Scholar

Oathes, D. J., Ray, W. J., Yamasaki, A. S., Borkovec, T. D., Castonguay, L. G., Newman, M. G., et al. (2008). Worry, generalized anxiety disorder, and emotion: evidence from the eeg gamma band. Biol. Psychol. 79, 165–170. doi: 10.1016/j.biopsycho.2008.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohme, R., Reykowska, D., Wiener, D., and Choromanska, A. (2010). Application of frontal eeg asymmetry to advertising research. J. Econ. Psychol. 31, 785–793. doi: 10.1016/j.joep.2010.03.008

CrossRef Full Text | Google Scholar

Oken, B. S., Salinsky, M. C., and Elsas, S. (2006). Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin. Neurophys. 117, 1885–1901. doi: 10.1016/j.clinph.2006.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Palva, S., and Palva, J. M. (2007). New vistas for α-frequency band oscillations. Trends Neurosci. 30, 150–158. doi: 10.1016/j.tins.2007.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Pope, A. T., Bogart, E. H., and Bartolome, D. S. (1995). Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195. doi: 10.1016/0301-0511(95)05116-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Putman, P., Verkuil, B., Arias-Garcia, E., Pantazi, I., and van Schie, C. (2014). EEG theta/beta ratio as a potential biomarker for attentional control and resilience against deleterious effects of stress on attention. Cogn. Affect. Behav. Neurosci. 14, 782–791. doi: 10.3758/s13415-013-0238-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Raufi, B., and Longo, L. (2022). An evaluation of the eeg alpha-to-theta and theta-to-alpha band ratios as indexes of mental workload. Front. Neuroinform. 16, 861967. doi: 10.3389/fninf.2022.861967

PubMed Abstract | CrossRef Full Text | Google Scholar

Ray, W. J., and Cole, H. W. (1985). Eeg alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. Science 228, 750–752. doi: 10.1126/science.3992243

PubMed Abstract | CrossRef Full Text | Google Scholar

Raymond, J., Varney, C., Parkinson, L. A., and Gruzelier, J. H. (2005). The effects of alpha/theta neurofeedback on personality and mood. Cogn. Brain Res. 23, 287–292. doi: 10.1016/j.cogbrainres.2004.10.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Reiley, C. E., and Hager, G. D. (2009). “Task versus subtask surgical skill evaluation of robotic minimally invasive surgery,” in International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer), 435–442. doi: 10.1007/978-3-642-04268-3_54

PubMed Abstract | CrossRef Full Text | Google Scholar

Ryu, K., and Myung, R. (2005). Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. Int. J. Ind. Ergon. 35, 991–1009. doi: 10.1016/j.ergon.2005.04.005

CrossRef Full Text | Google Scholar

Santamaria, J., and Chiappa, K. H. (1987). The eeg of drowsiness in normal adults. J. Clin. Neurophysiol. 4, 327–382. doi: 10.1097/00004691-198710000-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarker, S. K., and Vincent, C. (2005). Errors in surgery. Int. J. Surg. 3, 75–81. doi: 10.1016/j.ijsu.2005.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Schuetz, M., Gockel, I., Beardi, J., Hakman, P., Dunschede, F., Moenk, S., et al. (2008). Three different types of surgeon-specific stress reactions identified by laparoscopic simulation in a virtual scenario. Surg. Endosc. 22, 1263–1267. doi: 10.1007/s00464-007-9605-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Shou, G., and Ding, L. (2013). “Frontal theta eeg dynamics in a real-world air traffic control task,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Osaka: IEEE), 5594–5597.

PubMed Abstract | Google Scholar

So, W. K., Wong, S. W., Mak, J. N., and Chan, R. H. (2017). An evaluation of mental workload with frontal EEG. PLoS ONE 12, e0174949. doi: 10.1371/journal.pone.0174949

PubMed Abstract | CrossRef Full Text | Google Scholar

Staal, M. A. (2004). Stress, cognition, and human performance: A literature review and conceptual framework. Technical Memorandum (TM) 20060017835.

Google Scholar

Strijkstra, A. M., Beersma, D. G., Drayer, B., Halbesma, N., and Daan, S. (2003). Subjective sleepiness correlates negatively with global alpha (8-12 hz) and positively with central frontal theta (4-8 hz) frequencies in the human resting awake electroencephalogram. Neurosci. Lett. 340, 17–20. doi: 10.1016/S0304-3940(03)00033-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Sweller, J. (2011). Cognitive load theory. Psychol. Learn. Motivat. 55, 37–76. doi: 10.1016/B978-0-12-387691-1.00002-8

CrossRef Full Text | Google Scholar

Tao, L., Elhamifar, E., Khudanpur, S., Hager, G. D., and Vidal, R. (2012). “Sparse hidden markov models for surgical gesture classification and skill evaluation,” in International Conference on Information Processing in Computer-Assisted Interventions (Springer), 167–177. doi: 10.1007/978-3-642-30618-1_17

CrossRef Full Text | Google Scholar

Teigen, K. H. (1994). Yerkes-dodson: a law for all seasons. Theory Psychol. 4, 525–547. doi: 10.1177/0959354394044004

CrossRef Full Text | Google Scholar

Tempel, T., Frings, C., and Pastötter, B. (2020). EEG beta power increase indicates inhibition in motor memory. Int. J. Psychophysiol. 150, 92–99. doi: 10.1016/j.ijpsycho.2020.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Thut, G., Nietzel, A., Brandt, S. A., and Pascual-Leone, A. (2006). α-band electroencephalographic activity over occipital cortex indexes visuospatial attention bias and predicts visual target detection. J. Neurosci. 26, 9494–9502. doi: 10.1523/JNEUROSCI.0875-06.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Verma, G. K., and Tiwary, U. S. (2017). Affect representation and recognition in 3D continuous valence-arousal-dominance space. Multimed Tools Appl. 76, 2159–2183. doi: 10.1007/s11042-015-3119-y

CrossRef Full Text | Google Scholar

Vernon, D., Egner, T., Cooper, N., Compton, T., Neilands, C., Sheri, A., et al. (2003). The effect of training distinct neurofeedback protocols on aspects of cognitive performance. Int. J. Psychophysiol. 47, 75–85. doi: 10.1016/S0167-8760(02)00091-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, Z., and Majewicz Fey, A. (2018). Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int. J. Comput. Assist. Radiol. Surg. 13, 1959–1970. doi: 10.1007/s11548-018-1860-1

PubMed Abstract | CrossRef Full Text | Google Scholar

West, B. T., Welch, K. B., and Galecki, A. T. (2006). Linear Mixed Models: A Practical Guide Using Statistical Software. Chapman and Hall; CRC. doi: 10.1201/b17198

CrossRef Full Text | Google Scholar

Yu, D., Dural, C., Morrow, M., Yang, L., Collins, J. W., Hallbeck, S., et al. (2017). Intraoperative workload in robotic surgery assessed by wearable motion tracking sensors and questionnaires. Surg. Endosc. 31, 877–886. doi: 10.1007/s00464-016-5047-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Yurko, Y. Y., Scerbo, M. W., Prabhu, A. S., Acker, C. E., and Stefanidis, D. (2010). Higher mental workload is associated with poorer laparoscopic performance as measured by the nasa-tlx tool. Simulat. Healthcare 5, 267–271. doi: 10.1097/SIH.0b013e3181e3f329

PubMed Abstract | CrossRef Full Text | Google Scholar

Zander, T. O., Shetty, K., Lorenz, R., Leff, D. R., Krol, L. R., Darzi, A. W., et al. (2017). Automated task load detection with electroencephalography: towards passive brain-computer interfacing in robotic surgery. J. Med. Rob. Res. 2, 1750003. doi: 10.1142/S2424905X17500039

CrossRef Full Text | Google Scholar

Zhang, C., Zheng, C.-X., and Yu, X.-L. (2009). Automatic recognition of cognitive fatigue from physiological indices by using wavelet packet transform and kernel learning algorithms. Exp. Sys. App. 36, 4664–4671. doi: 10.1016/j.eswa.2008.06.022

CrossRef Full Text | Google Scholar

Zhou, T., Cha, J. S., Gonzalez, G., Wachs, J. P., Sundaram, C. P., and Yu, D. (2020). Multimodal physiological signals for workload prediction in robot-assisted surgery. ACM Trans. Hum. Rob. Interact. 9, 1–26. doi: 10.1145/3368589

CrossRef Full Text | Google Scholar

Zhou, T., Cha, J. S., Gonzalez, G. T., Wachs, J. P., Sundaram, C., and Yu, D. (2018). “Joint surgeon attributes estimation in robot-assisted surgery,” in Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (IEEE), 285–286. doi: 10.1145/3173386.3176981

CrossRef Full Text | Google Scholar

Zumer, J. M., Scheeringa, R., Schoffelen, J.-M., Norris, D. G., and Jensen, O. (2014). Occipital alpha activity during stimulus processing gates the information flow to object-selective cortex. PLoS Biol. 12, e1001965. doi: 10.1371/journal.pbio.1001965

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: robot-assisted surgery, medical robotics, human performance, surgical training, clinical assessment

Citation: D'Ambrosia C, Aronoff-Spencer E, Huang EY, Goldhaber NH, Christensen HI, Broderick RC and Appelbaum LG (2023) The neurophysiology of intraoperative error: An EEG study of trainee surgeons during robotic-assisted surgery simulations. Front. Neuroergon. 3:1052411. doi: 10.3389/fnrgo.2022.1052411

Received: 23 September 2022; Accepted: 19 December 2022;
Published: 09 January 2023.

Edited by:

Elena De Momi, Politecnico di Milano, Italy

Reviewed by:

Yuanyuan Gao, Boston University, United States
Arun Nemani, Draycon Labs, United States

Copyright © 2023 D'Ambrosia, Aronoff-Spencer, Huang, Goldhaber, Christensen, Broderick and Appelbaum. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Christopher D'Ambrosia, yes cdambros@eng.ucsd.edu

Download