Team performance during vacuum-assisted vaginal delivery: video review of obstetric multidisciplinary teams

Introduction Vacuum extraction is generally considered an operator-dependent task, with most attention directed toward the obstetrician’s technical abilities (1–3). Little is known about the effect of the team and non-technical skills on clinical outcomes in vacuum-assisted delivery. This study aimed to investigate whether the non-technical skills of obstetricians were correlated with their level of clinical performance via the analysis of video recordings of teams conducting actual vacuum extractions. Methods We installed between two or three video cameras in each delivery room at Aarhus University Hospital and Horsens Regional Hospital and obtained 60 videos of teams managing vacuum extraction. Appropriate consent was obtained. Two raters carefully reviewed the videos and assessed the teams’ non-technical skills using the Assessment of Obstetric Team Performance (AOTP) checklist, rating all items on a Likert scale score from 1 to 5 (1 = poor; 3 = average; and 5 = excellent). This resulted in a total score ranging from 18 to 90. Two different raters independently assessed the teams’ clinical performance (adherence to clinical guidelines) using the TeamOBS-Vacuum-Assisted Delivery (VAD) checklist, rating each item (0 = not done, 1 = done incorrectly; and 2 = done correctly). This resulted in a total score with the following ranges (low clinical performance: 0–59; average: 60–84; and high: 85–100). Interrater agreement was analyzed using intraclass correlation (ICC), and the risk of high or low clinical performance was analyzed on a logit scale to meet the assumption of normality. Results Teams that received excellent non-technical scores had an 81% probability of achieving high clinical performance, whereas this probability was only 12% among teams with average non-technical scores (p < 0.001). Teams with a high clinical performance often had excellent behavior in the non-technical items of “team interaction,” “anticipation,” “avoidance fixation,” and “focused communication.” Teams with a low or average clinical performance often neglected to consider analgesia, had delayed abandonment of the attempted vaginal delivery and insufficient use of appropriate fetal monitoring. Interrater reliability was high for both rater-teams, with an ICC for the non-technical skills of 0.83 (95% confidence interval [CI]: 0.71–0.88) and 0.84 for the clinical performance (95% CI: 0.74–0.90). Conclusion Although assisted vaginal delivery by vacuum extraction is generally considered to be an operator-dependent task, our findings suggest that teamwork and effective team interaction play crucial roles in achieving high clinical performance. Teamwork helped the consultant anticipate the next step, avoid fixation, ensure adequate analgesia, and maintain thorough fetal monitoring during delivery.


Introduction
Assisted vaginal delivery is employed in situations where the second stage of labor should be shortened due to maternal or fetal indications.The primary goal is to achieve expedited delivery while maintaining high-quality care.However, delivery wards worldwide have witnessed increased rates of cesarean sections in the second stage of labor.This trend may signify a decline in clinical skills or concerns regarding the risk of severe complications such as maternal perineal trauma, fetal scalp injuries, and even intracranial hemorrhages (1,2).While vacuum extraction remains the most utilized method for assisted vaginal delivery, we must be vigilant in addressing and mitigating these risks.
Vacuum extraction is commonly perceived as an operatordependent task, with emphasis placed on the technical skills of the obstetrician (1-3).However, there is a paucity of research exploring the importance of the non-technical skills employed by the entire delivery team (4)(5)(6).These non-technical skills have been categorized as cognitive, social, and personal resource skills.They include decision-making, situational awareness, communication, teamwork, leadership, and coping with stress and fatigue (7).These non-technical skills are important in acute emergency teams (8)(9)(10) and in obstetric teams managing postpartum hemorrhage (11), settings that clearly differ from delivery by vacuum extraction (12,13).
In the context of vacuum extraction, all research on the non-technical skills has been confined to simulation settings (4,5,14,15) which may not entirely capture the intricacies of real-word patient care.This underscores the imperative for a thorough evaluation of non-technical skills in the delivery suite (16).Therefore, this study analyzes videos of teams engaged in vacuum extractions during reallife deliveries, aiming to explore the non-technical skills and assess their potential correlation with the level of clinical performance.

Study design and setting
Two Danish hospitals participated in the study: Aarhus University Hospital, delivering level 3 maternity care for 5,000 deliveries per year, with 50 physicians, 100 midwives, and 20 technicians; and Horsens Regional Hospital, delivering level 2 maternity care for 2,000 deliveries per year, with 25 physicians, 45 midwives, and 2 technicians.The delivery rooms of the hospitals are well-equipped with two or three high-definition mini-dome surveillance cameras and a ceiling microphone at the center of the room.To minimize the video recordings of normal deliveries, we designed a video system that was activated by Bluetooth on the obstetrician's phone.All cameras recorded 5-min loops until the obstetrician entered the room; the Bluetooth signal activated the cameras, and the preceding 5 min and subsequent time spent in the delivery room were recorded.Videos were only included if consent was obtained from all participants within 48 h; otherwise, videos were automatically deleted from the server.
Written consent was obtained from all participants appearing in the videos (staff, patients, and relatives).Information and collection of consent was a two-step procedure.First, all women received information about the research project in our outpatient clinic some weeks before labor, with records of this information noted in the medical electronic records.Subsequently, in the delivery ward, the midwife responsible for the delivery ensured written informed consent was obtained.The inclusion criterion encompassed all assisted deliveries conducted in our delivery suites, and the exclusion criteria included the absence of informed consent or technical errors in sound or video recording.
Vacuum extractions in Denmark are mainly conducted in the delivery room, with the opportunity to swiftly move to the theater to perform a cesarean section in cases of failed extraction.The incidence of cesareans is 20%, with approximately half of these being emergencies and half elective.The incidence of attempted vacuum extractions is 6-7% of all deliveries and failed vacuum extractions is approximately 9% of these.The hospitals guideline state that vacuum extraction should be discontinued if no descent occurs after two pulls or two pop-offs, or if delivery is not imminent following three pulls.The vacuum extraction is often conducted by a multidisciplinary team, where the consultant performs the extraction, a midwife supports the perineum, a second midwife ensures fetal monitoring, and a nurse assistant ensures all equipment for the extraction is available (for example, different-sized metal cups or silicone alternatives and vacuum machine).Forceps is rarely used for vertex presentation.
The video recordings of teams were recorded between November 2014 and July 2016 and analyzed during the summer of 2017.

Clinical performance assessment by team OBS-VAD
We developed and validated a checklist to evaluate the clinical performance of assisted vaginal delivery via vacuum extraction.The checklist was the result of a Delphi process with 12 experts from the United Kingdom, Norway, Iceland, Sweden, and Denmark (17), and comprises 18 items, each with a weight of importance score: "Done correctly and in a timely manner" (2 points); "Done incorrectly or done correctly with delay" (1 point); "Not done" (0 points); "Cannot be assessed" (no value); or "Not indicated" (no value).The checklist results in a total score of 0-100% (100% = highest standard of care).The checklist also included a visual analog scale of patient safety ranging from 0 to 100%, where elements of management not included in the checklist were evaluated.The total clinical performance score was reported as: 'High' (85-100); ' Average' (60-84); or 'Low' (<60).Both raters (LH and LA) were senior obstetric consultants working in one of the two study hospitals.They were experienced in video review and received appropriate training, including an introduction to the TeamOBS-VAD checklist and a discussion of examples of high and low performance.Subsequently, the raters independently assessed all videos for clinical performance, and were blinded to each other's score.

Non-technical skills assessment by assessment of obstetric team performance
We used the AOTP (18) validated tool to assess the team's non-technical score.It comprises 18 items grouped into six categories: "Communication with the patient, " "Task management, " "Teamwork, " "Situation awareness, " "Communication with the team members, " and "Environment of the room." Each item was scored on a Likert scale (1-5; 1, poor; 5, excellent), resulting in a total score between 18 and 90 (list of all items in Supplementary Table S3).Both raters (LB and KH) were physicians who were experienced in video review and AOTP from a previous study (11).The raters assessed all the videos independently and were blinded to each other's scores.

Statistical analysis
We assessed the following criteria: (1) the interrater agreement by intraclass correlation (ICC) and Bland-Altman analysis; (2) the agreement on each item by percentage agreement and weighed Cohen's kappa; (3) the association between the non-technical score and clinical performance by a restricted cubic spline regression analysis with three knots at 2.5, 3, and 3.5; and (4) the mean difference in clinical performance between the lowest (2) and the highest non-technical score (5) by spline regression analysis (19,20).The regression models were checked using the diagnostic plots of the residuals.The confidence intervals for risk analysis were computed using a non-parametric percentile bootstrap.The potential confounding factors of parity, indication for vacuum (prolonged second stage or fetal compromise), classification (mid, low, and outlet), level of training (junior 1-5 years and senior >5 years), time of event, and hospital type were assessed using multiple linear regression analysis.We used STATA 15 (StataCorp LP, College Station, TX) for all statistical analysis.

Included videos
Women expecting to give birth in one of the two study hospitals were informed about the project, and videos between November 2014 and July 2016 were included.The main reason for exclusion was the absence of consent after 48 h (28 cases), often because the mother had been discharged from the hospital before consent was obtained.Consent was declined by five healthcare providers, six mothers, or relatives.We included 60 videos, with 48% eligible cases (Figure 1).The 60 teams comprised 178 different healthcare providers, 60 different team combinations, and different levels of difficulty with vaginal delivery (Table 1).

Interrater agreement
The interrater agreement was high, as raters assessing the non-technical skills had an ICC of 0.83 (95% CI: 0.71-0.88)and consultants assessing clinical performance had an ICC of 0.84 (95% CI: 0.74-0.90).Agreement among raters for assessing the specific non-technical skills was 0.69-0.94weighted kappa and agreement was visualized using Bland-Altman Plots and limits of agreement (Table 2; Supplementary Tables S1, S2).Four videos were discussed and reassessed by the two raters as their clinical performance assessments differed by >15%, which had been defined a priority as the maximum acceptable difference.This new consensus score was used in the risk analyses, but not in the ICC calculations.No videos were reassessed for non-technical scores.

Clinical performance and non-technical skills
Most teams achieved high clinical performance scores in ensuring the correct position of the mother (94%), appropriate number of staff present (91%), and delivery conducted within maximum four pulls (88%).The greatest challenge posed in clinical skill was considering analgesia, as 39% of the teams neither discussed nor mentioned analgesia as an option.
Clinical performance and non-technical skills correlated with a 'dose-response-like' association (Figure 2).Teams with excellent non-technical scores had an 81% probability of high clinical performance, whereas this probability was only 12% among teams with average non-technical scores (p < 0.001;  S3).These results were robust in terms of the following confounders: hospital, team size, time of day, and number of pulls during the delivery (Supplementary Table S4).

Discussion
Assisted vaginal delivery by vacuum extraction has been considered an operator-dependent task; however, our findings indicate that the non-technical skills of teams play an important role in achieving high clinical performance.Thus, teams with an excellent non-technical score had an 81% chance of achieving high clinical performance, whereas this probability was only 12% among teams with average non-technical scores.Teamwork helped the obstetrician to anticipate the next steps, avoid fixation, ensure analgesia, and ensure sufficient fetal monitoring during delivery.
The main strength of this study is the automatic inclusion of videos using Bluetooth in the obstetrician's telephone, which activated the video cameras whenever the obstetrician entered the delivery room (21).This potentially reduced selection bias.The sample size is another strength, as the sample included 60 different teams and 178 healthcare providers from two hospitals, operators of the vacuum at all levels of experience, all degrees of difficulty, every day of the week, and any time of day.Finally, the two pairs of raters used a validated checklist for systematic assessment and the interrater agreement was high.
Our study had certain limitations.We cannot exclude selection bias, as low-performing teams may have been less willing to provide their consent.However, it is reassuring that 95% of the healthcare providers gave consent to include their videos and that only two videos were deleted after staff withdrew their consent.We also acknowledge that our setup did not include all vacuum extractions, as the camera was activated by the consultant on call.There is a risk of assessment bias as AOTP raters may have been influenced by the actual clinical performance and vice versa.All teams were aware of the project, and this may have changed their behavior, a form of "Hawthorne effect" which might affect the observed association (22).Furthermore, despite having all the delivery rooms equipped with two or three cameras in the ceiling, the video could not capture all the details of the technical skills, and raters often missed tactile information gained during vaginal examination before the operator applied the cup (for example, molding, level, and position).Furthermore, our legal permission was restricted to analyze the videos; hence, we did not have access to any medical journal with maternal or neonatal outcomes.Finally, while our study found that clinical performance and non-technical scores were strongly correlated with a dose-response like association; this is no proof of causality.
The opportunity to review video recordings of teams managing vacuum extractions in real-life was a special experience that changed the way vacuum extractions were handled in our department (23).Epidural as pain relief in labor is available for every laboring woman in Denmark; however, uptake is driven by maternal request.Overall, approximately 27% of nulliparous women in labor request an epidural.We considered whether the high sound level on the recording reflected real life.Therefore, we tested our audio source with the help of a sound engineer and found that the quality was high and that the dB level in the room was reflected by the dB level in the video (24).These findings highlight the importance of improving our practice of offering women analgesia during vacuum extraction.Based on our study, we recommend prioritizing the wider use of pudendal block in cases of vacuum extraction and ensuring that women are adequately informed about assisted delivery preferable in the antenatal period (25).
Before this study, non-technical skills for vacuum extraction have only been investigated in simulated settings (6,15,(26)(27)(28), but did suggest that non-technical skills contribute to successful delivery assisted by vacuum extraction (4,14,29).However, these studies did not describe the specific aspects of behavior or the specific non-technical skills that support teams to achieve high clinical performance.In this study, we observed how enacting specific non-technical skills helped obstetricians to anticipate the next steps, such as offering analgesia to women without an epidural or calling the pediatric team to the room when the indication was suspected fetal distress.Moreover, the team helped the consultant to keep track of the time and the number of contractions, thereby helping the consultant decide when to abandon the attempted vaginal delivery in a timely manner.This calls for rethinking our learning path for vacuum extraction, ensuring that trainees learn to include effective teamwork behavior in training and practice.Thus, this study confirms the results on surgical (30), trauma (31,32), and resuscitation teams (33)(34)(35) regarding the importance of non-technical skills in providing high clinical performance.Clinical performance and non-technical skills.High-performing teams in this study used specific behaviors that differed from those of high-performing teams managing postpartum hemorrhage (11).High-performing teams managing vacuum extraction demonstrated high scores in team interaction, anticipation, avoidance of fixation, and focused communication, whereas highperforming teams managing postpartum hemorrhage had high scores in vigilance, role assignment, stress management, and leadership (11).This difference may reflect the fact that most vacuum extractions are a 'semi-elective' procedure, whereas postpartum hemorrhage is usually an urgent, acute emergency.Another interesting disparity between the two settings was the importance of avoiding fixation, which we could only demonstrate in the vacuum extraction setting but not in the postpartum hemorrhage setting.This is related to the fact that the accoucheur who is conducting the vacuum delivery is 'task-focused' and therefore cannot maintain wider situational awareness.This requires a member of the team to 'step back' to monitor the whole situation (that is, 'situational leadership').This tends to happen in teams managing acute emergencies, such as postpartum hemorrhage, where the situational leader steps back to co-ordinate and delegate tasks while ensuring that vigilance is maintained (7).Therefore, all team members should keep an eye on the overall situation while the accoucheur conducts the vacuum extraction and may need to consider whether an individual needs to 'step back' to maintain overall vigilance.
Video reviews offer a unique opportunity to review our performance, and future studies should investigate whether video reviews can help to improve our vacuum extraction performance (31,34).Obstetric trainees primarily learn to perform vacuum extraction from senior colleagues by craft apprenticeship.In the last few decades, trainees have experienced difficulties in developing and maintaining obstetrical competencies, such as vacuum extractions, due to reduced working hours and a reduced rate of instrumental deliveries in obstetric care (33).Elective video review could be a valuable learning method to junior obstetricians to improve performance as they can revisit and reflect on their performance and learn from experienced colleagues (36).When they become consultants, they primarily work alone or with a junior trainee, and do not always have the same opportunity to benchmark their performance with several consultants; feedback may not always be ideal.Similar to other studies, we found that our colleagues had a genuine interest in learning about the outcomes of the video review, and we think video reviews offer us new learning opportunities for vacuum extraction to improve and maintain high performance in obstetric teams.Future studies are needed to explore how (37).
The external validity of our findings must be considered before they can be applied in other settings.In the labor and delivery wards, where the standard analgesia is epidural/spinal, our findings on further analgesia may not be relevant.However, we believe that the key findings of this study may be applicable in other settings where teams manage vacuum extractions, and we recommend including the specific behaviors of team interaction, anticipation, avoiding fixation, and focused communication to complement any technical training on vacuum extraction.
In conclusion, mastering vacuum extraction is a core competency in delivery wards and is important for the safety of women and children.Our study demonstrated how video reviews of real-life vacuum extraction can be used to evaluate the clinical and non-technical performance of teams to identify areas that require particular attention during training.Although vaginal vacuum extraction is generally accepted as an operator-dependent task, our findings indicate that the teams' non-technical skills play an important role in ensuring high clinical performance, as the accoucheur who is conducting the vacuum delivery is "task-focused" and cannot maintain wider situational awareness.Therefore, the team members are required to "step back" to monitor the whole situation, and the key non-technical skills associated with high clinical performance were team interaction, anticipation, avoidance of fixation, and focused communication.

TABLE 1
Description of included teams.The American College of Obstetricians and Gynecologist classification of types of operative vaginal delivery (ref guideline ACOG). *

TABLE 2
Interrater agreement for clinical performance and non-technical performance.Intraclass correlation (ICC) between raters.**All 60 videos were analyzed by two raters.The average agreement represents the ICC between two raters and the other two raters. *

TABLE 3
Risk/chance of low/high clinical performance to the level of non-technical performance.These figures are based on 60 recordings from real-life teams managing major vaginal-assisted deliveries via vacuum extraction and risk (95% CI).*TeamOBS-VAD score below the minimum pass (score < 60).**TeamOBS-VAD score above high performance (score > 85).