Robotics and AI for Teleoperation, Tele-Assessment, and Tele-Training for Surgery in the Era of COVID-19: Existing Challenges, and Future Vision
- 1Canadian Surgical Technologies and Advanced Robotics (CSTAR), London Health Sciences Centre, and School of Biomedical Engineering, University of Western Ontario, London, ON, Canada
- 2Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
- 3Department of Electrical and Computer Engineering, University of Western Ontario, London, ON, Canada
- 4Department of Surgery, University of Western Ontario, London, ON, Canada
- 5Department of Electrical and Computer Engineering, New York University, New York, NY, United States
- 6Department of Mechanical and Aerospace Engineering, New York University, New York, NY, United States
The unprecedented shock caused by the COVID-19 pandemic has severely influenced the delivery of regular healthcare services. Most non-urgent medical activities, including elective surgeries, have been paused to mitigate the risk of infection and to dedicate medical resources to managing the pandemic. In this regard, not only surgeries are substantially influenced, but also pre- and post-operative assessment of patients and training for surgical procedures have been significantly impacted due to the pandemic. Many countries are planning a phased reopening, which includes the resumption of some surgical procedures. However, it is not clear how the reopening safe-practice guidelines will impact the quality of healthcare delivery. This perspective article evaluates the use of robotics and AI in 1) robotics-assisted surgery, 2) tele-examination of patients for pre- and post-surgery, and 3) tele-training for surgical procedures. Surgeons interact with a large number of staff and patients on a daily basis. Thus, the risk of infection transmission between them raises concerns. In addition, pre- and post-operative assessment also raises concerns about increasing the risk of disease transmission, in particular, since many patients may have other underlying conditions, which can increase their chances of mortality due to the virus. The pandemic has also limited the time and access that trainee surgeons have for training in the OR and/or in the presence of an expert. In this article, we describe existing challenges and possible solutions and suggest future research directions that may be relevant for robotics and AI in addressing the three tasks mentioned above.
The novel coronavirus has been declared a public health emergency of international concern by WHO in Jan 2020 (WHO, 2020). By the time of writing this paper, all countries are affected by the pandemic. The unprecedented shock wave of the virus spread has impacted regular health care service delivery. The extreme pressure on healthcare systems has exceeded capacity, and managing the pandemic has become a global issue that has drastically influenced most aspects of the healthcare system. The performance of surgeries (most of which are categorized as elective surgeries), training for surgery, and assessments for surgery are aspects that have been significantly impacted.
Due to the chance of false negatives in the pre-surgery COVID-19 testing of patients, all patients have to be treated as suspect cases. Dealing with infected or suspected patients requires precautions such as consideration for anesthesiologists (Willer et al., 2020), limitation of staff exposure to patients, and wearing PPE, which poses difficulties to operating theatre (Kumar et al., 2020). However, these procedures cannot guarantee the safety of staff and patients. Moreover, since hospital staff is in contact with several people each day, cross-infection through staff should also be considered.
New regulations have recommended a moratorium on elective surgery to avoid virus spread in hospitals by minimizing personal interactions and expenditure of medical resources for infected patients who need intensive care. Deferring elective surgeries is based on opinions and has secondary consequences. Progression of the disease continues when the patient is waiting for surgery. This has a substantial impact on the life quality of patients (Fu et al., 2020), results in higher treatment costs (Reyes et al., 2019), and may cause unexpected death (Zafar, 2020). On the other hand, elective surgeries are not optional and must be performed eventually. Thus, there may be a need for performing a deferred surgery during the COVID-19 time frame. Moreover, catching up with the 2 million backlogged elective surgeries worldwide each week will impose a huge burden on the healthcare system when elective surgeries resume (Szklarski, 2020).
This unprecedented scenario not only has affected surgeries, but has also influenced surgery-related activities profoundly. Surgical education has been affected adversely by the pandemic. There has been a gradual reopening of activities, including schools, but with the anticipated rise in the rate of infection it is anticipated that there will be some levels of shut down again in this sector. Trainees are banished from wards, and residents have lost their access to practical OR training (Ferrario et al., 2020; Ferrel and Ryan, 2020). In addition, emergency and non-deferable surgeries are being done by senior surgeons without the presence of trainees to reduce operation time and risk of complications, and mitigate the risk of residents’ exposure to COVID-19 (Bernardi et al., 2020). This situation has imposed mental anxiety and has slowed down the learning curve of residents and medical students, who will be needed in catching-up with deferred surgeries in the future (Ahmed et al., 2020).
Moreover, going into hospitals for pre- and post-surgery assessments is also a safety concern during the pandemic. There is always an infection risk for any minute that a non-COVID patient spends in a hospital. Consequently, hospitals try to discharge patients as soon as possible to reduce the risk of infection. Besides, the closure of medical offices has disturbed pre- and post-surgery assessments (Scaravonati et al., 2020).
Since there is no widely approved or sufficiently tested vaccine, there is a possible chance of second and third global waves in the Fall and Winter, and continuing lockdown regulations imposes an intolerable burden on the healthcare system. Several countries are therefore planning for a reopening guideline to resume safe delivery of surgical services (Dattani et al., 2020). However, it is not clear how the reopening phase will affect the quality of healthcare delivery and how the above-mentioned tasks should be performed safely while there is a lack of a clinically approved therapy for COVID-19. This perspective paper proposes robotics and artificial intelligence (AI) as a solution for the three above-mentioned tasks and investigates potential opportunities in the area to address the mentioned problems.
Minimally invasive surgery (MIS) has demonstrated superiority over open surgery. Less amount of blood loss, and shorter recovery and hospital stay are the main reasons for the preference of MIS when it is possible. Meanwhile, robotics-assisted MIS (RAMIS) has evolved and shown superhuman capabilities for teleoperation and has found its place in MIS. Teleoperation offers surgeons an ergonomically operating posture (Ballantyne, 2002), provides them with more dexterity than conventional laparoscopy (Moorthy et al., 2004), enhances the accuracy of motion beyond surgeons’ natural ability, etc. Besides these benefits, the main virtue that distinguishes teleoperation in the COVID-19 era is providing the ability to separate the surgeon’s console (leader robot) from the patient robot (follower robot) while keeping them connected through a communication interface (Challacombe et al., 2003).
Telesurgery affords physical separation of the surgeon from the patient, in a separate room avoiding bilateral infection transfer, which can be life-threatening. In addition, the number of bedside staff in RAMIS is less than in open surgery (Kimmig et al., 2020). This provides the safety of the patient and operating room by reducing inter-personal contacts to the lowest level possible. This performance has been shown experimentally. In the United States, a CorPath robotic intervention arm has been used in coronary intervention on a COVID patient to provide safety of the personnel (Tabaza et al., 2020).
Dealing with an infected or suspected patient requires a maximal level of protection (Liang, 2020). The physical disturbance caused by this level of protection has a negative influence on surgical performance. On the other hand, the COVID-19 situation increases the surgeon’s mental stress, which critically affects the surgeon’s performance. Elevated stress levels could likely be due to the fear of contracting the virus or spreading it to patients and the surgeon’s family (Tan et al., 2020). Studies have shown that depression, anxiety, insomnia, and stress have increased, especially among front-line healthcare providers during the pandemic (Lin et al., 2020). High stress levels may result in inappropriate responses, such as poor decision making and impaired psychomotor performance (Wetzel et al., 2006; Arora et al., 2010). The elevated psychological stress levels among healthcare providers may sustain even one year after the outbreak as it happened in 2004 with SARS (Lee et al., 2007). Not only telesurgery reduces surgeon’s stress by providing better ergonomics during surgery (Berguer and Smith, 2006), but also, during the COVID era, robotics-assisted surgery significantly reduces stress levels by providing higher infection protection through physical distancing between the patient and the surgeon; also, it reduces the number of needed medical staff to be present in close proximity to the patient and each other during prolonged surgeries (which can increase the possibility of infection transfer between a patient and staff and between staff members). It should be highlighted that robotics assisted surgery does not make the aforementioned interactions zero as there is always the need for some format of interaction between a patient and staff. However, it reduces the duration and the number of interactions.
Recently, the concept of semi-autonomous and autonomous surgery has attracted a great deal of interest thanks to advancements in the area of machine intelligence especially when combined with computer vision (Moawad et al., 2020). When compared with teleoperated robotic surgery, AI-based autonomous and semi-autonomous robotic systems has not been fully exploited in the literature as these are newer topics of the field. In the language of surgery project, it has been shown that combining AI with teleoperation can provide a semi-automated system that can recognize and perform tasks automatically when there is a pre-trained model for the recognized task, allowing for faster and high accuracy procedures (Bohren et al., 2011). Semi-autonomous robots have been used for orthopedic surgery, such as MAKOplasty, when preoperative images are fused with intraoperative information to provide surgeons with an augmented sensorimotor capability through production of dynamic virtual fixture in time and space. More recently, fully-autonomous robotic surgery has been discussed in medical robotic communities and preliminary experiments on ex-vivo tissue have shown promising results. The performance and accuracy of semi-autonomous surgical robots have been proved clinically (Hampp et al., 2019). For example, the MAKO (Stryker, 2020) and NAVIO (Smith and Nephew, 2020) robots guide the surgeon in joint arthroplasty semi-autonomously and prevent excessive bone loss. This guarantees proper bone preparation and precise implantation. However, autonomous surgical robots, despite their great accuracy in comparison to manual procedures (Shademan et al., 2016), are still in the non-clinical development phase. In the context of remote operation, the use of autonomous robots can provide a higher degree of separation while providing some additional accuracy through processing of multimodal intraoperative information. However, this is a technologically challenging field which should be investigated to provide more autonomy regarding management of surgery during a crisis such as COVID-19.
The other benefit of RAMIS in the COVID-19 era is that it increases the availability of intensive care unit (ICU) beds. The smaller incision for RAMIS shortens patient’s recovery time and hospital stay. This allows hospitals to dedicate more ICU beds to critically ill cases while handling surgeries. There is however a shortcoming in terms of the OR time usage for RAMIS as a result of the extra setting-up time and longer procedure times (Heemskerk et al., 2007; Cho et al., 2016; Lindfors et al., 2018). Nonetheless, a shorter post-surgery hospital stay is of paramount importance and outweighs the longer OR time, notably in the COVID-19 era. Moreover, deploying AI in robotic surgery has been shown to decrease soft tissue damage and consequently decrease recovery time (Wall and Krummel, 2020).
Due to abdominal pressure in laparoscopic surgery, there are some concerns about the possibility of aerosolization of viral particles and contamination through surgery smoke in laparoscopic surgery (Schwarz and Tuech, 2020; Van den Eynde et al., 2020). Although these methods of infection are not completely proved for COVID-19 yet, safety regulations should be considered to prevent possible infections. It should be noted that surgical smoke is also released in open surgeries; however, in RAMIS, it is easier to handle the smoke trapped in the patient’s body. Safety precautions to prevent these issues are 1) lowering the electrocautery power to reduce the amount of smoke production (Mottrie, 2020); 2) smoke evacuation and abdominal deflation through ultra-low penetrating air (ULPA) filter (Kimmig et al., 2020); and 3) reducing abdominal pressure to the lowest possible. RAMIS surgeries are feasible to perform with lower abdominal pressure than conventional laparoscopic surgery (Kimmig et al., 2020). To summarize, RAMIS is safer than MIS and open surgery in terms of contamination through aerosolization of viral particles for bedside staff.
Telerobotic surgical systems have solved several issues associated with conventional MIS and also provided the surgeon with new capabilities. These features are 1) depth perception; 2) dexterity enhancement; 3) improved accuracy; 4) better hand-eye coordination; and 5) and multiple tools delivery through a single incision (Atashzar and Patel, 2018). Moreover, in teleoperation, information and operation data can be saved and used for training purposes both for AI supervision and training of novice surgeons (Zemmar et al., 2020). The problem of degraded haptic feedback in conventional laparoscopy has not been solved yet. Better tracking accuracy and improved surgical performances have been achieved using the haptic feedback in RAMIS (Talasaz et al., 2014), (Currie et al., 2017). Related to this, the lack of haptic feedback increases the risk of tissue damage due to large unintentional forces. Other modalities of feedback such as visual force feedback of the tool (Tavakoli et al., 2006), a tactile sensor and tactile ultrasound (tactUS) instrument for palpation and tumor localization (Trejos et al., 2009; Naidu et al., 2017a; Naidu et al., 2017b), and skin stretch feedback (Schorr et al., 2013) are influential in robotic surgery, but none of them can completely make up for the absence of haptic feedback. Thus, enabling telerobotic surgical systems with force sensing and force reflecting modules is of high importance, which increases the quality of teleoperated surgery (Talasaz and Patel, 2013; Talasaz et al., 2017). A machine learning algorithm has been deployed to estimate the elongation of suture from knot type, initial suture length, and surgical thread type data, and visual feedback has been used to warn the surgeon of the risk of suture breakage (Dai et al., 2019). In particular, considering a larger number of surgeries that can benefit from teleoperated procedures using robots, during the era of COVID, addressing this challenge should be accelerated. This topic has seen ongoing research, and unfortunately, the current trend does not show a promise of an upcoming solution. With improving technology for haptic feedback, this can result in a major advance in the performance of teleoperated surgeries on a larger scale and can enlarge the domain of surgeries, which can be conducted teleoperatively, helping with the management of the current concerns regarding infection transfer during surgery in the time of COVID.
There are two characteristics associated with a good haptic teleoperation system: transparency and stability. In the last three decades, a significant amount of research has been done on developing a transparent control architecture. Four-Channel Lawrence (FCL) was proposed as the first transparent teleoperation system (Lawrence, 1993). It was modified to simpler architectures (Hashtrudi-Zaad and Salcudean, 2001), (Hashtrudi-zaad and Salcudean, 2002). Atashzar et al. proposed a simplified two-channel modified-ELFC (M-ELFC) architecture that provides a high degree of transparency (Atashzar et al., 2012). To deal with the stability issue, three categories of passivity-based controllers have been proposed: 1) the Wave Variable approach (Aziminejad et al., 2008); 2) Time Domain Passivity Approach (TDPA) (Ryu et al., 2010); and 3) Small-gain approach (Atashzar et al., 2017a). Both techniques stabilize the system; however, stabilization comes at the cost of compromising transparency. Considerable research has been done to improve the performance of teleoperation (Artigas et al., 2010; Chawda and Omalley, 2015; Atashzar et al., 2017b; Panzirsch et al., 2019; Singh et al., 2019), but the proposed stabilization methods are still far from ideal. The discussion above clarifies some of the technical challenges creating obstacles to realizing high-fidelity haptics-enabled teleoperated surgery. The potential of RAMIS in resolving the surgical issues caused by COVID-19 is calling for an accelerated trend of research and development, extending the performance of teleoperated surgical robotic systems for allowing more benefit of this technology in reducing the burden on the healthcare system during the pandemic and similar crises in the future.
Remark: AI has been extensively developed in the last decade and has revolutionized many industries. However, the application of AI in surgical procedures requires a significant amount of adaptation and consideration. Robotic surgery can take advantage of AI in the COVID era from three aspects; 1) increasing accuracy and reducing the risk of failure by providing shared and full autonomy in simple tasks (Rabinovich et al., 2020; Wall and Krummel, 2020); 2) allowing physical distancing by changing the surgeon’s role from executive and continuous control to supervisory and intermittent control; and 3) increasing the average number of surgical procedures, which will be required to address the backlogged surgeries caused by the shutdown of elective surgeries over a long period of time, thereby reducing the load on surgeons and allowing after-hour surgeries (Zemmar et al., 2020).
Tele-Examination of Patients
Preoperative examination for surgery preplanning and post-operative patient examination in the recovery time is another matter of concern in the COVID-19 era. In-person visits increase the risk of virus contraction for the patient and the surgeon. Keeping personal interactions as low as possible is the key factor in dealing with the pandemic.
Post-operative examinations may include patient’s assessment at home and ICU. Telepresence robots that are made for telehealthcare purposes allow physicians to interact with patients, and monitor patients’ vital signs without the physical presence of the surgeon in the ICU (Laniel et al., 2017). These systems have been used in Italy at COVID-19 patients’ bedside in the ICU (Bogue, 2020; Pullella, 2020). A similar robot has been used in Israel to communicate with quarantined patients (Marks, 2020). In terms of home healthcare, messages, phone, and video calls have been used for post-operative examination. It has been shown that telepresence robots could provide a stronger feeling of a person to person interaction for both users, in comparison to video and phone calls, and both physicians and patients have expressed satisfaction (Tavakoli et al., 2020), (Becevic et al., 2015).
Preoperative examinations have also been done with AI- and robotics-enabled telehealth, but applications are limited due to the lack of physical exams and the need for clinical imaging. However, it has been shown that for some specific conditions, diagnosis via telemedicine could be as accurate as an in-person diagnosis when examination through telemedicine is feasible subject to limitations (Asiri et al., 2018). For example, AI can be used for digital triage to direct patients to the most appropriate medical center based on the resources and their condition before they show up in emergency rooms (Lai et al., 2020). As another example, it has been shown that blood draw and injections can be done with portable robots using AI more accurately and faster than a manual procedure (Zemmar et al., 2020). Another example is the telerobotic system that has been used in China to perform cardiac and lung ultrasound on a COVID patient (Wang et al., 2020). These systems can help safeguard patients and staff by reducing the need for patient referral to hospital and physical distancing.
Robotics and AI have taken a step in the development of tele-examination of patients during pre- and post-operative phases. However, there is still room for adding new capabilities to tele-healthcare robots in order to lower the need for in-person examinations or patient referrals to hospital. Besides robots, focusing on smartphone-based or computer-based tele-examination systems would be useful because of their widespread use.
Tele-Training of Surgeons
The outbreak of COVID-19 has severely affected surgical training procedures. The most significant components of surgical training are comprised of theoretical, pre-clinical, and hands-on clinical training, but the lockdown caused by the pandemic has severely limited the opportunities for students and residents to acquire surgical training (Puliatti et al., 2020), (Bernardi et al., 2020). High-quality and intense healthcare support, which would be needed during and after COVID-19, requires precise training. Although some schools are in a gradual reopening phase, there would be some level of shut down again with the next wave of the pandemic.
In such extraordinary conditions, online learning, teleconferences, and webinars can be of benefit with regard to surgical education and fill the gap with regard to theoretical training issues (Dedeilia et al., 2020). The benefits of these online learning technologies have been shown prior to the COVID-19 pandemic.
On the other hand, robotics and AI could improve the quality of pre-clinical training. Pre-clinical training is conventionally performed through dry or wet lab practices. The use of robotic simulators based on virtual reality (VR) systems has shown a significant improvement in novice surgeons’ skills (Tergas et al., 2013). Hands-on-Surgical Training (HoST) provided by augmented reality (AR), and dual-user teleoperated system with virtual fixtures are more advanced simulators that help the novice surgeon to navigate using haptics-enabled cues outside the OR (Shahbazi et al., 2013; Kumar et al., 2015). Xperience Team Trainer developed by Mimic Simulations allows teamwork training in the OR at the pre-clinical stage (Mimics, 2020). This technology provides simultaneous training for the novice surgeon and the bedside assistant to improve coordination between the surgeon and assistant.
The preservation of acquired skills is another important issue in the COVID time. Surgical skills including motor and cognitive skills decay when a surgeon goes through a long period of time without using the acquired skills (Perez et al., 2013). Simulation-based medical education may fill the gap in surgical practice and prevent the loss of surgical skills during a lockdown (Higgins et al., 2020). In addition, AI can be employed to interpret the data collected from simulations for surgeons’ skill evaluation (Winkler-Schwartz et al., 2019).
Because the above-mentioned technologies provide high-quality training while keeping social distancing, they could be part of the solution for the educational gap in the COVID-19 era. An active line of research and development that can be accelerated would be to design and develop small, inexpensive, and portable sensorized robotic modules connected to cloud-based virtual reality surgical environments. A large number of trainee surgeons could then continue their hands-on practice/training when access to training facilities is significantly restricted. This is critical because sensorimotor learning is a continual process in the human brain, and a long pause before getting to the agency level can drastically result in fading of sensorimotor skills.
Theoretical and pre-clinical training may guide students to pass the cognitive phase of learning; however, the integration phase, which gives them appropriate motor skills to perform surgery, requires performing surgery under the supervision of an expert in the OR (Choi et al., 2020). Residents would have very limited access to this form of training due to cancelation of elective surgery (Imielski, 2020) or requirements of social distancing. For telesurgery which is more challenging for residents to perform than open surgery and requires specific training, a viable solution that can be achieved using existing systems can be developed using hand-over-hand haptic-enabled tele-training (realized by multilateral teleoperation systems). This would not only allow novice surgeons to perform surgery from a safe distance, but also give them the opportunity to be supervised by an expert at the same time (Shahbazi et al., 2018a; Shahbazi et al., 2018b). The dual console teleoperation system format shares the control of the operation between the expert and the trainee. Incorporating haptics-enabled feedback would then provide the trainee with real-time force feedback. This format of telesurgery training gives the resident experience through supervised surgery without jeopardizing the safety of the patient or the resident during the constraints imposed by COVID-19. Furthermore, these multilateral tele-training systems could also be set up to evaluate the motor skills of trainees based on their performance.
Discussion and Conclusion
The novel coronavirus has challenged the healthcare system across the globe. Social distancing has become a new normal and may remain for a significant length of time especially as a result of the lack of vaccine and treatment for a critical period of time. This has deeply impacted surgeries and surgically related activities which may revolutionize how future healthcare systems function. Canceling elective surgeries was an efficient policy to curb the spread of the virus; nevertheless, keeping to this plan could have a detrimental effect on the health of patients and the healthcare system. Currently, governments are working on reopening guidelines. In this unprecedented situation, robotics and AI could play an important role in the safe delivery of surgical services through the use of telesurgery, tele-examination, and tele-training environments. A summary of the existing technologies and required features is given in Table 1.
TABLE 1. Existing technologies and required features in telesurgery, tele-examination, and tele-training.
Regarding teleoperated robotic surgery, it should be noted that although there is a wide range of benefit for both patients and the surgeons, there still exist a spectrum of challenges which are open topics for research and development. Regarding benefits, in the context of laparoscopic surgery, it can be mentioned that besides reduced operation time, reduced blood loss, increased accuracy, and reduced recovery time, there are additional benefits that are more pronounced during the pandemic, including reduced time and frequency of interpersonal interaction between surgical staff, reduced number of staff, reduced interaction between patients and staff, all to reduce the risk of infection transfer and increase the safety of surgical procedures (Kimmig et al., 2020; Tavakoli et al., 2020). It should be noted that the current state of telesurgery and robotics-assisted surgery are advanced for abdominal surgery; however, for some categories such as orthopedic surgery, teleoperation has not been considered as a robust option. During the pandemic, any technology that reduces the duration of surgery directly or indirectly (for example, by increasing the accuracy which reduces the need for readjustments) can significantly reduce the chances of infection transmission. This is critical since, in general, surgeons operate on many patients in a short time, which can increase the risk of infection even between patients indirectly through their surgeon.
However, it should be noted that there is a wide range of challenges which have not been addressed yet, especially in the context of teleoperated surgery, and these for the future direction of research. One of the major challenges is the stability and transparency of force-feedback teleoperated robotic systems (Aziminejad et al., 2008; Ryu et al., 2010; Atashzar et al., 2017a). Due to the concerns of safety the existing commercialized telerobotic surgical systems, such as the da Vinci surgical system, do not enable force reflection, even though it is known that force reflection can significantly increase the quality of surgery by providing a much higher situational awareness for surgeons. Although a number of stabilizers and control algorithms have been reported in the literature, the existing algorithms result in deviation of motion tracking and force reflection, which reduces the accuracy of surgery and is often not acceptable (Artigas et al., 2010; Chawda and Omalley, 2015; Atashzar et al., 2017b; Panzirsch et al., 2019; Singh et al., 2019). Besides stability, instrumentation is another challenge. Attaching inexpensive, disposable, biocompatible, and miniaturizable force sensors to surgical tools for measuring multidimensional forces for reflection through a teleoperation medium is a major instrumentation challenge and an open line of research (Atashzar and Patel, 2018). Technologies such as optical force sensors are promising options and are the front line of research in this regard.
In addition, the introduction of AI in telesurgery is a new field of research and development which has attracted a great deal of interest in order to enable parts of surgical tasks to be automated, thereby reducing some cognitive and physical burden for the surgical team with the potential for reducing the operation time, increasing accuracy and reducing the number of needed staff in the operating room. The accuracy resulting from using AI in industrial applications has been shown; however, more research is required to prove its performance and build up confidence in the medical area (Wall and Krummel, 2020). Dealing with soft tissue is the main challenge when involving AI in the context of robotic and telerobotic surgery.
Regarding tele-examination, telepresence robots have been effective in improving post-operative patient-surgeon interactions and monitoring patient’s vital signs, mostly in ICUs. However, due to limitations, effective solutions for detailed pre- and post-operative tele-examination of patients have not been proposed in the literature. One of the main challenges in this area is the development of portable sensorized robots for detailed remote monitoring of patient’s signs. On the other hand, AI would be particularly useful in automating tele-examination devices to reduce the need for in-person pre- and post-operative examinations.
As for tele-training, simulation-based training systems using AR and HoST have provided a context for pre-clinical training while ensuring the safety of trainees and experts. In addition, simulation-based training can be effective in ensuring skill levels of surgeons in the presence of long periods of surgical inactivity. However, there are open areas for research in this field. Hand-over-hand training using multilateral teleoperation is one of the future research areas that can profoundly improve the quality of clinical surgical tele-training. The stability of delayed multilateral teleoperation and effective methods for sharing control between an expert and a trainee are directions for future researches (Shahbazi et al., 2018a; Shahbazi et al., 2018b). Besides hand-over-hand training, employment of AI for surgical skill training and assessment are open research areas.
In this perspective article, we have provided our opinions on some existing technologies which can be adopted rapidly to help with the current unprecedented situation and have given a perspective of the technologies required in hospitals. The intention in writing this article has been to initiate discussions between researchers, policymakers, and stakeholders to further investigate the use of robotic, telerobotic and AI-based solutions in a framework for enhancing the performance of surgery, surgical training, post-operative treatment, and monitoring under the severe restrictions imposed by COVID-19. The vision and opinions presented in this article are based on an extensive review of the literature concerning approaches through which Robotics and AI can play a significant role.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
The four authors (NF, MT, RP, SA) collaborated on the conceptualization of this article, conduction of the literature review, technical and technological analysis, and writing and editing the paper.
The work of NF and RP was funded by the Natural Sciences and Engineering Research Council (NSERC) of Canada under grant #RGPIN-1345 (awarded to RP) and the Tier-1 Canada Research Chairs Program (RP). The work of SA is supported, by the National Science Foundation (Award Number:2031594).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Arora, S., Sevdalis, N., Aggarwal, R., Sirimanna, P., Darzi, A., and Kneebone, R. (2010). Stress impairs psychomotor performance in novice laparoscopic surgeons. Surg. Endosc. 24 (10), 2588–2593. doi:10.1007/s00464-010-1013-2
Artigas, J., Ryu, J. H., and Preusche, C. (2010). “Position drift compensation in time domain passivity based teleoperation,” in IEEE/RSJ 2010 international conference on intelligent robots and systems, Taipei, Taiwan, October 18–22, 2010 (IEEE), 4250–4256. doi:10.1109/IROS.2010.5652691
Asiri, A., AlBishi, S., AlMadani, W., ElMetwally, A., and Househ, M. (2018). The use of telemedicine in surgical care: a systematic review. Acta Infor. Med. 26 (3), 201–206. doi:10.5455/aim.2018.26.201-206
Atashzar, S. F., Shahbazi, M., Talebi, H. A., and Patel, R. V. (2012). “Control of time-delayed telerobotic systems with flexible-link slave manipulators,” in IEEE International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, PT, October 7–12, 2012 (IEEE), 3035–3040. doi:10.1109/IROS.2012.6386170
Atashzar, S. F., Polushin, I. G., and Patel, R. V. (2017a). A small-gain approach for nonpassive bilateral telerobotic rehabilitation: stability analysis and controller synthesis. IEEE Trans. Robot. 33 (1), 49–66. doi:10.1109/TRO.2016.2623336
Atashzar, S. F., Shahbazi, M., Tavakoli, M., and Patel, R. V. (2017b). A passivity-based approach for stable patient-robot interaction in haptics-enabled rehabilitation systems: modulated time-domain passivity control. IEEE Trans. Cont. Syst. Technol. 25 (3), 991–1006. doi:10.1109/TCST.2016.2594584
Aziminejad, A., Tavakoli, M., Patel, R. V., and Moallem, M. (2008). Transparent time-delayed bilateral teleoperation using wave variables. IEEE Trans. Cont. Syst. Technol. 16 (3), 548–555. doi:10.1109/TCST.2007.908222
Becevic, M., Clarke, M. A., Alnijoumi, M. M., Sohal, H. S., Boren, S. A., Kim, S. M., et al. (2015). Robotic telepresence in a medical intensive care unit—clinicians’ perceptions. Perspect. Health Inf. Manag. 12, 1–9.
Berguer, R., and Smith, W. (2006). An ergonomic comparison of robotic and laparoscopic technique: the influence of surgeon experience and task complexity. J. Surg. Res. 134 (1), 87–92. doi:10.1016/j.jss.2005.10.003
Bernardi, L., Germani, P., Del Zotto, G., Scotton, G., and de Manzini, N. (2020). Impact of COVID-19 pandemic on general surgery training program: an Italian experience. Am. J. Surg. 220 (5), 1361–1363. doi:10.1016/j.amjsurg.2020.06.010
Bohren, J., Guerin, K., Xia, T., Hager, G. D., Kazanzides, P., and Whitcomb, L. L. (2011). “Toward practical semi-autonomous teleoperation: do what i intend, not what i do,” in Proceedings of IEEE workshop on advanced robotics and its social impacts, Menlo Park, CA, October 2–4, 2011 (IEEE), 20–23. doi:10.1109/ARSO.2011.6301974
Chawda, V., and Omalley, M. K. (2015). Position synchronization in bilateral teleoperation under time-varying communication delays. IEEE/ASME Trans. Mechatron. 20 (1), 245–253. doi:10.1109/TMECH.2014.2317946
Cho, J. N., Park, W. S., Min, S. Y., Han, S. A., and Song, J. Y. (2016). Surgical outcomes of robotic thyroidectomy vs. conventional open thyroidectomy for papillary thyroid carcinoma. World J. Surg. Oncol. 14 (1), 1–7. doi:10.1186/s12957-016-0929-y
Choi, B., Jegatheeswaran, L., Minocha, A., Alhilani, M., Nakhoul, M., and Mutengesa, E. (2020). The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey. BMC Med. Edu. 20 (1), 1–11. doi:10.1186/s12909-020-02117-1
Currie, M. E., Talasaz, A., Rayman, R., Chu, M. W. A., Kiaii, B., Peters, T., et al. (2017). The role of visual and direct force feedback in robotics-assisted mitral valve annuloplasty. Int. J. Med. Robotics 13 (3), 1–12. doi:10.1002/rcs.1787
Dai, Y., Abiri, A., Pensa, J., Liu, S., Paydar, O., Sohn, H., et al. (2019). Biaxial sensing suture breakage warning system for robotic surgery. Biomed. Microdevices. 21 (1), 5–10. doi:10.1007/s10544-018-0357-6
Dattani, R., Morgan, C., Li, L., Bennett-Brown, K., and Wharton, R. M. H. (2020). The impact of COVID-19 on the future of orthopaedic training in the UK. Acta. Orthop. 91 (6), 627–632. doi:10.1080/17453674.2020.1795790
Dedeilia, A., Sotiropoulos, M. G., Hanrahan, J. G., Janga, D., Dedeilia, P., and Sideris, M. (2020). Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In. Vivo 34 (3), 1603–1611. doi:10.21873/invivo.11950
Ferrario, L., Maffioli, A., Bondurri, A. A., Guerci, C., Lazzarin, F., and Danelli, P., (2020). COVID-19 and surgical training in Italy: residents and young consultants perspectives from the battlefield. Am. J. Surg. 220 (4), 850–852. doi:10.1016/j.amjsurg.2020.05.036
Fu, S. J., George, E. L., Maggio, P. M., Hawn, M., and Nazerali, R. (2020). The consequences of delaying elective surgery: surgical perspective. Ann. Surg. 272 (2), e79–e80. doi:10.1097/SLA.0000000000003998
Hampp, E. L., Chughtai, M., Scholl, L. Y., Sodhi, N., Stoker, B. M., Jacofsky, D. J., et al. (2019). Robotic-arm assisted total knee arthroplasty demonstrated greater accuracy and precision to plan compared with manual techniques. J. Knee Surg. 32 (3), 239–250. doi:10.1055/s-0038-1641729
Hashtrudi-Zaad, K., and Salcudean, S. E. (2001). Analysis of control architectures for teleoperation systems with impedance/admittance master and slave manipulators. Int. J. Robotics Res. 20 (6), 419–445. doi:10.1177/02783640122067471
Hashtrudi-zaad, K., and Salcudean, S. E. (2002). Transparency in time-delayed systems and the effect of local force feedback for transparent teleoperation. IEEE Trans. Robot. Automat. 18 (1), 108–114. doi:10.1109/70.988981
Heemskerk, J., De Hoog, D. E. N. M., Van Gemert, W. G., Baeten, C. G. M. I., Greve, J. W. M., and Bouvy, N. D. (2007). Robot-assisted vs. conventional laparoscopic rectopexy for rectal prolapse: a comparative study on costs and time. Dis. Colon Rectum 50 (11), 1825–1830. doi:10.1007/s10350-007-9017-2
Higgins, M., Madan, C., and Patel, R. (2020). Development and decay of procedural skills in surgery: a systematic review of the effectiveness of simulated-based medical education interventions. The Surgeon. doi:10.1016/j.surge.2020.07.013
Kimmig, R., Verheijen, R. H. M., and Rudnicki, M. (2020). Robot assisted surgery during the COVID-19 pandemic, especially for gynecological cancer: a statement of the society of european robotic gynaecological surgery (SERGS). J. Gynecol. Oncol. 31 (3), e59. doi:10.3802/jgo.2020.31.e59
Lai, L., Wittbold, K. A., Dadabhoy, F. Z., Sato, R., Landman, A. B., Schwamm, L. H., et al. (2020). Digital triage: novel strategies for population health management in response to the COVID-19 pandemic. Healthcare 8 (4), 100493. doi:10.1016/j.hjdsi.2020.100493
Laniel, S., Létourneau, D., Labbé, M., Grondin, F., Polgar, J., and Michaud, F. (2017). “Adding navigation, artificial audition and vital sign monitoring capabilities to a telepresence mobile robot for remote home care applications,” in IEEE International Conference on Rehabilitation Robotics, London, UK, July 17–20, 2017 (IEEE), 806–811. doi:10.1109/ICORR.2017.8009347
Lee, A. M., Wong, J. G. W. S., McAlonan, G. M., Cheung, V., Cheung, C., Chua, S. E., et al. (2007). Stress and psychological distress among SARS survivors 1 year after the outbreak. Can. J. Psychiatry 52 (4), 233–240. doi:10.1177/070674370705200405
Lin, K., Yang, B. X., Luo, D., Liu, Q., Ma, S., Huang, R., et al. (2020). The mental health effects of COVID-19 on health care providers in china. Am. J. Psychiatry 177 (7), 635–636. doi:10.1176/appi.ajp.2020.20040374
Lindfors, A., Åkesson, Å., Staf, C., Sjöli, P., Sundfeldt, K, and Dahm-Kähler, P. (2018). Robotic vs open surgery for endometrial cancer in elderly patients: surgical outcome, survival, and cost analysis. Int. J. Gynecol. Cancer 28 (4), 692–699. doi:10.1097/IGC.0000000000001240
Marks, J. R. (2020). Sheba showcases future medtech used now for COVID-19. Available at: https://themedialine.org/life-lines/sheba-showcases-future-medtech-used-now-for-covid-19/. (Accessed May 27, 2020).
Mimics (2020). Xperience team TrainerTM. Mimic simulation technology. Available at: https://mimicsimulation.com/xperience/ (Accessed July 8, 2020).
Moawad, G. N., Elkhalil, J., Klebanoff, J. S., Rahman, S., Habib, N., and Alkatout, I. (2020). Augmented realities, artificial intelligence, and machine learning: clinical implications and how technology is shaping the future of medicine. J. Clin. Med. 9 (12), 3811. doi:10.3390/jcm9123811
Moorthy, K., Munz, Y., Dosis, A., Hernandez, J., Martin, S., F. Bello, F., et al. (2004). Dexterity enhancement with robotic surgery. Surg. Endosc. Other Interv. Tech. 18 (5), 790–795. doi:10.1007/s00464-003-8922-2
Mottrie, A. (2020). ERUS (EAU robotic urology section) guidelines during COVID-19 emergency. Available at: https://uroweb.org/wp-content/uploads/ERUS-guidelines-for-COVID-def.pdf. (Accessed May 11, 2020).
Naidu, A. S., Naish, M. D., and Patel, R. V. (2017a). A breakthrough in tumor localization: combining tactile sensing and ultrasound to improve tumor localization in robotics-assisted minimally invasive surgery. IEEE Robot. Automat. Magaz. 24 (2), 54–62. doi:10.1109/mra.2017.2680544
Naidu, A. S., Patel, R. V., and Naish, M. D. (2017b). Low-cost disposable tactile sensors for palpation in minimally invasive surgery. IEEE/ASME Trans. Mechat. 22 (1), 127–137. doi:10.1109/TMECH.2016.2623743
Panzirsch, M., Ryu, J. H., and Ferre, M. (2019). Reducing the conservatism of the time domain passivity approach through consideration of energy reflection in delayed coupled network systems. Mechatronics 58, 58–69. doi:10.1016/j.mechatronics.2018.12.001
Pullella, P. (2020). Tommy the robot nurse helps keep Italy doctors safe from coronavirus. Available at: https://www.usnews.com/news/world/articles/2020-04-01/tommy-the-robot-nurse-helps-keep-italy-doctors-safe-from-coronavirus. (Accessed April 1, 2020).
Reyes, C., Engel‐Nitz, N. M., DaCosta Byfield, S., Ravelo, A., Ogale, S., and Bancroft, T. (2019). Cost of disease progression in patients with metastatic breast, lung, and colorectal cancer. Oncologist 24 (9), 1209–1218. doi:10.1634/theoncologist.2018-0018
Ryu, J. H., Artigas, J., and Preusche, C. (2010). A passive bilateral control scheme for a teleoperator with time-varying communication delay. Mechatronics 20 (7), 812–823. doi:10.1016/j.mechatronics.2010.07.006
Scaravonati, R., Diaz, E., Roche, S., Bertone, S., and Brandi, C. (2020). Strategies for follow up after hernia surgery during COVID 19 pandemia. Int. J. Surg. 79, 103–104. doi:10.1016/j.ijsu.2020.05.051
Schorr, S. B., Quek, Z. F., Romano, R. Y., Nisky, I., Provancher, W. R., and Okamura, A. M. (2013). “Sensory substitution via cutaneous skin stretch feedback,” in IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May 6–10, 2013 (IEEE), 2341–2346. doi:10.1109/ICRA.2013.6630894
Shademan, A., Decker, R. S., Opfermann, J. D., Leonard, S., Krieger, A., and Kim, P. C. W. (2016). Supervised autonomous robotic soft tissue surgery. Sci. Translational Med. 8 (337), 337ra64. doi:10.1126/scitranslmed.aad9398
Shahbazi, M., Atashzar, S. F., and Patel, R. V. (2013). “A dual-user teleoperated system with Virtual Fixtures for robotic surgical training,” in Proceedings - IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, May 6–10, 2013 (IEEE), 3639–3644. doi:10.1109/ICRA.2013.6631088
Shahbazi, M., Atashzar, S. F., Ward, C., Talebi, H. A., and Patel, R. V. (2018b). Multimodal sensorimotor integration for expert-in-the-loop telerobotic surgical training. IEEE Trans. Robot. 34 (6), 1549–1564. doi:10.1109/TRO.2018.2861916
Singh, H., Jafari, A., and Ryu, J. H. (2019). “Enhancing the force transparency of time domain passivity approach: observer-based gradient controller,” in Proceedings - IEEE International Conference on Robotics and Automation, Montreal, QC, 20-24 May 2019 (IEEE), 1583–1589. doi:10.1109/ICRA.2019.8793902
Smith, , and Nephew, n. d. (2020). Using NAVIO in total knee arthroplasty. Available at: https://www.smith-nephew.com/professional/microsites/navio/total-knee-arthroplasty/navio-total-knee/ (Accessed December 10, 2020).
Stryker, n. d. (2020). Mako robotic-arm assisted surgery. Available at: https://www.stryker.com/us/en/portfolios/orthopaedics/joint-replacement/mako-robotic-arm-assisted-surgery.html. (Accessed November 30, 2020).
Szklarski, C. (2020). Canada’s higher COVID-19 death rate tied to better chronic disease control. Available at: https://www.cbc.ca/news/health/covid-19-heart-stroke-1.5652003. (Accessed July 16, 2020).
Tabaza, L., Virk, H. U. H., Janzer, S., and George, J. C. (2020). Robotic-assisted percutaneous coronary intervention in a COVID-19 patient. Catheterization and cardiovascular interventions 97 (3), E343–E345. 10.1002/ccd.28982
Talasaz, A., and Patel, R. V. (2013). Integration of force reflection with tactile sensing for minimally invasive robotics-assisted tumor localization. IEEE Trans. Hapt. 6 (2), 217–228. doi:10.1109/TOH.2012.64
Talasaz, A., Trejos, A. L., Perreault, S., Bassan, H., and Patel, R. V. (2014). A dual-arm 7-degrees-of-freedom haptics-enabled teleoperation test bed for minimally invasive surgery. J. Med. Devices. 8 (4), 041004. doi:10.1115/1.4026984
Talasaz, A., Trejos, A. L., and Patel, R. V. (2017). The role of direct and visual force feedback in suturing using a 7-DOF dual-arm teleoperated system. IEEE Trans. Hapt. 10 (2), 276–287. doi:10.1109/TOH.2016.2616874
Tavakoli, M., Aziminejad, A., Patel, R. V., and Moallem, M. (2006). Tool/tissue interaction feedback modalities in robot-assisted lump localization. Conf Proc IEEE Eng Med Biol Soc 2006, 3854–3857. doi:10.1109/IEMBS.2006.260672
Tavakoli, M., Carriere, J., and Torabi, A. (2020). Robotics, smart wearable technologies, and autonomous intelligent systems for healthcare during the COVID‐19 pandemic: an analysis of the state of the art and future vision. Adv. Intell. Syst. 2 (7), 2000071. doi:10.1002/aisy.202000071
Tergas, A. I., Sheth, S. B., Green, I. C., and Giuntoli, R. L. (2013). A pilot study of surgical training using a virtual robotic surgery simulator. JSLS. 17 (2), 219–226. doi:10.4293/108680813X13654754535872
Trejos, A. L., Jayender, J., Perri, M. T., Naish, M. D., Patel, R. V., and Malthaner, R. A. (2009). Robot-assisted tactile sensing for minimally invasive tumor localization. Int. J. Robotics Res. 28 (9), 1118–1133. doi:10.1177/0278364909101136
Van den Eynde, J., De Groote, S., Van Lerberghe, R., Van den Eynde, R., and Oosterlinck, W. (2020). Cardiothoracic robotic assisted surgery in times of COVID-19. J. Robotic Surg. 14, 795–797. doi:10.1007/s11701-020-01090-7
Wall, J., and Krummel, T. (2020). The digital surgeon: how big data, automation, and artificial intelligence will change surgical practice. J. Pediatr. Surg. 55, 47–50. doi:10.1016/j.jpedsurg.2019.09.008
Wetzel, C. M., Kneebone, R. L., Woloshynowych, M., Nestel, D., Moorthy, K., Kidd, J., et al. (2006). The effects of stress on surgical performance. Am. J. Surg. 191 (1), 5–10. doi:10.1016/j.amjsurg.2005.08.034
Willer, B. L., Thung, A. K., Corridore, M., D’Mello, A. J., Schloss, B. S., Malhotra, P. S., et al. (2020). The otolaryngologist’s and anesthesiologist’s collaborative role in a pandemic: a large quaternary pediatric center’s experience with COVID-19 preparation and simulation. Int. J. Pediatr. Otorhinolaryngol. 136, 110174. doi:10.1016/j.ijporl.2020.110174
Winkler-Schwartz, A., Yilmaz, R., Mirchi, N., Bissonnette, V., et al. (2019). Machine learning identification of surgical and operative factors associated with surgical expertise in virtual reality simulation. JAMA Netw. Open 2 (8), e198363. doi:10.1001/jamanetworkopen.2019.8363
WHO (2020). WHO Timeline - COVID-19. Available at: https://www.who.int/news-room/detail/27-04-2020-who-timeline---covid-19. (Accessed April 27, 2020).
Zafar, A. (2020). The unintended consequences of surgery delays during COVID-19. Available at: https://www.cbc.ca/news/health/covid-surgery-delay-unintended-consequences-1.5629360. (Accessed June 29, 2020).
Keywords: COVID-19, robotics, surgery, teleoperation, tele-examination, tele-training
Citation: Feizi N, Tavakoli M, Patel RV and Atashzar SF (2021) Robotics and AI for Teleoperation, Tele-Assessment, and Tele-Training for Surgery in the Era of COVID-19: Existing Challenges, and Future Vision. Front. Robot. AI 8:610677. doi: 10.3389/frobt.2021.610677
Received: 26 September 2020; Accepted: 18 January 2021;
Published: 14 April 2021.
Edited by:Elena De Momi, Politecnico di Milano, Italy
Reviewed by:Selene Tognarelli, Sant'Anna School of Advanced Studies, Italy
Riccardo Muradore, University of Verona, Italy
Francesca Cordella, Campus Bio-Medico University, Italy
Copyright © 2021 Feizi, Tavakoli, Patel and Atashzar. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: S. Farokh Atashzar, firstname.lastname@example.org