COVID-19 Pandemic Spurs Medical Telerobotic Systems: A Survey of Applications Requiring Physiological Organ Motion Compensation
Lingbo Cheng
Mahdi Tavakoli- 1College of Control Science and Engineering, Zhejiang University, Hangzhou, China
- 2Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
The coronavirus disease 2019 (COVID-19) pandemic has resulted in public health interventions such as physical distancing restrictions to limit the spread and transmission of the novel coronavirus, causing significant effects on the delivery of physical healthcare procedures worldwide. The unprecedented pandemic spurs strong demand for intelligent robotic systems in healthcare. In particular, medical telerobotic systems can play a positive role in the provision of telemedicine to both COVID-19 and non-COVID-19 patients. Different from typical studies on medical teleoperation that consider problems such as time delay and information loss in long-distance communication, this survey addresses the consequences of physiological organ motion when using teleoperation systems to create physical distancing between clinicians and patients in the COVID-19 era. We focus on the control-theoretic approaches that have been developed to address inherent robot control issues associated with organ motion. The state-of-the-art telerobotic systems and their applications in COVID-19 healthcare delivery are reviewed, and possible future directions are outlined.
Introduction
On January 30, 2020, the World Health Organization (WHO) officially declared the coronavirus disease 2019 (COVID-19) outbreak as a public health emergency of international concern (World Health Organization, 2020). Subsequently, the COVID-19 was assessed by WHO as a pandemic. The pandemic resulted in public health interventions to limit the spread and transmission of the novel coronavirus, causing significant effects on the delivery of physical healthcare procedures worldwide. For instance, to slow the spread of disease by stopping chains of transmission of COVID-19 and preventing new ones from appearing, social and physical distancing measures are strongly recommended globally, which resulted in dramatic reductions of in-person visits of patients to clinics or professionals. As this unprecedented crisis is likely to last for a long time and will possibly have multiple waves until a vaccine is available, rapidly seeking and developing a global solution to address this issue (i.e., physical distancing restrictions) will build confidence in delivering healthcare services either remotely or in-person while observing physical distancing.
Intelligent robotic systems, especially telerobotic systems, can play a positive role in this pandemic as they can effectively improve the fully remote or physical distancing-aware healthcare procedures (Tavakoli et al., 2020). Specifically, robotic and telerobotic systems can significantly reduce the risk of infectious disease transmission to frontline healthcare workers by providing a way to triage, evaluate, monitor, and treat patients from a safe distance. Moreover, medical robots have inherent advantages including steady-hand, accuracy, motion scaling, and biomotion compensation, which lead them to be able to provide general support for patients and medical professionals and further alleviate the non-COVID-19 burden placed on healthcare systems during this crisis. In fact, given the growing demands for remote-based healthcare services in the age of COVID-19, a motivation to urgently develop and apply technologies for robotics-assisted surgery has emerged. Inspired by the abovementioned points, this paper focuses on a survey addressing the subject of teleoperation on medical applications.
Teleoperation naturally indicates operating at a distance, which can perfectly meet the requirements of fully remote or in-person distancing-aware healthcare services during the COVID-19 pandemic. Meanwhile, a medical telerobotic system is capable of extending the human capabilities such as the facilitation of motion and/or force scaling, offering advantages in minimally invasive surgery including repeatability, accuracy, dexterity, fine manipulation, etc. In a general single-master/single-slave medical telerobotic system, the human applies a force on the master consoler, which results in movement commands transmitted to the slave manipulator that in turn mimics the human's operations.
Designing a teleoperation system requires addressing many issues concerning sensors and actuators, communication media, time-delay problem, stability, and transparency. Most of the relevant work and surveys focused on teleoperation, which is assumed to include a stable environment on the slave site, aiming for perfect transparency, system stability, and solving time-delay-induced problems (Hokayem and Spong, 2006; Passenberg et al., 2010). However, a survey addressing problems for teleoperation with a moving environment is seldom studied. Differing from the most researched issues such as time-delay, stability, and transparency, the most critical problem for teleoperation with a moving environment is to synchronize the slave robot's motion with the movement of the object (the environment) so that the automatic robotic motion compensations can be deployed instead of manual ones by the human operators.
A typical application of teleoperation in the medical area is manipulating with physiological organ motion caused by cardiac and respiratory activity. The cardiac motion has important local effects on the heart and areas proximal to the heart. Respiratory motion affects the movement for the majority of the body, from the thorax to the abdomen (including heart, lungs, liver, pancreas, and kidneys), and from inside to outside (such as chest and breast) of the body. It has been reported that organ displacements may range from 10 to 40 mm in anterior–posterior, left–right, and superior–inferior directions during normal breathing (Keall et al., 2006). The physiological organ motion has significant effects on medical procedures such as (i) inside or outside moving-organ surgeries (the surgeon has to manually compensate for the organ motion), (ii) moving-organ evaluation (the ability to define accurate target volumes in radiation oncology is difficult), and (iii) image-based diagnosis and monitoring (image quality and quantitative accuracy are highly effected) (Uchinami et al., 2019). When the telerobotic systems (Ballantyne, 2002) are used for the applications requiring automatic organ motion compensation, the current medical performance may need to be improved.
Among many medical telerobotic systems (Avgousti et al., 2016; Evans et al., 2018), da Vinci® robot (Intuitive Surgical Inc.) is currently the most widespread robotic surgical system, which not only can be used for teleoperation over remote distance but also can perform a variety of surgeries, evaluations, diagnosis, and monitoring. Those functionalities involve scaling the operator's actions over a small distance and with a negligible communication delay. Experimentally, the communication delay will be kept within 5–10 ms, and the effects caused by communication delay is trivial and can be negligible. In the short-distance applications, the master console and the slave manipulators are generally placed in the same operating room or different operating rooms in the same clinic; the time-delay problem, therefore, is trivial and negligible.
In this paper, we narrow down the teleoperation systems to short-distance medical telerobotics with applications accompanied by physiological organ motion, and mainly focus on the issue of motion compensation. The aim of this survey is to present the state of the art of the medical telerobotic systems with applications requiring motion compensation and the related control strategies. The rest of the paper is organized as follows. Section Teleoperation for Organ Motion Compensation deals with the control strategies of robotic-assisted systems with an emphasis on the control for automatic motion compensation. Section Applications focuses on clinical applications with telerobotic systems and solutions to effectively deliver healthcare services during the COVID-19 pandemic. Section Discussions and Future Directions discusses the perspectives of future work and concludes the paper.
Teleoperation for Organ Motion Compensation
The mission of advancing medical telerobotic systems is to boost medicine performance by improving patient care, expanding access to high-quality therapy, and enhancing physician education, safety, and efficiency. For medical telerobotic systems with physiological organ motion such as respiratory and heartbeat motion, to minimize the risks of tool-tissue collision and tissue injury, an idea of automatic synchronization of the movement of robotic manipulator's end-tip with the moving organ is proposed. This inspires the development of telerobotic systems to provide compensation for the physiological organ motion to assist the human in performing operations accurately and safely. Indeed, if the robotic system can move a surgical instrument (e.g., catheter, ultrasound probe, forceps) in synchrony with the target tissue while the organ moves, it can provide significant benefits to the surgeon and give him/her a feeling of performing surgery on a stationary organ.
Teleoperation Systems
In a telerobotic system, the master console controls a remote slave robotic manipulator by sending position/velocity commands and receiving potential haptic/visual feedback signals, as well as the information of slave robot status. Teleoperation systems can be divided into three categories with their features: unilateral teleoperation systems, bilateral teleoperation systems, and multilateral teleoperation systems.
In a typical single-master/single-slave teleoperation system, if the slave does not possess a force sensor, which causes the human operator losing the sense of touch, then this system is called a unilateral teleoperation system. In contrast, if the slave possesses force sensors and is able to transmit the force feedback to the master, then this system is called a bilateral teleoperation system. In other words, the human can feel the interaction force between the slave robot and what it is touching, enabling the human to efficiently manipulate the master robot to provide appropriate commands. When a teleoperation system consists of more than one master console and/or slave manipulator and involves more than one sensed and command signal flow between the human operator and the environment, the system is called a multilateral teleoperation system. A multilateral framework not only allows for a one-to-one correspondence between the operator–master and the slave–environment sets but also realizes collaborative scenarios between multiple operator–master sets and/or multiple slave robots.
Physiological Organ Motion
The motion of a moving organ is primarily induced by respiratory and/or heartbeat motions with different frequency ranges. In order to mimic the physiological organ motion in experiments, the studies can be classified into two categories: organ simulators and living organ. By designing mechanical devices, the organ simulators can be controlled to mimic the moving organ's motion based on pre-acquired organ motion data (Yang et al., 2016; Cheng et al., 2019) or biological signals (Cheng and Tavakoli, 2018b). In vivo experiments use living porcine organ (Kesner and Howe, 2014) or dog organ (Mansouri et al., 2018) to demonstrate the control techniques. Specifically, in Yang et al. (2016), a stereo video of in vivo porcine heart, which recorded image sequence of a totally endoscopic coronary artery bypass graft from a da Vinci (Intuitive Surgical, CA) surgical platform, was used to measure the 3D heart positions offline by vision tracking. The quasi-periodic 3D heart motion signals were transmitted to a Motoman SIA-5F (Yaskawa America, Inc., Miamisburg, OH, USA) 7-DOF serial manipulator (Cheng et al., 2019) to control the manipulator to work as a real heart organ.
Motion Compensation Control Techniques
To compensate for the physiological organ motion and synchronize a robot's motion with the organ's motion, various control methods have been proposed for both handheld robotic systems (Yuen et al., 2009; Poulsen et al., 2012; Winter et al., 2015; Kolbitsch et al., 2018; Salehi et al., 2018; Ting et al., 2018) and telerobotic systems (Ginhoux et al., 2005; Gangloff et al., 2006; Cheng et al., 2018). In the paper, we mainly focus on motion compensation control methods for telerobotic systems, which generally falls into four categories: position control, force control, impedance control, and hybrid control.
Position Control
The position-based controllers need the real-time organ position and use that to synchronize the slave's movement with the organ's motion. For a teleoperation system, to further control the slave robot to mimic the human's operation, the summed positions of the master and the moving organ are used as a reference position for the slave robot to follow. A pure position-based telerobotic control system belongs to a unilateral teleoperation system as it provides the human without haptic feedback.
Before discussing robot controllers, a vital issue is to measure the moving organ position in real time. To this end, many image-based sensors have been widely used in research such as high-speed camera/laparoscopy (Nakajima et al., 2014), X-ray fluoroscopy (Ma et al., 2020), computed tomography (CT) (Su et al., 2013), magnetic resonance imaging (MRI) (Yang et al., 2014), positron emission tomography (PET) (Bettinardi et al., 2013), and ultrasound imaging (US) (Bowthorpe and Tavakoli, 2016a,b; Diodato et al., 2018). To get performance, hybrid imaging systems are also developed to measure precise organ motion including MRI/US imaging (Celicanin et al., 2018), MRI/CT imaging (Neumann et al., 2017), PET/CT imaging (Bettinardi et al., 2013; Pepin et al., 2014), and PET/MRI (Kolbitsch et al., 2018). The above-listed measurements have their advantages and limitations, which are elaborated in Table 1.
Table 1. Advantages and disadvantages of medical imaging measurements.
Non-image-based sensors have also been used to collect moving organ motion data such as sonomicrometric sensors (Bebek and Cavusoglu, 2007), optical measurement (Ruszkowski et al., 2016), and electromagnetic tracking system (Loschak et al., 2020).
Once the position of the moving organ is measured, robot controllers aimed for motion compensation can be deployed. Controllers for physiological organ motion compensation can be classified into error feedback controllers, predictive feedforward (prediction-based) controllers, and predictive feedback controllers, as elaborated in the following:
Error feedback controllers directly use the measured position as a reference signal for the medical robot. A proportional–integral–derivative (PID) controller is widely used to continuously calculate an error value (Murphy, 2004). However, the error feedback controller is found to be unable to reduce tracking error sufficiently if used solely.
Prediction-based controllers use the estimated current organ position as the setpoint to move the medical tools. It aims to develop accurate mathematical models of the organ's motion by using one or more previous measured motion dataset. The primary goal is to improve motion tracking performance by developing estimation methods. To this end, Taken's theorem (Ortmaier et al., 2005), artificial neural network (Cheng and Tavakoli, 2019b; Hirai et al., 2019), extended Kalman filter (EKF) (Liang et al., 2014), receding horizon model predictive controller (Bebek and Cavusoglu, 2007), and recursive least squares-based adaptive filter (Tuna et al., 2014) have been investigated in the developments of prediction-based controllers.
Predictive feedback controllers not only need the organ's current position but also take the tracking error into account. By considering the physiological organ motion as periodic disturbances, controllers such as model predictive controllers (MPCs) (Gangloff et al., 2006; Vrooijink et al., 2017), Smith predictor-based controllers (Bowthorpe et al., 2013; Bowthorpe and Tavakoli, 2016a), generalized predictive controllers (GPCs) (Bowthorpe and Tavakoli, 2016b), and repetitive-GPC (R-GPC) (Ginhoux et al., 2005) were used. As these methods rely on the known organ motion model, the robustness of the system to irregular organ motion is challenging.
Force Control
For applications that require tool-tissue contact such as ablation and biopsy with controlled depth, a significant breakthrough in medical telerobotics is facilitated by force-reflecting haptic feedback, which allows the human to perceive the forces applied by the slave robot on the environment (a moving target). Force feedback (haptic feedback) increases the transparency of the teleoperation, which enhances human operator's immersive maneuver on the master consoler. Such function, as mentioned earlier in the paper, requires sensory feedback information. In other words, a force/torque sensor should be mounted on the slave side to measure and transmit slave–environment interaction forces. To simultaneously compensate for the physiological organ motion, various force-based control methods are proposed. The control goal is to keep the slave–environment interaction forces at a constant value so that the human can have a haptic feeling that the environment is motionless through force feedback.
Moreira et al. (2014) proposed a force control scheme using an active observer (AOB) based on a viscoelastic interaction model (soft tissue model) to compensate for the physiological motion. The model-based force control used the AOB to estimate the system states and an extra state, which is employed to compensate for system disturbances and modeling errors. Dominici and Cortesão achieved motion compensation by designing a cascade MPC architecture with a Kalman AOB (Dominici and Cortesão, 2014). The AOB inner loop provides stable closed-loop dynamics, and the MPC outer loop generates reference forces for AOB control for autonomous motion compensation. The authors further proposed another force control scheme by using a double AOB architecture (Cortesão and Dominici, 2017). In the work of Mohareri et al. (2014), the authors developed an asymmetric force feedback control system for bimanual telerobotic surgery using the da Vinci surgical system. To avoid instability issues caused by the closed-loop system, the authors proposed to use one hand to exert force through the master robot and use the other hand to perceive force feedback from the slave robot. He et al. (2019) proposed a neural network-based force control scheme to compensate for the eyeball motion in retinal surgery, in which the tool–eyeball interactive force is fed into the neural network and the latter is trained to command the robot manipulator to move according to the eyeball movements.
Impedance Control
Different from the position control and the force control, which are utilized to control position or force variables separately, impedance control is a compliant control, which is employed to achieve desirable dynamic interaction between a robot manipulator and its environment. In other words, impedance control can control the dynamic relationship between robot motion and robot–environment interaction force as desired. For a robotic manipulator aiming to compensate for the organ's motion, the robot and the moving organ can be expressed as impedance and admittance, respectively (Hogan, 1984, 1985). The goal of impedance control is to regulate the dynamic relationship to achieve the requirements of automatically compensating for the organ motion while keeping the interaction force in a safe range.
Florez et al. (2012) proposed a method that uses an impedance control on a handheld robotic instrument to compensate for physiological motion. The handheld system allows the human to perform low-frequency motions that correspond to the task. At the same time, the part of the instrument contacting the moving organ actively moves in synchronism with the organ's motion to keep a constant contact. Zarrouk et al. (2010) proposed an adaptive control architecture based on model reference adaptive control to solve the 3D physiological motion compensation in beating-heart surgery. A reference impedance model and an adaptive controller were designed for the surgical robot. The aforementioned impedance-controlled systems are developed for handheld medical robotics instead of teleoperated systems. In the work of Cheng (Cheng and Tavakoli, 2018a; Cheng et al., 2018, 2019) and Sharifi et al. (2018), the model reference adaptive control was applied to the bilateral teleoperation systems separately. The authors designed two reference impedance models for the master and slave robots, respectively. The slave reference impedance model was used to make the slave robot compensate for the living organ's motion, while the master reference impedance model has the ability to ensure the human to perceive non-oscillatory robot–organ interaction force. The oscillatory haptic feedback caused by oscillatory motion and force sensor inertia is filtered out by the master reference impedance model.
Hybrid Control
Hybrid control combines two or more control schemes together, including hybrid position/force control, hybrid position/impedance control, etc. The goal of hybrid control is mainly to develop a compliant control scheme to achieve specific task requirements. In Yuen et al. (2010) and Kesner and Howe (2014), the authors separately incorporated position control and force control to achieve beating-heart motion compensation. These methods combined the US guidance with a force controller and are aimed to incorporate a feedforward term that contains the estimated motion of a beating heart. The US was used to measure the position of the moving organ, while the force controller was utilized to extend the device application from free motion to constrained contact motion. Nakajima et al. (2014) used visual servoing to compensate for the organ motion and performed haptic feedback using an acceleration-based bilateral control method. System stability was evaluated through frequency characteristics and root locus. In Cheng and Tavakoli (2018b), an impedance control combined with an US image-guided position control was developed in a teleoperation system. The US scanner estimated the moving organ position and transmitted it along with the master robot position to the slave robot as a position reference. For non-oscillatory haptic feedback, a reference impedance model was designed for the master robot to provide the human with a steady slave–organ interaction force.
Applications
Significant interest has been documented for both interventional (e.g., therapeutic treatments such as surgery and protontherapy) and diagnostic (e.g., US scan, X-ray scan, and biopsy) applications (Figure 1).
Figure 1. Scheme of the control methods for physiological motion compensation and the potential applications.
The corresponding telerobotic systems can significantly reduce the risk of infectious disease transmission to frontline healthcare workers by making it possible to evaluate, monitor, and treat patients from a safe distance. Moreover, the teleoperation techniques are able to provide general support for patients and medical professionals, alleviating the non-COVID-19 burden placed on healthcare systems during this crisis. The latter, i.e., secondary prevention and disease management of non-COVID-19 individuals who need therapeutic treatments or diagnosis during this difficult time, is of equal importance. Telerobotic and autonomous systems can support healthcare staff such as physiotherapists and surgeons during the COVID-19 pandemic through facilitating fully remote or in-person distancing-aware physical treatments and diagnosis services.
Teleoperated Intervention
Telesurgery
Telerobotics applications mostly involve articulated robot configurations with an interchangeable surgical tool that is mounted on the end effector of the slave robot (surgical robot). Robot systems have been developed from the first functional telesurgery system—ZEUS—to the da Vinci surgical system; the latter is currently the only commercially available surgical robotic system. However, telerobotics for applications with physiological organ motion are mostly in the domain of research yet.
Most of the proposed systems are application-specific medical telerobots, such as the telerobots used for beating-heart surgery (mitral valve prolapses and repair, atrial septal defect, atrial fibrillation) and percutaneous nephrolithotomy surgery (kidney stones, kidney cysts, kidney blockage). Compared to conventional surgery operations, the surgeries assisted by telerobotic systems requiring organ motion have significant advantages. First, the master–slave system enables remote or physical distancing-aware surgical procedures during the COVID-19 pandemic. Second, automatic compensation for complex physiological organ motion greatly reduces the difficulty of operation for surgeons and increases surgical accuracy, which turns to improve patient safety. Third, advanced technique introduces minimally invasive robotic surgery, which can be used for the surgeries mentioned above with benefits including small incisions, little pain, low risk of infection, short recovery time, and reduced blood loss. Last but not least, specifically for beating-heart surgery, robotic surgery has been found to have additional advantages over the conventional arrested-heart surgery; the latter has to employ a heart–lung bypass machine (Angelini et al., 2002).
Teleradiotherapy
When a tumor locates close to the vital organs (heart, lung, etc.), radiation therapy is generally recommended as a useful treatment to destroy cancer cells and slow tumor growth without harming nearby healthy tissue. The goal of radiation therapy is to assess the true volume of the tumor and its real motion and to obtain an accurate target delineation and an accurate and personalized definition of the treatment plan (Khan and Gibbons, 2014). As the breathing-induced motion has significant effects on organs (e.g., liver, lung, breast, kidney, prostate, and pancreas), radiation therapy, accurately and automatically compensating for continuous physiological respiratory motion of organs, is necessary. Indeed, if not correctly compensated, organ motion can lead to a spreading of the thermal dose, which is the cause of two severe issues: (i) loss of treatment efficiency, and (ii) generation of unplanned lesions in adjacent healthy tissues. Moreover, teleradiotherapy will be useful to get rid of the side effects of radiation therapy to the physicians.
Telediagnosis
Tele-Echography
Ultrasound is an imaging modality that plays a significant role in medical emergency and surgical decision diagnosis. To compensate for the limited availability of ultrasound experts in isolated areas (such as physical distancing-aware caused by COVID-19 pandemic), the use of robotic telemedicine systems is gaining attention. A commercial MELODY tele-ultrasound robotized system was developed by AdEchotech SME (France) (Vieyres et al., 2013) for long-distance US diagnosis. The slave robot is attached to an US probe through a probe holder. The human at the master site moves a fictive US probe as required for an echographic diagnosis. The MELODY system was designed to fulfill remote static organ diagnosis without considering issues such as moving organ motion compensation. Sharifi et al. (2017) developed a bilateral telerobotic system for echography in beating-heart surgery. Although it is just a proof of concept, the idea of the control scheme is worth to be considered for future commercial popularizing.
Teleradiography
Teleradiography allows radiologists or physicians to provide services without physically being at the location of the patient. Similar to tele-echography, by mounting the CT scanner or X-ray holder on the slave robot, the radiologists or physicians can remotely diagnose the patient's body without being exposed to radiation. Most importantly, robot-assisted organ motion compensation will be a benefit for accurate imaging and preventing the over-radiation of the patients.
Training and Education
The wide applicability of teleoperated interventions and telediagnosis will depend on not only the maturity of the technology but also the skill level of trained physicians. These applications require specialized skills compared to traditional methods. Moreover, it is essential that medical schools are equipped with such technologies to appropriately train physicians. Existing possibilities include the use of multilateral teleoperation systems with a multiple control console configuration to enable training or collaborative medical applications (Shahbazi et al., 2018; Cheng and Tavakoli, 2019a).
Discussions and Future Directions
Influenced by the COVID-19 pandemic, the presented review focuses on a potential solution for remote and physical distancing-aware healthcare delivery—medical telerobotics. The review studies the medical telerobotics for applications with physiological organ motion, and discusses control schemes for motion compensation, potential applications, and associated benefits. The medical telerobotics have been already employed in a wide range of diagnostic and interventional applications in different medical disciplines. To successfully apply medical telerobotic technologies to clinical practice, a significant issue is to develop appropriate control schemes for the specific application.
Solutions only involving visual servoing (image-based position control) are found to have several limitations: (i) artificial and natural landmarks occlusion will affect the measurements of the landmark-based sensors, (ii) tissue deformation during contact tasks will affect organ position measurement, and (iii) physiological motion induces oscillatory force feedback and will affect human's performance. To deal with those issues, a latest research proposed a novel printing procedure to fabricate an electrical-impedance-tomography strain sensor on an ex-vivo breathing lung. The authors integrate a visual sensing system with a 3D printer to track the time-varying 3D geometry of the lung (Zhu et al., 2020). The method presented in the abovementioned research could aid modern medical treatments in myriad ways, such as printing electrode arrays for neural interfaces and printing bioscaffolds with engineered cells for tissue regeneration.
Another issue of position control is that when a position controller is used for contact tasks, the contact constraints will be treated as a disturbance resulting in increasing position tracking error, which probably leads to excessive interaction force. Therefore, a position control scheme in position tracking task works better in free motion than constrained motion. Applications such as biopsy and percutaneous puncture are more suitable to be performed by a position-based telerobotic control scheme.
In clinical practice, most medical interventions require direct interaction between one or more tools with the patient's organs. To assure the patient's safety and provide the surgeon with a comprehensive perception, force feedback is necessary for medical telerobotics during robotized interventions. In addition to the limitations of sensors and systems available for force feedback teleoperations, the force control scheme has its drawbacks as well. As the goal of pure force control is to keep the contact force as the reference without position limitations, it leads to force control working in constrained motion.
In fact, both position control and force control can be treated as extreme situations of impedance control. Specifically, the position controller has infinite impedance, while the force controller has zero impedance. Those controllers may be appropriate for applications in which the work exchanged between the robot and its environment is negligible. For applications where power exchange cannot be ignored, hybrid position/force interaction control or impedance control can be used to provide techniques to accommodate the side effects.
Medical telerobotics for applications requiring physiological organ motion have been developed considerably for the last 20 years, and they will be necessarily developed much further in the coming years, especially in the field of surgery and diagnosis. However, further efforts are required to address both clinical and technological challenges.
An existing difficulty for the adoption of medical telerobotic technologies is to deliver accuracy and precision medical procedures, which require particular effort to overcome. For instance, precise interaction and force applied on the tissue by the robotic instrument, accurate dose delivery to the patients, and limited radiation exposure of the patients should be strictly controlled according to specific medical practice. Considering and regulating applicable requirements and specifications for medical procedures and devices will be a benefit for bridging the gap between engineering and medicine. Both patients' and physicians' safety are always the priority in clinical practice. Therefore, when adopting a medical device in clinical practice, the stability of the system and the robustness and reliability to an unforeseeable emergency such as irregular organ motion should be of great concern.
Another significant issue that limits the spread of telerobotic system in clinical practice is the high cost of the medical devices. As the system development requires interdisciplinary knowledge including medicine, engineering, computer science, and mathematics, the challenges and costs are doubtless high. As a result, mature technology and standard requirements of specifications would be beneficial.
Ultimately, medical telerobotics is a promising technology, which has significant advantages for healthcare delivery and can play a positive role in the COVID-19 pandemic as it can effectively improve the remote or physical distancing-aware healthcare procedures. The present review study of teleoperation for medical applications requiring physiological motion shows that various control methods have been proposed for specific applications. As a result, preliminary research toward this direction has already been achieved, but the deep potential of medical telerobotics for applications requiring organ motion remains largely unexploited.
Author Contributions
LC completed the work including literature review and preparation for the initial draft of the manuscript. MT provided guidance and valuable suggestions/discussions and was involved in the editing of the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This work was supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada under grants RGPIN-2019-04662 and RGPAS-2019-00106.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Angelini, G. D., Taylor, F. C., Reeves, B. C., and Ascione, R. (2002). Early and midterm outcome after off-pump and on-pump surgery in Beating Heart Against Cardioplegic Arrest Studies (BHACAS 1 and 2): a pooled analysis of two randomised controlled trials. Lancet 359, 1194–1199. doi: 10.1016/S0140-6736(02)08216-8
Avgousti, S., Christoforou, E. G., Panayides, A. S., Voskarides, S., Novales, C., Nouaille, L., et al. (2016). Medical telerobotic systems: current status and future trends. Biomed. Eng. Online 15:96. doi: 10.1186/s12938-016-0217-7
Ballantyne, G. H. (2002). Robotic surgery, telerobotic surgery, telepresence, and telementoring. Surg. Endosc. Other Intervent. Techn. 16, 1389–1402. doi: 10.1007/s00464-001-8283-7
Bebek, O., and Cavusoglu, M. C. (2007). Intelligent control algorithms for robotic-assisted beating heart surgery. IEEE Trans. Rob. 23, 468–480. doi: 10.1109/TRO.2007.895077
Bettinardi, V., De Bernardi, E., Presotto, L., and Gilardi, M. C. (2013). Motion-tracking hardware and advanced applications in PET and PET/CT. PET Clin. 8, 11–28. doi: 10.1016/j.cpet.2012.09.008
Bowthorpe, M., and Tavakoli, M. (2016a). Physiological organ motion prediction and compensation based on multirate, delayed, and unregistered measurements in robot-assisted surgery and therapy. IEEE/ASME Trans. Mech. 21, 900–911. doi: 10.1109/TMECH.2015.2482391
Bowthorpe, M., and Tavakoli, M. (2016b). Generalized predictive control of a surgical robot for beating-heart surgery under delayed and slowly-sampled ultrasound image data. IEEE Robot. Autom. Lett. 1, 892–899. doi: 10.1109/LRA.2016.2530859
Bowthorpe, M., Tavakoli, M., Becher, H., and Howe, R. (2013). Smith predictor-based robot control for ultrasound-guided teleoperated beating-heart surgery. IEEE J. Biomed. Health Inf. 18, 157–166. doi: 10.1109/JBHI.2013.2267494
Celicanin, Z., Manasseh, G., Petrusca, L., Scheffler, K., Auboiroux, V., Crowe, L. A., et al. (2018). Hybrid ultrasound-MR guided HIFU treatment method with 3D motion compensation. Magn. Reson. Med. 79, 2511–2523. doi: 10.1002/mrm.26897
Cheng, L., Fong, J., and Tavakoli, M. (2019). “Semi-autonomous surgical robot control for beating-heart surgery,” in IEEE 15th International Conference on Automation Science and Engineering (Vancouver, BC), 1774–1781. doi: 10.1109/COASE.2019.8843275
Cheng, L., Sharifi, M., and Tavakoli, M. (2018). Towards robot-assisted anchor deployment in beating-heart mitral valve surgery. Int. J. Med. Rob. Comput. Assist. Surg. 14:e1900. doi: 10.1002/rcs.1900
Cheng, L., and Tavakoli, M. (2018a). Switched-impedance control of surgical robots in teleoperated beating-heart surgery. J. Med. Rob. Res. 3:1841003. doi: 10.1142/S2424905X18410039
Cheng, L., and Tavakoli, M. (2018b). Ultrasound image guidance and robot impedance control for beating-heart surgery. Control Eng. Pract. 81, 9–17. doi: 10.1016/j.conengprac.2018.08.017
Cheng, L., and Tavakoli, M. (2019a). A multilateral impedance-controlled system for haptics-enabled surgical training and cooperation in beating-heart surgery. Int. J. Intell. Robot. Appl. 3, 314–325. doi: 10.1007/s41315-019-00099-y
Cheng, L., and Tavakoli, M. (2019b). “Neural-network-based heart motion prediction for ultrasound-guided beating-heart surgery,” in IEEE 15th International Conference on Automation Science and Engineering (Vancouver, BC), 437–442. doi: 10.1109/COASE.2019.8843136
Cortesão, R., and Dominici, M. (2017). Robot force control on a beating heart. IEEE/ASME Trans. Mech. 22, 1736–1743. doi: 10.1109/TMECH.2017.2696259
Diodato, A., Cafarelli, A., Schiappacasse, A., Tognarelli, S., Ciuti, G., and Menciassi, A. (2018). Motion compensation with skin contact control for high intensity focused ultrasound surgery in moving organs. Phys. Med. Biol. 63:035017. doi: 10.1088/1361-6560/aa9c22
Dominici, M., and Cortesão, R. (2014). “Cascade robot force control architecture for autonomous beating heart motion compensation with model predictive control and active observer,” in IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (São Paulo), 745–751. doi: 10.1109/BIOROB.2014.6913867
Evans, C. R., Medina, M. G., and Dwyer, A. M. (2018). Telemedicine and telerobotics: from science fiction to reality. Updat. Surg. 70, 357–362. doi: 10.1007/s13304-018-0574-9
Florez, J. M., Szewczyk, J., and Morel, G. (2012). “An impedance control strategy for a handheld instrument to compensate for physiological motion,” in IEEE International Conference on Robotics and Automation (Saint Paul, MN), 1952–1957. doi: 10.1109/ICRA.2012.6225103
Gangloff, J., Ginhoux, R., de Mathelin, M., Soler, L., and Marescaux, J. (2006). Model predictive control for compensation of cyclic organ motions in teleoperated laparoscopic surgery. IEEE Trans. Control Syst. Technol. 14, 235–246. doi: 10.1109/TCST.2005.863650
Ginhoux, R., Gangloff, J., de Mathelin, M., Soler, L., Sanchez, M. M. A., and Marescaux, J. (2005). Active filtering of physiological motion in robotized surgery using predictive control. IEEE Trans. Rob. 21, 67–79. doi: 10.1109/TRO.2004.833812
He, C., Patel, N., Shahbazi, M., Yang, Y., Gehlbach, P., Kobilarov, M., et al. (2019). Toward safe retinal microsurgery: development and evaluation of an RNN-based active interventional control framework. IEEE Trans. Biomed. Eng. 67, 966–977. doi: 10.1109/TBME.2019.2926060
Hirai, R., Sakata, Y., Tanizawa, A., and Mori, S. (2019). Real-time tumor tracking using fluoroscopic imaging with deep neural network analysis. Phys. Med. 59, 22–29. doi: 10.1016/j.ejmp.2019.02.006
Hogan, N. (1984). “Impedance control: an approach to manipulation: Part I—theory,” in 1984 American control conference (San Diego, CA), 304–313. doi: 10.23919/ACC.1984.4788393
Hogan, N. (1985). Impedance control: an approach to manipulation: Part II—implementation. J. Dyn. Syst. Meas. Control 107, 8–16. doi: 10.1115/1.3140713
Hokayem, P. F., and Spong, M. W. (2006). Bilateral teleoperation: an historical survey. Automatica 42, 2035–2057. doi: 10.1016/j.automatica.2006.06.027
Keall, P. J., Mageras, G. S., Balter, J. M., Emery, R. S., Forster, K. M., Jiang, S. B., et al. (2006). The management of respiratory motion in radiation oncology report of AAPM Task Group 76 a. Med. Phys. 33, 3874–3900. doi: 10.1118/1.2349696
Kesner, S. B., and Howe, R. D. (2014). Robotic catheter cardiac ablation combining ultrasound guidance and force control. Int. J. Rob. Res. 33, 631–644. doi: 10.1177/0278364913511350
Khan, F. M., and Gibbons, J. P. (2014). Khan's the Physics of Radiation Therapy. Philadelphia, PA: Lippincott Williams and Wilkins.
Kolbitsch, C., Neji, R., Fenchel, M., Schuh, A., Mallia, A., Marsden, P., et al. (2018). Joint cardiac and respiratory motion estimation for motion-corrected cardiac PET-MR. Phys. Med. Biol. 64:015007. doi: 10.1088/1361-6560/aaf246
Liang, F., Yu, Y., Cui, S., Zhao, L., and Wu, X. (2014). Heart motion uncertainty compensation prediction method for robot assisted beating heart surgery-master-slave Kalman filters approach. J. Med. Syst. 38:52. doi: 10.1007/s10916-014-0052-y
Loschak, P. M., Degirmenci, A., Tschabrunn, C. M., Anter, E., and Howe, R. D. (2020). Automatically steering cardiac catheters in vivo with respiratory motion compensation. Int. J. Rob. Res. 39, 586–597. doi: 10.1177/0278364920903785
Ma, H., Smal, I., Daemen, J., and van Walsum, T. (2020). Dynamic coronary road mapping via catheter tip tracking in X-ray fluoroscopy with deep learning based Bayesian filtering. Med. Image Anal. 61:101634. doi: 10.1016/j.media.2020.101634
Mansouri, S., Farahmand, F., Vossoughi, G., and Ghavidel, A. A. (2018). A hybrid algorithm for prediction of varying heart rate motion in computer-assisted beating heart surgery. J. Med. Syst. 42:200. doi: 10.1007/s10916-018-1059-6
Mohareri, O., Schneider, C., and Salcudean, S. (2014). “Bimanual telerobotic surgery with asymmetric force feedback: a daVinci® surgical system implementation” in IEEE/RSJ International Conference on Intelligent Robots and Systems (Chicago, IL), 4272–4277. doi: 10.1109/IROS.2014.6943165
Moreira, P., Zemiti, N., Liu, C., and Poignet, P. (2014). Viscoelastic model based force control for soft tissue interaction and its application in physiological motion compensation. Comput. Methods Programs Biomed. 116, 52–67. doi: 10.1016/j.cmpb.2014.01.017
Murphy, M. J. (2004). Tracking moving organs in real time. Semin. Radiat. Oncol. 14, 91–100. doi: 10.1053/j.semradonc.2003.10.005
Nakajima, Y., Nozaki, T., and Ohnishi, K. (2014). Heartbeat synchronization with haptic feedback for telesurgical robot. IEEE Trans. Ind. Electron. 61, 3753–3764. doi: 10.1109/TIE.2013.2287258
Neumann, W., Lietzmann, F., Schad, L. R., and Zöllner, F. G. (2017). Design of a multimodal (1H/23Na MR/CT) anthropomorphic thorax phantom. Z. Med. Phys. 27, 124–131. doi: 10.1016/j.zemedi.2016.07.004
Ortmaier, T., Groger, M., Boehm, D. H., Falk, V., and Hirzinger, G. (2005). Motion estimation in beating heart surgery. IEEE Trans. Biomed. Eng. 52, 1729–1740. doi: 10.1109/TBME.2005.855716
Passenberg, C., Peer, A., and Buss, M. (2010). A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics 20, 787–801. doi: 10.1016/j.mechatronics.2010.04.005
Pepin, A., Daouk, J., Bailly, P., Hapdey, S., and Meyer, M. E. (2014). Management of respiratory motion in PET/computed tomography: the state of the art. Nucl. Med. Commun. 35:113. doi: 10.1097/MNM.0000000000000048
Poulsen, P. R., Fledelius, W., Cho, B., and Keall, P. (2012). Image-based dynamic multileaf collimator tracking of moving targets during intensity-modulated arc therapy. Int. J. Radiat. Oncol. Biol. Phys. 83, e265–e271. doi: 10.1016/j.ijrobp.2011.12.053
Ruszkowski, A., Schneider, C., Mohareri, O., and Salcudean, S. (2016). “Bimanual teleoperation with heart motion compensation on the da Vinci® Research Kit: implementation and preliminary experiments.” in IEEE International Conference on Robotics and Automation (Stockholm), 4101–4108. doi: 10.1109/ICRA.2016.7487601
Salehi, N., Rahmim, A., Fatemizadeh, E., Akbarzadeh, A., Farahani, M. H., Farzanefar, S., et al. (2018). Cardiac contraction motion compensation in gated myocardial perfusion SPECT: a comparative study. Phys. Med. 49, 77–82. doi: 10.1016/j.ejmp.2018.05.004
Shahbazi, M., Atashzar, S. F., Ward, C., Talebi, H. A., and Patel, R. V. (2018). Multimodal sensorimotor integration for expert-in-the-loop telerobotic surgical training. IEEE Trans. Rob. 34, 1549–1564. doi: 10.1109/TRO.2018.2861916
Sharifi, M., Salarieh, H., Behzadipour, S., and Tavakoli, M. (2017). Tele-echography of moving organs using an Impedance-controlled telerobotic system. Mechatronics 45, 60–70. doi: 10.1016/j.mechatronics.2017.05.006
Sharifi, M., Salarieh, H., Behzadipour, S., and Tavakoli, M. (2018). Beating-heart robotic surgery using bilateral impedance control: theory and experiments. Biomed. Signal Process. Control 45, 256–266. doi: 10.1016/j.bspc.2018.05.015
Su, P., Yang, J., Lu, K., Yu, N., Wong, S. T., and Xue, Z. (2013). A fast CT and CT-fluoroscopy registration algorithm with respiratory motion compensation for image-guided lung intervention. IEEE Trans. Biomed. Eng. 60, 2034–2041. doi: 10.1109/TBME.2013.2245895
Tavakoli, M., Carriere, J., and Torabi, A. (2020). Robotics, smart wearable technologies, and autonomous intelligent systems for healthcare during the COVID-19 pandemic: an analysis of the state of the art and future vision. Adv. Intell. Syst. 2:2000071. doi: 10.1002/aisy.202000071
Ting, L. L., Chuang, H. C., Liao, A. H., Kuo, C. C., Yu, H. W., Zhou, Y. L., et al. (2018). Experimental king technique in radiation therapy. Phys. Med. 49, 11–18. doi: 10.1016/j.ejmp.2018.04.393
Tuna, E. E., Karimov, J. H., Liu, T., Bebek, Ö., Fukamachi, K., and Çavuşoglu, M. C. (2014). Towards active tracking of beating heart motion in the presence of arrhythmia for robotic assisted beating heart surgery. PloS ONE 9:e102877. doi: 10.1371/journal.pone.0102877
Uchinami, Y., Suzuki, R., Katoh, N., Taguchi, H., Yasuda, K., Miyamoto, N., et al. (2019). Impact of organ motion on volumetric and dosimetric parameters in stomach lymphomas treated with intensity-modulated radiotherapy. J. Appl. Clin. Med. Phys. 20, 78–86. doi: 10.1002/acm2.12681
Vieyres, P., Novales, C., Rivas, R., Vilcahuaman, L., Sandoval, J., Clark, T., et al. (2013). “The next challenge for world wide robotized tele-echography experiment (WORTEX 2012): from engineering success to healthcare delivery,” in Congreso Peruano De Ingeniería Biomédica, Bioingeniería, Biotecnología y Física Médica (TUMI II) (Lima Peru).
Vrooijink, G. J., Denasi, A., Grandjean, J. G., and Misra, S. (2017). Model predictive control of a robotically actuated delivery sheath for beating heart compensation. Int. J. Rob. Res. 36, 193–209. doi: 10.1177/0278364917691113
Winter, J. D., Wong, R., Swaminath, A., and Chow, T. (2015). Accuracy of robotic radiosurgical liver treatment throughout the respiratory cycle. Int. J. Radiat. Oncol. Biol. Phys. 93, 916–924. doi: 10.1016/j.ijrobp.2015.08.031
World Health Organization (2020). WHO Coronavirus Disease (COVID-19) Dashboard. Available online at: https://covid19.who.int (accessed November 2, 2020).
Yang, B., Liu, C., Zheng, W., and Liu, S. (2016). Motion prediction via online instantaneous frequency estimation for vision-based beating heart tracking. Inf. Fusion 35, 58–67. doi: 10.1016/j.inffus.2016.09.004
Yang, B., Roys, S., Tan, U. X., Philip, M., Richard, H., Gullapalli, R., et al. (2014). Design, development, and evaluation of a master-slave surgical system for breast biopsy under continuous MRI. Int. J. Rob. Res. 33, 616–630. doi: 10.1177/0278364913500365
Yuen, S. G., Kettler, D. T., Novotny, P. M., Plowes, R. D., and Howe, R. D. (2009). Robotic motion compensation for beating heart intracardiac surgery. Int. J. Rob. Res. 28, 1355–1372. doi: 10.1177/0278364909104065
Yuen, S. G., Perrin, D. P., Vasilyev, N. V., Pedro, J., and Howe, R. D. (2010). Force tracking with feedforward motion estimation for beating heart surgery. IEEE Trans. Rob. 26, 888–896. doi: 10.1109/TRO.2010.2053734
Zarrouk, Z., Chemori, A., and Poignet, P. (2010). “Adaptive force feedback control for 3d compensation of physiological motion in beating heart surgery,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (Taipei), 1856–1861. doi: 10.1109/IROS.2010.5650684
Keywords: COVID-19, healthcare, physical distancing, teleoperation, telerobotics, telemedicine, motion compensation, robot control
Citation: Cheng L and Tavakoli M (2020) COVID-19 Pandemic Spurs Medical Telerobotic Systems: A Survey of Applications Requiring Physiological Organ Motion Compensation. Front. Robot. AI 7:594673. doi: 10.3389/frobt.2020.594673
Received: 14 August 2020; Accepted: 14 October 2020;
Published: 09 November 2020.
Edited by:
Bin Fang, Tsinghua University, ChinaReviewed by:
Haiyuan Li, Beijing University of Posts and Telecommunications (BUPT), ChinaZhen Deng, University of Hamburg, Germany
Copyright © 2020 Cheng and Tavakoli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Mahdi Tavakoli, mahdi.tavakoli@ualberta.ca