Model-free control for autonomous prevention of adverse events in robotics

Introduction: Preventive control is a critical feature in autonomous technology to ensure safe system operations. One application where safety is most important is robot-assisted needle interventions. During incisions into a tissue, adverse events such as mechanical buckling of the needle shaft and tissue displacements can occur on encounter with stiff membranes causing potential damage to the organ. Methods: To prevent these events before they occur, we propose a new control subroutine that autonomously chooses a) a reactive mechanism to stop the insertion procedure when a needle buckling or a severe tissue displacement event is predicted and b) an adaptive mechanism to continue the insertion procedure through needle steering control when a mild tissue displacement is detected. The subroutine is developed using a model-free control technique due to the nonlinearities of the unknown needle-tissue dynamics. First, an improved version of the model-free adaptive control (IMFAC) is developed by computing a fast time-varying partial pseudo derivative analytically from the dynamic linearization equation to enhance output convergence and robustness against external disturbances. Results and Discussion: Comparing IMFAC and MFAC algorithms on simulated nonlinear systems in MATLAB, IMFAC shows 20% faster output convergence against arbitrary disturbances. Next, IMFAC is integrated with event prediction algorithms from prior work to prevent adverse events during needle insertions in real time. Needle insertions in gelatin tissues with known environments show successful prevention of needle buckling and tissue displacement events. Needle insertions in biological tissues with unknown environments are performed using live fluoroscopic imaging as ground truth to verify timely prevention of adverse events. Finally, statistical ANOVA analysis on all insertion data shows the robustness of the prevention algorithm to various needles and tissue environments. Overall, the success rate of preventing adverse events in needle insertions through adaptive and reactive control was 95%, which is important toward achieving safety in robotic needle interventions.

Introduction: Preventive control is a critical feature in autonomous technology to ensure safe system operations.One application where safety is most important is robot-assisted needle interventions.During incisions into a tissue, adverse events such as mechanical buckling of the needle shaft and tissue displacements can occur on encounter with stiff membranes causing potential damage to the organ.
Methods: To prevent these events before they occur, we propose a new control subroutine that autonomously chooses a) a reactive mechanism to stop the insertion procedure when a needle buckling or a severe tissue displacement event is predicted and b) an adaptive mechanism to continue the insertion procedure through needle steering control when a mild tissue displacement is detected.The subroutine is developed using a model-free control technique due to the nonlinearities of the unknown needle-tissue dynamics.First, an improved version of the model-free adaptive control (IMFAC) is developed by computing a fast time-varying partial pseudo derivative analytically from the dynamic linearization equation to enhance output convergence and robustness against external disturbances.
Results and Discussion: Comparing IMFAC and MFAC algorithms on simulated nonlinear systems in MATLAB, IMFAC shows 20% faster output convergence against arbitrary disturbances.Next, IMFAC is integrated with event prediction algorithms from prior work to prevent adverse events during needle insertions in real time.Needle insertions in gelatin tissues with known environments show successful prevention of needle buckling and tissue displacement events.Needle insertions in biological tissues with unknown environments are performed using live fluoroscopic imaging as ground truth to verify timely prevention of adverse events.Finally, statistical ANOVA analysis on all insertion data shows the robustness of the prevention algorithm to various needles and tissue environments.Overall, the success rate of preventing adverse events in needle insertions through adaptive and reactive control was 95%, which is important toward achieving safety in robotic needle interventions.

Introduction
Autonomous robots can perform complex tasks without human intervention using sensing, estimation, and control methodologies with the goal of improving the quality, efficiency, and safety of a system.These systems have vast applications in healthcare, commercial industries, transportation, aerospace, and defense.However, achieving human-like capabilities of dealing with high unpredictability, rapidly changing environments, and predicting adverse scenarios in a timely and safe fashion has been somewhat elusive for fully autonomous systems.Therefore, developing a combined approach of predictive control with adverse event prevention models, adaptive learning, decision making, and human-robot interaction techniques is an important step toward achieving smart and safe autonomy in various sectors of manufacturing, aerospace, and healthcare (Schwarting et al., 2018;Reis et al., 2021;Walter et al., 2022).
An application area where safety is of utmost importance during autonomous behavior is in the field of robot-assisted minimally invasive surgery (RMIS).In RMIS, incision tools are maneuvered by a robot to assist surgeons in performing surgical operations more accurately while maintaining smaller incision cuts and improving patient outcomes (Poon et al., 2018;Khandalavala et al., 2019).Despite many research efforts to develop robotic systems for medicine, such as the da Vinci (Intuitive Surgical Inc., Sunnyvale, CA), Senhance (TransEnterix, Morrisville, NC, United States), and Flex ® (Medrobotics Corp., Raynham, MA, United States) robotic systems, an effective commercial system for deep-tissue needle insertion has not yet been realized due to complex needle-tissue interactions and the frequency of adverse events (Misra et al., 2010a;Majewicz et al., 2012).One such event is the buckling of a needle shaft that occurs due to rapid increase in frictional forces when the needle encounters a hard membrane or an obstacle (Narayan et al., 2018a).On further insertion, buckling could lead to tissue ruptures (Gruijthuijsen et al., 2018).Another event that could prevent surgeons from reaching desired targets within the tissue is the tissue deformation and displacements caused by the insertion forces of the needle (Oldfield et al., 2015).To address these difficulties, several steering control methods and novel needle designs have been proposed that accurately control the needle tip to reach desired targets (Hadjerci et al., 2016;Sprang et al., 2016;Li et al., 2017b;Fallahi et al., 2017).There have been efforts to autonomously predict adverse situations and correct for faults using sensor readings in unmanned vehicles (Huang et al., 2022;Ryu et al., 2022); however, much work is still needed for safe autonomous predictions of adverse events in robotic surgery.While reducing targeting errors is almost a solved problem, preventing and controlling adverse events such as needle buckling and tissue displacements before occurrence still remain a significant challenge to be addressed in real time (Leibinger et al., 2016;de Baere et al., 2022).With appropriate prediction and control subroutines, these adverse events can be prevented before they occur and damage to the tissue could be reduced, as demonstrated in Figure 1.
In prior work, we developed novel sensor forecasting algorithms based on which needle buckling events were predicted before occurrence (Narayan and Fey, 2020).Furthermore, with unknown needle-tissue interaction dynamics and computation complexities, implementing model-based or learning-based controllers would be challenging (Li et al., 2017a).Therefore, we implement a modelfree adaptive control (MFAC) technique (Hou and Jin, 2011) for steering control of needles and minimizing adverse events.This technique relies on the principle of linearizing a nonlinear system with a single parameter called the pseudo partial derivative (PPD) parameter which is slowly time varying in nature.However, this technique is not robust to changes in system parameters which affect real-time stability and output convergence.Two key contributions of this paper are: 1) developing an improved version of MFAC called improved model-free adaptive control (IMFAC) for general nonlinear systems by introducing fast time-varying PPD to guarantee continuous real-time stability, output convergence, and robustness against any external disturbance and 2) developing an action subroutine that integrates IMFAC with prior adverse event prediction methods (Narayan and Fey, 2020) to autonomously prevent needle buckling and tissue displacement events during robotic needle interventions.
This paper is structured as follows: Section 2 discusses the literature for adverse event prevention methods and data-driven control models related to robot-assisted needle insertions.Section 3 discusses the theoretical development of IMFAC, stability analysis, and simulation tests on a general class of nonlinear systems.Section 4 summarizes control algorithms to prevent adverse events.Section 5 discusses experimental methods.Section 6 discusses adverse event prevention results during needle insertions in gelatin and biological tissues.Finally, Section 7 concludes the paper.

Related work
This section discusses existing methods to prevent adverse events specific to robotic needle interventions, as well as state-ofthe-art data-driven control methods.

Prevention and control of adverse events
Over the years, several strategies have been implemented to avoid adverse events within the tissue, such as needle buckling and target displacements (Rossa and Tavakoli, 2017).For example, to prevent needle buckling events as a result of increases in the needle-tissue interaction forces, novel designs for steerable needles were developed.The designs increase the critical buckling force of the needle shaft and eventually reduce insertion forces on the needle during insertions (Webster et al., 2006;Van De Berg et al., 2017).However, these methods do not provide real-time buckling control.It is more important to track the difference between insertion force and critical buckling force relative to the needle position in the tissue, rather than just the critical buckling force (Van De Berg et al., 2017;Hulburt et al., 2019).For this purpose, needle-tissue biomechanics models are used to derive a relationship between insertion forces and needle displacements (Tang et al., 2007;Misra et al., 2010b;Sakes et al., 2016).However, biomechanics models require prior information of tissue properties, which is challenging for intra-operative procedures when tissue environments are unknown.Similarly, imaging techniques (Abolhassani et al., 2006;Hadjerci et al., 2016;Li et al., 2017bLi et al., ), 10.3389/frobt.2023.1271748 .1271748

FIGURE 1
Importance of control for timely prevention of adverse events in robotic needle interventions.(A) When the tissue is stationary and the needle is inserted at time, t 1 .(B) On further insertion, when the needle encounters an obstacle or the tissue starts displacing by l 1 at time, t 2 > t 1 .(C) With no control, the needle either buckles or the tissue displaces further by l 2 > l 1 , causing severe displacements of the tissue.However, with preventive control, needle buckling or tissue displacement is prevented.mechanical devices and probes as tissue constraints (Brown et al., 2010;Kobayashi et al., 2012), and path planners (Vrooijink and Abayazid, 2014;Li et al., 2017b;de Baere et al., 2022) have been implemented to control the deformations and displacements of the tissue.Though these methods have shown success in achieving accurate target placement of the needle tip, they rely on finite element and biomehanics models.Accurate characterization of needle-tissue dynamics can be time consuming, and thus, it can be challenging to implement real-time control (Rossa and Tavakoli, 2017;Barnoy et al., 2022).
Recently, research groups have taken inspiration from parasitic invertebrates such as wasp ovipositor and mosquito to design needles that can penetrate solid substrates easily.Inspired by the mechanism of the ovipositor, a thin needle composed of several wires sliding alongside the other was developed to maneuver needles with high curvatures while minimizing impact on the surrounding tissues (Sprang et al., 2016;Scali et al., 2017).Similarly, Li et al. (2020) developed a needle-cannula system that mimics the mosquito proboscis mechanism of incremental motions to reduce tissue deformations and displacements during insertion.They also used harpoon notches at the needle tip to reduce needle-tissue frictional forces (Li et al., 2020).These bio-inspired mechanisms of needle insertions can generally improve maneuverability, while reducing critical needle-tissue interaction events such as needle buckling, tissue deformations, target displacement, and needle deflections.However, they have complicated designs and require several fixtures to prevent buckling of individual wires.Furthermore, a systematic study to select design parameters is required, which usually change due to varying heterogeneous tissue substrate (Leibinger et al., 2016;Matheson and Rodriguez y Baena, 2020).
These limitations call for alternative solutions that do not depend on needle designs and unknown tissue structures, such as model-free techniques.Thus, optimizing on needle insertions while minimizing adverse events in real time using data-driven control techniques would be the focus of this work.

Data-driven control methods
In the last decade, much attention has been given to the application of data-driven control techniques for modeling complex nonlinear processes (Abouaïssa and Chouraqui, 2019).The common classical technique being the proportional, integral, and derivative (PID) controller has shown to be generalizable to many systems and finds applications in most regulatory-based process control industries (Åström and Hägglund, 1995;McMillan, 2012).However, tuning of the gains is performed offline and is affected by process disturbances and the time-varying behavior of the system.This makes PID less robust and challenging to implement in real time.To address these issues, intelligent PID was proposed to predict dynamic system behavior through estimation techniques and online numerical differentiator, eliminating offline tuning and prior identification procedure (Fliess and Join, 2009).Yet, its performance is limited by issues of sensor noise, sampling rates, and computation power as the size of data increases (Hou and Jin, 2011).
There has been increasing interest in unconventional control strategies such as neural networks, fuzzy logic, and genetic algorithms (Thomas and Poongodi, 2009;Heidari et al., 2013;He et al., 2015;Bing et al., 2018;Glida et al., 2021).Neural network control is based on learning from the mapping between system input-output data, whereas fuzzy control is based on learning from past experience and expert knowledge to predict and control system behavior (Abouaïssa and Chouraqui, 2019).Genetic algorithms tune control models based on stochastic optimization techniques that mimic biological evolution (Rodríguez-Abreo et al., 2020).The advantage is that these methods do not require any mathematical model of the system; however, determining parameters that 10.3389/frobt.2023.1271748accurately satisfy system behavior can be computationally expensive.Moreover, methods to identify system stability with learning models are still yet to be developed and fully analyzed (Singh and Sukavanam, 2012).Efforts have been made to develop learningbased control techniques with low computation complexity, to optimize needle insertions in tissue while minimizing undesired needle deflections and tissue displacements (Buzurovic et al., 2010;Rossa et al., 2017;Pratt and Petruska, 2022).Yet, their methods do not entail prevention of these events before occurrence.
An alternative technique to control any complex nonlinear system without using learning models is MFAC (Hou and Jin, 2011).This technique uses the dynamic linearization principle to simplify the system nonlinearities and requires only system input-output data.Due to the ability to perform stability analysis online, simplified mathematical models, and robustness against external disturbances, MFAC finds applications in unmanned aerial vehicles, automated manufacturing, and mobile robots (Hou and Jin, 2019).Recently, Li et al. (2017a) derived a model-free adaptive control from Kalman filter techniques to predict needle buckling and control positions of a flexible robot.However, they have not provided a mechanism yet to minimize an adverse event when it occurs.To the best of our knowledge, model-free adaptive control has not yet been implemented on robotic systems that could control needle-tissue interactions while preventing or minimizing adverse events in real time.

Improved model-free adaptive control
In the original MFAC framework, the online stability of a system depends on the choice of system and control parameters (Hou and Jin, 2011).Since these parameters are tuned offline, real-time robustness against system uncertainties and output convergence cannot be guaranteed.Therefore, online tuning of control parameters is implemented with MFAC to improve realtime output convergence and stability in the presence of unexpected disturbances, and we name this technique as IMFAC.This section focuses on the formulation of IMFAC and parametric selection methods through theoretical analysis of stability and output convergence.

Derivation
If a discrete-time nonlinear system with output y and input u c satisfies the Lipschitz condition for timestamps k i ≠ k j and i, j > 0, then the nonlinear system can be approximated as a dynamic linearization model, where Θ c is a slow time-varying parameter called PPD that compresses system nonlinearity.This approximation is useful in predicting the output of a nonlinear system without using dynamic models.PPD is a scalar for single-input single-output (SISO) systems and a vector for multi-input multi-output (MIMO) systems.For tracking and regulation-based control, the following quadratic cost function is employed to obtain control effort, Δu c (k), as a function of prediction errors, y* − y(k + 1): where y* is the desired output of the system.The weighting factor λ c > 0 restricts Δu c (k) and, thus, is an important tuning parameter for output convergence and system stability.Substituting y(k + 1) from Eq. 2 and minimizing Eq. 3 with respect to u c , the optimal input for tracking is computed as where Θ c (k) is determined using numerical approximation methods due to unknown system dynamics.The following cost function is used to obtain an approximate Θ c (k) by minimizing the error between the modeled output from Eq. 2 and observed output, y(k): (5) The second term in the cost function ensures robustness against system nonlinearity.Minimizing this cost with respect to Θ c (k) yields where Δu c (k) = u c (k) − u c (k − 1).However, Eq. 6 is not robust to process errors such as arbitrary initialization of system parameters, which affect output convergence.This issue is resolved when Θ c (k) is assigned as a rapid time-varying parameter The Lipschitz constant, b , is chosen from the theoretical analysis of system stability and output convergence.Finally, Eqs 2-7 are computed iteratively.This technique is similar to the existing framework (Hou and Jin, 2011); however, the novelty lies in the introduction of the rapid varying PPD and the methods of initializing control parameters, (λ c , Θ c (0)), and system parameters, (u c (0), y(0)), for continuous real-time stability discussed in Section 3.2.

Stability analysis and parameter initialization
In the traditional MFAC scheme, Θ c (0), y(0), u c (0), and λ c are hand-tuned using trial and error methods which can be tedious.This issue can be resolved if a systematic approach to parametric initialization is followed under certain assumptions of the system stated below.
This assumption holds for a class of systems that cannot move beyond the commanded external force, unless affected by disturbance factors.For example, in robotic needle steering, a tissue cannot displace beyond the needle insertion velocity unless affected by other factors such as patient breathing.This condition resembles the bounded-input bounded-output stability where energy in the system output cannot increase beyond the energy levels in the system inputs according to energy conservation law.Hence, the absolute values of Θ c from Eq. 2 are also bounded between 0 and 1 according to Eq. 8.
Next, the sign of Θ c depends on the system response to inputs, which is stated in the following assumption.
This assumption holds for systems whose outputs are affected by multiple inputs.For example, if two inputs (say, u c1 and u c2 ) influence an output such that an increase in u c1 and decrease in u c2 drive the system to desired outputs, then 0 < Θ c (0) < 1 for u c1 and −1 < Θ c (0) < 0 for u c2 .
Third, if initial values of the system input and output are assigned close to the operating points, then output convergence is guaranteed.
The operating points (a, ã ) are usually identified during prior simulations.For physical systems, they are easily identified as safe operating limits provided by the system manufacturer.
Finally, output convergence also depends on the range of the input weighting factor, λ c , stated as follows.
Assumption 4: If system (1) is controlled for a regulation problem, then there exists a minimum value, λ min c = 1 4 such that |y* − y(k + 1)| → 0 for all k and λ c > λ min c .There also exists a maximum value, λ max c , above which the system settles but with steady-state errors.
Proof.The proof is straightforward where the boundedness property of b from Assumption 1 is used to maintain both system stability and convergence.If the steady-state error is defined as then substituting y(k + 1) in Eq. 9 from Eq. 1, we have Substituting Δu c (k) from Eq. 4 in Eq. 10 and from the absolute value theorem, (11) Now, using the boundedness of Θ c from Assumption 2, we know that Since b cannot exceed 1 from Eqs 11, 12, we assume | as the maximum limit to find λ min c .Then, using the general concept of quadratic equation, Hence, λ min c = 0.25 using the relation, 1 2√λ c = 1 in Eq. 13.Finding an exact maximum value, λ max c , is not important, as λ c > 1 leads to lower rising times and larger steady-state errors for many systems (Hou and Jin, 2011).Since many real-time applications require fast rising time and low steady-state errors, λ max c = 1 is considered.Furthermore, Δu c (k) needs to be constrained as most actuators have finite excitation.Thus, λ c can be chosen arbitrarily in the range (λ min c , 1) to balance between constrained inputs and tracking errors.

Simulation tests
The improved framework (IMFAC) is validated on two test nonlinear systems to compare performance with MFAC (Hou and Jin, 2011).All equations are coded in the MATLAB ® 2018Rb environment and tested on simulated SISO and MIMO nonlinear systems.The algorithms are compared using the same initial conditions and control parameters.
For a SISO system with some nonlinear input-output model, Let the initial conditions of the system be u c (1) = u c (2) = 0 and y(1) = y(2) = 0 with a constant reference output as y* = 0.5.The output of the system is expected to change at k = 500, 750 according to Eq. 14, and thus, some disturbances due to these transitions are expected.Simulation results of SISO are shown in Figures 2A, B, where solid green plots correspond to IMFAC, dashed red plots correspond to original MFAC, and the dash-dot blue plot is the reference signal.For λ c = 0.1 < λ min c , IMFAC converged to the desired value within k = 790 after disturbance at k = 750, while MFAC showed lack of output convergence due to sustained oscillations (upper plot of Figure 2A).When |Θ c | > 1, IMFAC showed faster convergence at the disturbances, whereas MFAC showed slower convergence at k = 830 after disturbance at k = 750 (upper plot of Figure 2B).For the same control parameters and initial conditions, IMFAC showed better stability and faster convergence due to the corrective action of the rapid time-varying PPD, compared to MFAC with slow time-varying PPD (lower plots of Figures 2A, B).
We consider a two-input (u c1 , u c2 ) one-output (y) MIMO system with some nonlinear model in Eq. 15.where u c = [u c1 , u c2 ] T ∈ ℝ 2×1 .Arbitrary initial conditions of the system are u c (1) = u c (2) = [0.5, 0.6] T , y(1) = 0, y(2) = 0.9, and Θ c (1) = [0.5, −0.2].Let λ c1 = λ c2 = λ c = 0.5 for each input.We can have λ c1 ≠ λ c2 ; however, for the ease of validating Assumption 2, we choose λ c1 = λ c2 .A constant reference trajectory close to zero (y* = 0.01) is considered since ideal y* = 0 would cause control inputs to be zero which cannot drive the system to desired outputs.Therefore, near zero reference is used to simulate stationary or minimum movement of the system.Simulation results show that negative Θ c2 = −0.2decreases u c2 and positive Θ c1 = 0.5 increases u c1 , which drive the system to the desired output (y* = 0.01), see Figure 2C.It should be noted that initial Θ c can be assigned to control the system inputs in the desired direction only if there is basic knowledge about system response to the inputs.
Finally, the performance of IMFAC is compared with original MFAC in terms of frequency of unstable events computed as the number of occurrences of |Δy shows that system stability was not affected by arbitrary initialization of λ c and other disturbances.Otherwise, these results show the comparable performance of IMFAC with MFAC using a systematic approach to parameter initialization.On average, the convergence time of IMFAC was 20% faster than that of MFAC.This shows that irrespective of the control parameters and initial conditions, system stability and fast output convergence are always achieved with IMFAC.

Application to robotic needle steering
The MIMO system in Section 3.3 is similar to the needle intervention applications where tissue motion, y = l, is controlled by two inputs: a) insertion velocity ν and b) duty-cycle factor as a measure of the number of needle shaft rotations per insertion, D. Tissue motion can be minimized with a decrease in insertion and same λ c = 0.5 will be used for both the inputs (ν and D) for all needle insertions.Initial values of the inputs are commanded by a user.The range or operating points of these inputs is decided by the maximum rating of the motors used to drive the needle following Assumption 3. Since the output (tissue displacement) is zero when excitation (insertion inputs) is zero for needle-tissue systems, we will assume zero initial values, l(0) = 0, for all needle insertions.The next section will discuss application of IMFAC to prevent adverse events during needle insertions in a tissue.

Prevention of adverse events
In this section, we first summarize the methods to predict adverse events using a previously developed sensor data forecasting technique called compact-form dynamic linearization model-free prediction (CFDL-MFP) (Narayan and Fey, 2020).We then develop optimal needle steering algorithms using IMFAC and detection algorithms from the work of Narayan et al. (2018b) for the overall prevention of adverse events.

Prediction of adverse events from sensor data forecasts
Let the time series data obtained from a sensor at current time k be S(k).The steps to generate sensor data forecasts, Ŝ N (k + 1), for some future prediction horizon, N, using CFDL-MFP are The data vector S represents sensor data.For example, previous N samples of force data measured from a force sensor at time instant k are represented as S N (k), and Ŝ N (k + 1) represents N-step ahead forecasts of force data at the next time instant k + 1.The forecast parameters ΔU N (k) and ΦN (k) are computed following the procedure in our previous work (Narayan and Fey, 2020): where • is array element-wise operation, ⃗ 1 is an N-dimension vector of ones, and Δ Ŝ (k) = Ŝ (k) − Ŝ (k − 1).The sensor data S N (k) contain previous N observed data points, and the reference vector S * N (k) is the running average of the observed data points.
Forecasts of force sensor data are generated when a slight increase in force is detected, and if force forecasts show patterns of rapid increase, then needle buckling is predicted (Narayan and Fey, 2020).Similarly, tissue displacements can be predicted using forecasts of tissue position data.Tissue displacement is characterized based on the intensity of motion such as mild tissue displacement (MTD) and severe tissue displacement (STD).STD occurs when the needle tip stops moving relative to the tissue frame (Narayan et al., 2018a).To predict STD before occurrence, N-step forecasts of the tissue position relative to fixed frame l twN and the needle-tip position relative to fixed frame l nwN are generated using Eqs 16, 17.It should be noted that l tw and l nw are the Euclidean norm of 3D positions measured from position sensors ⃗ p tw and ⃗ p nw , respectively.The norm of the needle-tip position relative to the tissue frame is obtained as l nt = l nw − l tw , and ( l twN , l nwN , l ntN ) are the forecasts of the corresponding position norms.STD events are then predicted using these norm position forecasts with the sigmoid slope detection methods in Eq. 18.
The needle insertion distance ŝ is linearly projected into the prediction horizon N. The thresholds m 1 and m 2 are experimentally determined (Narayan et al., 2018b).

FIGURE 4
Flowchart for the overall prevention of adverse events.Critical events such as STD and needle buckling are prevented using reactive control (e.g., robot stop and retract needle).When mild events are detected, needle insertions continue through optimal from IMFAC to prevent potential severe events before occurrence.

Real-time prevention algorithm
To prevent critical events such as needle buckling and STD before they occur, it is necessary for the robot to respond immediately to the prediction algorithms.For this purpose, reactive control is implemented where the robot stops and retracts the needle when buckling is predicted, and the robot stops insertion when STD is predicted to avoid potential damage to the tissue.However, when a mild event such as MTD is detected, adaptive control (IMFAC) computes optimal steering inputs (insertion velocity and duty cycle) to steer needles such that the occurrence of mild events is minimized and severe events are prevented.This strategy ensures continuity of the insertion procedure which otherwise is disrupted by reactive control.A complete event prevention algorithm with a combination of reactive and adaptive control is shown in Figure 4.
The pseudo code for adverse event prevention is presented in Algorithm 1.The code is an integration of the proposed methods with the previous event detection, sensor forecasting (CFDL-MFP), and prediction algorithms (Narayan et al., 2018a;Narayan and Fey, 2020).The inputs to the algorithm are the filtered sensor data at current time k, CFDL-MFP prediction horizon N, user-commanded initial needle insertion velocity ν, and duty-cycle factor D. Before iteration, the IMFAC control parameters are initialized according to Section 3.2, and the initial event prediction flag is set to zero, see steps 3-4.The total insertion distance s is user-defined prior to the insertion procedure.Encoder readings s(k) are used to check if the measured insertion distance reached user-defined s, see step 5.All sensor data are filtered in real time using the Kalman filtering scheme from the work of Narayan et al. (2018b).For every iteration, the Euclidean norm of 3D position sensors and optimal mean forces f * p (k) are computed, since CFDL-MFP and detection algorithms work on univariate time series data, see steps 6-7.The Optimal Mean function is the running average of the observed sensor data (Narayan et al., 2018a).Checks for needle buckling detection are performed for every k, see step 8.If a force increase is alerted by the detection algorithm, N-step forecasts of force data faN (k) are generated with CFDL-MFP and the buckling prediction algorithm checks for any buckling event using the force sensor forecasts (Narayan and Fey, 2020).If buckling is predicted, the robot stops insertions and needle is retracted, see steps 9-16.Similarly, checks for tissue displacement detection are performed, see step 17.If MTD is detected (k = k m ), the displacement prediction algorithm checks for STD events.When STD is predicted, the robot stops insertions, see steps 18-24.Otherwise, steering control inputs for the needle are continuously updated until the end of insertion distance s to minimize or prevent further displacements of the tissue, see steps 25-29.The algorithm is executed in a multi-threaded framework to ensure simultaneous prediction and prevention of adverse events in real time.

Experimental methods
This section describes in detail the robot hardware setup and protocols used to validate the adverse event prevention algorithm during needle insertions in artificial and biological tissues.

Robot and setup
The needle steering robot for needle insertion experiments is shown in Figure 5A.The robot is controlled with a Dell Inspiron 15 Gaming Laptop through custom C++ multi-threaded   application (QT Creator 4.6.0) on Ubuntu Linux 16.04.The code manages low-level PID motor control and the execution of dutycycled flipping algorithms to enable different needle steering directions (Majewicz et al., 2014).The robot is equipped with a 6-axis force-torque sensor (Nano-17, ATI Inc.) at the needle base to track axial forces due to needle-tissue interaction and an electromagnetic (EM) tracker (trakSTAR, NDI Inc.) to track needletip position.Data from these sensors are streamed at 50 Hz similar to (Narayan et al., 2018b).Algorithm 1 is coded as modules in Python 3.5 and interfaced with a low-level motor control code in QT Creator C++ through pybind11, an open-source Python-C++ wrapper that calls Python modules from C++ directly enabling multi-threaded operations.Real-time plotting of sensor data and prediction results of Algorithm 1 during needle insertions are displayed on QT GUI using the QCustomPlot library.

Gelatin tissues to simulate adverse events
A homogeneous gelatin tissue of stiffness 60 kPa (1:3 gelatin to sugar in 500 mL water) and dimensions 14 × 8 × 7.5 cm was created to validate tissue displacement prevention methods from Algorithm 1.The tissue is placed on a platform smeared with soap water to simulate a slippery surface for the sliding motion of the tissue, as shown in Figure 5B.For short-hand notation, let this sample be denoted as T 1 .Another gelatin sample of varying stiffness 60 and 20 kPa (1:3 gelatin to sugar in 1000 mL water) was created by embedding an obstacle to validate needle buckling prevention methods in heterogeneous tissue with a known environment.To prevent damage to the needle and the EM needle position sensor, a standard rectangular Scotch-Brite dish scrubber of thickness 0.7 cm is used as an obstacle.The 20 kPa region is on one side of the obstacle at 7.1 cm from a short edge, and the stiffer 60 kPa region is on the other side of the obstacle at 6 cm from the opposite short edge as shown in Figure 5C.This tissue denoted as T 2 is also placed on the slippery platform to validate Algorithm 1 during buckling and tissue displacement events.For more details on gelatin preparation methods, refer to the work of Narayan et al. (2018a).Both T 1 and T 2 are used for experiments within 12 h of refrigeration.A trakSTAR sensor was taped on the surface of these tissues to track displacements.

Steerable needles
Two steerable needles with different dimensions and tips are used to validate the proposed prevention methods for robustness to needle types.The first steerable needle (N 1 ) shown in Figures 5D is made from a hollow nitinol tube with dimensions 250 mm length, 0.8 mm OD, and 0.6 mm ID and a 30°prebent tip of 4 mm length (Majewicz et al., 2014).For integration with the robot, an EM needle tracker was inserted into the hollow end of the needle and secured with Luer locks.The second needle (N 2 ) is a clinically available prostrate seeding needle with a stiff hollow tubing of length 175 mm and thickness 1.27 mm.A stainless steel wire of thickness 1.2 mm was cut into a small piece of length 10 mm and inserted into the tip of the hollow seeding tube such that 5 mm was jutting out of the tube and the rest 5 mm was inside the tube.The tip was ground to create a bevel angle of 60°to enable cutting through the tissue.Another EM needle tracker was inserted into the hollow end of the needle and secured with Luer locks for integration with the robot similar to N 1 , as shown in Figure 5E.

Insertion protocol in gelatin and biological tissues
To specifically evaluate IMFAC control and tissue displacement prevention methods against different needle types, a total of 30 insertions with prebent needle N 1 and six insertions with straight needle N 2 were performed in tissue T 1 .To validate both buckling prevention and tissue displacement prevention algorithms against various needle and tissue types, a total of 10 insertions with needle N 1 and four insertions with N 2 were performed in tissue T 2 .These insertions were performed using four initial user-commanded insertion velocities (0.25,0.5,1,1.5)cm/s,and three initial usercommanded duty cycle (0,50,100)%.Videos of the insertions were recorded using an iPhone 11 camera focused on the top surface of the tissue and a digital camera (Sony W800/B20MP 5X Optical Zoom) focused on the overall robot-tissue setup.The videos were used as ground truth for verification of the prevention algorithms.For viewing and analysis, motion images were generated in MATLAB at 30 frames per second and overlaid using Adobe Photoshop CC 2020.Around 16 needle insertions were performed with the ex vivo goat liver tissue labeled as T 3 in Figure 5F at the University of Texas Southwestern Medical Center.The liver tissue was placed on a clean platform.Live fluoroscopic videos were collected using a 9-in XRII OEC series 9600 fluoroscope (X-ray source and CCD camera) and saved with a Canopus ADVC 110 frame grabber, as shown in Figure 5G.The fluoroscopic videos were used as ground truth to check if adverse events were detected or prevented using Algorithm 1.

Results and discussion
A good result of an optimal needle steering in the event of a mild tissue displacement is shown in Figures 6A-C.As the prebent needle N 1 steered within the tissue T 1 , MTD was detected in the direction of the tip showing curved tissue motion from the motion image in Figure 6A.Each motion image was generated by overlaying three frames of motion above the other to show the translation displacement of the tissue and insertion or retraction motions of the needle.As soon as MTD was detected at 11.5 s (pink-shaded region) from Figure 6B, optimal steering inputs were calculated such that MTD stopped for short periods of time as can be seen in the discontinuities around 20 s from Figure 6C.The insertion procedure completed with reduced insertion velocity (blue plot) and increased duty-cycle (green plot) as no severe events were predicted.It should be noted that the insertion inputs were constrained where the velocity cannot reduce below 0.25 cm/s nor increase beyond 1.5 cm/s due to motor ratings.Similarly, there are only limited dutycycled steering options between 0% and 100%.If the needle was inserted with the lowest insertion velocity and MTD was detected, then the duty-cycle increased to reduce tissue motion.
Buckling prediction results in tissues with embedded obstacle T 2 are shown in Figures 6D-F.The motion images are obtained by overlaying two frames above the other to show the insertion and retraction motions of the needle during a predicted event.The frames corresponding to the interaction of the needle tip at the obstacle and the final retraction step are overlaid for better visibility since overlaying more than two images causes blur.In all the plots shown, buckling events were predicted at those timestamps when a force increase (not shown) was alerted by the detection algorithms as soon as the needle tip touched the obstacle, and the needle retracted as soon as a buckling was predicted at this time of force increase alert at 6.7 s.Using Algorithm 1, the robot stopped insertions and the needle was retracted by 2 cm.The retraction distance of 2 cm was chosen arbitrarily.This choice would depend on the path planning algorithms based on preoperative data or the operating surgeon.Results in the 20 kPa region of tissue T 2 show that prediction algorithms are robust to tissue environments and that they solely depend only on the force readings.However, the limitation is that sometimes, force does not increase even on encounter with an obstacle.In this case, the algorithms would neither detect nor predict any buckling event, explained in the next paragraph.
An example of an optimal needle steering with reactive control is shown in Figures 6G-I.When N 1 was inserted into tissue T 1 , MTD was detected at 27.2 s as soon as the tip approached the obstacle.Optimal steering inputs were calculated to reduce MTD, but the robot stopped motion as severe displacement of the tissue was predicted, represented by the yellow-shaded plot.It should be noted that buckling of the needle shaft was expected as soon as the tip encountered the obstacle; however, instead of a rapid force increase, a mild displacement was detected which resulted in severe displacement.Therefore, even when the needle appeared to have buckled, the axial force did not increase rapidly as expected, which caused both the buckling prediction and detection algorithms to fail.However, the overall procedure worked because of a displacement prediction and the insertion stopped on time.This example shows the advantage of developing unified methods for adverse event predictions.
Another example of a unified prediction is shown in Figure 7.When the needle N 1 was inserted for 0% duty cycle, the needle became curved due to the high stiffness of the tissue.Moreover, since the surface was slippery, mild displacements were detected as the needle steered further into the tissue.Due to the high steering curvature of the 0% duty-cycled needle, a buckling was predicted and a slight bend of the needle shaft can be seen in the motion image.Even though this could be a false buckling alarm, it is still safer to stop the needle as it gives time to the operating surgeon to decide if the insertion procedure should continue or stop.
Validation results in the ex vivo goat liver T 3 are shown in Figure 8.The prebent needle N 1 was used for all insertions in T 3 .The timestamps of the live fluoroscopic videos match the timestamps of the real-time plots of the predicted adverse events, confirming the timely prevention of these events in heterogeneous tissues.All insertion data in gelatin and goat liver tissues with both the types of needles were combined for statistical analysis.The Kruskal-Wallis statistical method was performed on the data to study the effectiveness and robustness of the prevention algorithm against different needles and tissue environments.Prevention rate is computed as the ratio of the number of adverse events prevented to the total number of insertions for each tissue sample and needle type (blue diamond bars), see Figure 9A.Similarly, the detected event rate (red cross bars) is the ratio of the number of adverse events detected by the total number of insertions for each tissue sample and needle type, and the rate of no events is computed as the ratio of the number of insertions with no adverse events detected or predicted by the total number of insertions for each tissue sample and needle type (green square bars).Error bar plots show that the largest percentage of the needle insertions was optimized for control and prevention of adverse events (blue diamond bars).According to the two-way ANOVA analysis from Figure 9B, prevention rates did not depend on the needle or tissue type nor the needle-tissue interaction effects (p > 0.01), proving the robustness of the algorithms.The rate of detected events shows that early prediction of adverse events was not successful; however, the robot still stopped the insertion procedure through a reactive control mechanism on detection of the adverse event.The rate of controlled needle insertions (prevented events) in the goat liver was comparable to the rate of the detected events because of the local deformations of the tissue which were detected as severe tissue displacement events.Tissue deformation is a separate problem that is not covered in this work; however, real-time tissue mapping could be used to obtain control inputs for optimal needle steering with minimal tissue deformation and motion.
To study the effectiveness of the adverse event prevention algorithm to robotic needle steering applications, needle insertions were grouped into a) completed insertions with an optimal steering mechanism to prevent severe adverse events, b) aborted insertions due to the prediction or detection of a severe event, and c) completed insertions with no adverse events predicted or detected.The bar plots in Figure 9C show that the majority of needle insertions were aborted due to the reactive control mechanism of Algorithm 1 which prevented severe events such as STD and needle buckling.In about 25% of the needle insertions, MTD was detected and optimal steering inputs were continuously updated to complete the insertion procedure while minimizing MTD and preventing severe events.About less than of the needle insertions did not encounter any adverse events, which shows that reactive and adaptive mechanisms are important to be considered in robot-assisted needle procedures to ensure patient safety.
A major limitation of this study is that there were no targets assigned to evaluate the accuracy and time of the insertion procedure.Steering control inputs should be optimal not only in minimizing adverse events but also in procedural time; that is, time taken for the needle to reach desired targets, while also minimizing adverse events simultaneously, needs to be extensively studied and is a scope for future work.This work is a significant step to ensure procedural safety through data-driven techniques.When combined with efficient computer vision and tactile feedback techniques, our prediction and control methods can significantly improve patient safety and reduce procedural time and effort for the operating surgeon. 10.3389/frobt.2023.1271748

Conclusion and future work
We improved the existing model-free adaptive control technique (IMFAC) for general nonlinear systems to ensure continuous system stability and robustness against changes in system parameters and external disturbances.Comparing the simulation results with existing MFAC, IMFAC was more stable and robust to arbitrary values of system and control parameters.Then, we developed a preventive control subroutine by integrating IMFAC with previously developed event prediction methods from the work of Narayan and Fey (2020) to minimize adverse events in real time.Choice between reactive and adaptive control techniques was autonomous depending on the severity of the event.The robot implemented reactive methods such as needle retraction and stopped insertions when a needle buckling or a severe tissue displacement was predicted.During a mild tissue displacement, the robot adaptively steered the needle into the tissue to minimize and prevent future severe displacements so that continuity of the insertion procedure is ensured.Statistical results across different tissues and needles show robustness and generalizability of our algorithms to incision tool types and unknown tissue environments.
Since the algorithms rely on sensor data, the prediction and prevention subroutines might fail if a sensor gets faulty, resulting in more severe adverse events and injuries to the tissue.To address this issue, future work will focus on integrating augmented visual feedback techniques with our data-driven models.Moreover, the technique would also help identify the source of adverse events so that more intelligent actions can be implemented to ensure better continuity and safety of the procedure.
FIGURE 2 Simulation results on test nonlinear systems: (A,B) SISO system showing stable response and faster convergence of improved MFAC (solid green, upper plots) even when λ c < λ min c or |Θ c | > 1, due to the rapid time-varying behavior of Θ c (solid blue, lower plots).(C) MIMO system response through the directional control of inputs (u c1 , u c2 ) and initial Θ c , where the output is controlled by increasing u c1 with positive Θ c and decreasing u c2 with negative Θ c .(D) Stability analysis comparing improved MFAC (blue diamond) and original MFAC (red cross) for different λ c and initial Θ c .
(k + 1)| > |Δu c (k)| per simulation time, for different values of control parameters (λ c and Θ c ) as shown in Figure 2D.The blue diamond plot (IMFAC) for λ = 0.1 < λ min c FIGURE 3Model-free control to minimize tissue motion l.

Frontiers
FIGURE 5 Experimental setup: (A) Needle steering robot with force and EM sensors.(B,C) Gelatin tissues (T 1 , T 2 ) on a slippery platform smeared with soap water to elicit adverse events.(D,E) Prebent-tip needle (N 1 ) for insertions in all tissues and the straight bevel needle (N 2 ) for insertions in T 1 and T 2 .(F) Goat liver (T 3 ) on a non-slippery platform to validate event prevention algorithms.(G) Fluoroscopic imaging to track adverse events during insertions in T 3 .

FIGURE 6
FIGURE 6 in gelatin tissues: (A-C) Motion image and plots showing controlled needle insertions with the prebent needle 1 ) in tissue T 1 to minimize tissue motion, by changing needle insertion velocity and duty cycle (tip orientations) during the detection of MTD.(D-F) Motion image and plots showing the retraction of the straight needle (N 2 ) at the obstacle in T 2 , and buckling was predicted.(G-I) Motion image and plots showing controlled needle insertions with needle N 1 in tissue T 2 during MTD, followed by reactive control (robot stopped insertion) when severe displacement (STD) was predicted.
FIGURE 7 Optimal needle steering control with buckling prevention: (A) Motion image showing optimal needle insertions in gelatin T 1 to minimize MTD and prevent needle buckling.(B) Control inputs change when MTD is detected to avoid STD.(C) Normalized position plots of the needle and tissue showing MTD detections.(D) Buckling predicted due to which the robot stops insertion and retracts the needle.

FIGURE 8
FIGURE 8 Prevention of adverse events in biological tissue: (A-C) Fluoroscopic motion image and plots showing controlled insertions with the change in insertion velocity and duty cycle (tip orientation) during MTD, followed by reactive control (robot stopped insertion) when STD was predicted.(D-F) Motion images and plots showing needle retraction when a drastic increase in force was predicted.