Improving Precision in Navigating Laparoscopic Surgery Instruments toward a Planar Target Using Haptic and Visual Feedback

1. Introduction

Adopted due to improved patient outcomes, laparoscopic surgery is a form of Minimally Invasive Surgery (MIS) during which the surgeon manipulates thin elongated instruments inserted into the patient’s abdomen through ports (trocars) while monitoring the operating site through an endoscopic camera (laparoscope). The significantly reduced access trauma during such surgery benefits patients through better cosmesis and reduced recovery times (Li et al., 2014), which also contribute to cutting costs for hospitals. However, surgeons face new challenges stemming from reduced dexterity and diminished perception of the operating field (Bholat et al., 1999; Picod et al., 2005; Xin et al., 2006; Nisky et al., 2012). Notably, the unnatural camera point and field of view, loss of visual depth cues combined with complicated hand-eye coordination seriously complicate surgical tool navigation within the abdomen.

This complication of the task is not only tedious for the surgeon but can actually lead to sub-optimal performance and potentially dangerous errors during surgery (Deziel et al., 1993). For example, in their study on the causes of errors in endoscopic surgery, Joice et al. (1998) identify “Too much force/rotation/displacement” as leading causes of operative injury and/or post-operative complications. Because of this, we propose to study methods for assisting surgeons during intra-corporeal tool navigation by supplying relevant information to them, allowing them to correct their errors and to optimize their gestures.

Visual and vibrotactile feedback (Bark et al., 2011) as well as kinesthetic feedback (Feygin et al., 2002) have been considered as means for improving performance in gesture guidance and learning. Haptic guidance has been widely explored in rehabilitation scenarios [e.g., Causo et al. (2012)] and teaching of complex gestures (e.g., musical instrument playing) (Grindlay, 2006; Holland et al., 2010). An often reached conclusion is that congruent visual and vibrotactile feedback improves the quality of gestures, whereby vibrotactile feedback alone allows for faster responses than visual feedback, probably because of the lower induced cognitive load (Rodríguez et al., 2010). Vibrotactile cues for guidance have also been explored for pedestrian (Elliott et al., 2010) and vehicle (Van Erp et al., 2004; Ege et al., 2011) navigation, concluding that tactile feedback functions best in situations with high visual cognitive load and concurrent tasks.

Haptic feedback for surgical tool navigation has been explored mostly in the context of Robotic Minimally Invasive Surgery (RMIS), using mainly kinesthetic feedback (Li et al., 2007; Feng et al., 2009). However, tactile feedback has also been considered as a viable solution, either for co-manipulation (Rosen et al., 1999; Yao et al., 2004) or as a form of sensory substitution – mainly for the feedback of force information [e.g., Wurdemann et al. (2013)]. Of course, haptic feedback has been explored beyond RMIS both for feeding back force (Li et al., 2015) and navigation information. Bluteau et al. (2010) studied the use of vibrotactile cues for guiding a tool along a 3D trajectory in traditional (open) Computer-Assisted Surgery (CAS). Similar work by Hansen et al. (2013) investigated such forms of feedback for improving surgical navigation during resection tasks. Brell and Hein (2007) review work and design considerations for tactile feedback to augment surgical gestures based on preoperative information, noting that tactile feedback is a promising alternative to visual guidance as the cues are private, intuitive, and can easily code complex spatial information. While visual feedback alone yields lower errors than tactile feedback alone, it also significantly prolongs Task Completion Times (TCT). They achieve best results through combined visual and tactile feedback, with extremely low error rates, although TCTs are longer than for tactile feedback alone.

Thus, to our knowledge, work on using informative feedback to assist surgical gestures remain largely focused on compensating the specific shortcomings of Robot-assisted Minimally Invasive Surgery (RMIS) and have anecdotally been explored in conventional surgery settings. However, results from these domains as well as from applications outside of the surgical domain hint at the possibility that haptic, visual, and multi-modal feedback may indeed find applications in assisting conventional laparoscopic gestures.

In this paper, we discuss a series of experiments focusing on assistance for guiding the tip of a laparoscopic instrument toward a target plane. This situation could arise for example during a laparoscopic hepatectomy, during which the surgeon must delineate a plane bisecting the liver and cut the organ along said plane while ensuring the best accuracy and planarity of the cut (Franco, 2001). Inaccurate cuts pose potential health hazards through insufficient resection of pathological tissue or through excess resection of healthy tissue. Non-planar cuts on the other hand are dangerous due to the resulting poor vascularization of protruding liver tissue, which may lead to post-operative complications.

Chapter 2 outlines the materials and methods for two consecutive series of experiments aimed at comparing performance improvements in navigating an instrument toward a planar target under provision of various types of haptic and visual feedback. The observed differences are presented in detail in chapter 3 and subsequently discussed in chapter 4. We conclude that although navigation assistance through a co-manipulator implementing soft guidance virtual fixtures delivers results far superior to those achieved through exclusively informative visual and/or tactile feedback, the latter still show promise for simple, low-cost solutions for assistance, which may best benefit experienced surgeons.

2. Materials and Methods

In the following, we present the experimental set-up and procedures for both series of experiments evaluating the benefits of haptic and visual feedback in laparoscopic instrument navigation.

2.1. Experimental Set-up and Task

We placed subjects in front of a laparoscopic trainer (EndoSim LaproTrain, see Figures 1 and 2 for details) and let them manipulate standard laparoscopic forceps inserted through 5-mm trocars while observing the endoscopic image on a 24″ screen placed directly in front of them. Three different sized pegs (A, B, and C, respectively – see Figure 1) were set up vertically within the trainer so that their tips formed a steeply inclined plane similar to a resection plane for a hepatectomy.

FIGURE 1

Figure 1. Schematic of the laparoscopic trainer containing the three pegs (A–C) forming an inclined plane with their tips. The subject attempts to follow a given trajectory between the peg tips (dark blue line) without deviating from the target plane. Normal deviation from the target (d, in millimeter) is computed at every instant and fed back to the subject as per the current feedback condition. The plot on the right shows an example of a deviation curve obtained for the blue trajectory drawn on the left. A fixed cylindrical obstacle is placed on the [AC] trajectory segment to prevent straight line movements between both pegs and complicate the task for the user.

FIGURE 2

Figure 2. Setup as seen from the point of view of the subject, with detail of the three means for providing feedback.

Subjects were tasked with following random trajectories beginning at one peg, passing through both other pegs before returning to the peg of origin (see Figure 1). They were instructed to thereby keep the tip of their instrument as close as possible to the target plane while trying to finish the trial as quickly as possible.

The laparoscopic forceps were fitted with passive optical markers allowing for tracking of the instrument handle position in space thanks to an NDI Polaris optical tracking system (25 Hz frame rate, accurate to <1 mm). For experimental conditions involving a robotic co-manipulator (see section 1 for a detailed listing of experimental conditions), the instrument handle was attached to the wrist of a 6-degrees of freedom (DoF) haptic interface (Haption Virtuose 6D) capable both of applying forces to the instrument and recording its position in space. Instrument handle position data were recorded through custom software, which allowed for real-time computing of the instrument tip position in space thanks to a prior calibration step, which provided us with the transformation matrix between instrument handle and tip.

The positions of the three-peg tips in the laparoscopic trainer as well as the position of the insertion point (trocar position) were recorded prior to the experiments, allowing for computation of the normal deviation of the instrument tip with respect to the target plane and thus enabling the generation of appropriate feedback signals indicating this deviation to the subject.

A fixed physical obstacle consisting of an 80-mm diameter plastic cylinder was placed on the trajectory segment between pegs A and C so as to prevent subjects from following a simple straight line trajectory on this segment with the hopes of highlighting respective benefits and disadvantages of various feedback conditions in assisting following of non-intuitive trajectories.

At the end of the experiments series, the instrument tip position data and the computed deviations were analyzed using Matlab as per evaluation metrics presented in the following in section 2.

2.2. Sample Populations and Trial Randomization

In a first experiment (Experiment 1 in the following) discussed in part in Howard and Szewczyk (2014), 23 (16 males and 7 females) novice right-handed subjects performed 5 repeats of the task presented previously for the 11 different feedback conditions (i.e., a total of 55 trajectories per subject) partially listed in Table 1. For practical reasons, conditions were grouped in three blocks: open surgery (1 condition), laparoscopic surgery (6 conditions), and robot-assisted laparoscopic (4 conditions) (see Table 1 in the following for details). Subjects began with the open surgery block, the order of the subsequent blocks was randomized and the order of the conditions within the blocks was fully randomized. This procedure was chosen in order to minimize bias from learning effects and fatigue, under the assumption that the task ergonomics sufficiently differed from one block to another to not require full randomization of all 11 conditions.

TABLE 1

Table 1. Evaluated feedback conditions.

A second experiment (Experiment 2 in the following) was conducted to take a closer look at improvements observed in experiment 1 when providing visual feedback, continuous vibrotactile feedback, and their combination (only six of the initial 11 conditions were analyzed further – these are marked with a (*) in Table 1). As the number of conditions was reduced, subjects were able to complete 6 trajectories per condition. The order in which subjects performed the six conditions was fully randomized in order to minimize potential influence of short-term learning effects on our results. For this experiment, we recruited a new sample of 11 (8 male, 3 female) novice, right-handed subjects, with no previous experience in laparoscopy or with our experiments.

For exploratory purposes, an intern with extensive laparoscopic surgery training (male, right-handed, age 28) was asked to complete 10 trajectories, respectively, for the following conditions (see Table 1 for details):

• laparoscopic surgery without provision of feedback (L),

• laparoscopic surgery with added visual feedback of current deviation (V),

• laparoscopic surgery with added tactile feedback of current deviation (T),

• laparoscopic surgery with added combined tactile and visual feedback of current deviation (TV),

• laparoscopic surgery with active robotic co-manipulator implementing virtual fixtures (K),

• and laparoscopic surgery with active robotic co-manipulator implementing virtual fixtures with added visual feedback of current deviation (KV), in order to gain insights into the generalizability of our results to a population of surgeons.

All subjects received detailed explanations on the experimental protocol and presented forms of feedback prior to beginning the experiments and provided written informed consent for their participation.

2.3. Evaluated Forms of Feedback

Feedback informed the users of their normal deviation to the plane in various manners. We consider conditions as being “without feedback” when the user is presented with only the endoscopic image. Table 1 lists the feedback conditions relevant to our current analysis.

Visual feedback was provided in the form of a horizontal bargraph displayed on the screen (see Figure 2 top left). The bargraph consisted of a horizontal grey container displayed beneath the endoscopic image, within which a green bar changed length either to the left or right starting from the center so as to display the current deviation magnitude, which was also shown as a numerical value in [millimeter] at the center of the bargraph. The absence of a green bar signaled the fact that the user was on target, i.e., at a deviation between −1 mm and +1 mm from the plane. Otherwise, the green bar length varied continuously between 0 and its maximum length (i.e., half the gray container length) to indicate deviation.

Cutaneous vibrotactile feedback was provided to the user via an Eccentric Rotating Mass (ERM) motor [Precision Microdrives™ Pico Vibe 307-100 (Microdrives, 2014)] strapped to the inner side of the index finger holding the instrument (see Figure 2 right). This placement is interesting in the context of integration of vibrotactile feedback to the handle of serial co-manipulators for laparoscopic surgery. The distance to the target plane was encoded as a linear increase in vibration intensity, proportional to the magnitude of the deviation (range: 0–7 g for deviations from 0 to 30 mm). For ERM motors, vibration amplitude and frequency are inherently linked, so that the frequency of the vibrotactile feedback varied almost linearly between 25 and 260 Hz (Microdrives, 2014) (for detailed information on the amplitude/voltage and frequency/voltage relationships for the employed ERM motor).

We implemented soft guidance virtual fixtures using a Haption Virtuose 6D™ haptic interface setup as a parallel co-manipulator (see Figure 2 bottom left). The haptic interface was attached to the instrument just below the handle held by the user and applied forces in order to guide the instrument tip toward the plane. These forces were calculated as per equation (1) in order to simulate a spring (k = 400 N/m) attached between the instrument tip and the plane.

$F_{\mathit{\text{wrist}}}^{\to }=l_{\mathit{\text{out}}}/l_{\mathit{\text{in}}}\cdot (-k\cdot \overrightarrow{n}\cdot d)$ (1)

where l_out and l_in, respectively, denote the lengths of instrument shaft inside and outside of the point of insertion (calculated based on the robot wrist position, the known instrument length and the trocar position defined in the robot coordinate system prior to the experiment), d is the current deviation from the plane and $\overrightarrow{n}$ is the plane’s normal vector. The choice of a relatively low stiffness (i.e., soft virtual fixtures) was made with the clinical imperative of leaving the surgeon in control of their action in mind, as minor deviations from the pre-operative plan may sometimes be necessary. This also has the advantage of keeping the virtual fixtures mainly informative at low deviations, for better comparison with the other forms of feedback.

We evaluated forms of feedback listed in Table 1 below, along with two unlisted tactile feedback conditions in the laparoscopic block (omitted for lack of any positive results) and two conditions in the robot-assisted block (inactive haptic interface, with and without visual feedback, with the purpose of verifying the transparency of the haptic interface with regards to task execution).

2.4. Feedback Thresholds

Pilot tests showed that maximum deviations around 30 mm were attained when performing the task in a laparoscopic setting without feedback (L), hence the choice of this value as the maximum displayed deviation, be it for the bargraph or tactile feedback (see section 2 for details on feedback implementation). The optical tracking system provided us with measurement accuracies just below 1 mm around the instrument tip position, thus any deviation computed as below 1 mm was considered as on-target. The feedback levels varied linearly from minimum to maximum over the resulting range between 1 and 30 mm deviation from the target. To allow for good comparison, the stiffness of the virtual spring implemented by the haptic interface was chosen so as to not allow any deviation beyond 30 mm.

2.5. Evaluation Metrics

Resulting trajectories were analyzed both in terms of precision and time criteria. Relevant precision criteria encompassed both:

• on-target precision (using a “relative Time on Target” (rToT) score, defined as the percentage of TCT (Task Completion Time) during which the instrument tip was under 1 mm normal deviation from the target plane),

• and amplitude of deviations (i.e., maximum peak-to-peak normal deviations).

The evaluated time criterion was:

• TCT (Task Completion Time), which serves as an adequate measure of task completion speed since the nominal trajectory length did not vary between trials.

In experiment 2, a questionnaire (see Table 2) presented as a five-point Likert scale was filled out by subjects after performing the task in each feedback condition. The results are detailed and discussed in section 4 and shown in the form of graphs in Figure 6.

TABLE 2

Table 2. Questionnaire filled out by subjects after performing the task in each condition.

2.6. Statistical Analyses

We studied mean performances of each subject for each feedback condition, providing us with 23 observations per condition for experiment 1 and 11 observations per conditions for experiment 2. As a general rule, data for each condition were not normally distributed but only slightly skewed and sample variance was large enough for us to assume unequal population variances. Although samples are relatively small, the limited skewness led us to provide t-intervals for our estimation of 95% confidence intervals for population means (shown as blue vertical lines in Figures 3, 4 and 5) and calculate statistical significance of observed differences in means using paired-sample t-tests with subsequent Bonferroni correction of the obtained p-values.

FIGURE 3

Figure 3. Mean subject rToT (in percentage) for each condition. Results from experiment 1 are shown in yellow, results from experiment 2 in orange, performances for the intern in green; data points shown as black crosses, interquartile range shown by colored boxes, sample means as horizontal red lines and confidence intervals for population means as vertical blue lines. The spread of the intern’s performances is shown by the vertical green bars, his mean performance by the green circle with blue contour.

FIGURE 4

Figure 4. Mean subject DA (in millimeter) for each condition. Results from experiment 1 are shown in yellow, results from experiment 2 in orange, performances for the intern in green; data points shown as black crosses, interquartile range shown by colored boxes, sample means as horizontal red lines and confidence intervals for population means as vertical blue lines. The spread of the intern’s performances is shown by the vertical green bars, his mean performance by the green circle with blue contour.

FIGURE 5

Figure 5. Mean subject TCT (in seconds) for each condition. Results from experiment 1 are shown in yellow, results from experiment 2 in orange, performances for the intern in green; data points shown as black crosses, interquartile range shown by colored boxes, sample means as horizontal red lines and confidence intervals for population means as vertical blue lines. The spread of the intern’s performances is shown by the vertical green bars, his mean performance by the green circle with blue contour.

3. Results

In order to verify that the co-manipulated setting (Robot-assisted condition K, with added visual feedback KV) did not differ significantly from standard laparoscopic setting (L, V, T, TV), we evaluated subject performance when manipulating a tool attached to an inactive haptic interface, with and without visual feedback (R, RV). No significant differences were observed between conditions R (inactive robotic co-manipulator) and L (laparoscopic reference condition) and RV (inactive robotic co-manipulator with added visual feedback) and LV (laparoscopic surgery with added visual feedback), respectively for all of the metrics considered, leading us to conclude that any observed differences between K (Robot-assisted condition), KV (Robot-assisted condition with added visual feedback), and the conditions in the laparoscopic block are likely to be due to the action of the virtual fixtures. Because of this and for the sake of clarity, the results presented below omit the observations for conditions R (inactive robotic co-manipulator) and RV (inactive robotic co-manipulator with added visual feedback).

3.1. Precision Criteria – Mean Relative Time Spent on Target

Figure 3 shows the results obtained for on-target precision, measured by the percentage of TCT spent at deviations below 1 mm (rToT). Results are grouped by experimental condition, with results from experiment 1 shown by the yellow boxplots, and those from experiment 2 shown by the orange boxplots. The intern’s mean performances are shown with green dots for comparison. Sample means are shown by the thick horizontal red lines, accompanied by confidence intervals for population means shown by vertical blue lines. The actual data points are shown by the black crosses, and interquartile ranges for the respective samples by the colored boxes.

Results are coherent between experiments, although subjects performed worse in the mean for condition L (laparoscopic surgery without feedback) in the second experiment, leading to greater relative improvements obtained from feedback.

In condition O (open surgery without feedback), subjects perform the task with moderate precision (around 35% time spent on target). Precision is greatly reduced for the task in condition L (between 15 and 25% time spent on target), and introduction of solely informative visual, tactile, or combined feedback (V, T, TV) tends to improve precision once again without returning on-target precision to levels obtained in the open setting (O).

The use of soft guidance virtual fixtures greatly increases on-target precision (around 50% of TCT spent on target), with performances significantly improved even over that of condition O.

Concerning the intern, it is surprising to note that for conditions L (laparoscopic surgery, no feedback), V, T, and TV (laparoscopic surgery with visual, tactile, or combined feedback, respectively), he shows performances close yet below average of those of novice subjects despite his experience in laparoscopy. However, his performances in conditions with feedback tend to follow a similar trend to those observed in novice subjects, and improvements from soft guidance virtual fixtures are particularly marked in his case. Of course these results are anecdotal in comparison with those presented for our novice subjects and cannot serve as a basis for any statistically valid conclusions.

Table 3 sums up the observed differences in means between conditions along with associated statistical significance.

TABLE 3

Table 3. Differences in mean relative Time on Target (rToT) between conditions and associated statistical significance.

3.2. Precision Criteria – Mean Deviation Amplitude

Figure 4 shows the results obtained for our second precision criterion – mean deviation amplitudes (DA), using the same boxplot layout as discussed in the previous section (Results from experiment 1 in yellow, results from experiment 2 in orange, mean performances for intern in green; data points shown as black crosses, interquartile range shown by colored boxes, sample means as horizontal red lines and confidence intervals for population means as vertical blue lines).

As expected, the passage from open to laparoscopic surgery leads to a great degradation in precision (increase in deviation amplitudes). Providing solely informative feedback (conditions L, V and TV) tends to improve performance over the reference laparoscopic condition (L), although this is only consistently significant for combined visual and tactile feedback (TV). Surprisingly, deviation amplitudes are generally smaller for condition L in experiment 2 despite the fact that subjects tended to move at higher speeds [see following section on Task Completion Times (TCT)] and had worse on target precision [see section on relative Time on Target (rToT)] than in experiment 1. This is not the case for the conditions with feedback (V, T, TV), where the lower deviation amplitudes also came at the cost of prolonged Task Completion Times (TCT). The use of soft guidance virtual fixtures significantly improves performance both over the reference laparoscopic condition (L) and even the reference open surgery condition (O).

Again, the intern performs more or less on par with novices regarding this criterion, with his mean performances either at or slightly above those of novice subjects, except for conditions K and KV (Robot-assisted laparoscopy, respectively, without and with added visual feedback) where his deviation amplitudes are exceptionally low. Once again, these results remain anecdotal and do not carry the same significance as those obtained with the novice subjects.

Table 4 sums up the observed differences in means between conditions along with associated statistical significance.

TABLE 4

Table 4. Differences in mean deviation amplitudes (DA) between conditions and associated statistical significance.

3.3. Speed Criteria – Time to Complete Task

Performances in terms of speed (evaluated as TCT – Task Completion Time) are shown in Figure 5, using the same boxplot layout as discussed in the previous section (results from experiment 1 in yellow, results from experiment 2 in orange, mean performances for intern in green; data points shown as black crosses, interquartile range shown by colored boxes, sample means as horizontal red lines, and confidence intervals for population means as vertical blue lines).

Once again, results appear coherent between experiments, though subjects performed the task faster in the mean for condition L in experiment 2, which may explain the difference in performance between experiments with regards to precision for this condition.

In condition O (open surgery without feedback), subjects naturally performed the task at high speeds [12.5 s mean Task Completion Time (TCT)], and moving to laparoscopic setting greatly reduced task execution speed (mean TCT between 33.2 s and 42.4 s in condition L [laparoscopic surgery without feedback)]. The addition of solely informative visual, tactile, or combined feedback (conditions V, T, and TV) again reduced task execution speed when compared with the reference laparoscopic condition (L). Finally, the use of soft guidance virtual fixtures both without and with added visual feedback (conditions K, KV) slightly increased mean task execution speeds, although these were still far from those obtained in the reference open condition (O).

The intern, however, performed the task notably faster than novice subjects in all conditions. Also, the reductions in speed observed when providing visual, tactile, or combined feedback (conditions V, T, and TV) are very limited when compared to novice performances. As with both previous criteria, these results remain anecdotal and do not carry the same significance as those obtained with the novice subjects.

Table 5 sums up the observed differences in means between conditions along with associated statistical significance.

TABLE 5

Table 5. Differences in mean task completion time (TCT) between conditions and associated statistical significance.

3.4. Influence of Trajectory Complexity

The physical obstacle placed on the trajectory segment between pegs A and C (see Figure 1) forced the subjects to follow a highly non-intuitive curved trajectory instead of the easier straight line on said segment. We observed no notable differences in tip movement speeds between the segments without an obstacle ([AB] and [BC]) and the segment with the obstacle ([AC]). As expected, the introduction of the obstacle led to an increase in mean deviation amplitudes and a reduction of mean relative Time on Target (rToT) (though never significant). Contrary to our initial expectation, we obtained no difference in improvements from feedback between segments with and without the obstacle.

3.5. Confounding Factors

As we are aware of evidence supporting possible influence of factors such as musicianship and experience with video games and Virtual Reality (VR) (Lynch et al., 2010) on performances in laparoscopic surgery or similar tasks, we compared performances for each of the metrics presented in section 1 while grouping subjects, respectively, by gender, musicianship, experience with video games, and prior experience with haptic interfaces.

Our sample populations were distributed as follows according to the criteria mentioned above.

3.5.1. Experiment 1

Gender: 16 males, 7 females; experience with video games and VR: 11 experienced, 12 inexperienced; musicianship: 6 musicians, 17 non-musicians; experience with haptic interfaces: 14 experienced, 9 inexperienced.

3.5.2. Experiment 2

Gender: 7 males, 4 females; experience with video games and VR: 7 experienced, 4 inexperienced; musicianship: 3 musicians, 8 non-musicians; experience with haptic interfaces: 6 experienced, 5 inexperienced.

No notable differences were observed between groups for the three metrics we considered, i.e., relative Time on Target (rToT), Deviation Amplitudes (DA), and Task Completion Times (TCT). Thus, we treated our populations as homogeneous in the following analysis of results.

4. Discussion

4.1. Differences in Performances between Experiments

The most notable differences observed between experiments are those in the baseline performance in laparoscopic surgery (condition L): in experiment 2, subjects perform the task faster in the mean (lower Task Completion Times) but with less on-target precision (lower relative Time on Target) than in experiment 1. This can be explained by differences in the distributions of speed-accuracy trade-off, which, however, does not amount to the claim that subjects in experiment 2 consistently chose a different speed-accuracy trade-off when compared to experiment 1 as the datasets for both experiments significantly overlap.

A surprising observation in the reference laparoscopic condition (L), however, is the fact that although subjects in experiment 2 performed the task faster than in experiment 1, they also displayed far less spread in their Deviation Amplitudes (DA), resulting in better precision than for experiment 1 concerning this criterion. We therefore believe that there may have been fatigue effects or a possible negative after-effect from performing in the reference open surgery condition (O) beforehand in experiment 1 (it should be noted that the number of task repetitions was only 5 in experiment 1, so any after-effect would probably have consequences visible in the results). These factors could negatively influence the spread in deviation amplitudes and rToT in experiment 1, and the redesign of the protocol in experiment 2 would have remedied this.

The improvement patterns in the means between conditions L, V, T, TV (respectively: laparoscopic surgery without, then with visual, tactile, and combined feedback) are consistent between both experiments for all three criteria [relative Time on Target (rToT), Deviation Amplitude (DA), and Task Completion Time (TCT)] despite being quantitatively different, leaving us confident that they are a good basis for the conclusions drawn in the present paper.

Concerning the quantitative nature of these differences, the general trend is toward spread in performance being equal or lower in experiment 2 compared to experiment 1 for any given condition and criterion pair. Furthermore, differences in means between condition pairs for a criterion are also larger or equal in experiment 2 when compared to experiment 1. This is why we believe the protocol in experiment 1 introduced a significant fatigue bias, which would tend to increase spread in subject performances and thereby mask observable differences in mean performance introduced by the differences in the provided feedback.

4.2. Degradation in Performance between Open and Laparoscopic Surgery

Relative Time on Target (rToT) is significantly worse in all laparoscopic conditions (except robot-assisted laparoscopic without and with visual feedback – K and KV) than in the reference open surgery condition (O) though spread is largely reduced. In laparoscopic settings, whether informative feedback is presented or not, it is significantly harder for a novice subject to maintain the tip of the instrument close to the target while moving than it is in open surgery. This is understandable when considering the limitations in dexterity discussed in the introduction. The larger spread in condition O can be attributed to the fact that though it is easier to maintain the instrument tip on target, condition O also allows faster movement between pegs, which can easily exacerbate errors in on-target accuracy. The reduction in possible movement speeds between open and laparoscopic surgery appears particularly clearly in the significant increase in Task Completion Times (TCT) between conditions O and the reference laparoscopic condition L (1.65× and 2.38×, respectively), which cannot be explained by additional attention to precision [significant reductions in relative Time on Target (rToT) as well as increases in Deviation Amplitudes (DA)].

4.3. Improvement of Performance through Tactile and Visual Feedback

The introduction of solely informative visual, tactile, and combined feedback (conditions T, V, and TV) leads to consistent improvements in terms of precision (both on-target and concerning deviation amplitudes) but at the cost of reduced task execution speeds for novice subjects. This is to be expected as the availability of feedback leads subjects to pay more attention to their deviation and take the time necessary in correcting the movement of their instrument.

Deviation Amplitudes (DA) are significantly reduced in laparoscopic settings thanks to provision of combined visual and tactile feedback (TV). This leads us to conclude that both forms of feedback either effectively communicate information allowing for correction through redundancy over different sensory channels or that both forms of feedback somehow compensate for each other’s weaknesses when combined. The observed minor drop in task execution speed observed in condition TV when compared to the tactile feedback condition (T) or the visual feedback condition (V) is associated with better performances in terms of both precision criteria, thus hinting at a greater focus on precision objectives rather than speed objectives when redundant multi-modal feedback was available.

The intern showed improvement patterns comparable to those observed in novices yet with only a very limited degradation in task execution times. This would seem to indicate that although the intern favored a speed-accuracy trade-off where task execution speed was prioritized, he was able to take advantage of the provided information to improve his precision without negative impacts on his speed. As such, it would appear that exclusively informative feedback may be of most use to experienced laparoscopic surgeons.

Based on our questionnaire evaluating user perception of the benefit of this feedback (see Figure 6 for complete results), combining vibrotactile with visual feedback (TV) seems to significantly correlate with a drop in self-assessed performance [mean score drop by 0.1 (p < 0.05)] despite the improvement in measured performance. These results may indicate a negative effect from a perceived excess of information in condition TV, leading to more mental fatigue and perceived confusion for the user. The visual feedback condition (V) was evaluated as being significantly clearer than the tactile feedback condition T [mean score difference of 0.18 (p < 0.05)], which in turn outperformed condition TV (though not significantly). Condition TV was evaluated as significantly easier to use than condition T [mean score difference of 0.1 (p < 0.05)], but harder to use than condition V [mean score difference of 0.27 (p < 0.01)]. This result could also reflect the perceived complexity of dealing with tactile and visual cues simultaneously.

FIGURE 6

Figure 6. Subjective evaluation of feedback and performance as per the questionnaire described in Table 2. The results for question 3 “Did you understand the feedback” are omitted as the question served the purpose of evaluating the reliability of other results and was consistently answered with “Agree” (4) or “Strongly agree” (5). Median responses for each condition are shown in blue, and the distribution of responses is shown through red circles varying in diameter proportionally to the number of responses collected at each response level for each condition (the larger the circle, the more responses collected – see legend on figure for precise numeric significance). (A) shows the distribution of responses for Question 1 of Table 2 (“I felt the task was difficult to perform in this condition”), (B) shows the distribution of responses for Question 2 of Table 2 (“I believe I performed well in this condition”), (C) shows the distribution of responses for Question 4 of Table 2 (“I felt the feedback helped me accomplish the task”), (D) shows the distribution of responses for Question 6 of Table 2 (“I was disturbed by the feedback”), (E) shows the distribution of responses for Question 7 of Table 2 (“I felt the feedback was easy to use”), and (F) shows the distribution of responses for Question 5 of Table 2 (“I thought the feedback was intuitive”).

4.4. Improvement of Performance through Co-Manipulation

The use of a parallel co-manipulator implementing soft virtual fixtures improved both novices and the intern’s performance in terms of speed and accuracy. This is understandable as subjects are physically limited in their deviation from the target and may relinquish a degree of control to the robotic co-manipulator in order to achieve high precision without needing to slow their movements.

The addition of visual feedback to the robotic assistance through virtual fixtures (condition KV) led to insignificant improvements over condition K (robotic assistance without visual feedback), leading us to believe that in such a setup virtual fixtures are sufficient and there is little to be gained from multimodal feedback.

For both conditions K and KV, the intern showed greater relative improvements in performance (both for speed and accuracy) than those observed with novice subjects, hinting at a possible greater benefit of robotic assistance for experienced users.

Although performance was far better in conditions K and KV than in all other feedback conditions, it should be noted that providing such feedback in the operating room is limited by the cost of the devices used, potential safety issues raised through the shared control between robot and surgeon and problems of clutter and installation times in the operating room. If the necessary hardware becomes available in operating rooms and the safety issues that arise are dealt with, there is no doubt that such assistance is optimal, but until then forms of feedback similar to those presented in conditions V, T, and TV [visual, tactile, and combined feedback] may be interesting low-cost and easily implemented alternatives for improving surgeon performance.

4.5. Benefits of Feedback in Following Non-intuitive Trajectories

We observed no difference in the effect of providing feedback between performances on trajectory segments without obstacles and with obstacles. This may be due both to insufficient complexity of the obstacle and to novices relatively poor manipulation skills, which likely prevent them from getting the most out of the feedback provided.

5. Conclusion

In this paper, we confirm previous results indicating that in a 1D guidance task, visual and cutaneous vibrotactile feedback as well as their combination leads to improved performances in terms of precision at the cost of increased TCTs. Our shortened and fully randomized experimental protocol minimized contributions from learning effects in the observed differences.

We also compared data for novice subjects with an intern’s performances, showing no significant differences in terms of precision but a significantly lower Task Completion Times (TCT) at equal precision. Overall, the patterns of improvement over the reference condition obtained in novice subjects for conditions V, T, TV, K, and KV (respectively: visual, tactile, combined visual and tactile, robot assistance without and with visual feedback) can be found again in the intern’s performance, leading us to believe in a good chance of our results being generalizable to a population of surgeons with similar results. Interestingly, the intern performed better in the tactile conditions T (tactile) and TV (tactile with added visual feedback) than in condition V (visual feedback alone), which may indicate a lower cognitive load when using feedback presented though tactile cues instead of visual cues. Similarly to our previous experiments, we note that visual feedback still seems beneficial, particularly in avoiding larger deviation amplitudes. Finally, the intern’s TCTs seemed much less affected by the provision of feedback, which stands out as a particularly interesting feature when considering clinical applications of such feedback.

Analysis of novice subject’s perception of the usability and impact of the provided feedback on their performance revealed that presence or lack of feedback does not seem to have any significant effect on the perceived difficulty of the task. The self-assessed performance of the users is, however significantly improved in the visual feedback condition V over the combined visual and tactile feedback condition TV, indicating a potentially disturbing effect from the excess of information provided in condition TV. Overall, the subjects seemed to understand the feedback well in all conditions, with significantly better understanding reported for the visual feedback condition V when compared to the tactile feedback condition T. We believe this to be linked to the fact that our visual feedback through the bar-graph provided additional directional information whereas the vibrotactile feedback only provided information on the magnitude of the deviation from the target and was harder to interpret. When comparing perceived assistance from the various forms of feedback, all conditions seem to do equally well. All forms of feedback are perceived as equally easy to use, with the exception of the combined visual and tactile feedback condition TV, which scores significantly lower than the tactile feedback condition T, hinting at complexity arising from an excess of information. This is reflected in the perceived intuitiveness of the provided feedback, where TV again scores significantly lower than the visual feedback condition V. When asked about potential disturbances in task execution arising from the feedback, subjects tend to be undisturbed, with no significant differences between forms of feedback.

These promising initial results for the use of cutaneous feedback are leading us to consider extending the evaluation of such forms of feedback to more complex guidance tasks (i.e., 2D and 3D trajectories), while performing a comparative evaluation of various forms of tactile feedback in order to improve performance and intuitiveness of the feedback. Furthermore, there was little to no contact between the instrument and structures placed within the trainer in this study. In order to assess the viability of such forms of feedback for clinical applications, we are currently working on evaluating their use in tasks involving physical interaction within the trainer, e.g., dissection or suturing tasks. Developing an experimental protocol around a more complex task involving actual interaction with the environment within the laparoscopic trainer will also require subjects to focus more on the task at hand and the laparoscopic image, thus highlighting potential effects of various forms of feedback on the attention given to the task being performed. Finally, we aim to test generalizability of our results to a population of surgeons.

Author Contributions

All authors participated in the design of the presented experiments, analyses of the results, and drafting and revision of the article. The experiments and data analyses were conducted by TH. Both authors agree to be accountable for all aspects of the work.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This work was supported by French state funds managed by the ANR within the Investissements d’Avenir programme (Labex CAMI) under reference ANR-11-LABX-0004.

References

Bark, K., Khanna, P., Irwin, R., Kapur, P., Jax, S. A., Buxbaum, L. J., et al. (2011). “Lessons in using vibrotactile feedback to guide fast arm motions,” in World Haptics Conference (WHC), 2011 IEEE (Istanbul: IEEE Comput. Soc), 355–360.

Google Scholar

Bholat, O. S., Haluck, R. S., Murray, W. B., Gorman, P. J., and Krummel, T. M. (1999). Tactile feedback is present during minimally invasive surgery. J. Am. Coll. Surg. 189, 349–355. doi:10.1016/S1072-7515(99)00184-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Bluteau, J., Dubois, M.-D., Coquillart, S., Gentaz, E., and Payan, Y. (2010). “Vibrotactile guidance for trajectory following in computer aided surgery,” in 32nd Annual International Conference of the IEEE EMBS (Buenos Aires: Institute of Electrical and Electronics Engineers (IEEE)), 2085–2088.

Google Scholar

Brell, M., and Hein, A. (2007). Positioning tasks in multimodal computer-navigated surgery. Multimedia Signal Process. Syst. Healthc. Life Sci. 14, 42–51. doi:10.1109/MMUL.2007.81

CrossRef Full Text | Google Scholar

Causo, A., Tran, L. D., Yeo, S. H., and Chen, I.-M. (2012). “Vibrotactile motors on stationary arm as directional feedback to correct arm posture,” in IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (Kaohsiung: Institute of Electrical and Electronics Engineers (IEEE)), 202–207.

Google Scholar

Deziel, D. J., Millikan, K. W., Economou, S. G., Doolas, A., Ko, S. T., and Airan, M. C. (1993). Complications of laparoscopic cholecystectomy: a national survey of 4,292 hospitals and an analysis of 77,604 cases. Am. J. Surg. 165, 9–14. doi:10.1016/S0002-9610(05)80397-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Ege, E. S., Cetin, F., and Basdogan, C. (2011). “Vibrotactile feedback in steering wheel reduces navigation errors during GPS-guided car driving,” in IEEE World Haptics Conference, WHC 2011 (Istanbul: Institute of Electrical and Electronics Engineers (IEEE)), 345–348.

Google Scholar

Elliott, L. R., van Erp, J. B. F., Redden, E. S., and Duistermaat, M. (2010). Field-based validation of a tactile navigation device. IEEE Trans. Haptics 3, 78–87. doi:10.1109/TOH.2010.3

CrossRef Full Text | Google Scholar

Feng, C., Rozenblit, J. W., Hamilton, A. J., and Wytyczak-Partyka, A. (2009). “Defining spatial regions in computer-assisted laparoscopic surgical training,” in Proceedings of the International Symposium and Workshop on Engineering of Computer Based Systems (Logan, UT: Institute of Electrical and Electronics Engineers (IEEE)), 176–183.

Google Scholar

Feygin, D., Keehner, M., and Tendick, R. (2002). “Haptic guidance: experimental evaluation of a haptic training method for a perceptual motor skill,” in Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002 (Orlando, FL: Institute of Electrical and Electronics Engineers (IEEE)), 40–47.

Google Scholar

Franco, D. (2001). Right Hepatectomy. Available at: http://www.websurg.com/doi-ot02en155.htm

Google Scholar

Grindlay, G. C. (2006). The impact of haptic guidance on musical motor learning. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA.

Google Scholar

Hansen, C., Black, D., Lange, C., Rieber, F., Lamadé, W., Donati, M., et al. (2013). Auditory support for resection guidance in navigated liver surgery. Int. J. Med. Robot. 9, 36–43. doi:10.1002/rcs.1466

PubMed Abstract | CrossRef Full Text | Google Scholar

Holland, S., Bouwer, A. J., Dalgleish, M., and Hurtig, T. M. (2010). “Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit,” in Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (Cambridge, UK: ACM Press), 21–28.

Google Scholar

Howard, T., and Szewczyk, J. (2014). “Visuo-haptic feedback for 1-d guidance in laparoscopic surgery,” in Biomedical Robotics and Biomechatronics 2014 5th IEEE RAS EMBS International Conference On (Saõ Paulo: Institute of Electrical and Electronics Engineers (IEEE)), 58–65.

Google Scholar

Joice, P., Hanna, G. B., and Cuschieri, A. (1998). Errors enacted during endoscopic surgery–a human reliability analysis. Appl. Ergon. 29, 409–414. doi:10.1016/S0003-6870(98)00016-7

CrossRef Full Text | Google Scholar

Li, J., Ji, Z., and Li, Y. (2014). Comparison of laparoscopic versus open procedure in the treatment of recurrent inguinal hernia: a meta-analysis of the results. Am. J. Surg. 207, 602–612. doi:10.1016/j.amjsurg.2013.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, M., Ishii, M., and Taylor, R. H. (2007). Spatial motion constraints using virtual fixtures generated by anatomy. IEEE Trans. Robot. 23, 4–19. doi:10.1109/TRO.2006.886838

CrossRef Full Text | Google Scholar

Li, M., Konstantinova, J., Secco, E. L., Jiang, A., Liu, H., Nanayakkara, T., et al. (2015). Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue. Med. Biol. Eng. Comput. 53, 1177–1186. doi:10.1007/s11517-015-1309-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Lynch, J., Aughwane, P., and Hammond, T. M. (2010). Video games and surgical ability: a literature review. J. Surg. Educ. 67, 184–189. doi:10.1016/j.jsurg.2010.02.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Microdrives, P. (2014). PicoVibe 307-100. Available at: https://www.precisionmicrodrives.com/sites/default/files/307-100-datasheet.pdf.

Google Scholar

Nisky, I., Huang, F., Milstein, A., Pugh, C. M., Mussa-Ivaldi, F. A., and Karniel, A. (2012). “Perception of stiffness in laparoscopy – the fulcrum effect,” in Studies in Health Technology and Informatics, Vol. 173, 313–319.

Google Scholar

Picod, G., Jambon, A. C., Vinatier, D., and Dubois, P. (2005). What can the operator actually feel when performing a laparoscopy? Surg. Endosc. 19, 95–100. doi:10.1007/s00464-003-9330-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodríguez, J., Gutiérrez, T., Casado, S., and Sánchez, E. J. (2010). “Training strategies for learning a 3D trajectory with accuracy,” in IEEE International Symposium on Haptic Audio-Visual Environments and Games (HAVE), 2010 (Phoenix, AZ: Institute of Electrical and Electronics Engineers (IEEE)), 1–6.

Google Scholar

Rosen, J., Hannaford, B., MacFarlane, M. P., and Sinanan, M. N. (1999). Force controlled and teleoperated endoscopic grasper for minimally invasive surgery–experimental performance evaluation. IEEE Trans. Biomed. Eng. 46, 1212–1221. doi:10.1109/10.790498

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Erp, J., Jansen, C., Dobbins, T., and Van Veen, H. (2004). “Vibrotactile waypoint navigation at sea and in the air: two case studies,” in Proceedings of EuroHaptics (Munich: Institute of Automatic Control Engineering), 166–173.

Google Scholar

Wurdemann, H. A., Secco, E. L., Nanayakkara, T., Althoefer, K., Lis, K., Mucha, L., et al. (2013). “Mapping tactile information of a soft manipulator to a haptic sleeve in RMIS,” in 3rd Joint Workshop on New Technologies for Computer/Robot Assisted Surgery (CRAS 2013), Verona.

Google Scholar

Xin, H., Zelek, J. S., and Carnahan, H. (2006). “Laparoscopic surgery, perceptual limitations and force: a review,” in First Canadian Student Conference on Biomedical Computing, Kingston, ON.

Google Scholar

Yao, H.-Y., Hayward, V., and Ellis, R. E. (2004). “Chapter A tactile magnification instrument for minimally invasive surgery,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2004: 7th International Conference, Saint-Malo, France, September 26-29, 2004. Proceedings, Part II, eds C. Barillot, D. R. Haynor, and P. Hellier (Berlin, Heidelberg: Springer Berlin Heidelberg), 89–96.

Google Scholar