A Bio-inspired Grasp Stiffness Control for Robotic Hands

1. Introduction

The human hand shows a high level of dexterity that allows to perform numerous complex tasks. Therefore, it is often used as inspiration for designing robots to operate in human environments.

The five-fingered Shadow Dexterous Hand^TM1 and the Awiwi Hand (Grebenstein, 2014) are examples of such a bio-inspired conception. Their anthropomorphic design endows the robotic hands with a similar kinematic workspace to that of a human hand. Moreover, these robots use actuation and perception systems that allow to imitate the dynamic movements of human hands and to grasp a wide variety of objects. Nevertheless, the required number of sensing and actuation units in such poly-articulated hands has contributed to an increased cost and complexity in their manufacturing.

With the aim to introduce principled simplification strategies in the design or control of robotic hands, recent works use underactuated and compliant mechanisms. These are endowed with compliant elements and fewer actuators than degrees of freedom, proving to be an effective way to reduce the design complexity of robotic hands and prostheses, while maintaining their bio-inspired shape (Controzzi et al., 2017; Mottard et al., 2017).

Alternatively, other approaches reproduced the human ability to generate coordinated movements in the joint space using the concept of postural synergies (Santello et al., 1998). These synergies have been tackled not only from the hardware perspective (Ajoudani et al., 2014; Catalano et al., 2014), but also from the software viewpoint (Gioioso et al., 2013).

In this paper, we focus on another important feature of human grasp which so far has drawn less attention: a principled simplification of grasp stiffness control. Previous works like Cutkosky and Kao (1989) and Prattichizzo and Trinkle (2016) have tackled the issue through an extensive mathematical analysis of robotic grasp compliance and its components. Still, they dealt more with the study of the main influencing components on the grasp compliance, rather than providing a high-level control to modulate it. Sauser et al. (2012) and Li et al. (2014) proposed different ways to learn and adapt the grasp based on a desired stiffness or external disturbances, respectively. These methods used force-sensors as input information, and relied on human demonstrations that needed to be re-defined when the object changed.

While these studies addressed the grasp stiffness problem from a more classic robotics point of view, here we propose a bio-inspired approach that aims to reproduce the human stiffening behavior in a robotic hand. In fact, humans have developed effective strategies to grasp tools depending on the task to be performed, and in particular on the task requirements in terms of stiffness. Drilling a wall, for example, requires to be very stiff in the direction perpendicular to the surface, while remaining compliant in the other directions to comply to possible external disturbances. Instead, in a fine painting task, the grasp should be stiff on the plane parallel to the wall, while being compliant in the perpendicular direction to the surface to comply with the disturbances caused by the wall irregularities.

Two main concepts are at the basis of how humans control the stiffness of the fingers that results in the object grasp stiffness: (i) the Configuration Dependent Stiffness (CDS), and (ii) the Common Mode Stiffness (CMS). CDS and CMS mechanisms were firstly observed in the human arm stiffness control. Patterns of stiffness variation depending on the arm posture (CDS) were observed for the human arm in Mussa-Ivaldi et al. (1985), while the work in Milner (2002) showed that adjusting posture is more effective than using the joint stiffness to stabilize the hand position in the presence of external disturbances. These observations were successfully exploited in the control framework of a robot arm (Ajoudani et al., 2013, 2015a). Moreover, in Ajoudani et al. (2013), a CMS variable was used to implement a coordinated activation across the arm joints, and, simultaneously, a CDS variable was employed to control the redundant kinematic DoFs. Assuming an ellipsoidal representation of the endpoint stiffness (Mussa-Ivaldi et al., 1985), the CMS variable regulated the volume of the endpoint stiffness ellipsoid, whereas the CDS variable modified its geometry by controlling the nullspace velocity of the manipulator.

Similarly, regarding the stiffness control of the human hand, Milner and Franklin found that humans can modify the endpoint stiffness geometry by varying the fingers posture (CDS), being this process more energy-efficient than using muscle co-contraction (Milner and Franklin, 1998). In a complementary study, Rossi et al. (2015), presented one of the first attempts to explore the concept of human hand synergies in the stiffness coordinates, showing the existence of a coordinated stiffening pattern in human fingertips during a tripod grasp (CMS). Authors estimated the human fingertip stiffness profiles using external stochastic perturbations, and illustrated them by ellipsoids. The preliminary results of this study suggested that the co-activations of the forearm muscles contribute to a coordinated stiffening of the fingers, leading to an increase in the amplitudes of the major axes of the stiffness ellipsoids with minor effects on their orientations. Therefore, in humans, CDS allows for a more energy-efficient stiffness control than co-contraction, while CMS limits the number of control inputs needed to achieve complex manipulation tasks.

The approach detailed in this paper proposes to exploit the advantages given by the CDS and CMS concepts in robotic hands. The aim is to obtain bio-inspired and energy-effective grasps with a simplified control strategy where n degrees of freedom are used for the robot position control, and one to implement the common-mode stiffness (n+1 actuation principle). Furthermore, the method targets to use the minimum required sensory information, taking as input only the joint positions.

This work completes and extends the preliminary study presented in Ruiz Garate et al. (2017). Though the general idea remains the same, validation experiments have been performed, and several key factors of the control strategy have been improved. The most relevant one is that, being able to represent the grasp stiffness by an ellipsoid, the method proposed here focuses on reproducing the main geometrical features of such an ellipsoid instead of the overall stiffness matrix numerical values targeted before. Moreover, the stability of the final pose is assured during the process, whereas previously, stability was only checked once the final solution was found. In addition, a wider variety of objects are exploited and tested in real experiments with the current method, using a generalization of their shapes Previously, the method had only been tested with spherical objects in a simulation environment. These points will be further clarified in the following sections.

On the other hand, the approach presented in this paper differs from the one in Ruiz Garate et al. (2018) from a fundamental methodology point of view. The method in Ruiz Garate et al. (2018) provides full joint trajectories assuring grasp stability based on a simultaneous optimization of the hand pose (joint positions) and joint stiffness (CDS/CMS). The current approach, instead, performs a preliminary exploration of all feasible grasp configurations by analyzing the entire fingers workspace, and thus providing a first basis for the CDS. Then, an optimization is done for each of these configurations trying to maximize the grasp robustness while best approximating the desired stiffness. The stable grasp configuration giving the nearest solution to the desired grasp stiffness is kept in the end. So, this method directly outputs a final grasp configuration, differently from Ruiz Garate et al. (2018), where the method outputted the full trajectory through several iterations. Hence, with the current method it is afterwards needed to compute a trajectory from the initial pose to the final found one.

Therefore, this paper presents the first attempt to provide a complete generalized method for the regulation of the grasp stiffness, based on the exploration of the grasp workspace. After a thorough explanation of our control approach (section 2), we show its performance using the fully actuated Allegro Hand, a robotic hand with 4 fingers and 16 DoFs² (sections 2, 3).

2. Materials and Methods

2.1. Problem Definition

The bio-inspired control strategy proposed in this paper exploits the introduced concepts of CDS and CMS to regulate the grasp stiffness of a robotic hand.

Before detailing the control policy, some mathematical background regarding grasp stiffness is needed. Grasp stiffness is a fundamental tool for modeling and controlling compliant robotic grasps. Its main characteristics are discussed in Kao and Ngo (1999), and more recently in Malvezzi and Prattichizzo (2013), where also under-actuation is taken into account. The object grasp stiffness matrix K∈ℝ^{6 × 6} is a symmetric matrix that relates the wrench Δw∈ℝ⁶ applied to an object to its displacement Δu∈ℝ⁶:

$\begin{array}{rc}\Delta \text{w}=\text{K}\Delta \text{u}=(\text{G}{\text{K}}_c{\text{G}}^T)\Delta \text{u},& (1)\end{array}$

where G is the grasp matrix relating the contact forces and moments transmitted through the contact points, to the set of wrenches that the hand applies on the object. In our case, $\text{G}\in {ℝ}^{6\times 3n_c}$, as we assume Hard Finger contacts (Prattichizzo and Trinkle, 2016), being n_c the number of contact points on the object. ${\text{K}}_c\in {ℝ}^{3n_c\times 3n_c}$ is the equivalent contact stiffness matrix taking into account all the system compliance sources (Malvezzi and Prattichizzo, 2013) and incorporating the fingers and object structural elasticity (Bicchi, 1994):

$\begin{array}{rc}{\text{K}}_c={({\text{C}}_s+\text{J}{\text{K}}_q^{-1}{\text{J}}^T)}^{-1},& (2)\end{array}$

with ${\text{K}}_q\in {ℝ}^{n_q\times n_q}$ being the diagonal matrix representing the joint stiffness, and $\text{J}\in {ℝ}^{3n_c\times n_q}$ the hand Jacobian matrix. n_q is the total number of finger joints in the hand³. ${\text{C}}_s\in {ℝ}^{3n_c\times 3n_c}$ is the structural compliance matrix.

The endpoint stiffness of the fingers K_c as well as the resulting grasp stiffness K can be represented by translational and rotational stiffness ellipsoids. Each of these are characterized by a particular geometry (defined by the ellipsoid axes) and volume (magnitude).

The objective of the proposed method is to manipulate the grasp stiffness K toward a desired K_d while preserving the grasp contact points. We focus on the translational part of the desired grasp stiffness, i.e., K_{d, t}, influencing the translational interaction force profiles that play an important role in manipulation of daily objects. For example, doing a precise object placement in a hole, the grasp should be most stiff in the hole direction, whereas it should be compliant in the other ones.

To attain the desired grasp stiffness, the proposed method controls the hand pose q (CDS) and the joint stiffness K_q (CMS). The CDS control is implemented by manipulating the vector q that contains the joint angular positions q = [q₁, q₂, …q_nq]. The changes in the hand pose reflect into geometrical variations of the grasp stiffness ellipsoid. On the other hand, the CMS control regulates the synergistic finger stiffness K_q, mainly providing modifications of the stiffness ellipsoid volume. For every finger f, K_q = αΓ_f, where α is the CMS parameter (Nm/rad) and Γ_f is a constant normalized vector implementing the coordinated stiffening of the hand fingers (Rossi et al., 2015). Therefore, the maximum achievable grasp stiffness volume is limited by the maximum applicable α.

Being K_t represented by a 3D ellipsoid, differently from Ruiz Garate et al. (2017), where the whole translational matrix numerical values were targeted, here we define the task requirements by means of high-level features of the grasp stiffness ellipsoid geometry. Our method is therefore designed to target, in descending order of importance:

• main axis orientation of the desired grasp stiffness ellipsoid,

• secondary axis orientation of the desired grasp stiffness ellipsoid,

• length ratio of the two main axes.

To measure the performance of our method in matching these features, we define three indexes:

$\begin{array}{rcl}{\theta }_1& =min\left\{\left|arccos({\text{U}}_{1,d}{\text{U}}_1)\right|,\left|arccos(-{\text{U}}_{1,d}{\text{U}}_1)\right|\right\}& (3)\end{array}$

$\begin{array}{rcl}{\theta }_2& =min\left\{\left|arccos({\text{U}}_{2,d}{\text{U}}_2)\right|,\left|arccos(-{\text{U}}_{2,d}{\text{U}}_2)\right|\right\}& (4)\end{array}$

$\begin{array}{rcl}\beta & =\left|\frac{D_{d,1}}{D_{d,2}}-\frac{D_1}{D_2}\right|& (5)\end{array}$

θ₁ and θ₂ are the difference in orientation between the desired and obtained main and secondary axes, respectively. The orientation of these axes is defined from the eigenvectors of K_t. U₁ represents the main eigenvector and U₂ the secondary one. β is the difference between the desired and obtained length ratio. To compute this ratio, the eigenvalues of K_t are used, where D₁ stands for the main eigenvalue and D₂ for the secondary one.

Besides matching the geometrical features of K_t, the control strategy must ensure grasp stability. To this aim, the stiffness ellipsoid volume can be tuned so to maximize an index of grasp robustness. We decided to use a simplified version of the Potential Contact Robustness (PCR) index (Pozzi et al., 2017). By using this index, the method will be now able to assure robust grasps. The PCR is based on the distance of the contact force from the friction cone boundaries, and increases as the grasp becomes more robust. We adjust the equation to focus only on the Coulomb friction constraint, maximizing the distance to the friction cone (Pozzi et al., 2017). Every contact point c is considered to be stable if:

$\begin{array}{rc}ϵ{\lambda }_c-{\lambda }_{n,c}<s,& (6)\end{array}$

where λ_c represents the contact force, λ_{n, c} the normal component of the forces, s is a security margin set to −0.01, and $ϵ=\frac{1}{\sqrt{1+{\mu }^2}}$ being μ = 0.8 the coefficient of friction.

2.2. CDS and CMS Control Strategy

Based on the introduced concepts of stiffness geometry and grasp stability, the proposed algorithm consists of four main consecutive steps outlined in Figure 1. A preliminary process should be done to explore the workspace of each of the hand fingers. Fingertip locations FT_ws with respect to the world frame together with their corresponding transformation matrices T_ws and joint positions Q_ws are stored (Figure 1). Then, the main steps are, in consecutive order:

1. Stabilization of the initial grasp.

2. Search of compatible configurations preserving the contact points.

3. Grasp optimization (CDS+CMS).

4. Trajectory generation.

FIGURE 1

Figure 1. Control scheme. The left dashed box contains the prelimiary steps to be done for each new hand and grasp: from the fingers workspace, configurations keeping similar contact points to the initial grasp are found and stored as Q_c. Once these configurations are found they can be re-used for every new desired stiffness K_d. The right dashed box contains the optimization of the found configurations Q_c to find the minimum joint stiffness that stabilizes the grasp while trying to obtain the main features of the desired grasp stiffness K_d. The stable grasp giving the closest features to the desired ones is kept and a trajectory is generated for the finger joints toward it.

These processes are described in more detail in the following subsections. For every new grasp, the first two steps need to be executed. Then, the desired stiffness can be changed only requiring to perform the two last ones.

It must be noted that during the process we refer to two type of hands: (i) the actual hand based on the measurement of the joint positions q, and (ii) the virtual hand from which the reference joint positions q_ref are defined such that the actual hand slightly squeezes the object and stabilizes the grasp. This last pose drives the fingers of the virtual hand inside the object (see Figure 2).

FIGURE 2

Figure 2. Initial grasp stabilization scheme. Once the hand is in a grasping position, the encoders are read. These values (q_encoders) are sent to create a virtual model of the hand and object. From this information, joint reference values are found that stabilize the grasp. Those values (q_ref) together with the chosen initial joint stiffness K_q are command to the hand resulting in the initial stable grasp.

The encoder readings of the joints positions are the only sensory information the method relies on. This means that no information needs to be acquired from the real hand in terms of force pressure, orientation, object position, etc. The fact that no extra force or visual information is required makes the method generalizable to almost any hand and object. However, to do so the following assumptions are made:

• We focus on fingertip grasps, as with power grasps the possibility of having in-hand manipulation is very limited. For the sake of simplicity all contacts are modeled as hard finger contacts where only forces (and no moments) are transmitted (Prattichizzo and Trinkle, 2016).

• Due to the lack of visual information, to find the object location (position and orientation), it has to be attached to one of the fingers, similarly to Sundaralingam and Hermans (2017). The thumb is chosen for this purpose. This means that at every evaluated configuration i, the transformation between the object frame and that of the thumb remains constant and equal to the one of the initial configuration (${}^{\mathrm{\text{thumb}}}{T}_{\mathrm{\text{obj}},0}={}^{\mathrm{\text{thumb}}}{T}_{\mathrm{\text{obj}},i}$). Therefore, from any location of the thumb fingertip, the new object location can be retrieved.

• Due to possible model inaccuracies, for the main stiffness computations the contact locations are approximated to be exactly at the fingertips, whose positions can be retrieved from direct kinematics.

• To analyse the stability and check if any contact was lost, contacts between the virtual reference hand and the object are searched for. As q_ref should drive the fingers toward the inside of the object to stabilize the grasp (Figure 2), the contact points must be always detectable. If they are not found, then certainly the contact has been lost. These contact points are obtained analytically as the intersection between the fingertips of the virtual stabilized hand, modeled as spheres of a known radius r_s, and the object. This method takes into account the fingertip volume, differing from the one in Ruiz Garate et al. (2017), where contacts where only detectable if an intersection occurred between the the main axis of the last finger link and the object.

2.2.1. Stabilization of the Initial Grasp

The initial grasp serves as starting point for the exploration for the CDS control, as we look for new configurations holding the same contact points as the initial one. It can be specified by setting the finger joints in a desired configuration, or by closing the hand with a grasp planner and retrieving the actual values. This initial configuration is sent to the algorithm, that creates a virtual model of both the hand and the object (see Figure 2, upper arrow). Due to the lack of visual information, the object is supposed to be initially aligned with the centre of the fingertip of the thumb. From the obtained virtual grasp model, the contact points are defined.

Similarly to Ruiz Garate et al. (2018), at this stage a hand configuration is looked for such that it squeezes the object and stabilizes the initial grasp. To do so, an optimization is carried out where a cost function based on the PCR index is computed using the method in Gabiccini et al. (2011). This optimization computes a set of internal forces such that the contact forces ${\lambda }_c=-{\text{G}}^{\#}\text{w}+\text{E}\text{y}$ are as far as possible from their friction cone limits. w stands for the external wrench applied and E represents the basis of the controllable subspace of internal forces. y is a vector that parametrizes the homogeneous part of the solution of the equation w = −Gλ_c. Once an optimal set of internal forces λ_opt = Ey_opt is found, the displacement of the end-effector (fingertips) corresponding to that force can be found as $d\text{x}={\text{K}}_c^{-1}{\lambda }_{opt}$. The complementary joint displacement dq = J^T(J^JT)−1dx is then applied to the virtual hand model, while keeping the object in place. Figure 2 shows a schematic layout of this process.

A final check is performed to assure that the newly found configuration is stable using Equation (6) and λ_opt = Ey_opt. If the grasp proves to be stable, this configuration is finally commanded to the real hand, assuring the stability of the initial grasp (Figure 2, lower arrow).

2.2.2. Search of Compatible Configurations

From the fingers workspace obtained preliminary, configurations keeping similar fingertip locations as the initial ones are stored (Q_c in Figure 1). For this step, the fingertip locations used are the ones before the stabilization driving them inside the object. A maximum of 5000 configurations of the workspace are evaluated as a compromise between the number of possible solutions and the required time.

To find similar fingertip configurations, the distance from the object center to the fingertips is checked. Therefore, it is first necessary to determine the object location, and afterwards define the fingers position with respect to the object frame. The initial transformation matrix of the thumb with respect to the object can be retrieved from the initial ones of the object and the thumb with respect to the world frame: ${}^{\text{thumb}}{T}_{\text{obj},0}={{(}^w{T}_{\text{thumb},0})}^{-1}{}^w{T}_{\text{obj},0}$. The rest of initial fingertip positions with respect to the object frame are obtained as: ${}^{\text{obj}}{P}_{f,0}={{(}^w{T}_{\text{obj},0})}^{-1}{}^w{P}_{f,0}$. The values of ${}^w{\text{P}}_{f,0}$ and ${}^w{\text{T}}_{\mathrm{\text{thumb}},0}$ can be obtained using the D-H parameters of the hand and solving the forward kinematics.

Similarly, at every new hand configuration q_i of the explored workspace, ${}^w{\text{P}}_{f,i}$ and ${}^w{\text{T}}_{\mathrm{\text{thumb}},i}$ are retrieved. As the transformation matrix between the thumb and the object remains fixed, the new object frame is estimated:

$\begin{array}{cc}{}^w{T}_{\text{obj},i}{=}^w{T}_{\text{thumb},i}{}^{\text{thumb}}{T}_{\text{obj},0}& (7)\end{array}$

and the new fingertip positions with respect to the object frame can be retrieved:

$\begin{array}{cc}{}^{\text{obj}}{P}_{f,i}={{(}^w{T}_{\text{obj},i})}^{-1}{}^w{P}_{f,i}& (8)\end{array}$

Configurations satisfying ${\parallel }^{\text{obj}}{P}_{f,0}{-}^{\text{obj}}{P}_{f,i}\parallel$ mm are stored in Q_c = {q₁, q₂, …, q_c}. This threshold is chosen as a compromise between the need to keep the same grasp contacts and allowing a reasonable movement of the fingertip locations.

Though this process can be time consuming, it only needs to be executed once when the grasp is first defined. Then, for all possible targeted stiffness, these values can be re-used (Figure 1).

This process provides all the possible hand configurations that hold similar contact points to the initial one. However, at each of this configurations the resulting grasp stiffness geometry are different, providing the basis of the CDS control.

2.2.3. Grasp Optimization

As outlined in Figure 1, for all feasible configurations found in the previous step, an optimization is performed to find a synergistic joint stiffness α that maximizes the the grasp robustness (PCR) while trying to stay as close as possible to the desired stiffness geometry. Therefore, the function to be optimized f(α) is defined as:

$\begin{array}{r}\begin{array}{cc}{\displaystyle \underset{\alpha }{\mathrm{\text{minimize}}}}& f(\alpha )=10^6{K}_e+\frac{1}{PCR}\\ \mathrm{\text{subject to}}& {\alpha }_{min}\le \alpha \le {\alpha }_{max}& (9)\end{array}\end{array}$

To obtain PCR from each configuration, depending on α, new reference finger joint positions are found that stabilize the grasp. This is done following the same procedure as in section 2.2.1. In order to run this optimization, the function fminbnd (MATLAB and Statistics Toolbox, The MathWorks, Inc., Natick, Massachusetts, United States) is used, optimizing α by first computing the grasp stiffness and then trying to find the stabilized grasp.

The stiffness error K_e is defined as a weighted function on the difference between the desired and actual stiffness parameters described previously (Equations (3)–(5)):

$\begin{array}{rc}{K}_e={\theta }_1+0.5{\theta }_2+0.01\left|\frac{D_{d,1}}{D_{d,2}}-\frac{D_1}{D_2}\right|& (10)\end{array}$

This step provides the CMS control for each of the previously found configurations adapting mainly the volume of the grasp stiffness depending on the attainable stability. If the found α is smaller than the one defined for the initial grasp, the latter is kept. Nonetheless, this value can be afterwards manipulated by the user depending on task specifications or external inputs.

The optimization stops if incompatible contact points are found during the stabilization and previous values are kept. From all the feasible configurations that result in a stable grasp, finally the one with the lowest K_e is kept as the optimal. In this way, the achievement of the desired stiffness is prioritized in the process (CDS control).

2.2.4. Trajectory Generation

The previous steps provide a final stable configuration that best matches the desired stiffness characteristics in terms of orientation and shape. To move from the original configuration to the final one, a trajectory is needed (Figure 1).

Based on the work of Sundaralingam and Hermans (2017), we compute a smooth trajectory of 40 steps with Δt = 0.05 s. At each step, the following function is optimized:

$\begin{array}{l}\underset{\stackrel{.}{q}}{\text{minimize}}\text{f}(q)={\displaystyle \sum (|q_d-q_i|)}+{10}^2{\parallel }^{\text{obj}}{P}_{f,0}{-}^{\text{obj}}{P}_{f,i}\parallel \\ \text{subject to}{\stackrel{.}{q}}_{min}\le \stackrel{.}{q}\le {\stackrel{.}{q}}_{max}& (11)\end{array}$

${\stackrel{°}{\text{q}}}_{min}$ and ${\stackrel{°}{\text{q}}}_{max}$ are set to −30 and 30 deg/s respectively for every joint, and the initial joint angular velocity to a random value between 0 and 30 deg/s. A high weight is given to the second term of the equation to assure that the contact points are not lost. For this optimization, the fmincon function (MATLAB and Statistics Toolbox, The MathWorks, Inc., Natick, Massachusetts, United States) is used with a maximum of 50 iterations.

The joint stiffness obtained from the grasp optimization is applied from the beginning of the trajectory. This value is always equal or higher than the initial stiffness to provide better stability.

2.3. Experimental Validation

To evaluate the proposed method, we carried out experiments with the Allegro Hand (SimLab Co., Ltd.), a 4-fingered robotic hand with 16 independent torque-controlled joints, 4 joints per finger⁴ (see Figure 3A). One of its main features is the“roll-pitch”-type MCP joint of the fingers, that gives the hand high dexterity (Lee et al., 2017). The torque controlled fully actuated Allegro hand allows to simulate the CDS/CMS control via active joint impedance control. This control is implemented as:

$\begin{array}{rc}\tau ={\text{k}}_q\delta \text{q}+{\text{k}}_d\delta \stackrel{°}{\text{q}}+{\tau }_g& (12)\end{array}$

where ${\text{k}}_q=diag({\text{K}}_q)\in {ℝ}^{16}$ is the vector of joint stiffness values, and ${\text{k}}_d\in {ℝ}^{16}$ contains the damping parameters found from already provided stable coefficients c_f as ${\text{k}}_d=\sqrt{\frac{{\text{k}}_p}{{\text{c}}_f}}$. τ_g is the gravity torque vector compensating for the weight of the hand. This value was provided with the hand software. In this way, when the joint stiffness is set to very low values, the hand is controlled in gravity compensation mode.

FIGURE 3

Figure 3. (A) Picture of the hand with all joints set to 0 deg. In red the joints directions for the index (q_{i, 1…4}) and the thumb (q_{t, 1…4}) are displayed. The middle and little finger have similar joints to the index. (B) Set of objects used during the experiments.

To compute the grasp stiffness, the joint stiffness synergy is defined as Γ_f = [0.7, 0.95, 1, 0.8] for the index, middle, and little gingers, and Γ_f = [1, 0.95, 1, 0.95] for the thumb. These values are chosen based on the mechanical characteristics of the hand and the selected vertical set-up (the pedestal of the hand being fixed to a table).

For the stability optimization, a minimum and maximum contact force of 2 and 5 N respectively are specified. This avoids getting a reference position that goes too much inside the object which, due to the hand configuration, would provide a big push from the upper fingers and prevent the lower finger (the thumb in this case) from moving. Based on the technical specifications from the hand, during the optimization α is allowed to vary between 2 and 6 Nm/rad. The initial value of α is set to 4 Nm/rad.

To test the different aspects of the algorithm, we decided to use several objects from the YCB set (Calli et al., 2015): lemon, orange, apple, gelatin box, banana, metal mug, and pudding box, displayed in Figure 3B. These objects are modeled as generic shapes: spheres, cuboids, or cylinders. With this generalization, the method is able to cover a wider variety of objects than it was possible in Ruiz Garate et al. (2017), where only spheres where studied, and Ruiz Garate et al. (2018), where yet no cylinders where available. Different grasps (2, 3, 4 fingertip grasps) are chosen depending on the shape and weight of the object. This decision is done together with the different desired stiffness to demonstrate the full potential of our control strategy. The combinations of selected grasp types and target stiffness are detailed in Table 1.

TABLE 1

Table 1. Selected objects for the experiments, type of grasp implemented, and chosen desired stiffness.

3. Results

Experimental results are evaluated in terms of:

• difference in orientation between the desired and obtained main axis of the stiffness ellipsoid: θ₁,

• difference in orientation between the desired and obtained secondary axis of the stiffness ellipsoid: θ₂,

• difference between the desired and obtained length ratio of the two main axes: β,

• initially chosen and finally minimum found CMS parameter α,

• Potential Contact Robustness index (PCR).

Table 2 shows the obtained values corresponding to these parameters. Results concerning the principal features of grasp stiffness orientation (θ₁ and θ₂) are also visually depicted in Figure 4. Figures 5–11 show the initial and final virtual grasp configurations together with the initial, desired, and final grasp stiffness ellipsoids in the performed experiments.

TABLE 2

Table 2. Initial values and results of the algorithm when manipulating the different objects.

FIGURE 4

Figure 4. Initial and final values of (A) the difference in orientation between the desired and obtained main axis (θ₁), and (B) the difference in orientation between the desired and obtained secondary axis (θ₂).

FIGURE 5

Figure 5. (A) Lemon virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness. The scale for the desired stiffness ellipsoid graph is set smaller for visualization purposes.

FIGURE 6

Figure 6. (A) Orange virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness.

Difference between desired and initial main axis orientation θ₁ decreases for all tested cases. This decrease ranges from 50.14% in the worst case to 99.24% in the best. Similar decreases are observed for the secondary axis, where the reduction spans from 66.06 to 91.94%.

For the cases where a manual translational stiffness is defined (lemon and pudding box), the error in the length ratio β is decreased (by 84.58 %, for the lemon) or remains almost the same (difference of 0.4889 %, for the pudding box). In the case when the objective stiffness is defined as a rotation of the initial one, the initial error in length ratio is zero. Therefore any change will increase it. This error is in any case kept quite low (below 0.9) except for the apple and the metal mug, where it reaches a value of 4. However, as it can be observed in Figures 7, 10 these differences do not have a major influence on the overall stiffness geometry.

FIGURE 7

Figure 7. (A) Apple virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness.

The minimum required synergistic joint stiffness parameter α (Table 2) is generally lower than the initial one, except for the gelatin box and the metal mug. The minimum values are of 2 Nm/rad while the maximum almost reaches the limit of 6 Nm/rad.

PCR varies from an initial range of [0.00241–0.5352] to a final of [0.0036–1.8541]. If the minimum found stiffness is considered, PCR decreases for all cases except the apple and gelatin box. However, if the lower bound on the initial stiffness is considered, PCR only decreases significantly in the case of the banana.

Finally, Figure 12 shows the performed experiments for each object as sequences of frames. The video available in “Supplementary Video 1” shows the presented CDS/CMS stiffness regulation method for the lemon, the apple, and the gelatin box cases.

4. Discussion

From the results shown in Table 2 and Figure 4, it can be seen how the error in the main considered features (the stiffest directions) always decreases. In the worst case, the error is reduced by 50% of the initial one and is almost zero in the the best case. These results are generally better than those in Ruiz Garate et al. (2018). In Ruiz Garate et al. (2018), a trajectory is generated based on the simultaneous optimization of all parameters with no pre-knowledge of the hand workspace, whereas the proposed algorithm does a preliminary exploration of the fingers workspace and an evaluation of the hand configurations based on such exploration. This has the advantage of being sure to find a global optimum based on the proposed criteria. On the other hand, it has the drawback of only giving the final grasp configuration. This means that a trajectory must be generated, which is done based on the method of Sundaralingam and Hermans (2017). However, no proper stability criterion is included in this trajectory generation, which could be done, if required, by adding an extra condition to Equation (11). Nonetheless, in our experiments keeping the fingertip distance constant and a minimum joint stiffness was enough to generate stable trajectories.

Regarding the error in the length ratio, as we prioritize the axes orientation, β is only improved if it does not cause major impact in these orientations. Therefore, while in the case of the lemon (Table 2, Figure 5) the error is significantly reduced, for the pudding box (Table 2, Figure 11) this error is almost unchanged. Moreover, a particular orientation might not be possible together with a specific length ratio, and so the first prevails. In any case, this error is kept quite low.

With respect to the synergistic joint stiffness parameter α, Table 2 shows that the main influence on the minimum required joint stiffness is not the size of the object or its weight (the gelatin box of 97 g gave a value of 4.066 while the pudding box of 187 g is only 3.393 Nm/rad). Also, the lemon required a higher α than the orange or the apple, being it smaller. This is in concordance with the fact that the stability values are computed based on the relation between the normal and tangential contact forces (Equation 6). This means that the distribution of these forces plays a major role. Therefore, the arrangement of the contacting fingers and their relative position with respect to the object surface are the main factors to assure stability. It must be pointed out that, as shown in Equation (12), a gravity compensation term is already included in the hand command. Hence, there is not need for additional torque coming from the joint stiffness term to hold the grasp pose. Nonetheless, some minimum stiffness is needed to comply with the object weight and inaccuracies of the gravity compensation.

In spite of that, though the joint stiffness magnitude might not be the most important factor to determine stability, it does play a significant role. Actually, as it can be seen from the PCR values in Table 2, for the cases where the found minimum α is smaller than the initial one, the PCR is lower than what we would obtain keeping α to the initial value of 4 Nm/rad. This means that increasing the joint stiffness has a positive impact in the overall grasp robustness. However, if the increase is not properly handled it can saturate the motors. Thus, a hint on the minimum stiffness value required can be useful.

During the experiments, the three-finger grasp is more widely tested because it clearly shows the changes on the hand configuration while being able to grasp a big diversity of objects. Only one four-finger grasp is tested. This is due to the constraint put by the stability criterion. We consider that the grasp is stable if all contact points are within the limits of the friction cone. However, this condition might be too strict for the cases with many contacts, as some might not necessarily be stable, whereas the overall grasp is. Other stability criteria could be studied to deal with numerous contact cases, for example by starting from the one presented in Pozzi et al. (2018). Here, the presented four-finger grasp serves as a proof of the generalization of the overall method. The control law that was used to regulate the joint impedance (Equation (12)) is based on a simple PD torque control scheme. This control could be enhanced, e.g., taking the joint torque limits into account (Ajoudani et al., 2015b), or the instability of joint stiffness parameters (Maekawa et al., 1990). In de Jesús Rubio (2018), a method enhancing stability is tested with a robotic arm in which the states to regulate are more than the available inputs. Similar methods to that one or the one proposed in Pan et al. (2016b) could be studied to assure tracking stability and faster response for the robotic hand joints. Also, more advanced controls could be though of, like e.g., the one proposed in Pan et al. (2016a), joining a PD control term in the feedback loop and a radial-basis-function (RBF) neural network (NN) in the feedforward loop, including therefore a learning control mechanism.

As stated in the introduction, whereas in Ruiz Garate et al. (2017) a grasp is found and then checked to be stable, the proposed method includes the stability as a condition during the search of the optimal grasp configuration. This assures that any final found configuration will be stable.

Moreover, differently from Ruiz Garate et al. (2017) and following the line of Ruiz Garate et al. (2018), the method does not try to achieve the overall stiffness matrix. Instead, based on the property of the stiffness matrix to be defined as an ellipsoid, high-level features are chosen as objective. In this way, the desired stiffness can be defined based on the task requirements in a simple way: which translational directions of the grasp should be stiffer or more compliant. Likewise, if the stiffness requirements are being transferred from a teleoperation set-up, the master can have very different structure, configuration, mass, etc. Hence, the volume of the stiffness ellipsoid might not be reproducible, while the high-level features of the task requirements can still be transferred.

As we start from the premise that no sensory information is available apart from the encoder position, there is no visual information regarding the object shape. Therefore, we generalize the objects as spheres, cuboids, or cylinders. Experiments show that this generalization is sufficient for most daily objects (see Figures 5–11). However, in some cases like the banana (Figure 9), the generalization only applies if the fingers are positioned in the close vicinity of each other. In case of such irregular objects, a more detailed object model would be needed. However, such implementation would be computationally costly, not only for the modeling, but also when determining if the contact points are maintained and stable.

FIGURE 8

Figure 8. (A) Gelatin box virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness.

FIGURE 9

Figure 9. (A) Banana virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness.

FIGURE 10

Figure 10. (A) Metal mug virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness. The scale for the final stiffness ellipsoid graph is set larger for visualization purposes. As the difference between the ellipsoids are not very visible due to the 3D perspective, at the top of each graph the projection of each ellipsoid in the “XY” plane is plotted.

FIGURE 11

Figure 11. (A) Pudding box virtual grasp in the initial and final configurations, (B) Initial, objective, and final stiffness. θ₁ indicates the difference between the initial/final main ellipsoid axis orientation and that of the desired stiffness.

FIGURE 12

Figure 12. Sequential captures of the configuration changes on the different grasps depending on the initial pose and desired stiffness. (A) Lemon, (B) orange, (C) apple, (D) gelatin box, (E) banana, (F) metal mug, and (G) pudding box.

Another point to be noticed is that no self-collision detection method is included in the algorithm. So far, the inherent joint limits of the hand, and the fact that an object is being held, prevent the fingers from colliding with each other. In the case of manipulating very small objects, or needing a particular configuration of the fingers that are not in contact with the object, a collision detection should be implemented. This would require either contact sensors or a full modeling of the volumes and contacts between all finger links.

Possible applications of our method range from more classical teleoperation scenarios, where geometry of the grasp stiffness of the master can be intuitively transferred onto the slave's one. Then, the grasp stability can be regulated based on the local sensory feedback. More innovative applications could be for example, a robotic system that exploits the environmental constraints, contributing to the CDS control (Eppner et al., 2015).

5. Conclusions

This manuscript proposed a control method to achieve a desired grasp stiffness based on high-level features of the task to execute. As the grasp stiffness can be represented by a 3D ellipsoid, these features were defined from the geometrical aspects of such ellipsoid, namely main and secondary axes orientation, and their length ratio. These features were targeted while assuring the stability of the grasp.

The presented control approach was based on the bio-inspired concepts of CDS and CMS that characterize the way humans regulate the fingers endpoint stiffness. The CDS control changed the hand pose influencing the overall geometry of the stiffness ellipsoid. The CMS control modified the synergistic joint stiffness and consequently the volume of the ellipsoid, which had a more important role in assuring the stability. This algorithm was constrained to keep the same grasp contacts so that the object was always held from the same points.

The method demonstrated to reduce the error between the desired and obtained main stiffness geometry features. Moreover, it proved to work in the absence of sensory information (except for the joint encoders), based on the proposed assumptions on the object shapes and contact points.

Future work will envisage the study of alternative and less conservative criteria to analyze the grasp stability. Also, the application of the control paradigms to other robotic hands would be interesting. At a higher level, the human hand−arm system could be studied during grasp stiffness regulation in manipulation tasks, for a possible development of a CDS/CMS control of the equivalent artificial hand−arm composite.

Author Contributions

VRG and MP significantly contributed to the development of algorithm, the execution of experiments, the analysis of results, and the writing of the manuscript. AA significantly contributed to the development of the algorithm, the analysis of results, and the writing of the manuscript. DP significantly contributed to the analysis of results and the writing of the manuscript. All the authors approved the submitted version of the manuscript.

Funding

This work is supported in part by the EU H2020 projects SOMA: SOft MAnipulation (no. 645599), and SoftPro: Synergy-based Open-source Foundations and Technologies for Prosthetics and RehabilitatiOn (no. 688857).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frobt.2018.00089/full#supplementary-material

Supplementary Video 1. This video shows how the proposed method was able to regulate the grasp stiffness. In particular we show the directions in which the final grasp is more stiff or more compliant. Three examples are chosen: the lemon grasp, the apple grasp, and the gelatin box grasp. The video is in .mpg format.

Footnotes

1. ^ www.shadowrobot.com/products/dexterous-hand/

2. ^ http://wiki.wonikrobotics.com/AllegroHandWiki/index.php/Allegro_Hand

3. ^ This work only considers finger joint movements, keeping a fixed wrist configuration.

4. ^ http://wiki.wonikrobotics.com/AllegroHandWiki/index.php/Allegro_Hand

References

Ajoudani, A., Fang, C., Tsagarakis, N. G., and Bicchi, A. (2015a). “A reduced-complexity description of arm endpoint stiffness with applications to teleimpedance control,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (Hambourg), 1017–1023.

Google Scholar

Ajoudani, A., Gabiccini, M., Tsagarakis, N. G., and Bicchi, A. (2013). “Human-like impedance and minimum effort control for natural and efficient manipulation,” in 2013 IEEE International Conference on Robotics and Automation (Karlsruhe), 4499–4505.

Google Scholar

Ajoudani, A., Godfrey, S. B., Bianchi, M., Catalano, M. G., Grioli, G., Tsagarakis, N., et al. (2014). Exploring teleimpedance and tactile feedback for intuitive control of the Pisa/IIT softhand. IEEE Trans. Haptics 7, 203–215. doi: 10.1109/TOH.2014.2309142

PubMed Abstract | CrossRef Full Text | Google Scholar

Ajoudani, A., Tsagarakis, N. G., and Bicchi, A. (2015b). “On the role of robot configuration in cartesian stiffness control,” in 2015 IEEE International Conference on Robotics and Automation (ICRA) (Seattle, WA), 1010–1016.

Google Scholar

Bicchi, A. (1994). On the problem of decomposing grasp and manipulation forces in multiple whole-limb manipulation. Robot. Auton. Syst. 13, 127–147.

Google Scholar

Calli, B., Singh, A., Walsman, A., Srinivasa, S., Abbeel, P., and Dollar, A. M. (2015). “The YCB object and model set: towards common benchmarks for manipulation research,” in 2015 International Conference on Advanced Robotics (ICAR) (Istanbul), 510–517.

Google Scholar

Catalano, M., Grioli, G., Farnioli, E., Serio, A., Piazza, C., and Bicchi, A. (2014). Adaptive synergies for the design and control of the Pisa/IIT softhand. Int. J. Robot. Res. 33, 768–782. doi: 10.1177/0278364913518998

CrossRef Full Text | Google Scholar

Controzzi, M., Clemente, F., Barone, D., Ghionzoli, A., and Cipriani, C. (2017). The SSSA-myhand: a dexterous lightweight myoelectric hand prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 459–468. doi: 10.1109/TNSRE.2016.2578980

PubMed Abstract | CrossRef Full Text | Google Scholar

Cutkosky, M. R., and Kao, I. (1989). Computing and controlling compliance of a robotic hand. IEEE Trans. Robot. Autom. 5, 151–165.

Google Scholar

de Jesús Rubio, J. (2018). Robust feedback linearization for nonlinear processes control. ISA Trans. 74, 155–164. doi: 10.1016/j.isatra.2018.01.017

CrossRef Full Text | Google Scholar

Eppner, C., Deimel, R., Álvarez Ruiz, J., Maertens, M., and Brock, O. (2015). Exploitation of environmental constraints in human and robotic grasping. Int. J. Robot. Res. 34, 1021–1038. doi: 10.1177/0278364914559753

CrossRef Full Text | Google Scholar

Gabiccini, M., Bicchi, A., Prattichizzo, D., and Malvezzi, M. (2011). On the role of hand synergies in the optimal choice of grasping forces. Auton. Robots 31, 235–252. doi: 10.1007/s10514-011-9244-1

CrossRef Full Text | Google Scholar

Gioioso, G., Salvietti, G., Malvezzi, M., and Prattichizzo, D. (2013). Mapping synergies from human to robotic hands with dissimilar kinematics: an approach in the object domain. IEEE Trans. Robot. 29, 825–837. doi: 10.1109/TRO.2013.2252251

CrossRef Full Text | Google Scholar

Grebenstein, M. (2014). The Awiwi Hand: An Artificial Hand for the DLR Hand Arm System, Cham: Springer International Publishing.

Google Scholar

Kao, I., and Ngo, C. (1999). Properties of the grasp stiffness matrix and conservative control strategies. Int. J. Robot. Res. 18, 159–167.

Google Scholar

Lee, D. H., Park, J. H., Park, S. W., Baeg, M. H., and Bae, J. H. (2017). Kitech-hand: a highly dexterous and modularized robotic hand. IEEE/ASME Trans. Mechatron. 22, 876–887. doi: 10.1109/TMECH.2016.2634602

CrossRef Full Text | Google Scholar

Li, M., Yin, H., Tahara, K., and Billard, A. (2014). “Learning object-level impedance control for robust grasping and dexterous manipulation,” in 2014 IEEE International Conference on Robotics and Automation (ICRA) (Hong Kong), 6784–6791.

Google Scholar

Maekawa, H., Kaneko, M., Yokoi, K., Tanie, K., and Imamura, N. (1990). Grasp control for a multifingered hand with kinematic redundancy : experiments on stable grasp by the direct compliance control method. JSME Int. J. 33, 546–552. doi: 10.1299/jsmec1988.33.546

CrossRef Full Text | Google Scholar

Malvezzi, M., and Prattichizzo, D. (2013). “Evaluation of grasp stiffness in underactuated compliant hands,” in 2013 IEEE International Conference on Robotics and Automation (Karlsruhe), 2074–2079.

Google Scholar

Milner, T. E. (2002). Contribution of geometry and joint stiffness to mechanical stability of the human arm. Exp. Brain Res. 143, 515–519. doi: 10.1007/s00221-002-1049-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Milner, T. E., and Franklin, D. W. (1998). Characterization of multijoint finger stiffness: dependence on finger posture and force direction. IEEE Trans. Biomed. Eng. 45, 1363–1375.

PubMed Abstract | Google Scholar

Mottard, A., Laliberté, T., and Gosselin, C. (2017). “Underactuated tendon-driven robotic/prosthetic hands: design issues,” in Proceedings of Robotics: Science and Systems (Cambridge, MA).

Google Scholar

Mussa-Ivaldi, F., Hogan, N., and Bizzi, E. (1985). Neural, mechanical, and geometric factors subserving arm posture in humans. J. Neurosci. 5, 2732–2743.

PubMed Abstract | Google Scholar

Pan, Y., Liu, Y., Xu, B., and Yu, H. (2016a). Hybrid feedback feedforward: an efficient design of adaptive neural network control. Neural Netw. 76, 122–134. doi: 10.1016/j.neunet.2015.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Pan, Y., Tairen, S., and Haoyong, Y. (2016b). Composite adaptive dynamic surface control using online recorded data. Int. J. Robust Nonlinear Control 26, 3921–3936. doi: 10.1002/rnc.3541

CrossRef Full Text | Google Scholar

Pozzi, M., Malvezzi, M., and Prattichizzo, D. (2017). On grasp quality measures: Grasp robustness and contact force distribution in underactuated and compliant robotic hands. IEEE Robot. Autom. Lett. 2, 329–336. doi: 10.1109/LRA.2016.2612304

CrossRef Full Text | Google Scholar

Pozzi, M., Salvietti, G., Bimbo, J., Malvezzi, M., and Prattichizzo, D. (2018). The closure signature: a functional approach to model underactuated compliant robotic hands. IEEE Robot. Autom. Lett. 3, 2206–2213. doi: 10.1109/LRA.2018.2810946

CrossRef Full Text | Google Scholar

Prattichizzo, D., and Trinkle, J. C. (2016). “Grasping,” in Springer Handbook of Robotics, eds B. Siciliano and O. Khatib (Birmingham: Springer Science & Business Media), 955–988.

Google Scholar

Rossi, M., Altobelli, A., Godfrey, S. B., Ajoudani, A., and Bicchi, A. (2015). “Electromyographic mapping of finger stiffness in tripod grasp: a proof of concept,” in 2015 IEEE International Conference on Rehabilitation Robotics (ICORR) (Singapore), 181–186.

Google Scholar

Ruiz Garate, V., Pozzi, M., Prattichizzo, D., Tsagarakis, N., and Ajoudani, A. (2018). Grasp stiffness control in robotic hands through coordinated optimization of pose and joint stiffness. IEEE Robot. Autom. Lett. (accepted).

Google Scholar

Ruiz Garate, V., Tsagarakis, N., Bicchi, A., and Ajoudani, A. (2017). “On the common-mode and configuration-dependent stiffness control of multiple degrees of freedom hands,” in 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids) (Birmingham), 113–120.

Google Scholar

Santello, M., Flanders, M., and Soechting, J. F. (1998). Postural hand synergies for tool use. J. Neurosci. 18, 10105–10115.

PubMed Abstract | Google Scholar

Sauser, E. L., Argall, B. D., Metta, G., and Billard, A. G. (2012). Iterative learning of grasp adaptation through human corrections. Robot. Auton. Syst. 60, 55–71. doi: 10.1016/j.robot.2011.08.012

CrossRef Full Text | Google Scholar

Sundaralingam, B., and Hermans, T. (2017). “Relaxed-rigidity constraints: in-grasp manipulation using purely kinematic trajectory optimization,” in Proceedings of Robotics: Science and Systems (Cambridge, MA).

Google Scholar