Postural Hand Synergies during Environmental Constraint Exploitation

Humans are able to intuitively exploit the shape of an object and environmental constraints to achieve stable grasps and perform dexterous manipulations. In doing that, a vast range of kinematic strategies can be observed. However, in this work we formulate the hypothesis that such ability can be described in terms of a synergistic behavior in the generation of hand postures, i.e., using a reduced set of commonly used kinematic patterns. This is in analogy with previous studies showing the presence of such behavior in different tasks, such as grasping. We investigated this hypothesis in experiments performed by six subjects, who were asked to grasp objects from a flat surface. We quantitatively characterized hand posture behavior from a kinematic perspective, i.e., the hand joint angles, in both pre-shaping and during the interaction with the environment. To determine the role of tactile feedback, we repeated the same experiments but with subjects wearing a rigid shell on the fingertips to reduce cutaneous afferent inputs. Results show the persistence of at least two postural synergies in all the considered experimental conditions and phases. Tactile impairment does not alter significantly the first two synergies, and contact with the environment generates a change only for higher order Principal Components. A good match also arises between the first synergy found in our analysis and the first synergy of grasping as quantified by previous work. The present study is motivated by the interest of learning from the human example, extracting lessons that can be applied in robot design and control. Thus, we conclude with a discussion on implications for robotics of our findings.


INTRODUCTION
The human hand is a remarkably complex system, with many joints, ligaments, muscles, and sensory receptors contributing to its wide dexterity. The control of such an abundance or redundancy is classically referred to as Bernstein's problem (Bernstein, 1967). Several neuroscientific findings (Mussa-Ivaldi, 1999;Saltiel et al., 2001;Latash, 2008;Overduin et al., 2014;Stratmann et al., 2016) suggest that the human nervous system is able to cope with such complexity leveraging on a control space of reduced dimensionality. Under this regard, synergies were defined as principal patterns of actuation. Through them, the human Central Nervous System, also leveraging upon peripheral constraints, can generate movements by combining pre-organized patterns, i.e., by simultaneously activating different degrees of freedom, instead of acting separately on each joint or muscle. These patterns are implemented at various levels of the motor control architecture, i.e., either they are represented by cortical mechanisms, hardcoded in low-level neural circuits, or generated by the mechanical organization of human musculoskeletal system (Tresch et al., 1999;Santello et al., 2013;Leo et al., 2016). The synergy concept has been investigated in different tasks. Examples regarding hand control are grasping imagined objects (Santello et al., 1998), reachto-grasp (Mason et al., 2001), and precision grip (Grinyagin et al., 2005).
Another key aspect in the generation of meaningful motor actions was recognized in the interplay between synergistic components and the external environment (Feldman et al., 2015). Indeed, from a sensory point of view, it plays a crucial role to build up our knowledge of the world. Under this regard, a well-known characterization of hand movements during the exploration of the environment is represented by the Exploratory Procedures identified by Lederman and Klatzky (1990). Exploratory Procedures are stereotyped hand motions characteristic of human haptic exploration of the environment. By applying these procedures, subjects try to maximize the amount of haptic information acquired from external objects. Thakur et al. (2008) analyzed Exploratory Procedures through Principal Component Analysis, identifying a reduced set of postural synergies generating the observed behavior.
Focusing instead on the use of constraints for effective motor task accomplishment, Environmental Constraint Exploitation (ECE) primitives are defined in the study of . We will refer in the following to ECE strategies as actions which aim to manipulate or grasp an object and involve the use of a constraint as central element of the strategy. Figure 1 shows some examples reproducing experimentally observed behaviors, where a subject exploits a flat surface. In Puhlmann et al. (2016), a taxonomy of ECE primitives is proposed. In , the authors also show that impairing the use of the environment during grasping and manipulation sensibly decreases subjects' performance, suggesting the central role of these strategies in humans. This way of manipulating and grasping is very different from the classical strategies used in robotics, where the contact with the environment is avoided as much as possible (Bonilla et al., 2014). Thus, it is authors' opinion that a deeper understanding of ECE can directly affect the performance of robotic manipulators.
In this work, we investigate the presence of a synergistic behavior underlying the generation of hand postures during ECE, execution, and planning. We also investigate the effect that cutaneous impairment might have on such synergies. We were motivated in doing this by the many studies in the literature highlighting the central role of tactile information in grasping (Johansson and Westling, 1984;Westling and Johansson, 1987;Nowak et al., 2004).
We performed experiments with six participants, who were asked to grasp a set of objects from a flat surface. We selected this task since it represents a good trade-off between analytic complexity and richness of kinematic behavior induced by ECE, as accounted, e.g., by Figure 1 and in the study of . Two experimental conditions were considered: with and without cutaneous impairment. In the first case, participants were requested to wear rigid shells at their fingertips. Results were processed through Principal Component Analysis, as it is common in literature (Santello et al., 2013), in order to check the existence of a set of Principal Components describing the recorded data. We performed the analysis during the actual contact with the environment and on pre-shaping postures, i.e., during the approaching movement that precedes the hand-object-table interaction.
A marked synergistic behavior results in all the considered experiments. Both the interaction with the external environment and the cutaneous impairment appear to modify only the higher order Principal Components. Furthermore, the first synergy exhibits a good resemblance with the one observed in grasping of imagined objects (reported in Santello et al. (1998)).
In recent years, the idea of hand synergies (with special focus on grasping) has been successfully applied in robotics for a simplified yet effective design, sensing, and control of artificial systems. For a detailed review, we refer the interested reader to the studies of Alessandro et al. (2013) and Santello et al. (2016). In accordance with this previous successful experience of mutual inspiration between neuroscience and robotics, we believe that the here proposed results can inform the design and control of artificial hands able to take full advantage of the Exploitation of Environmental Constraints. We discuss these aspects in the final part of the work.

Participants
We tested six able-bodied volunteers (three females, three males; age range: 23-27 years, mean 25.17 years). All subjects were tested on their dominant hand (right hand; self-reported hand dominance). All participants were naive to the experimental purpose of the study and had no history of neuromuscular disorders. Before data collection, subjects signed an informed consent to participate in the experiment. The experimental protocols were approved by the Institutional Review Board of University of Pisa, in accordance to the Declaration of Helsinki.

Apparatus
For the present investigation we employed a multimodal acquisition setup (see Figure 2), composed of: • Phase Space Motion Capture System; we record kinematic data using a commercial system for 3D motion tracking with active LED markers, the Phase Space. 1 Ten stereo cameras working at 480 Hz tracked 3D positions of 24 markers fastened to hand and phalanges as shown in Figure 2A. The LED frequency is in the visible red. • A fingertip shell ( Figure 2B); we use the shell of ThimbleSense sensor (Battaglia et al., 2016). Subjects wore ThimbleSenses in all fingertips during the tactile impairment experiments, as in Figure 2B. Each shell is connected to the correspondent fingertip as in classical thimbles. Please refer to Battaglia et al. (2016) for a more extensive description of the impairment effects of the shells. Also force information is acquired, to be used in future works. Subjects wore Thim-bleSenses on all fingertips, during the tactile impairment experiments. • A set of 21 objects ( Figure 2D); this is composed of 2 euro coin, button badge, key, credit card, CD, comb hair color, salt shaker, tape, chessman (queen), knob, matchbox, screw, match, cigarette, rubber band, marker, screw driver, shashlik, glasses, coffee mug, and plate. • Sensorized platform (600 mm × 400 mm), which includes the force-torque sensor ATI mini45E mounted as in Figure 2C. It allows the sensing of interaction forces/torques with the table where objects were placed. • Two cameras (Logitech hd 1080p) to record from two viewpoints the experiments, as in Figure 3B. The recordings were used to visually check the effectiveness of the segmentation. We use blue filters to improve the quality of the recording, which was reduced by the poor light conditions and the red light emittance due to the motion capture system.
In contrast with previous work on grasping real objects (e.g., Mason et al., 2001), we chose objects that are difficult to grasp and therefore necessitate exploitation of the flat surface constraint. A similar set of objects was used in  in an analogous table-top scenario, demonstrating the effectiveness of eliciting a rich set of ECE strategies.
The acquisition is implemented through a custom application developed in C++ employing: • Boost libraries (Schäling, 2011) to perform the synchronization between Phase Space data and force/torque sensors. • Phase Space OWL library to get the optical tracking system data. • A custom library providing an interface to acquire the force/torque sensor data ).
The acquisition system allows organizing and synchronizing data with respect to an absolute clock with period 0.025 s.

Experiments
The experiments were designed with the aim of identifying kinetostatic primitives used by humans in grasping and manipulation tasks involving ECE. The subject was asked to comfortably seat in front of the table, as in Figure 3A. The subject was instructed to pose her/his hand in the initial hand position location, positioned at the right side of the sensorized surface as in Figure 3B. The distance between the subject's hand starting position and the site where the object was placed was 60 cm. For each trial, subjects were asked to reach the object posed in the center of the sensorized surface. Once the hand had reached the object, subjects were asked to grasp, lift (~20 cm height), hold (~1 s), put it back on the table, and place the hand back to its starting position. The experimenter gave the starting signal to subjects. In the instructions, the experimenter emphasized that the whole movement should be performed in a natural fashion, i.e., the object should be grasped as if the subject was about to use it, in accordance with Santello et al. (1998). Two trials were performed for any of the 21 objects grasped. The object order was randomized for every subject. The sequence of trials was repeated two times, with and without tactile impairment, for a total of 84 trials per subject. Each subject performed the whole experiment in a single day. The experiments with tactile impairment were performed in the morning, the experiments without tactile impairment in the afternoon, in agreement with previous experiments (Battaglia et al., 2016). Each subject performed the experiment in an independent day w.r.t the other ones.

Data Pre-Processing and Analysis
Data processing is organized in two phases. First, data are preprocessed to reduce the noise and to evaluate joint angles and contact points. Second, we perform statistical analyses to identify the presence of a synergistic behavior in ECE. We perform the analyses in pre-shaping and during the whole contact with the environment. To quantify the role of tactile impairment, we consider separately the impaired and unimpaired cases. The analysis was performed considering the whole group of participants. In Section 3.4, we report the CI for the dot products between the synergies' vectors.

Data Processing
Raw data collected from the experimental setup were force and torque from the six axes F/T sensor ATImini45 and 24 LED positions from the motion capture system, both with and without tactile impairment.

Kinematic Model of the Human Hand
An accurate description of the human hand is challenging due to the high number of bones and joints composing the human hand. As a trade-off between accuracy and complexity, in this work we considered a 20 DoF kinematic model of the human hand ( Figure 4A). Each long finger is described by a set of four angles: two DoFs for flexion-extension and abduction-adduction in metacarpophalangeal joints, one DoF for flexion-extension in proximal and distal intraphalangeal joints. The thumb is described with four angles: two DoFs for the trapeziometacarpal, one DoF for the metacarpophalangeal, and one DoF for the interphalangeal. For the sake of space, we do not report here the mathematical form of the kinematics, 2 which can be easily derived from the Denavit-Hartenberg parameterization in Figure 4B (see, e.g., Murray et al. (1994)). In the Appendix, we report the direct kinematics derivation for the led positioned on a long finger nail.
A key characteristic of the model is that it shares the 15 DoFs of the model used in Santello et al. (1998), allowing an easy comparison with the classical postural synergies of grasp, as done, e.g., in Gabiccini et al. (2013).
FIGURE 4 | Kinematic model of the human hand considered in this work. The model has 20 DoFs, which include the 15 DoFs of the hand model used in Santello et al. (1998). The left panel graphically describes the kinematics, while the right one specifies the Denavit-Hartenberg parameterization of thumb and the long fingers. We denoted with j the joint index, starting from the proximal joint to the distal, while q 1 , q 2 , q 3 , q 4 are the joint angles and a 2 , a 3 , a 4 are the phalanx lengths.

Model Calibration
To reconstruct realistic values of the joint angles from marker data, the kinematic model employed should reproduce as closely as possible the actual kinematics of the subject being recorded. To achieve this goal, we implement an identification procedure that follows what was done in Gabiccini et al. (2013). The data set is acquired by asking the subject to perform the Kapandji test (Kapandji, 1985), i.e., touching the four long fingers with the tip of the thumb. The test is repeated two times. The parameters to be identified are: a B ∈ R 30 collecting the length of each phalanx and the three space positions of the abduction joints (TR/TA, IM/IA, MM/MA, RM/RA, and LM/LA in Figure 4A) w.r.t. the local frame, and a G ∈ R 45 collecting the location of each marker placed on the phalanxes w.r.t. the joint to which the LED is connected.
Their values are evaluated as ones minimizing the Root Mean Square Error (RMSE) between measured marked positions y k and the estimated ones from the hand kinematics where x k ∈ R 20 is the vector collecting the estimation of joint angles at the kth time step, x k > 0 is to be considered element-wise, x is the vector collecting all the joint estimations in the considered time steps. We selected k id as 60 equidistant frames. The constraint x k > 0 accounts for biomechanical joint limits, in order to achieve a more robust estimation. The optimization problem is solved through the MatLab function fmincon. The initial guess for the parameters a G and a B are evaluated through direct measurements, done with a caliper on each subject's hand before the experiment, while the initial value of x is the null vector. Since we do not have direct access to the real joint angles, the quality of the calibration is evaluated as the RMSE between the measured led positions, and the ones resulting from the identification, as, e.g., in Gabiccini et al. (2013). The average error across trials and subjects is 2.2 mm for the unimpaired case and 3.1 mm for the impaired case.

Joint Angles Estimation
Articulated hand postures typically produce marker occlusion to cameras. We use here Piecewise Cubic Hermite Interpolating Polynomials (De Boor et al., 1978) to interpolate missing values. To estimate the hand postures from marker position we propose a Constrained Extended Kalman Filter (Sircoulomb et al., 2008). Kalman filtering was already employed for a similar scope in Fu and Santello (2010). We rely on the identified hand model, representing joint evolution as a random walk. In each time step, the filter estimates the hand posture from the measure of the marker position in space and previous state estimation. Joint limits are considered as constraints. To increase the robustness against marker losses (we had an average of 2.47 marker loss per frame, and a maximum number of consecutive marker loss of~15), we multiply at each step the Kalman Filter observation noise covariance matrix by the number of consecutive missing measures.
The filter is initialized with the open hand posture and a null state covariance matrix. The observation noise covariance matrix is 0.001 I 20×20 , and the state noise covariance matrix is 0.0005 I 24×24 . Both the matrices are heuristically tuned. The quality of the reconstruction is evaluated as the RMSE between measured and estimated LED positions. The mean values are 2.9 mm for the unimpaired case and 3.2 mm for the impaired case.

Force Data
The force/torque data from ATI Mini45 (i.e., sensorized surface) are filtered through a moving average filter based on the Savitzky-Golay method (Schafer, 2011). The window width is heuristically tuned as the 1.5% of total data length. We then use the knowledge of surface form, to evaluate the centroid of contact of Force/Torque data. 3

Analysis
We segment pre-processed data into three main phases, also described in Figure 5: (i) pre-shaping, where the object is reached and the hand posture is shaped in order to purposefully interact with the environment, (ii) contact, in which the constraint is exploited in order to manipulate and grasp the object, and (iii) post-contact, when the object is grasped and lifted from the table.
By considering non-adhesive interactions with the environment, we can assume any change occurring to the force orthogonal to the surface as due to an interaction. We thus segmented the actions searching for a change in the corresponding force measured by the sensorized surface (see Section 2.2). The cutoff from the first and the second phase is identified by the first contact with the table, when the force starts to increase. To accurately detect this point we consider both to the signal and to its derivative. The cutoff identifying the end of the contact phase is taken as the first time in which the contact force returns to zero.
The aim of the analysis is to identify a subspace of reduced dimensionality embedding the hand postures, to test the hypothesis of presence of a synergistic behavior in Environmental Constraint Exploitation. Principal Component Analysis (PCA) is a valuable tool to achieve this goal (Santello et al., 1998(Santello et al., , 2013Mason et al., 2001;Thakur et al., 2008). Given a set of data, described by a correlation matrix C and a mean m, PCA derives an orthonormal base of the data space, whose first element S 1 indicates the direction where data present the greatest variability. In turn, each successive component S i has the highest variability under orthogonality constraint. The ith element of the base is referred to as Principal Component of the data set. The normalized percentage of the data variability projected on each Principal Component is called explained variance of the component.
We evaluate PCA as the singular value decomposition of the data correlation matrix C, i.e., by finding an orthonormal matrix Σ, which brings C in Jordan form through the similitude transformation Σ T C Σ. In that case, the Principal Components are the column of the matrix Σ = [S 1 , . . ., S n ], and the explained variances are the corresponding eigenvalues. For further details on PCA we refer the interested reader to the study of Jolliffe (2002). If PCA is used to analyze hand postures in joint space, explained variances can be used to understand if the hand moves on a reduced set of the configurations by looking if there are few principal directions that explain the major part of the data. If this is the case we refer to such principal directions as synergies.
It is worth noticing that the use of the same calibrated kinematic model for every subject enables a coherent description of the hand configuration space as R 20 for all the experimental conditions. We consider cosine of the angle between synergy's directions as the metric to compare results of the different analyses. We evaluate it as the absolute value of the normalized dot product between the synergy vectors.
We also compare synergies resulting from our analysis with those employed for the execution of grasping, as shown in Santello et al. (1998). The kinematic model in Santello et al. (1998) takes into account a subset of the joints considered in this work (see Section 2.4.1.1). Thus, the configuration space of grasping synergies is a subset of dimension 15 of the hand configuration space considered in this work. To compare vectors, we projected full hand configurations in the corresponding subspace. This is equivalent to simply neglect the values corresponding to joints ID, MD, RD, and LD in Figure 4A.
Since our main focus is to extract lessons that can inform the robotic design (as discussed in Section 4.2), a complete study of inference is beyond the scope of the present work. However, we considered Student's t test to infer the general validity of some of our results. Please refer to the study of Hogg and Craig (1995) for an exhaustive introduction.

Pre-Shaping Analysis
The pre-shaping analysis is done with the purpose of identifying kinematic regularities in the generation of hand postures for the ECE exploitation. In Santello et al. (1998), PCA is performed for constant postures acquired in grasping. These authors took out effects due to interaction with the objects, by asking subjects to grasp imagined ones. In this analysis, we aim to achieve the same goal by performing PCA on a data set composed of the last poses before the contact with the environment in each trial, i.e., the last pose in pre-shaping phases when a purposeful interaction with the environment is planned. Unimpaired and impaired cases are separately analyzed. Both data sets are composed of 252 poses (six subjects, 21 objects, and two trials).

Contact Analysis
In order to evaluate if there are some kinematic regularities in the strategies employed by the subjects during actual exploitation of the surface (see, e.g., Figure 1), we perform PCA on data collected during the contact phase. We perform the analysis separately in the impaired and unimpaired case. To characterize the effect of cutaneous impairment we also evaluated the mean amount of time in which subjects stay in contact with the table, mean time for task accomplishment, and the mean norm of interaction forces, by averaging the correspondent values for every subject and every object. The data sets for both impaired and unimpaired conditions are composed of a variable number of poses depending on the strategy execution time. Each ECE generates an amount of postures equal to 40 times the execution time (see acquisition rate in Section 2.2). All 21 objects, two trials, and six subjects were considered. We indicate with "x" the DoFs that were not considered in Santello et al. (1998).

Differences between Pre and during Contact
We investigate the persistence of the same basic ingredients of hand posture before and during the contact with the environment for both bare fingers and cutaneous impairment conditions. The dot product of synergies evaluated in previous sections was computed in order to quantify their similarities. Figure 6 shows the explained variance associated with the PCs on the poses during the pre-shaping phase. For the unimpaired case, the first Principal Component explains about 54% of the variance, while the first three pre-shaping synergies explain more than 72%. Figure 7 shows the graphical representation of the first three resulting postural synergies. The same analysis in the impaired case shows that the first synergy explains about 42% of the variance, while three Principal Components explain more than 68% of the variance. Thus, for our data set, the variance explained by the first main synergies is lower in the impaired condition. In Figure 7, we present the graphical representations of the movements corresponding to the first three synergies of pre-shaping without tactile impairment. Table 1 presents the numerical values of the first ECE synergy of pre-shaping with and without impairment, in comparison with the first synergy of grasp (Santello et al., 1998). Figure 8 presents the movement corresponding to the second synergy of pre-shaping with tactile impairment. In all the considered cases, the first synergy describes an opening-closing behavior of the whole hand, while the second synergy corresponds to a closure of the distal joints mostly of index and medial fingers and a closing of the thumb. The third synergy is similar to the second, but concerning the little and ring fingers. Figure 9A shows the scalar product between the first grasp synergy in Santello et al. (1998) and the ones found in this work for FIGURE 8 | Graphical representation of the movements associated with the second ECE synergy. Pre-shaping impaired, contact unimpaired, and contact impaired conditions are considered. The mean posture is referred to as m, the second synergy as S 2 . We do not report here the first synergies, since they do not present visible discrepancies from each other. The present figure shows also a good coherence in the behavior described by the second synergy, among the considered conditions. both sensory conditions. What is noticeable is that there is a high level of consistency between the main synergy of grasping and preshaping of human hand in impaired and unimpaired conditions (≥0.9). The similarity is reduced for the second synergy, as shown in Figure 9B, and so on for the other orders. Figure 9 reports also a high correlation between pre-shaping synergies with and without tactile impairment. The presence of impairment does not alter the first two synergies during the pre-shaping phase. However, the similarity strongly drops when the synergy order increases further, reaching 0.36 for the third and 0.003 for the forth.

Contact Analysis
In the unimpaired condition, the first Principal Component explains about the 49% of the variance, and the first three synergies more than 73%. The same analysis in the impaired case returns a first synergy explaining about the 39% of the variance, and the three Principal Components explaining more than 65% of the variance. Also in this case, for our data set the percentage of variance explained in the impaired condition is lower. Table 1 presents the numerical values of first ECE synergy during the contact with the environment, with and without impairment, in A B FIGURE 9 | Dot products between first and second grasp synergies and first and second ECE synergies evaluated during pre-shaping, with and without tactile impairment. The gray scale graphically codes the product value: black is 1, i.e., very similar, white is 0, i.e., very different. A high correlation between the first grasping and ECE synergies is evidenced. The ECE synergies with and without impairment present high similarity, which, however, drops for higher order synergies. (A) First synergies and (B) second synergies.
FIGURE 10 | Dot products between the ECE synergies in pre-shaping and in contact with the environment, in the bare finger case. The gray scale graphically codes the product value: black is 1, i.e., very similar, white is 0, i.e., very different. A tendency to maintain the first main components before and after the contact results clearly from this analysis. The results for the impaired case are analogous.
Frontiers in Neurorobotics | www.frontiersin.org August 2017 | Volume 11 | Article 41 comparison with the first synergy of grasp (Santello et al., 1998). Figure 8 presents the movement corresponding to the second synergy with and without tactile impairment. The analysis also demonstrates that subjects with tactile impairment are in contact with the table for an average time of 4.2 ± 3.1 s, while for the unimpaired case the average time is 2.4 ± 2.4 s. The complete task is performed in 13.9 ± 2.7 s for the tactile impairment case, while it is performed in 11.5 ± 2.0 s in the other case. Finally, the contact force is different for the experiments considered. Mean value of norm of contact forces for the tactile impairment case is 23.2 ± 8.6 N, while mean value is 12.3 ± 5.7 N in the other case.

Differences between Pre and during Contact
The result shows that the first Principal Components in unimpaired and impaired conditions are very similar w.r.t. the corresponding ones of the pre-shaping analysis. The similarity tends to decrease with the increase in synergy order (correlation ≥ 0.75 till 9th synergy). Dot products are graphically reported in Figure 10 for the unimpaired case. Thus, ECs induce only changes for the high-order synergies, leaving unaltered the main ones, regardless of availability of tactile input. Note that the first synergy is still equivalent to the one in Santello et al. (1998).

Inference and Statistical Relevance
In the previous sections, we presented results from six subjects. Despite such a moderate number of participants, findings and analyses are in line with the existing literature in the field, see e.g., Santello et al. (1998); Mason et al. (2001);and Naceri et al. (2016). To generalize, we here report additional statistical analyses that provide Student's t test based confidence intervals (CI) with 95% probability. CI refers to dot products performed on the PCs extracted for the different conditions and the ones obtained in grasping.
CI for the dot product between the first synergy for impaired and unimpaired conditions is [0.81, 0.96]; CI for the dot product between the first grasping synergy and the unimpaired first ECE synergy is [0.88, 0.94]; CI for the dot product between the first grasping synergy and the impaired first ECE synergy is [0.79, 0.9]. Regarding the contact analysis of Section 3.3, the dot product between the first Principal Component before and during contact results in a CI of [0.97, 0.99] (unimpaired case).

Implications for Motor Control
The explained variance reported in Figure 6 suggests the presence of an underlying synergistic behavior in the purposeful exploitation of environmental constraints. Indeed, the dimensionality of the space required to approximate hand posture is considerably smaller than the number of degrees of freedom. In particular, the first Principal Component shows a marked predominance, with a maximum total variance explained of 54% in the unimpaired pre-shaping case. The three first synergies explain more than 65% of the total variance in all the considered conditions. In the considered experiments, during contact with the environment, the amount of variability accounted by higher order synergies increases. This could be explained by observing that the interaction can shape the subject hand beyond its nominal kinematics. This behavior is also in agreement with the findings in Santello et al. (2002), where the synergistic analysis of Santello et al. (1998) was performed on grasped real objects instead of imagined ones.
Despite our inference analysis (Section 3.4) is limited to the first synergy, a series of characteristics of our data set can be pointed out, which are in accordance with existing neuroscientific findings. Future works will focus on different experimental procedures and tasks to further investigations. In our data set, the first two synergies in the impaired and unimpaired conditions are very similar, as shown in Figures 8 and 9. This suggests that the presence of tactile impairment, while modifying the strategies themselves, does not substantially modify the most basic kinematic ingredients commonly used to generate hand postures. Subjects are aware of the presence of the tactile impairment, so it is reasonable to expect that they might have changed their planning in accordance to that. Indeed, the drop of such similarity after the third synergy suggests that cutaneous impairment affects posture refinement, which can be likely ascribed to higher order synergies (as described in Santello et al. (1998)). However, the amount of data collected does not allow a sound statistical characterization of this behavior. To assess whether higher order synergies are primarily noise or they actually contribute to hand postures, we will resort to the usage of discriminant analysis and information theory, as in Santello et al. (1998), as future works.
Analogously, Figure 10 shows a high similarity in the first Principal Components during contact and pre-shaping, while differences can be observed for higher order synergies. Uncontrolled Manifold theory (Scholz and Schöner, 1999;Latash et al., 2007) suggests that the central nervous system selects in the space of joint angles a subset of variables of interest, which are regulated, purposefully leaving free the remaining variables. The persistence of main postural synergies of pre-shaping during the contact with the environment can be interpreted in the light of this theory by considering the first set of ECE synergies as the variables of interest for the considered task, which remain constant when an external disturbance occurs. The subspace individuated by the higher order synergies is instead left free to adapt to the external environment. Moreover, such behavior could also be due to peripheral constraints embedded in the musculoskeletal system, as discussed, e.g., in Santello et al. (2013).
Data shown in Figure 9 and Table 1 demonstrate that a strong resemblance exists between the first synergy, resulting from the analysis of ECE strategies, and the first synergy of grasp as found in Santello et al. (1998).
This could suggest the presence of underlying synergies, which are integrated with task specific ones. This was proposed, e.g., in Gorniak et al. (2007), and it is in agreement with experimental results presented in Thakur et al. (2008), where task independent synergies are estimated by a set of unconstrained tasks (see CI estimates in Section 3.4).

Implications for Robotics
As it can be widely observed in the literature, neuroscientific results of synergistic behavior of human hands have been successfully translated and applied to robotics to inform the design, control, and sensing of artificial systems, with special focus on grasping, see Bicchi et al. (2011) and Bianchi and Moscatelli (2016). One of the first notable applications of synergies to robotics was in Brown and Asada (2007), where authors propose to use grasp synergies to derive actuation patterns for an underactuated robotic hand. In Gabiccini et al. (2011), the use of hand synergies for the choice of grasping forces is discussed. In Ciocarlie et al. (2007) and later in Amor et al. (2012); Malhotra et al. (2012); and Villani et al. (2012), a synergy based low-dimensional synergistic space is considered to obtain effective pre-grasp shapes for fully actuated robotic hands.
Recently, synergy-inspired actuation has been combined with the introduction of compliance in the structure Xu et al., 2014;Della Santina et al., 2015;Chen and Xiong, 2016) (according to the soft synergy framework ). The availability of robotic hands embedding elasticity in their mechanics has also led to a shift in their control philosophy, accounted, e.g., in Bonilla et al. (2014) and . In the classical planning, suitable points are selected on the object to be grasped or manipulated, generating a nominal grasp of good quality. Trajectories are then executed, to correctly position the fingertips while avoiding contacts with the environment. On the contrary, soft manipulation has changed this scheme. The hand-environment contacts are no more avoided but exploited to successfully shape the hand around the object. Under this regard, the study of environmental exploitation in humans could inform the design, planning, and control of robotic hands to take full advantage from the external environment.
The most direct implication of the presented results could leverage upon the observation that the first synergy of grasping is very similar to the first synergy for Environmental Constraint Exploitation (ECE). Thus, the implementation of the first synergy of grasp as degree of actuation can target the twofold goal of realizing underactuated robotic hands that can effectively grasp objects and, at the same time, are able to exploit Environmental Constraints. To increase hand functionalities beyond the first degree of actuation, we could implement additional ECE synergies, possibly in combination with the grasp synergies. For some examples of the implementation of synergies for the design of underactuated robotic hands, we refer the interested reader to Grioli et al. (2012); Della Santina et al. (2015); and Piazza et al. (2016).
Looking at the differences between the impaired and unimpaired conditions, the key kinematic ingredients seem to remain unaltered at least for the gross movements. However, the time for task accomplishment and the force exchanged with the environment is higher for the impaired case. This result could indicate possible sensing strategies for soft robotic hands, i.e., to detect contact with the environment, e.g., through IMU sensors, which can lead to the development of planning and control laws aiming at minimizing force execution on external objects.

ETHICS STATEMENT
Before data collection, subjects signed an informed consent to participate in the experiment. The experimental protocols were approved by the Institutional Review Board of University of Pisa.

A.1. Finger Kinematics
The forward kinematics of a serial chain with n degrees of freedom is described by post-multiplicating different rotation-translation matrices with the following form where R i−1,i is the rotation matrix between frame i − 1 and frame i and h i−1,i is the translation vector between frame i − 1 and frame i. Then the whole forward kinematics is calculated as T 0,n = T 0,1 T 1,2 . . . T n−1,n .
For the long finger (its kinematics is introduced in section 2.4.1.1 and parametrized with Denavit-Hartenberg in Figure 4B) we have where C i stands for cos(q i ), S i stands for sin(q i ), a i is the length of the phalanx i − 1 and q i is the ith joint rotation. Then, the forward kinematics is obtained as T 04 = T 01 T 12 T 23 T 34 = ( R 0,4 h 0,4 0 1×3 1 ) .
The extended expression of T 04 is neglected for simplicity and can be easily calculated by the reader. The final position ( x LF y LF z LF ) T of the external phalanges can be extracted from T 04 by selecting h 0,4 :   x LF y LF z LF   =   C 1 (a 2 C 2 + a 3 C 23 + a 4 C 234 ) S 1 (a 2 C 2 + a 3 C 23 + a 4 C 234 ) −a 2 S 2 − a 3 S 23 − a 4 S 234   , where C 23 stands for cos(q 2 + q 3 ), C 234 stands for cos(q 2 + q 3 + q 4 ), S 23 stands for sin(q 2 + q 3 ), S 234 stands for sin(q 2 + q 3 + q 4 ). The same procedure can lead to the description of the thumb forward kinematics. We have and then a 4 S 4 (C 3 S 1 − C 1 C 2 S 3 ) + a 2 C 1 C 2 + a 3 S 1 S 3 +a 4 C 4 (S 1 S 3 + C 1 C 2 C 3 ) + a 3 C 1 C 2 C 3 a 2 C 2 S 1 − a 4 S 4 (C 1 C 3 + C 2 S 1 S 3 ) −a 3 C 1 S 3 − a 4 C 4 (C 1 S 3 − C 2 C 3 S 1 ) + a 3 C 2 C 3 S 1 a 4 S 2 S 3 S 4 − a 3 C 3 S 2 − a 4 C 3 C 4 S 2 − a 2 S 2         .
(A1) It is worth to be noticed that, while linear w.r.t. the parameters a 2 , a 3 , a 4 , equation (A1) defines a strongly non-linear relationship between joint angles q 1 , q 2 , q 3 , q 4 and LED position.