Original Research ARTICLE
Intra-Auditory Integration Improves Motor Performance and Synergy in an Accurate Multi-Finger Pressing Task
- 1Department of Kinesiology, University of Maryland, College Park, MD, USA
- 2Department of Physical Education, Hanyang University, Seoul, South Korea
- 3Neuroscience and Cognitive Science Program, University of Maryland, College Park, MD, USA
- 4Applied Mathematics and Statistics, and Scientific Computation Program, University of Maryland, College Park, MD, USA
- 5Department of Mechanical Engineering, Kyung Hee University, Yongin-Si, South Korea
- 6Department of Stroke Rehabilitation, National Rehabilitation Center, Seoul, South Korea
- 7Department of Bioengineering, University of Maryland, College Park, MD, USA
Humans detect changes in the air pressure and understand the surroundings through the auditory system. The sound humans perceive is composed of two distinct physical properties, frequency and intensity. However, our knowledge is limited how the brain perceives and combines these two properties simultaneously (i.e., intra-auditory integration), especially in relation to motor behaviors. Here, we investigated the effect of intra-auditory integration between the frequency and intensity components of auditory feedback on motor outputs in a constant finger-force production task. The hierarchical variability decomposition model previously developed was used to decompose motor performance into mathematically independent components each of which quantifies a distinct motor behavior such as consistency, repeatability, systematic error, within-trial synergy, or between-trial synergy. We hypothesized that feedback on two components of sound as a function of motor performance (frequency and intensity) would improve motor performance and multi-finger synergy compared to feedback on just one component (frequency or intensity). Subjects were instructed to match the reference force of 18 N with the sum of all finger forces (virtual finger or VF force) while listening to auditory feedback of their accuracy. Three experimental conditions were used: (i) condition F, where frequency changed; (ii) condition I, where intensity changed; (iii) condition FI, where both frequency and intensity changed. Motor performance was enhanced for the FI conditions as compared to either the F or I condition alone. The enhancement of motor performance was achieved mainly by the improved consistency and repeatability. However, the systematic error remained unchanged across conditions. Within- and between-trial synergies were also improved for the FI condition as compared to either the F or I condition alone. However, variability of individual finger forces for the FI condition was not significantly decreased as compared to I condition alone. This result indicates an improvement in motor performance is consistent with Bayesian estimation, and changes in multi-finger interaction mostly result in the enhanced motor performance. These findings provide evidence that the central nervous system can take advantage of the intra-auditory integration in a statistically optimal (Bayesian) fashion to enhance motor performance by improving multi-finger synergy.
For nearly every real-world case of motor behavior, the central nervous system (CNS) simultaneously receives information from a variety of sensory modalities, including visual, tactile, and auditory signals (Ernst and Banks, 2002; Stein and Stanford, 2008; Berniker and Kording, 2011), and is required to coordinate multiple effectors (e.g., motor units, muscles, joints, limbs) to effectively perform the movement (Babinski, 1899; Castiello, 1997; Alexandrov et al., 1998; Shim et al., 2003b). Many theories of motor control argue that integration of these multiple sensory modalities with the synergistic control of multiple effectors is critical for successful or optimal motor performance (Ernst and Banks, 2002; Berniker and Kording, 2011; Pantall et al., 2012; Scholz et al., 2012). In effectively integrating multi-sensory information and controlling multi-effector system, the CNS faces at least two challenges: it must estimate the state of the system with sensory information that is corrupted by noise and transmission delays (Körding and Wolpert, 2004; Faisal et al., 2008), and it must solve the motor redundancy problem when deciding how effectors are used from a theoretically infinite set of possibilities (Bernstein, 1967; Gel'Fand, 1971; Ting and Macpherson, 2005; Latash et al., 2007).
Concerning the first problem (state estimation), previous studies on inter-sensory integration (e.g., visual and auditory modalities) have suggested that the CNS combines multiple sensory modalities to produce minimum variance in the state estimate. Bayesian inference is a promising approach to interpret integration of multiple sensory modalities and to define the statistically optimal way to maximize accuracy of the state estimation (Ernst and Banks, 2002; Berniker and Kording, 2011). Relatedly, each individual sensory modality consists of multiple components that can be perceived distinctively by the CNS, such as the color and shape of an object in vision, or the frequency and intensity of a sound in audition (Helmholtz and Ellis, 1885; Bloj et al., 1999). However, our knowledge of how the CNS integrates these different components within a modality (i.e., intra-sensory integration) is very limited. For example, most previous studies on intra-sensory integration were on vision (Jacobs, 1999; Knill and Saunders, 2003; Hillis et al., 2004), with relatively little known about intra-sensory integration of auditory feedback. Sound is physically a pressure wave transmitted through the air, and humans can normally hear sound waves with frequencies up to ~10,000 Hz and intensities up to ~120 dB. These two physical quantities (frequency, intensity) are the most salient features of sound that contribute to its perception, and they appear to be perceived independently by the CNS (Helmholtz and Ellis, 1885; Zagorski, 1975). However, it remains unclear if and how the CNS integrates auditory frequency and intensity for state estimation.
Concerning the second problem (motor redundancy), the control of multiple motor effectors has been studied at the level of muscles (Babinski, 1899; Smith, 1993; Ting and McKay, 2007), joints (Castiello, 1997; van der Kamp and Steenbergen, 1999; Latash and Jaric, 2002), and body segments (Alexandrov et al., 1998; Kim et al., 2012), including fingers (Latash, 2000; Latash et al., 2002, 2007; Shim et al., 2003b, 2005b, 2008). From these studies, the notion of motor synergy has been developed to describe the interactions between multiple motor effectors for the successful completion of a given motor task (Latash et al., 2007). One of our previous simulation studies suggested a central back coupling hypothesis showing that it was theoretically possible to generate the synergistic actions between motor effectors in a feedforward system without an active involvement of sensory feedback (Latash et al., 2005). However, several previous studies suggested that sensory feedback plays an important role in synergistic interactions between motor effectors (Ranganathan and Newell, 2008; Shim et al., 2012; Koh et al., 2015). Ranganathan and Newell (2008) found that the removal of visual sensory feedback from combined visual and tactile feedbacks resulted in decreased synergistic actions between finger forces in both within-trial (online) and between-trial (offline) during multi-finger pressing. In addition, our recent work showed that the removal of tactile feedback in multi-finger pressing tasks resulted in decreases in multi-finger synergies for online control (Koh et al., 2015). These studies suggest that sensory feedback plays critical roles in multi-finger synergies and sensory integration may also systematically influence the synergistic interactions between motor effectors. Especially, the results of these studies imply that sensory integration may lead to improvements in multi-finger synergy (i.e., decrease covariance between finger forces).
Previous studies have shown that multi-finger actions are controlled in a hierarchical manner with at least two hierarchical levels: individual finger actions at the lower level, and virtual finer (VF: an imagined finger producing the same mechanical effect as all fingers together) actions through the combined mechanical outcomes of individual finer actions at the higher level (Arbib, 1985; MacKenzie and Iberall, 1994; Baud-Bovy and Soechting, 2001; Shim et al., 2005a,b), at the higher level. In our recent work, we developed hierarchical variability decomposition (HVD) model to quantify the hierarchical organization of multi-finger actions with the separate analysis of within-trial (online) and between-trial (offline) motor behaviors (Koh et al., 2015). The HVD model decomposes the variability in the motor system into mathematically independent components each of which quantifies a distinct motor behavior. In the HVD model, the systematic error, consistency, and repeatability are quantified at the higher level. In a constant force production task, the systematic error reflects the CNS's ability to estimate the target force, and consistency reflects the CNS's ability to perform the task on a moment-to-moment basis (i.e., online variability), while repeatability reflects the ability to reproduce the same task goal on trial-to-trial basis (i.e., offline variability). The analysis of online and offline variability were previously used to infer different aspects of the control mechanism in redundant motor systems (Scholz et al., 2003; Ranganathan and Newell, 2008; Koh et al., 2015). In the HVD model, both consistency and repeatability can be explained by the sum of individual finger force variances and the amount of error compensation (or amplification) among individual finger forces (i.e., motor synergy; Latash et al., 2005; Martin et al., 2009) at the lower level. Here, we tested how the intra-auditory integration affects these several aspects of multi-finger actions quantified through the HVD model.
In the present study, we therefore investigated the effect of intra-auditory integration between frequency and intensity of sound on motor performance and motor synergy during a constant multi-finger force production. We hypothesized that motor task errors would decrease when both frequency and intensity components are presented to subjects as compared to the conditions that present only one of these auditory components to them, which would support Bayesian integration between frequency and intensity of sound. We also hypothesized that intra-auditory integration would be associated with enhanced multi-finger synergy, which would indicate an evidenced role of intra-sensory integration in the interactions between redundant motor effectors in humans.
Ten right-handed male volunteers (age 24.2 ± 1 years) participated in the study. Participants were free of neurological, psychiatric, speech-language, and motor impairments. No participant had more than 1 year of musical training. Participants provided written informed consent. All procedures were approved by the University of Maryland College Park Institutional Review Board.
Finger pressing forces at the distal phalanges were collected using load cells (Nano 17, ATI Industrial Automation, Apex, NC, US) at a sampling frequency of 1000 Hz with data acquisition hardware (6024E, National Instruments Corporation, Austin, TX, US) using a custom LabVIEW program (LabVIEW 8.2, National Instruments Corporation, Austin, TX, US). The program interfaced with a function generator (Agilent 33522A, Keysight Technologies, Inc., Santa Rosa, CA, US) to register the individual finger forces and calculate the virtual finger (VF) force as the sum of individual finger forces. The program also generated auditory signals played through the left and right ears of headphones worn by the subjects (AE2, Bose Corporation, Framingham, MA, US).
In order to minimize distortion of sound due to headphone frequency response characteristics (Jackson and Vinegar, 1979), the auditory signal was calibrated to produce a constant intensity across all frequencies. Calibration was performed in a soundproof room by manipulating frequency from 20 to 10,000 Hz in 1 Hz increments and normalizing intensity at each increment.
Subjects sat on a chair, wore the headphones, and placed the tips of their right hand's fingers (index, middle, ring, and little) on the load cells (Figure 1). The subjects were asked to use these fingers to produce a constant VF force of 18 N over 20 s while they received auditory feedback tones on the reference force (left ear) and on the VF force (right ear). The tone for the reference force (i.e., the reference tone) had frequency of 1000 Hz and intensity of 70 dB. The tone for VF force (i.e., the tracking tone) varied in three different experimental conditions:
1) Frequency condition (F): the frequency of the tracking tone was modulated with the deviation of the subject's VF force from 18 N, while the intensity of the tracking tone was kept constant at 70 dB.
2) Intensity condition (I): the intensity of the tracking tone was modulated with the deviation of the subject's VF force from 18 N, while the frequency of the tracking tone was kept constant at 1000 Hz.
3) Frequency and Intensity condition (FI): both frequency and intensity of the tracking tone was modulated with the subject's VF force.
Figure 1. Experimental setup. The subjects sat and placed their right hand finger tips on the sensors, wearing the headphones (A). During the task, the reference force (18 N) was provided as an auditory signal, a sinusoid signal with frequency 1000 Hz and intensity 70 dB (i.e., reference tone), played to the subject's left ear (B in red). The force generated by the subject was also provided as a sinusoid signal (i.e., tracking tone) to their right ear according to three feedback conditions: frequency (F) in which the frequency of the tracking tone changed, intensity (I) in which the intensity of the tracking tone changed, and frequency and intensity condition (FI) in which both the frequency and the intensity of the tracking tone changed according to the force the subject generated (B in blue).
The frequency of 1000 Hz for the reference tone was used to minimize the influence of binaural beats (Oster, 1973; Wahbeh et al., 2007), and the intensity of 70 dB for the reference tone was selected as mid-level hearing. The feedback gains of frequency and intensity were 7 Hz/N and 0.7 dB/N, respectively, according to “just noticeable differences” of frequency and intensity previously reported assuming a change in force of 1 N (Moore, 1973; Ozimek and Zwislocki, 1996). Prior to the data collection, participants were given five familiarization trials that were excluded from analysis. From each 20-s trial, the 11-s window from 6 to 17 s where the VF force was relatively constant was extracted for analysis (Koh et al., 2015) in order to avoid the initial force stabilization in the beginning of each trial and premature cessation of force production at the end. Subjects completed 15 trials and with rest 30 s rest between trials to prevent fatigue. The order of conditions was balanced across the subjects.
Analysis and Bayesian Model
The Bayesian approach in sensory integration research describes how sensory information can be used to update our knowledge of some quantity of interest (Jacobs, 1999; Ernst and Banks, 2002; Berniker and Kording, 2011). Our study had the following assumptions for analysis:
1. The Bayesian model for the sensory integration appropriate for static performance can also be used for dynamic performance (Ronsse et al., 2009). Let the random variable S be the true unknown VF force. If S has the specific value s, then (i) the estimated VF force based on frequency feedback, ŜF, and estimated VF force based on intensity feedback, ŜI, are independent Gaussian random variables with means s+bF and s+bI and variances and , respectively, where bF and bI and are biases; (ii) the estimated VF force based on both frequency and intensity feedback, ŜFI, is the Bayesian estimate assuming a uniform prior over the range of feasible VF forces and ignoring the biases bF and bI (Scarfe and Hibbard, 2011; Shi et al., 2013). Let s+bFI and denote the mean and variance, respectively, of the random variable ŜFI, where bFI is the bias of the combined estimate.
2. The variances of VF force in the F, I, and FI conditions primarily reflect the estimation variances, , , and , respectively. Additional variability in VF force caused by noise in other sensory systems and noise in the motor system is assumed to be small in all experimental conditions. The biases bF, bI, and bFI equal fT minus the mean VF force in the F, I, and FI conditions, respectively, where fT is the reference force of 18 N.
Based on these assumptions of the Bayesian model, the posterior probability density describing our knowledge of S given specific measurements ŜF = ŝF and ŜI = ŝI is:
The posterior has a Gaussian distribution. Its mean, ŝFI, is a weighted sum of the estimates ŝF and ŝI:
where and . The variance of the posterior, , is smaller than the variances of the frequency estimate, , and intensity estimate, :
To test whether intra-auditory components, frequency and intensity, are integrated for the motor performance enhancement, we calculated the overall mean-squared error (OMSE) from the experimental data, the averaged squared deviation of the VF force from the reference force as a measure of motor performance:
where yi(t) is VF force at trial i, and τ is the duration of yi(t).
In order to examine if the intra-auditory integration follows the Bayesian model, OMSEFI was also estimated according to Bayesian model using the bias and variance values we experimentally obtained from the F condition and I condition:
OMSEFI estimated from the model was then compared to OMSE obtained from the experiment (Figure 2).
Figure 2. Left figures show VF force time courses for all 15 trials from a representative subject for F condition (red), I condition (green), and FI condition (blue). Right Figure shows best fitted Gaussian distribution of VF force for all conditions and a Gaussian distribution calculated by the Bayesian model (black).
Hierarchical Structure and Decomposition of Motor Variability
The VF force for trial i, yi(t), was modeled as the sum of three components:
where Xi(t) is the demeaned VF force for trial i, m is the mean VF force after averaging over all time steps of all 15 trials, and Ei is the difference between the mean VF forces for trial i and m.
OMSE was partitioned into three error components that influence the motor performance following the HVD model (Koh et al., 2015): the “online variance” , defined as the variance within a trial, averaged over trials, the “offline variance” , defined as the variance between trials, and the “systematic error,” b2, defined as squared deviation between the reference force and m. The sum of online and offline variances is the variance of the VF force (), and the systematic error is the squared bias of the VF force (). The online and offline variances can also be defined in terms of the individual finger force variances and covariances as shown below.
where n is the number of task fingers (n = 4), and σxi, xj are a variance of within-trial ith finger force and a covariance between within-trial ith and jth finger forces, respectively, and and σei, ej are a variance and covariance for between-trial finger forces, respectively. The overhead bars indicate means over trials.
According to the HVD model (Figure 3), the online and offline errors can be further defined as the sum of the individual finger force variances, plus between-finger covariances. Here, we use the covariances between finger forces, and ∑σei, ej, as indices of multi-finger synergy for online and offline, respectively (Koh et al., 2015). Negative covariance indicates that the individual finger forces synergistically act to compensate each other errors and attenuate the VF force error (i.e., error compensation), while positive covariance means that individual finger forces act synergistically, but each finger's errors are accumulated and the VF force error increases (i.e., error amplification; Shim et al., 2003a; Latash et al., 2007). The covariances between finger forces, and ∑σei, ej that we use as an index of multi-finger synergy are mathematically similar to the motor synergy indices used in other previous studies (Shim et al., 2004; Latash, 2008, 2010; Ranganathan and Newell, 2008; Karol et al., 2011). One advantage of using covariances is that the total variance can be linearly decomposed into individual finger variances and inter-finger covariances (Koh et al., 2015). Therefore, covariances were used in this study as indices of motor synergies. We define motor synergy as task-specific interactions between motor effectors employed by CNS for enhancement of motor performance and utilization of degrees of freedom in a redundant motor system (Latash et al., 2007; Shim et al., 2008). The sum of individual finger force variances is the total variance in the motor task, while the between-finger covariance reflects synergistic interactions of individual finger forces. The total variance of the VF force (i.e., the sum of individual finger variances) was also decomposed into task-relevant and task-irrelevant variances according to the Uncontrolled Manifold theory (Scholz and Sch¨oner, 1999; Shim et al., 2004; Latash, 2010) to examine the task-relevant and task-irrelevant variances related to motor synergy.
Figure 3. Hierarchical organization of multi-finger force variability. The overall mean squared error (OMSE) is composed of or the linear sum of the variance of the VF force (σ2) and the systematic error (b2). The variance of the VF force (σ2) is composed of or the linear sum of the intra-trial moment-to-moment variance (online variance; ), the time-averaged trial-to-trial variance (offline variance; ), and at the virtual finger (VF) level where the task is performed with the sum of all finger forces (VF force). The online and offline variances at the VF level are composed of or the linear sum of individual finger (IF) force variances and between-finger force covariances at the IF level.
Paired t-test was used to compare OMSEFI experimentally obtained and OMSE statistically estimated. One-way repeated-measures ANOVA with Fisher's post hoc pairwise multiple comparisons were used to compare three feedback conditions. A simple regression analysis was performed to determine whether either of mean or variance of VF force at each trial increase or decrease as a function of trial. The level of statistical significance was set at p = 0.05.
Effects of Intra-Auditory Integration on Motor Performance
We found that intra-auditory integration followed Bayesian model. The OMSEFI calculated from the Bayesian model did not differ from the experimentally obtained OMSE [t(9) = 0.896, p = 0.393]. There were also no differences between the model and the experimental data for VF variance [t(9) = 0.614, p = 0.554] or the systematic error [t(9) = 2.10, p = 0.065; Figure 4]. We also found that intra-auditory integration enhanced motor performance quantified as OMSE. Repeated-measures ANOVA showed that OMSE experimentally obtained from the FI condition was smaller than both OMSEF and OMSEI from the F condition and the I condition, respectively [F(2, 9) = 12.76, p < 0.001; FI vs. F: p = 0.003, and FI vs. I: p = 0.004]. Using HDV model, OMSE decomposed into online and offline variance of the VF force, and the systematic error, which are reflected as consistency, reproducibility, and accuracy of motor task, respectively. We found that the enhanced motor performance achieved by improving consistency and reproducibility of the VF force with unchanged accuracy. The variance of the VF force in the FI condition was also smaller than those from the F condition and the I condition for both online control [F(2, 9) = 8.92, p = 0.002; FI vs. F: p = 0.009, and FI vs. I: p = 0.002] and offline control [F(2, 9) = 10.07, p = 0.001; FI vs. F: p = 0.003, and FI vs. I: p = 0.015]. The systematic errors did not differ across the feedback conditions [F(2, 9) = 2.088, p = 0.153]. Thus, the decreases in OMSE were mainly due to the reduction of both online and offline variances, rather than systematic errors.
Figure 4. VF level variables: overall mean square error (OMSE), varaince, sestematic error, and online and offline variances at the VF level for F condition (red), I condition (green), and FI condition (blue). These variables are associated with task performance. Error bars represent s.e.m. across subjects. OMSE calculated from the Bayesian model is also shown. Asterisk indicates statistical significance (P < 0.05).
Effects of Intra-Auditory Integration on Motor Synergy
The VF variance in this model is mathematically equivalent to the sum of individual finger variances and covariances for both online and offline control, which allows us to investigate the source of changes in VF variances. We found that intra-auditory integration enhanced both online and offline motor synergy. Repeated-measures ANOVA revealed that online individual finger variance [F(2, 9) = 6.86, p = 0.006; FI vs. F: p = 0.019, and FI vs. I: p = 0.016] and covariance [F(2, 9) = 7.38, p = 0.005; FI vs. F: p = 0.011, and FI vs. I: p = 0.003] were lower in the FI condition than in the F and I conditions (Figure 5). In contrast, offline IF variance in the FI condition was smaller than the F condition, but there was no difference between the FI condition and the I condition [F(2, 9) = 9.92, p = 0.001; FI vs. F; p = 0.002, and FI vs. I; p = 0.064]. Offline individual finger covariance was smaller in the FI condition than the F and I conditions [F(1.276, 9) = 9.85, p = 0.006; FI vs. F: p = 0.004, and FI vs. I: p = 0.019]. Note that smaller individual finger covariance represents smaller error amplification between individual finger forces (Koh et al., 2015). In analyses of task-relevant and task-irrelevant spaces, both online and offline individual finger variances in the task-relevant space were smaller in the FI condition as compared to the F and I conditions [online: F(2, 9) = 8.92, p = 0.002; FI vs. F: p = 0.009, and FI vs. I: p = 0.002, and offline: F(2, 9) = 10.07, p = 0.001; FI vs. F: p = 0.003, and FI vs. I: p = 0.015], while both online and offline individual variances in task-irrelevant space remained unchanged throughout the feedback conditions [online: F(2, 9) = 2.725, p = 0.093, and offline: F(2, 9) = 1.001, p = 0.387]. Online individual finger covariance in both F and I conditions was statistically greater than zero [F: t(9) = 3.020, p = 0.014 and I: t(9) = 3.997, p = 0.003] while online covariance in FI condition did not significantly differ from zero [t(9) = 1.429, p = 0.187]. Offline individual finger covariance in all conditions was statistically greater than zero [F: t(9) = 6.056, p < 0.001, I: t(9) = 6.845, p < 0.001, and FI: t(9) = 4.326, p = 0.002].
Figure 5. Multi-finger force covariance and individual finger force variance for online and offline controls for F condition (red), I condition (green), and FI condition (blue), at the individual finger (IF) level. The individual fingers' contributions to the overall covariance and variance values are shown in the 4-dimensional diamond graphs (A,C,E,G). Error bars represent s.e.m. across subjects. The accompanying bar graphs show the overall covariance and variance values (B,D,F,H). The covariance and variance, only in online control, showed statistically significant differences between FI and either F or I conditions. The overall variance values are decomposed into task-relevant and task-irrelevant variance values (I–L). Task-relevant variance for both online and offline controls showed statistically significant differences between FI and either F or I condition. Asterisk indicates statistical significance (p < 0.05).
We also tested whether a learning effect exists using a simple linear regression. The mean and variance of VF force across time at each trial was calculated to test if these values significantly increase or decrease as a function of trial. No significant regression was found for mean of VF force in any of the conditions [F: F(1, 13) = 4.296, p = 0.059, I: F(1, 13) = 0.050, p = 0.827, and FI: F(1, 13) = 0.108, p = 0.748], along with no significant regression for variance [F: F(1, 13) = 4.515, p = 0.053, I: F(1, 13) = 2.313, p = 0.152, and FI: F(1, 13) = 2.392, p = 0.146].
Previous studies on sensory integration have mainly focused on the CNS mechanisms of inter-sensory integration, e.g., visual and auditory, visual and tactile, or visual and proprioceptive (Ernst and Banks, 2002; Alais and Burr, 2004; Helbig and Ernst, 2007; Butler et al., 2010; Fetsch et al., 2010; Reuschel et al., 2010). The current study, for the first time, showed how intra-sensory integration of frequency and intensity in auditory feedback affected motor performance and multi-finger synergy in a constant finger force production task. We reported that motor performance could be enhanced through intra-auditory integration as evidenced by improved tracking of the target force by the VF force. In addition, we report that multi-finger synergy in producing that VF force is enhanced through intra-auditory integration as evidenced by decreased covariance between individual finger forces.
Our study employed the hierarchical organization of multi-finger actions and used the HVD model previously proposed by our group for the separate analysis of within-trial (online) and between-trial (offline) motor behaviors (Koh et al., 2015). We found that providing feedback on both intensity and frequency was associated with the decreased covariances among individual finger forces in both online and offline control. This result indicates that the CNS responded to the presentation of feedback on multiple auditory components by decreasing error amplification among individual finger forces, resulting in both greater consistency and greater repeatability of the VF force.
Positive covariance between individual finger forces indicates that individual finger forces synergistically interact (Latash, 2010), but amplify the VF force error in the constant multi-finger pressing task, while negative covariance implies that individual finger forces interact to attenuate the VF force error, or to maintain a relatively constant VF force when individual finger forces are fluctuating. The ability to produce a particular VF with fluctuating individual finger forces can be viewed as a sign of versatility or flexibility within the motor system. The current study showed that the intra-auditory integration was associated with decreased positive covariance between individual finger forces, i.e., decreased error amplification at VF level. Many previous studies that employed visual force feedback often reported negative covariance between individual finger forces in similar tasks of constant VF force production (Shim et al., 2003b, 2004; Latash, 2010; Karol et al., 2011). Contrary to these studies, our study showed positive covariance between individual finger forces. This covariance difference may be due to the differences in the level of feedback uncertainty in sensory feedback the CNS received, leading to relatively high task uncertainty introduced by the auditory feedback used in our study as compared to the visual feedback used in the previous studies. This speculation is consistent with the study by Ranganathan and Newell (2008) in which they demonstrated that the covariances between individual finger forces could be systematically modulated by the uncertainty of the sensory feedback. Reduction in positive covariance between individual finger forces with intra-auditory integration would also be consistent with the principle of minimal interactions (Gelfand and Tsetlin, 1966), where interactions between low-level elements of a hierarchy are organized such that the inputs from one element to another (i.e., interactions) are minimized in an optimal system.
In addition, it was observed that offline individual finger variances and covariances were 2–5 times greater than online counterparts. The large differences between online and offline control suggest that the individual finger forces fluctuated more and the coupling between individual finger forces were stronger between trials as compared to within a trial. This observation indicates that the CNS used larger “workspaces” of individual finger forces and the inter-finger interactions were greater between individual finger forces in offline control than in online control. The large fluctuation of individual fingers forces in offline may be contributed by two different mechanisms: active utilization of the redundant degrees of freedom in the multi-finger system in offline control, and/or greater noise in the offline part of the control system. According to the Bayesian model, prior knowledge about the relative likelihood of different stimulus values can be integrated with new sensory information to improve estimates of stimulus values (Körding and Wolpert, 2004). Prior knowledge is task-specific and achieved through experience with that particular task. Although our analysis assumes no prior knowledge, it may be possible that prior knowledge gained from previous moments within a trial or previous completed trials may have contributed to the motor outcomes. If it is indeed the case, it is more likely that the prior knowledge influenced motor behaviors more within a trial as the sensory feedback is continuously provided online contrary to the offline counterpart. Future research specifically on prior knowledge and its impact on both online and offline control may enhance our understanding of the role of intra-auditory integration in motor performance and motor synergy.
Sensory integration research has often used congruent or incongruent multi-sensory information to provide sensory feedback for the same or a different state of physical property, respectively. For example, sound and light signals can come from a same position (congruency) or different positions (incongruency) in an experiment where a subject is instructed to identify the location of the sensory source (Battaglia et al., 2003). The previous studies showed that both congruent and incongruent feedback conditions follow the Bayesian model (Knill and Pouget, 2004; Körding and Wolpert, 2004). Our research investigated only the congruent auditory feedback where we provided the information about the same state of sound due to the technical difficulties to provide different reference force levels required for incongruent auditory feedback. According to the previous studies, force magnitude is associated with force variability (Schmidt et al., 1979; Galganski et al., 1993; Enoka et al., 1999). In other words, the force variability can affect motor performance and potentially motor synergy as well. Because of these reasons, our experiment did not involve incongruent auditory feedback in the effort to prevent adding additional complexity in the study design.
The feedback gains used in the current study were set according to “just noticeable differences” previously reported assuming a change in force of 1 N. The use of the feedback gains through individual auditory sensitivity test may have provided more accurate measures into the intra-auditory integration. In addition, we manipulated two physical quantities, frequency, and intensity. These quantities are transformed as physiological quantities, pitch, and loudness. The psychophysical transformation from frequency-intensity space to pitch-loudness space may not be a homeomorphism. However, the sensitivity test during a constant force production task demands a new method for accurate quantification through careful experiments and modeling because the just noticeable difference may be subject to changes during the dynamic process of finger force production.
In conclusion, we found that both motor performance and motor synergy during a constant multi-finger force production were enhanced through intra-auditory integration, which supports the idea that the CNS can integrate multiple sensory components within the auditory modality to enhance motor outputs consistent with the Bayesian model. In addition, our results provide new evidence that the intra-auditory integration is also associated with improved motor synergy. Although the central back coupling hypothesis proposed in our previous study demonstrated the theoretical possibility of generating synergistic actions between motor effectors without an active involvement of sensory feedback (Latash et al., 2005), the current study provides evidences that the auditory feedback and intra-auditory integration can indeed play an important role multi-finger synergy.
Study conception and design: KK, HK, and JS. Acquisition of data: KK, HK. Analysis and interpretation of data: KK, HK, and JS. Drafting of manuscript: KK and JS. Critical revision: YP, TK, RM, YK, and JS.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This study has been supported by National Research Foundation of Korea (NRF-2012R1A6A3A04040457 and NRF-2013R1A1A2004812) and the Translational Research Center for Rehabilitation Robots, Korea National Rehabilitation Center, Ministry of Health and Welfare, Korea (NRCTR-EX15007).
Battaglia, P. W., Jacobs, R. A., and Aslin, R. N. (2003). Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1391–1397. doi: 10.1364/JOSAA.20.001391
Enoka, R. M., Burnett, R. A., Graves, A. E., Kornatz, K. W., and Laidlaw, D. H. (1999). Task- and age-dependent variations in steadiness. Prog. Brain Res. 123, 389–395. doi: 10.1016/S0079-6123(08)62873-3
Fetsch, C. R., Deangelis, G. C., and Angelaki, D. E. (2010). Visual-vestibular cue integration for heading perception: applications of optimal cue integration theory. Eur. J. Neurosci. 31, 1721–1729. doi: 10.1111/j.1460-9568.2010.07207.x
Gelfand, I. M., and Tsetlin, M. L. (1966). On mathematical modeling of the mechanisms of the central nervous ysstem, in Models of the Structural-Functional Organization of Certain Biological Systems, eds I. M. Gelfand, V. S. Gurfinkel, S. V. Fomin, and M. L. Tsetlin (Cambridge, MA: MIT Press), 9–26.
Karol, S., Kim, Y. S., Huang, J., Kim, Y. H., Koh, K., Yoon, B. C., et al. (2011). Multi-finger pressing synergies change with the level of extra degrees of freedom. Exp. Brain Res. 208, 359–367. doi: 10.1007/s00221-010-2486-x
Kim, M. J., Karol, S., Park, J., Auyang, A., Kim, Y. H., Kim, S., et al. (2012). Inter-joint synergies increase with motor task uncertainty in a whole-body pointing task. Neurosci. Lett. 512, 114–117. doi: 10.1016/j.neulet.2012.01.072
Koh, K., Kwon, H. J., Yoon, B. C., Cho, Y., Shin, J. H., Hahn, J. O., et al. (2015). The role of tactile sensation in online and offline hierarchical control of multi-finger force synergy. Exp. Brain Res. 233, 2539–2548. doi: 10.1007/s00221-015-4325-6
Latash, M. L., Shim, J. K., Smilga, A. V., and Zatsiorsky, V. M. (2005). A central back-coupling hypothesis on the organization of motor synergies: a physical metaphor and a neural model. Biol. Cybern. 92, 186–191. doi: 10.1007/s00422-005-0548-0
Martin, J. R., Latash, M. L., and Zatsiorsky, V. M. (2009). Interaction of finger enslaving and error compensation in multiple finger force production. Exp. Brain Res. 192, 293–298. doi: 10.1007/s00221-008-1615-2
Ozimek, E., and Zwislocki, J. J. (1996). Relationships of intensity discrimination to sensation and loudness levels: dependence on sound frequency. J. Acoust. Soc. Am. 100, 3304–3320. doi: 10.1121/1.416993
Pantall, A., Teulier, C., and Ulrich, B. D. (2012). Changes in muscle activation patterns in response to enhanced sensory input during treadmill stepping in infants born with myelomeningocele. Hum. Mov. Sci. 31, 1670–1687. doi: 10.1016/j.humov.2012.04.003
Reuschel, J., Drewing, K., Henriques, D. Y., Rösler, F., and Fiehler, K. (2010). Optimal integration of visual and proprioceptive movement information for the perception of trajectory geometry. Exp. Brain Res. 201, 853–862. doi: 10.1007/s00221-009-2099-4
Ronsse, R., Miall, R. C., and Swinnen, S. P. (2009). Multisensory integration in dynamical behaviors: maximum likelihood estimation across bimanual skill learning. J. Neurosci. 29, 8419–8428. doi: 10.1523/JNEUROSCI.5734-08.2009
Schmidt, R. A., Zelaznik, H., Hawkins, B., Frank, J. S., and Quinn, J. T. Jr. (1979). Motor-output variability: a theory for the accuracy of rapid motor acts. Psychol. Rev. 47, 415–451. doi: 10.1037/0033-295X.86.5.415
Scholz, J. P., Kang, N., Patterson, D., and Latash, M. L. (2003). Uncontrolled manifold analysis of single trials during multi-finger force production by persons with and without down syndrome. Exp. Brain Res. 153, 45–58. doi: 10.1007/s00221-003-1580-8
Scholz, J. P., Park, E., Jeka, J. J., Schöner, G., and Kiemel, T. (2012). How visual information links to multijoint coordination during quiet standing. Exp. Brain Res. 222, 229–239. doi: 10.1007/s00221-012-3210-9
Shim, J. K., Karol, S., Kim, Y. S., Seo, N. J., Kim, Y. H., Kim, Y., et al. (2012). Tactile feedback plays a critical role in maximum finger force production. J. Biomech. 45, 415–420. doi: 10.1016/j.jbiomech.2011.12.001
Shim, J. K., Latash, M. L., and Zatsiorsky, V. M. (2003a). The human central nervous system needs time to organize task-specific covariation of finger forces. Neurosci. Lett. 353, 72–74. doi: 10.1016/j.neulet.2003.08.079
Shim, J. K., Latash, M. L., and Zatsiorsky, V. M. (2003b). Prehension synergies: trial-to-trial variability and hierarchical organization of stable performance. Exp. Brain Res. 152, 173–184. doi: 10.1007/s00221-003-1527-0
Shim, J. K., Latash, M. L., and Zatsiorsky, V. M. (2005b). Prehension synergies: trial-to-trial variability and principle of superposition during static prehension in three dimensions. J. Neurophysiol. 93, 3649–3658. doi: 10.1152/jn.01262.2004
Shim, J. K., Lay, B. S., Zatsiorsky, V. M., and Latash, M. L. (2004). Age-related changes in finger coordination in static prehension tasks. J. Appl. Physiol. 97, 213–224. doi: 10.1152/japplphysiol.00045.2004
Wahbeh, H., Calabrese, C., and Zwickey, H. (2007). Binaural beat technology in humans: a pilot study to assess psychologic and physiologic effects. J. Altern. Complement. Med. 13, 25–32. doi: 10.1089/acm.2006.6196
Keywords: intra-auditory integration, multi-finger synergy, multi-finger pressing, motor performance, hierarchical variability
Citation: Koh K, Kwon HJ, Park YS, Kiemel T, Miller RH, Kim YH, Shin J-H and Shim JK (2016) Intra-Auditory Integration Improves Motor Performance and Synergy in an Accurate Multi-Finger Pressing Task. Front. Hum. Neurosci. 10:260. doi: 10.3389/fnhum.2016.00260
Received: 14 January 2016; Accepted: 17 May 2016;
Published: 08 June 2016.
Edited by:Srikantan S. Nagarajan, University of California, San Francisco, USA
Reviewed by:Tjeerd W. Boonstra, University of New South Wales, Australia
Edmund C. Lalor, Trinity College Dublin, Ireland
Marcos Duarte, Federal University of ABC, Brazil
Copyright © 2016 Koh, Kwon, Park, Kiemel, Miller, Kim, Shin and Shim. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jae Kun Shim, email@example.com