A Review on the Relationship Between Sound and Movement in Sports and Rehabilitation
- 1Department of Movement and Training Science, Institute for Human Movement Science, University of Hamburg, Hamburg, Germany
- 2Music and Health Science Research Collaboratory, Faculty of Music, University of Toronto, Toronto, ON, Canada
The role of auditory information on perceptual-motor processes has gained increased interest in sports and psychology research in recent years. Numerous neurobiological and behavioral studies have demonstrated the close interaction between auditory and motor areas of the brain, and the importance of auditory information for movement execution, control, and learning. In applied research, artificially produced acoustic information and real-time auditory information have been implemented in sports and rehabilitation to improve motor performance in athletes, healthy individuals, and patients affected by neurological or movement disorders. However, this research is scattered both across time and scientific disciplines. The aim of this paper is to provide an overview about the interaction between movement and sound and review the current literature regarding the effect of natural movement sounds, movement sonification, and rhythmic auditory information in sports and motor rehabilitation. The focus here is threefold: firstly, we provide an overview of empirical studies using natural movement sounds and movement sonification in sports. Secondly, we review recent clinical and applied studies using rhythmic auditory information and sonification in rehabilitation, addressing in particular studies on Parkinson’s disease and stroke. Thirdly, we summarize current evidence regarding the cognitive mechanisms and neural correlates underlying the processing of auditory information during movement execution and its mental representation. The current state of knowledge here reviewed provides evidence of the feasibility and effectiveness of the application of auditory information to improve movement execution, control, and (re)learning in sports and motor rehabilitation. Findings also corroborate the critical role of auditory information in auditory-motor coupling during motor (re)learning and performance, suggesting that this area of clinical and applied research has a large potential that is yet to be fully explored.
In the last decades, research in the fields of sport, neuroscience, and psychology, has sought to better understand the role of sounds on perceptual-motor processes from multiple angles of investigation. In applied research, there has been a great interest in how auditory information affect the production of complex movements and how it may be used in sports training and movement rehabilitation to improve motor performance in athletes, healthy individuals, and patients affected by neurological or movement disorders (e.g., Dubus and Bresin, 2013; Sigrist et al., 2013; Murgia et al., 2015; Pizzera and Hohmann, 2015; Sors et al., 2015; Thaut et al., 2015; Ghai and Ghai, 2018; Ghai et al., 2018a,b). However, this body of research is scattered both across time and scientific disciplines. Therefore, the aim of this paper is to provide an overview about the interaction between movement and sound and review the current literature regarding the effect of acoustic information to improve movement execution, control, and (re)learning in sports and motor rehabilitation.
The first section of the paper (Key topic 1) focuses on sports movements and presents an overview of studies investigating the effect of natural movement sounds and sonification in athlete performance enhancement. Natural movement sounds refer to real-time and naturally occurring acoustic information in the form of auditory reafferences, such as the sound resulting from the contact phase of the feet meeting the ground or the physical impact of limbs or equipment with air/ground/water/ball (Kennel et al., 2015; Pizzera and Hohmann, 2015). Natural auditory signals provide a large amount of information about movements that are readily available to the listener (Gaver, 1993a,b) and may be used in sport training to inform or enhance task-intrinsic feedback (Dubus and Bresin, 2013; Sigrist et al., 2013; Sors et al., 2015). Another line of research is dedicated to the development of perceptual strategies based on auditory information to assist movement execution and control through sonification. Sonification involves the transformation of kinematic and dynamic movement parameters into non-speech artificially produced sounds in order to improve motor perception and performance (Effenberg, 2005).
The second section (Key topic 2) addresses the use of sounds in motor rehabilitation. Firstly, we focus on rehabilitation methods that administer auditory rhythmic cues to improve motor function in Parkinson’s disease (PD) and post-stroke, such as Rhythmic Auditory Stimulation (RAS) (Thaut and Hoemberg, 2014; Murgia et al., 2015). Secondly, we consider studies investigating the application of movement sonification (i.e., real-time artificially produced sounds or musical sonification) to assist in the rehabilitation of motor functions in PD and stroke. Note that musical sonification differs methodologically from music-supported therapy, where the former is a relatively novel approach that uses measuring systems (e.g., inertial sensors) to map different movement parameters using musical components, and the latter involves repetitive exercises using musical instruments to retrain motor functions, thus not providing continuous real-time movement feedback (see for review, De Dreu et al., 2012; Zhang et al., 2016; Sihvonen et al., 2017). Studies addressing background music or applying music as auditory feedback are beyond the scope of this review.
In the third section (Key topic 3), we provide an overview of current evidence regarding the neural mechanisms involved in auditory-motor coupling. Particularly, we describe brain regions involved in auditory-motor coupling and address the role of mechanisms such as auditory-motor entrainment, auditory mirror neurons, and sensorimotor integration. By organizing and providing a critical appraisal of the current research, we attempt to develop a framework for future applied and clinical research on the effects of auditory information for motor control and (re)learning.
The systematic searches included numerous electronic literature databases (e.g., MEDLINE, EMBASE) and trial registers, as well as hand-searching of major journals, abstract books, conference proceedings and reference lists of retrieved publications. Also, potentially relevant texts known to the reviewers were included.
The search and screening process for relevant literature is shown in Figure 1. The titles of all retrieved publications were checked, duplicates were removed, and those publications related to other fields of research were excluded. The initial screening resulted in 345 remaining publications, which were further screened for eligibility based on the following criteria: (a) the work must be published in full in English language, (b) must be based on original data, and (c) must be related to the field of auditory information within the context of sport or sport-related activities, and rehabilitation. Publication abstracts and full texts were used to perform a thorough check of these criteria. After this step, 222 publications were identified and included in this paper, of which 131 papers are clinical or applied studies investigating the effect of auditory information in sports and motor rehabilitation.
Key Topic 1: Natural Movement Sounds and Movement Sonification in Sports
Natural Movement Sounds
The role of natural movement sounds in auditory action-perception coupling has been studied in sports domains and daily physical activity as part of more general research. Among the topics investigated, studies have examined the influence of natural movement sounds on movement execution (Agostini et al., 2004; Kennel et al., 2015), sense of agency (self vs. other) (Murgia et al., 2012a; Kennel et al., 2014a,b), action anticipation (Cesari et al., 2014; Allerdissen et al., 2017; Camponogara et al., 2017; Sors et al., 2017, 2018a,b; Cañal-Bruland et al., 2018), and motor learning (Pizzera et al., 2017).
Natural movement sounds carry rich auditory information that has direct physical correspondence to their referent event(s), providing crucial information that may be used to inform or enhance task-intrinsic feedback (Dubus and Bresin, 2013; Sigrist et al., 2013; Sors et al., 2015). One of the direct effects of the presence of natural movement sounds is improving athletes’ movement execution, as shown in a study investigating hammer throwing (Agostini et al., 2004). The role of auditory information on movement execution has also been investigated by manipulating the amount or the temporal features of feedback provided to athletes. It has been shown, for instance, that deprivation of auditory feedback hindered the performance of experienced tennis players by decreasing receiving service precision (Takeuchi, 1993). Kennel et al. (2015) examined whether the sounds of the steps during running would influence hurdling performance and found that temporally delayed auditory feedback decreased athletes’ performance by slowing down the time to complete the track and affecting the motion sequence during the first trials where the manipulation was presented. However, there were no differences in movement execution when comparing normal real-time auditory feedback condition and white noise.
Natural movement sounds also provide fundamental information about agency and facilitate the discrimination of one’s own from another person’s movement. The role of specific sound features on the sense of agency has been recently investigated in sports such as golf (Murgia et al., 2012a) and hurdling (Kennel et al., 2014a,b). Murgia et al. (2012a) found that expert golfers could identify the recorded sounds of their own golf swings from those of other athletes based on the temporal features of the movement sound, such as the overall action duration (i.e., how long the swing movement lasted from beginning to end) and the rhythmic patterns of the backswing and downswing movement. Kennel et al. (2014a,b) also found that athletes could distinguish between their own hurdling movements from those of others’ on the basis of the auditory information, using a variety of sound characteristics (e.g., hurdling step structure, amplitude of the sounds) to build a holistic representation of their own and others’ movements.
Research has also shown that athletes are able to extract relevant information from the sounds generated by their own or others’ movements to predict and anticipate actions based on changes in the environment or the opponents’ behaviors. It has been demonstrated, for example, that expert basketball players can detect the movement intentions of an opponent and prediction their running direction based on the sounds generated by the opponent’s movements (Camponogara et al., 2017). Cesari et al. (2014) found that the ability to precisely anticipate and reproduce a skateboarding jump based only on movement sounds was superior for experienced athletes than for non-experts. Specifically, only experts were able to modify their underfoot force and apply muscle synergies that were essentially similar to those used during a real jump on a skateboard only by hearing the movement sounds. Similarly, studies have also demonstrated that auditory information generated by movements may be used to predict a attack movement in fencing (Allerdissen et al., 2017), the shot power in soccer (Sors et al., 2017, 2018a), and the length of volleyball serves (Sors et al., 2018b). These behavioral data collectively indicate that the auditory-motor coupling generated during extensive training significantly interacts with athletes’ internal motor simulation as experienced athletes are not only able to extract highly specific information from action-related sounds but also use this information to anticipate another person’s movements based on action prediction mechanisms.
The short- and long-term effects of acoustic reafference to improve movement control and learning of complex movements has been recently investigated. Pizzera et al. (2017) tested a training protocol where natural step sounds produced during hurdling were recorded and presented before each trial with modulated tempo in relation to baseline: faster tempo, slower tempo, or normal tempo. Results showed that the presentation of the auditory information increased overall performance for all groups at short-term, enhancing running time and movement technique. When considering the long-term effects, findings suggested that only the groups that received acoustic information with modulated tempo (faster or slower) further increased performance at a 10-week retention test, whereas the performance of the group who trained with normal auditory feedback declined. These results indicate that, while acoustic information during training have immediate effects on athletes’ performance, repeated training with modified temporal acoustic information may be more effective and contribute to the development of a richer internal representation of the movement.
Sonification, as the transfer of movement data into non-speech audio signals, refers to the mapping of physiological and physical data onto psychoacoustic parameters (i.e., loudness, pitch, timbre, harmony and rhythm) in order to provide on- and/or offline access to biomechanical information otherwise not available (for an overview see Effenberg et al., 2011, 2016; Dubus and Bresin, 2013; Sigrist et al., 2013; Kos et al., 2015; Pizzera and Hohmann, 2015). Movement sonification thus aims to assist movement control, execution, and planning by improving self-awareness of physiological processes underlying movement execution and optimizing movement regulation and control (Effenberg, 2005).
The potential use of real-time movement sonification has motivated researchers to investigate the effectiveness of sonification as additional real-time acoustic information in sport training to enhance athletic performance in a wide range of sports (see Supplementary Table 1), including: running (Eriksson and Bresin, 2010; Bolíbar and Bresin, 2012; Boyd and Godbout, 2012; Sanderson and Hunt, 2016), aerobics (Hermann and Zehe, 2011), rowing (Dubus and Bresin, 2010; Schaffert and Mattes, 2011; Wolf et al., 2011; Cesarini et al., 2014b), swimming (Hermann et al., 2012; Cesarini et al., 2014a), sailing (Tarnas and Schaffert, 2017), cycling (Sigrist et al., 2016; Schaffert et al., 2017), speed skating (Godbout and Boyd, 2010; Stienstra et al., 2011; Boyd et al., 2012; Godbout et al., 2014), skiing (Kirby, 2009; Hasegawa et al., 2012), golf (Kleiman-Weiner and Berger, 2006; Nylander et al., 2014), juggling (Bovermann et al., 2007), German wheel (Hummel et al., 2010), squat jumps (Newbold et al., 2017), motorsport (Powell and Lumsden, 2015), recreational sports (Barrass et al., 2010), postural control (Avissar et al., 2013), slackline (Anlauff et al., 2013), handball (Höner et al., 2004), basketball (Ramezanzade et al., 2014), elastic trampoline (Pugliese and Takala, 2015), and manual wheelchair training and operation (Almqvist Gref et al., 2016).
Investigations examining the use of sonification in elite or high-performance sports have demonstrated that the presentation of artificially generated sounds optimize movement control and execution (e.g., stability, velocity, pattern and force symmetry) in sports such as swimming (Chollet et al., 1988, 1992), rowing (Schaffert et al., 2010, 2011; Schaffert and Mattes, 2011, 2015b, 2016), and cycling (Sigrist et al., 2016; Schaffert et al., 2017). For instance, Chollet et al. (1988, 1992) examined the effects of the presentation of concurrent auditory signals of hydrodynamic pressure exerted by the athlete’s hand during the propulsive action in crawl swimming. Movement data were transformed into auditory signals of equal amplitude and mapped on to pitch so that higher pressure was displayed as a higher pitch. The study results indicated that real-time sonification allowed swimmers to maintain stroke velocity improving movement stability and control (Chollet et al., 1988, 1992). Schaffert and colleagues investigated the influence of acoustic feedback in elite rowing (Schaffert et al., 2010, 2011; Schaffert and Mattes, 2011, 2015b, 2016) and elite para-rowing (Schaffert and Mattes, 2015a). For that, we measured the propulsive boat acceleration trace and converted this information into pitch changes so that athletes perceived an increase in pitch the more the boat accelerated. These studies repeatedly found that movement sonification led to faster boat speeds, increased distances traveled per stroke, and improved crew synchronization compared to training without additional auditory information (Schaffert et al., 2010, 2011; Schaffert and Mattes, 2011, 2015a,b, 2016). In cycling, Schaffert et al. (2017) demonstrated that the continuous real-time auditory information allowed cyclists to perceive fluctuations in forces applied on the pedals and consequently adapt muscle activation to maintain a consistent movement execution pattern and symmetry.
Real-time auditory signals may also enhance athletes’ self-awareness during movement execution by providing auditory feedback otherwise not available. This has been shown in studies evaluating the effect of sonification on exerted muscle power in resistance training and weightlifting (Murgia et al., 2012b; Yang and Hunt, 2013, 2015), precision rifle shooting (Konttinen et al., 2004) and inter-limb coordination in gymnastics (Baudry et al., 2006). Yang and Hunt (2013, 2015) examined the potential of real-time sonification to improve the quality of resistance training. Muscular activity (biceps curl) was measured with electromyographic sensors and sonified in relation to the biceps contractions and extensions so that the more effort was exerted the brighter the tone of the sound. The results showed that the auditory information provided concomitant with the movement helped athletes to maintain the pacing of their movement and improve exercise metrics with greater average repetition range and total effort. Murgia et al. (2012b) also showed that high-intensity sounds presented during the bench-press phase of weightlifting facilitated exerted mean muscle power compared with no sonification. Konttinen et al. (2004) investigated the effects of augmented auditory feedback on precision rifle shooting. The auditory signal informed shooters about rifle alignment by mapping the distance between their aiming point and target center. The study results showed improved shooting performance in the post- and retention tests (after 10 and 40 days) compared to a control group, suggesting that the auditory feedback enhanced shooters’ ability to detect errors in body alignment and modify their movements to improve rifle stability and shooting precision. The presentation of auditory information not usually available to athletes’ also improved inter-limb coordination in gymnastics. Baudry et al. (2006) examined the effects of auditory concurrent feedback on body segmental alignment and inter-limb coordination on experienced male gymnasts during the performance of a circle on a pommel horse. A two-part device (with one piece placed on the upper back and the other – a spring – placed on the knee’s backside, both linked with a cable) informed gymnasts about the bent position of the body with an auditory signal. Positive effects on body segmental alignment were found after 2 weeks of training, with gymnasts in the experimental group improving their percentage of maximum body segmental alignment whereas no gains in body alignment were observed for the control group.
Sonification has also been applied during sports training to inform athletes about performance error/deviation in real-time. Collectively, studies in sports training such as handball (Höner et al., 2004), sailing (Tarnas and Schaffert, 2017), speed skating (Godbout and Boyd, 2010; Godbout et al., 2014), and basketball (Ramezanzade et al., 2014) indicate that the availability of real-time auditory feedback enhances online error-correction mechanisms during movement execution and facilitate the learning of a new motor skill. In speed skating, for instance, Godbout and Boyd (2010) provided corrective sonic feedback to an elite athlete with difficulties to perform the cross-over stride movement. The skating stride was matched to a model skater and the differences were sonified. Based on this sonification model it was possible to provide warning cues, timing, and body position information in real-time, allowing the athlete to make corrections and adjustments during movement execution. Sonification modeling was also tested to improve jump shot in basketball with 20 novice participants (Ramezanzade et al., 2014). For that, one group received visual information from a professional player (model) as well as additional auditory information derived from the angular speed of the elbow joint of the player, whereas the second group only received visual information from the player. The findings indicated that the group who received audiovisual information outperformed the group that received only visual information in both the acquisition and retention tests, suggesting that auditory information may facilitate the acquisition and retention of a new motor skill.
Research indeed suggests that real-time auditory feedback supports the learning and retention of new motor skills. Studies collectively indicate that the acquisition of a new skill or movement technique (e.g., swimming stroke technique, precision shooting, inter-limb coordination in gymnastics, rowing technique, and basketball jump shots) is facilitated when auditory information is provided during the acquisition of a new motor skill (e.g., Chollet et al., 1992; Konttinen et al., 2004; Baudry et al., 2006; Ramezanzade et al., 2014; Schaffert and Mattes, 2014). Moreover, with ongoing training, the sonification of the movement is integrated into an internal representation of that skill, thus enhancing the efficacy of motor learning (Effenberg et al., 2016).
Key Topic 2: Rhythmic Auditory Stimulation and Movement Sonification in Rehabilitation
Rhythmic Auditory Stimulation (RAS)
Rehabilitation programs use rhythmic auditory cues as a means to enhance auditory-motor synchronization and promote sustained functional changes to movement (e.g., Thaut, 2005; Thaut and Hoemberg, 2014; Murgia et al., 2015). In particular, rhythm-based techniques use rhythmic patterns to prime the motor system by providing continuous time references that generate expectations for when auditory events will occur or when a movement needs to be performed. The foreknowledge of the duration of the cues allows movement anticipation and motor preparation, hence increasing the quality and precision of the motor responses (Thaut et al., 2015). Specifically, RAS is a rehabilitation technique that involves the utilization of rhythmic cues (metronome or rhythmically accentuated music with embedded metronome clicks) to facilitate rehabilitation of intrinsically rhythmical movements (Thaut et al., 1999; Thaut and Hoemberg, 2014). RAS can be used as an immediate entrainment stimulus providing rhythmic cues during movement or as a facilitating stimulus for training to achieve more functional movement patterns. This technique typically uses simple metronome beats matched to the patient’s baseline gait, but walking cadence can also be facilitated by using metronome beats embedded in musical patterns that are 5–10% faster than baseline (Thaut et al., 1996). Alternative versions of RAS include metronome sounds embedded in expert-selected (McIntosh et al., 1997) or patient-selected music (Thaut et al., 1996). In these studies, it was proposed that the musical texture would provide additional timing information compared with metronome alone, thus facilitating detection, anticipation, and synchronization to the beat (Thaut et al., 1997). A modification of RAS can also be found in the literature as Rhythmic Auditory Cueing (RAC), which is defined as the application of repetitive isochronous beats. Although the terminology may differ in different disciplines, the basic underlying principle of these techniques is the same.
There is robust evidence of the effectiveness of RAS to improve movement in PD patients (reviewed in Rubinstein et al., 2002; Lim et al., 2005; Rochester et al., 2010; Thaut and Abiru, 2010; Spaulding et al., 2013; Wittwer et al., 2013; Rocha et al., 2014; Schaefer, 2014; Murgia et al., 2015; Thaut et al., 2015; Ghai et al., 2018b), stroke (for review, Thaut and Abiru, 2010; Yoo and Kim, 2016), traumatic brain injury (e.g., Hurt et al., 1998), multiple sclerosis (e.g., Conklyn et al., 2010; Shahraki et al., 2017; reviewed in Ghai and Ghai, 2018), and cerebral palsy (e.g., Kwak, 2007; Kim et al., 2011; Baram and Lenger, 2012; Kwak and Kim, 2013; Ghai et al., 2018a for an overview). As the scope of this paper does not allow for a thorough description of all relevant clinical literature using RAS on motor rehabilitation, here we provide a brief overview of representative clinical studies using RAS in PD and post-stroke.
Gait disturbances such as shuffling, freezing of gait, instability (asymmetry and variability between steps) and general difficulties in walking movements and posture are among the most apparent symptoms of PD (Bloem et al., 2004; Rodger and Craig, 2016). Typically, PD patients with impaired gait have difficulty in regulating stride length (Morris et al., 1996) and tend to walk with reduced velocity and increased cadence or step rate (Knutsson, 1972). One probable origin of gait impairment in PD is deficient internal motor timing mechanisms due to basal ganglia dysfunction. Studies have also suggested that the irregular timing of walking pace may be associated with disturbances of coordinated rhythmic locomotion (Ebersbach et al., 1999; Thaut et al., 2001; Skodda et al., 2010) and sensorimotor synchronization (Bieńkiewicz and Craig, 2015, 2016).
Thaut et al. (1996) first described the effect of rhythmic entrainment on gait patterns in PD by demonstrating that patients who underwent 30 min of daily home-based gait training with RAS significantly improved their gait velocity, stride length, and step cadence after 3 weeks of intervention in relation to controls. These findings were later confirmed by several studies (e.g., McIntosh et al., 1997; Freedland et al., 2002; Del Olmo et al., 2006; Nieuwboer et al., 2007; Arias and Cudeiro, 2008, 2010; Hove et al., 2012; Song et al., 2015; Pau et al., 2016). Studies have also found that RAS training can have positive carry-over effects on movement from a few minutes to up to 4 weeks (McIntosh et al., 1997, McIntosh et al., 1998; Nieuwboer et al., 2009). Other beneficial outcomes include increase in the symmetry of muscle activation in upper and lower limbs (Malcolm et al., 2009; Bailey et al., 2018), and reduction of timing variability (Miller et al., 1996), resulting in more stable walking (Thaut et al., 1999; Hausdorff et al., 2007; Hove et al., 2012). A recent study also found positive effects of RAS for the facilitation of gait relearning (Uchitomi et al., 2013). Additionally, there are indications that RAS is superior in maintaining gait performance during dual-tasks due to low cognitive attentional load (Baker et al., 2008). This robust body of literature has been recently summarized and analyzed in systematic and meta-analysis studies, which concluded that rhythmic auditory information is generally an effective therapeutic tool for treating gait disturbances in PD (see Spaulding et al., 2013; Schaefer, 2014; Ghai et al., 2018b).
Although the application of rhythmic auditory information in gait training is well-established, the use of rhythm-based interventions to improve PD symptoms such as freezing of gait and risk of falls is still under investigation. In relation to freezing of gait, Willems et al. (2006) found no beneficial effects of RAS on freezing of gait in patients with less severe symptoms, but Delval et al. (2014) and Plotnik et al. (2014) reported positive effects of RAS on gait initiation and freezing of gait in PD patients. A recent review (Ginis et al., 2017) also concluded that cue-augmented training can reduce the severity of freezing in PD patients, but limitations in long-term consolidation and transfer of the effects to untrained tasks need to be considered in this population. RAS has been also recently applied to reduce falls or risk of falls in healthy elderly (Hurt-Thaut, 2014) and PD patients (Thaut et al., 2018). These studies collectively found that RAS training significantly reduced the number of falls in healthy individuals and PD patients by modifying key kinematics in gait control, thus suggesting that RAS may be beneficial to address the risk of falls.
A recent line of research has focused on whether specific parameters of the acoustic cues can influence the results of rhythm-based interventions for PD by comparing, for instance, differences between music and isochronous sounds (i.e., metronome) with interactive cueing systems that adapt to the patient’s gait (see review in Ashoori et al., 2015; Hove and Keller, 2015). For instance, Murgia et al. (2018) compared whether the nature of the stimulus presented would influence the effectiveness of RAS by providing ecological footstep sounds as auditory information. For that, one group of PD patients completed 5 weeks of supervised rehabilitation training that included walking while listening to ecological footsteps sounds, whereas the second group of patients walked listening to artificial stimuli (e.g., metronome). The overall conclusion of the study was that biological motion sounds such as footsteps are as effective as the metronome, but exploratory analyses of biomechanical measures suggested that there may be some differences in improvement linked to the type of auditory stimuli. Similarly, Dotov et al. (2017) tested biological variability in auditory stimulus vs. isochronous cues and found superiority of biologically variable auditory cues in fostering natural gait variability in PD patients; however, the authors limited their analysis to only immediate and likely transient effects of cueing. Young et al. (2016) found that action-relevance was a more dominant factor in facilitating improvements in gait parameters than acoustic continuity. Finally, Baram et al. (2016) examined the effectiveness of a device that provided a clicking sound generated in response to every step taken by the patient and found that closed-loop auditory feedback produced better results than open-loop auditory cues (e.g., metronome) in relation to gait speed. It is important to note, however, that the use of closed-loop auditory feedback information stands in contrast to metronome-based approaches in relation to a critical component, that is, the use of external auditory cues as predictable feedforward information transmitted by the steady rhythmic information.
Negative effects of RAS were reported when auditory cues were presented at rates much slower (e.g., 20%) or much higher than the patient’s preferred gait (Del Olmo and Cudeiro, 2005; Nombela et al., 2013). Arias and Cudeiro (2008) and Dalla Bella et al. (2017) also suggested that RAS efficacy may depend on individual characteristics, including severity of disease symptoms and impaired ability to synchronize to the beat. However, there are indications that beat perception may be of lesser importance due to evidence of the primacy of period entrainment over phase/beat entrainment during small tempo perturbations (e.g., Thaut et al., 1998a,b; Roberts et al., 2000; Thaut and Kenyon, 2003).
Overall, research using RAS and rhythmically enhanced-music show consistent evidence of the improvement of motor function in PD. Moreover, recent studies have extended the application of RAS to other non-motor functions (for review, see Thaut and Abiru, 2010). For instance, studies have indicated that RAS training enhances patients’ performance in both motor timing (movement synchronization, tapping) and in perceptual timing tasks (duration discrimination, beat detection in music), supporting the hypothesis that RAS engages brain networks involved in both perceptual and motor timing (Benoit et al., 2014; Dalla Bella et al., 2015).
Motor impairment is one the most widely recognized consequences of stroke, which include reduced movement coordination, decreased postural control, and decreased upper-limb function (Langhorne et al., 2009). Such significant impairments in locomotive function can lead to limitations in independent mobility, thus strongly affecting patients’ quality of life (Michael et al., 2005).
There is strong evidence that RAS can be effectively applied for timely motor control during gait training for stroke patients (Thaut et al., 1993, 1997, 2007; Hayden et al., 2009; for review see Thaut and Abiru, 2010; Hollands et al., 2012; Thaut and McIntosh, 2014; Nascimento et al., 2015; Yoo and Kim, 2016). Thaut et al. (1993) found that patients who walked with RAS matched to their baseline gait cadence showed decreased stride time variability and more balanced muscular activation pattern between the paretic and non-paretic limbs. Recent studies also indicate significant effects of RAS on standing balance (Suh et al., 2014), and gait coordination and symmetry during normal overground walking (Prassas et al., 1997; Roerdink et al., 2011; Lee et al., 2012; Yang et al., 2016) and treadmill training (Roerdink et al., 2007; Park et al., 2015; Yoon and Kang, 2016; Mainka et al., 2018). Immediate effects of RAS training with tempo changes were also found on gait kinematics (Cha et al., 2014) and in relation to the lesion site (Kobinata et al., 2016). Finally, there is growing support for the use of RAS in gait training during the chronic phase of stroke to improve walking speed and functional mobility (e.g., Shin et al., 2015; Ko et al., 2016; Wright et al., 2016, 2017).
Studies have also reported significant improvements in upper limb function after training with RAS (e.g., Whitall et al., 2000; Thaut et al., 2002a,b; Luft et al., 2004; Jeong and Kim, 2007; Malcolm et al., 2009; Chen et al., 2016). For instance, Malcolm et al. (2009) reported a significant decrease in compensatory reaching movements after a 2-week RAS training program, which consisted of patients moving between at least two targets by touching the digits of their affected hand to the assigned targets in synchrony with the auditory rhythmic stimuli. Another line of interventions has used RAC to prime and facilitate bilateral arm training, also known as BATRAC (for review Wolf et al., 2014; Choo et al., 2015). As an example of BATRAC training, in Whitall et al. (2000) participants pushed and pulled bilaterally two independent bar handles in synchrony or alternation with rhythmic auditory cues. The authors found significant improvement in isometric strength, range of motion, and functional motor performance of the paretic arm after 6 weeks of intervention and also at an 8-week follow-up assessment. Additionally, there are indications that music-supported training using musical instruments can improve motor recovery of arm movements after stroke by inducing auditory-sensorimotor co-representation of movements (e.g., Thaut et al., 2002a,b; Schneider et al., 2007, 2010; Rodríguez-Fornells et al., 2012; Altenmüller et al., 2009; Altenmüller and Schlaug, 2013; Amengual et al., 2013; for a review on music-support training, see Zhang et al., 2016).
Technology-assisted therapy and rehabilitation seek to help patients in regaining the ability to independently perform daily activities and to facilitate their reintegration into social and domestic life by using advances in smart technologies or robotics (for reviews on robotic-assisted therapy, see Lum et al., 2002; Prange et al., 2006; Kwakkel et al., 2008; Marchal-Crespo and Reinkensmeyer, 2009; Secoli et al., 2011; Pennycott et al., 2012; Rosati et al., 2013).
One of the first applications of sonification in a rehabilitation context was developed by Pauletto and Hunt (2006, 2009). They sonified muscular activity using the temporal patterns in electromyography (EMG) by converting electrical impulses from muscles into auditory information. The goal of this sonification approach was to assist therapists to audibly analyze the complex signals originating from multiple EMG-sensors during physical activity. Several sonification methods and system prototypes have been developed in recent years (e.g., Chiari et al., 2005; Dozza et al., 2005, 2007; Vogt et al., 2010; Tissberger and Wersenyi, 2011; Matsubara et al., 2012; Franco et al., 2013; Torres et al., 2013; Ghai et al., 2018c; see Supplementary Table 2 for details). A growing body of recent research generally agrees that sonification is a promising feedback tool for patients and therapists, complementing existing analytical components in therapy (such as visual displays) (for an overview, see Huang et al., 2006; Dubus and Bresin, 2013). The following sections present an overview of current investigations in sonification for movement rehabilitation in PD and stroke.
There has been growing interest in the application of sonification systems in neurologic rehabilitation focusing on improving gait in PD patients (e.g., Batavia et al., 2001; Miyake, 2009; Torres et al., 2013; Contreras Lopez et al., 2014; Young et al., 2014; Horsak et al., 2016; Schedel et al., 2016; see Supplementary Table 2 for details). A sonification system that has received significant attention in recent years is the use of instrumented footwear (Bresin et al., 2010; Fischer et al., 2017; see review in Maculewicz et al., 2016). These systems comprise of interactive shoes with embedded sensors that collect gait information (e.g., cadence, velocity, stride length), which are then used to trigger auditory cueing stimuli to inform both the therapist and the patient about the user’s current state. Recently, Gorgas et al. (2017) tested the effect of an instrumented shoe-insole-device for real-time sonification of gait (SONIGait; see also Horsak et al., 2016). This sonification system mapped individual walking characteristics on to musical notes in order to provide gait spatiotemporal information. Results indicated that a 5-min practice phase with sonification increased gait velocity and cadence, opening the possibility for further testing of this real-time sonification device in large controlled trials. Rodger et al. (2014) tested two sonification systems using synthesized walking sounds to guide and improve gait coordination in PD. The first approach used computer-generated sounds of footsteps on gravel in order to convey ecological information regarding step lengths and duration, whereas the second approach involved real-time sonification of the swing-phase of gait by using motion-capture and audio processing software. Study results suggested that both methods had an effect on step length variability but did not alter step duration variability, suggesting that the presentation of auditory information within the patient’s normal step duration range had an effect only on spatial characteristics of gait rather than temporal parameters.
A recent innovative line of motor learning based interventions have combined action observation and sonification to improve freezing of gait (see Gilat et al., 2018 for review). For instance, Mezzarobba et al. (2018) presented videos showing an actor performing gait-related gestures while simultaneously presenting the sonification generated by the kinematics of each gesture. Patients were then asked to imitate the movements shown. This training protocol was completed twice a week for a total of 8 weeks by a group of 12 patients, whereas the control group practiced the motor gestures by means of visual (stripes on the floor) or auditory cues (metronome). Assessments conducted after the intervention and 3 months after the treatment suggested that the multisensory treatment significantly reduced the number of episodes and duration of freezing facilitating the priming effect generated by action observation, whereas no significant difference was observed for all mobility indices in the control group.
External real-time auditory feedback has been extensively applied in upper-limb rehabilitation post-stroke (e.g., Maulucci and Eckhouse, 2001; Chen et al., 2006; Wallis et al., 2007; Dailly et al., 2012; Immoos et al., 2013; Bruckner et al., 2014; Fujii et al., 2016; reviewed in Ghai, 2018; see Supplementary Table 2 for details). For instance, Chen et al. (2006) and Wallis et al. (2007) tested a real-time multimodal sonification system which provided visual and auditory information in order to motivate arm reaching training for stroke patients. Specifically, arm movements triggered musical feedback that provided information about movement smoothness/jerkiness and speed of reach such that the acceleration of the motion during reaching and returning changed the musical intervals and harmonic progressions presented. Test results with three stroke patients reported in Wallis et al. (2007) suggested the feasibility of such sonification systems, opening new avenues for the application of this system in large-scale studies.
Scholz et al. (2015, 2016) investigated the effectiveness of a musical sonification therapy protocol to train gross motor function of upper extremities. For that, patients’ arm movements were sonified in real-time using two inertial sensors placed at the wrist and upper-arm of the affected side. The 3D-movement data were transformed into sounds so that upward movements resulted in an ascending C major scale, vertical movements into changes in brightness/timbre of the sounds, and sagittal movements into changes in loudness. The final goal of the training was to teach patients to play simple melodies by moving their arm in a 3D-sonification space. Patients received an average of 10 days of musical sonification therapy or a sham sonification training that did not include auditory feedback. The study results indicated that patients in the music group improved in measures of motor function relating to the smoothness of reaching but no significant changes were observed in other arm-function measures. Additionally, findings suggested a reduction of joint pain in a subgroup of patients who presented lower pain scores prior to the commencement of the musical sonification therapy.
Schmitz et al. (2014) tested an expanded concept for sonification in upper-limb stroke rehabilitation which included a mobile sonification system that provided 4D information about arm positions and trajectories as captured by inertial sensors. Specifically, hand position was mapped onto four acoustic parameters: arm velocity was mapped onto amplitude; elevation angle onto frequencies between 133.3 and 266.6 Hz; radial arm amplitude changed the impression of sound brightness; and azimuth angle determined the interaural intensity difference. Test results with seven patients indicated the potential application of this sonification system in larger clinical trials (see Schmitz et al., 2018).
Robertson et al. (2009) investigated the effect of sonification on upper limb movements after stroke. Patients performed a reaching task that involved reciprocal pointing to 9 targets while a sensor fixed to the hand processed online kinematic data and modulated the auditory feedback presented during movement. The study reported that the sonification had a positive effect on movement performance such as movement smoothness and trajectory curvature for patients with right hemisphere damage, while it worsened the performance of patients with left hemisphere damage. This result thus suggests that responses to auditory feedback may differ when the side of the lesion after stroke is taken into consideration.
Key Topic 3: Cognitive Mechanisms and Neural Correlates Underlying Auditory-Motor Coupling
There is robust evidence from multiple lines of inquiry that auditory information has a profound effect on the motor system. Physiological and neuroimaging research has demonstrated that one of the factors underlying this strong interaction is the widely distributed neuroanatomical network connecting the auditory and motor systems at the spinal cord, subcortical and cortical levels (Nayagam et al., 2011; Theunissen and Elie, 2014; Bizley, 2017). For instance, studies investigating reflexive motor responses to sound have described neural pathways formed by descending (efferent) fiber tracts originating in the ventral cochlear nucleus that project bilaterally to sensorimotor tracts in the spinal cord via reticulospinal connections (Rossignol and Melvill Jones, 1976; Huffman and Henson, 1990; Delwaide and Schepens, 1995; Marinovic et al., 2014; Marinovic and Tresilian, 2016). Neuroimaging research has also identified rich neuroanatomical interconnectivity between several distant cortical and subcortical brain areas, including the cerebellum, basal ganglia, thalamus, supplementary motor area (SMA) and pre-SMA, premotor cortex, and the auditory cortex (for review, see Teki et al., 2012; Chauvigné et al., 2014; Merchant et al., 2015; Lusk et al., 2016; Petter et al., 2016; Braun Janzen and Thaut, 2018; Koshimori and Thaut, 2018). Specifically, cortico-cerebellar networks have been shown to be predominantly engaged in movement synchronization to externally cued stimuli (Buhusi and Meck, 2005; Brown et al., 2006; Del Olmo et al., 2007; Thaut et al., 2008, 2009; Witt et al., 2008; Manto et al., 2012; Chauvigné et al., 2014), whereas basal ganglia-thalamo-cortical networks seem particularly involved in beat-based timing and self-paced or internally driven rhythmic movements (Halsband et al., 1993; Cunnington et al., 1996; Rao et al., 1997; Grahn and Rowe, 2009, 2013). Furthermore, recently emerging evidence also indicate that auditory and motor areas have direct routes of communication at cortical level via the arcuate fascicle, a white matter fiber tract with direct projections from the auditory cortex to motor areas, including primary motor cortex and premotor cortex (Fernández-Miranda et al., 2015; Wang et al., 2016).
Another crucial aspect is that the functional and structural architecture of the auditory system is built to rapidly detect temporal patterns of periodicity in acoustic signals. There is considerable evidence that the temporal resolution of the auditory system is superior to other sensory modalities (e.g., Repp and Penel, 2002, 2004; Grondin and McAuley, 2009; Shelton and Kumar, 2010; Grahn et al., 2011; Stauffer et al., 2012; Ammirante et al., 2016). Recent electrophysiological research has demonstrated that the temporal information of acoustic signals is highly preserved at all levels of the auditory processing stream and elicit a periodic neural response at the exact same frequency of the stimuli (for review, see Nozaradan, 2014). Moreover, listening to auditory rhythmic stimuli primes the motor system, increasing the neural efficiency of the motor cortex through a process of auditory-motor entrainment (Crasta et al., 2018). That is, the firing rates of auditory neurons triggered by auditory rhythmic information, such as the beat of the music or a metronome, entrains the firing patterns of neurons in the motor cortex. The oscillatory coupling of neural impulses in the cortical loop between auditory and motor regions generates temporal predictions that are crucial for the perception of, and entrainment to, auditory rhythms (Large and Snyder, 2009; Fujioka et al., 2012; Large et al., 2015; Merchant et al., 2015; Ross et al., 2016, 2017; Morillon and Baillet, 2017). Therefore, the continuous time reference of the rhythmic auditory cues provides predictable feedforward information that allows movement anticipation and motor preparation (Thaut et al., 2015). Additionally, it has been shown that external rhythmic auditory input also changes the pattern of muscle activation through changes in corticospinal excitability (Thaut et al., 1992, 1999; Miller et al., 1996; Wilson and Davey, 2002; Stupacher et al., 2013), modulates beta (β) brain oscillations (Fujioka et al., 2012; Merchant et al., 2015; Ross et al., 2016, 2017), and promotes neural-plasticity (Luft et al., 2004). Collectively, these findings provide strong evidence of the neurobiological mechanisms underlying the effects of RAS on motor planning and execution.
The use of real-time movement information extends the benefits of discrete rhythmic auditory stimuli by adding an auditory component to the movement cycle either with natural movement sounds or movement sonification (Effenberg, 2005; Sigrist et al., 2013; Effenberg et al., 2016; Bevilacqua et al., 2016; Dyer et al., 2017a). Robust evidence suggests that merely listening to action-related sounds activates the neural processes necessary to produce those sounds (e.g., Aziz-Zadeh et al., 2004; Lewis et al., 2005; Pizzamiglio et al., 2005; Aziz-Zadeh et al., 2006; Gazzola et al., 2006; Caetano et al., 2007; Pazzaglia et al., 2008; Alaerts et al., 2009; Engel et al., 2009; Ticini et al., 2012; reviewed in Aglioti and Pazzaglia, 2010). Kohler et al. (2002) provided the first empirical evidence that premotor neurons in monkeys respond to the sound of a familiar action, expanding the notion that movements and their perceptual consequences are intrinsically coupled in the brain (Fadiga et al., 1995; Rizzolatti and Craighero, 2004; Schütz-Bosbach and Prinz, 2007; Rizzolatti and Sinigaglia, 2010). In humans, research shows that acoustic information are sufficient to evoke accurate representations of complex movements (Repp and Knoblich, 2004; van der Zwan et al., 2009; Lewis et al., 2011; Murgia et al., 2012a; Sevdalis and Keller, 2014; Kennel et al., 2014a; reviewed in Pizzera and Hohmann, 2015), activating superior and medial posterior temporal regions involved in human motion recognition (Bidet-Caulet et al., 2005; Baumann and Greenlee, 2006; Saarela and Hari, 2008; Scheef et al., 2009; Schmitz et al., 2013). Importantly, motor resonance is associated with and strengthened by one’s experience and familiarity with the actions observed/perceived, as demonstrated by studies comparing expert and novice responses to specific sports- or dance-related sounds (e.g., Agostini et al., 2004; Hohmann et al., 2011; Tomeo et al., 2012; Woods et al., 2014; Murgia et al., 2017). Further evidence of the role of learning and expertise has been provided by research showing that a network comprising areas such as dorsolateral and inferior frontal cortex (including Broca’s area), superior temporal gyrus, and motor areas including supplementary motor and premotor areas, is engaged when experienced musicians listen to well-rehearsed music (Haueisen and Knösche, 2001; Bangert et al., 2006; D’Ausilio et al., 2006; Harris and De Jong, 2014; see also Proverbio et al., 2014) or watch silent video recordings of known music pieces (Lotze et al., 2003; Hasegawa et al., 2004; Baumann et al., 2007; Bianco et al., 2016; reviewed in Maes et al., 2014; Novembre and Keller, 2014). Activation of this network was also found when non-musicians listened to a music piece they had learned to play after a short period of training (Lahav et al., 2007; see also Bangert and Altenmüller, 2003). These findings thus suggest that strong auditory-motor associations are developed during sound-making experiences, providing support for the use of real-time auditory feedback to enhance sensorimotor representations and facilitate movement (re)-acquisition.
It is also thought that the continuous availability of information provided by mapping different dynamic or kinematic movement parameters onto distinct sound components (e.g., pitch, loudness, rhythm, timbre) improves movement quality and motor (re)learning through the integration of multiple congruent perceptual streams (Scholz et al., 2015; Effenberg and Schmitz, 2018; Ghai et al., 2018c), resulting in a richer and more effective internal representation of the movement (Shams and Seitz, 2008; Wolpert et al., 2011; Schmitz et al., 2013; Effenberg et al., 2016). Furthermore, the availability of real-time auditory feedback also enhances online error-correction mechanisms (Dyer et al., 2015; Hossner et al., 2015; Sigrist et al., 2015; van Vugt and Tillmann, 2015), increases cognitive-emotional functioning (Van Vugt et al., 2014; Altenmüller and Schlaug, 2015; Sihvonen et al., 2017), and may supplement perceptual deficits (Tinazzi et al., 2002; van Vugt and Tillmann, 2015; Danna and Velay, 2017; Ghai et al., 2018c).
The studies here reviewed examined the relationship between sound and movement in the context of sports training and movement rehabilitation. Our narrative synthesis focused specifically on the literature regarding the effect of natural movement sounds, movement sonification, and rhythmic auditory information. The current state of knowledge here summarized provides promising evidence of the effect of auditory information on sporting performance and motor (re)learning.
The availability of auditory information in the form of natural sounds occurring as a byproduct of a movement or as additional real-time acoustic feedback driven by movement dynamic or kinematic parameters has significant implications for motor execution and control of skilled performances. The large body of research here reviewed indicates that auditory information provides crucial information about agency (Murgia et al., 2012a; Kennel et al., 2014a,b), movement control and timing (e.g., Chollet et al., 1988, 1992; Schaffert and Mattes, 2011, 2016; Sigrist et al., 2016; Schaffert et al., 2017), movement execution (e.g., Agostini et al., 2004; Konttinen et al., 2004; Baudry et al., 2006; Murgia et al., 2012b; Yang and Hunt, 2013, 2015; Kennel et al., 2015), and performance error/deviation (e.g., Höner et al., 2004; Godbout and Boyd, 2010; Wolf et al., 2011; Godbout et al., 2014; Ramezanzade et al., 2014; Tarnas and Schaffert, 2017). Behavioral data also suggest that the auditory-motor coupling generated during extensive training significantly interacts with athletes’ internal motor simulation (Murgia et al., 2012a; Kennel et al., 2014a,b; Pizzera et al., 2017), as shown by studies demonstrating that skilled athletes are able to extract highly specific information from action-related sounds (e.g., Roberts et al., 2005) and predict another person’s movements based on action prediction mechanisms (e.g., Cesari et al., 2014; Camponogara et al., 2017; Allerdissen et al., 2017). These findings corroborate a robust body of neuroimaging and neurophysiological studies indicating that the mirror neuron system and a widely distributed neuroanatomical network is involved in the processing of action sounds (e.g., Fadiga et al., 1995; Kohler et al., 2002; Aziz-Zadeh et al., 2004, 2006; Bidet-Caulet et al., 2005; Lewis et al., 2005; Pizzamiglio et al., 2005; Pazzaglia et al., 2008; Ticini et al., 2012; Schmitz et al., 2013).
Studies also demonstrated positive effects of auditory information on motor (re)learning in sports and rehabilitation. Research findings revealed that real-time auditory feedback facilitates learning and improves retention of new motor skills (e.g., Chollet et al., 1992; Konttinen et al., 2004; Baudry et al., 2006; Ramezanzade et al., 2014; Schaffert and Mattes, 2014; Pizzera et al., 2017). There is growing support for the application of movement sonification to increase upper-limb functions after stroke (e.g., Wallis et al., 2007; Immoos et al., 2013; Schmitz et al., 2014, 2018; Scholz et al., 2015, 2016; Ghai, 2018), and to improve gait in PD patients using, for instance, instrumented footwear (e.g., Batavia et al., 2001; Rodger et al., 2014; Horsak et al., 2016; Maculewicz et al., 2016; Gorgas et al., 2017). These sonification approaches rely on the transformation of dynamic and kinematic movement parameters onto distinct sound components (e.g., pitch, loudness, rhythm, timbre) to increase cross-modal stimulation (Scholz et al., 2015, 2016; Ghai et al., 2018c) and sensorimotor representation of the movement to be (re)learned (Shams and Seitz, 2008; Schmitz et al., 2013; Effenberg et al., 2016).
On the other hand, another line of clinical studies summarized in this review focuses primarily on the rhythmic patterns of sound, making use of metronome or beat-enhanced music to facilitate rehabilitation of intrinsically rhythmical movements (Thaut, 2005; Thaut and Hoemberg, 2014; Murgia et al., 2015; Ghai et al., 2018b). This robust body of research evidence indicates that RAS has immediate effects on gait velocity, step cadence, and stride length (e.g., Thaut et al., 1996; McIntosh et al., 1997; Freedland et al., 2002; Nieuwboer et al., 2007; Arias and Cudeiro, 2008, 2010; Hove et al., 2012; Song et al., 2015; Pau et al., 2016), reducing gait variability (Miller et al., 1996) and improving walking stability in PD (Thaut et al., 1999; Hausdorff et al., 2007; Hove et al., 2012) and stroke (Thaut et al., 1993, 1997, 2007; Hayden et al., 2009; for review see Thaut and Abiru, 2010; Hollands et al., 2012; Thaut and McIntosh, 2014; Nascimento et al., 2015; Yoo and Kim, 2016). Studies have also demonstrated that auditory cueing significantly improves upper-limb function after stroke by reducing movement variability and reliance on compensatory movements (e.g., Whitall et al., 2000; Thaut et al., 2002a,b; Luft et al., 2004; Jeong and Kim, 2007; Malcolm et al., 2009; Chen et al., 2016). It has been proposed that the continuous time reference provided by the rhythmic auditory cues facilitates movement retraining by priming the motor system, allowing movement anticipation and motor preparation (Thaut et al., 2015), and potentially bypassing damaged areas through the activation of alternative pathways (Hoemberg, 2005; Dalla Bella et al., 2017; Braunlich et al., 2018).
We also identified a small number of studies that evaluated other variables influencing the effect of auditory information on motor performance, such as physiological arousal and motivation (Murgia et al., 2012b; Bood et al., 2013; Immoos et al., 2013; Pugliese and Takala, 2015; Scholz et al., 2016; Sanderson and Hunt, 2016; Newbold et al., 2017). Murgia et al. (2012b) found that athletes’ maintained peak performance and reduced performance variability in trials where high-intensity sounds were presented during the pressing phase of weightlifting exercises, and Bood et al. (2013) reported changes in psychophysical and physiological outcome measures due to the motivational aspects of the stimuli during running. Novel therapeutic approaches, such as musical sonification (Scholz et al., 2016), also considered the motivational aspects of adding real-time auditory feedback to stimulate patients and improve treatment compliance, thus opening new avenues to systematically examine the role of physiological arousal, motivation, reward, and mood in larger clinical trials. The potential use of interactive sonification systems in sports and rehabilitation has motivated researchers and engineers to develop applications and system prototypes for exercise and physical activity, rehabilitation, and entertainment (e.g., Barrass et al., 2010; Lécuyer et al., 2011; Franco et al., 2013; Bruckner et al., 2014; Contreras Lopez et al., 2014; Pugliese and Takala, 2015; Newbold et al., 2017; see Supplementary Material). These studies explore a wide range of devices and applications where the playful character of music or the competitive component of sports has inspired new technology-enabled forms of play (e.g., exertion or computer games) and therapy. Future applications of this technology in sports, recreation, and rehabilitation are yet to be fully explored.
The large body of literature here reviewed clearly shows an emerging area of clinical and applied research. However, there are important research gaps that need to be addressed in future research. Firstly, there is a clear need to better understand what auditory components and amount of information are most relevant in motor training and rehabilitation. This is not trivial, particularly in sonification applications, as research suggests that an overload of auditory information has detrimental effects on task performance (e.g., Wolf et al., 2011) and that task-irrelevant auditory stimuli are strong distractors (Parmentier, 2014). The use of meaningful auditory information is, therefore, determinant for the user’s experience (Effenberg et al., 2016; Dyer et al., 2017b) and needs to be considered in a clear framework for sonification mapping derived from a better understanding of the processes underlying motor learning/control from a basic research perspective (Dyer et al., 2015). Research in the field of auditory information processing has great potential to promote active crosstalk between basic and applied research, with findings generated in the laboratory providing insights for the application in real-life situations, that being in sports training or therapy and rehabilitation, and vice-versa. Secondly, we have identified few studies using natural movement sounds or sonification in elite or high-performance sports. A challenge for future investigations is to evaluate novel applications in ecologically valid and real-life situations that closely resemble the athlete’s movement technique and training conditions in order to better identify what type of information is most relevant and improve equipment setup, thus acquiring more reliable results. This depends directly on the development of procedures that are feasible for the systematic use in daily training. In addition, future research should also consider the way in which auditory information is presented to athletes and patients (loudspeaker vs. earplugs) in order to avoid, for instance, perceptual overload, and to ensure that the feedback information is properly delivered. From the clinical perspective, although there is growing attention on the application of sonification systems to improve motor function in PD and post-stroke, we have identified a relatively small number of controlled trials, revealing the need to further examine the effectiveness and feasibility of sonification methods and devices in larger controlled clinical studies.
This review examined the relationship between sound and movement in the context of sports training and movement rehabilitation. The findings here summarized provide evidence of the effect of natural movement sounds, movement sonification, and rhythmic auditory information on sporting performance and motor (re)learning. This emerging area of clinical and applied research demonstrates large underutilized potential, warranting further investigation of the promising application of auditory feedback information in sports and rehabilitation.
All authors listed have made a substantial intellectual contribution to the work and approved it for publication. NS conceived and drafted the first version of the manuscript. TBJ contributed to the writing and revision of the manuscript. MT and KM supervised and revised the manuscript.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2019.00244/full#supplementary-material
Agostini, T., Righi, G., Galmonte, A., and Bruno, P. (2004). “The relevance of auditory information in optimizing hammer throwers performance,” in Biomechanics and Sports. CISM Courses and Lectures, Vol. 473, ed. P. B. Pascolo (Vienna: Springer), doi: 10.1007/978-3-7091-2760-5_9
Alaerts, K., Swinnen, S. P., and Wenderoth, N. (2009). Interaction of sound and sight during action perception: evidence for shared modality-dependent action representations. Neuropsychologia 47, 2593–2599. doi: 10.1016/j.neuropsychologia.2009.05.006
Allerdissen, M., Güldenpenning, I., Schack, T., and Bläsing, B. (2017). Recognizing fencing attacks from auditory and visual information: a comparison between expert fencers and novices. Psychol. Sport Exerc. 31, 123–130. doi: 10.1016/j.psychsport.2017.04.009
Almqvist Gref, A., Elblaus, L., and Falkenberg Hansen, K. (2016). “Sonification as catalyst in training manual wheelchair operation for sports and everyday life,” in Proceedings of the Sound and Music Computing Conference, Stockholm, 9–14.
Altenmüller, E., Marco-Pallares, J., Münte, T. F., and Schneider, S. (2009). Neural reorganization underlies improvement in stroke-induced motor dysfunction by music-supported therapy. Ann. N. Y. Acad. Sci. 1169, 395–405. doi: 10.1111/j.1749-6632.2009.04580.x
Amengual, J. L., Rojo, N., de Las Heras, M. V., Marco-Pallarés, J., Grau-Sánchez, J., Schneider, S., et al. (2013). Sensorimotor plasticity after music-supported therapy in chronic stroke patients revealed by transcranial magnetic stimulation. PLoS One 8:e61883. doi: 10.1371/journal.pone.0061883
Ammirante, P., Patel, A. D., and Russo, F. A. (2016). Synchronizing to auditory and tactile metronomes: a test of the auditory-motor enhancement hypothesis. Psychon. Bull. Rev. 23, 1882–1890. doi: 10.3758/s13423-016-1067-9
Anlauff, J., Cooperstock, J. R., and Fung, J. (2013). “Augmented feedback for learning single-legged stance on a slackline,” in Proceedings of the International Conference on Virtual Rehabilitation (ICVR) (Philadelphia, PA: IEEE), 162–163. doi: 10.1109/ICVR.2013.6662104
Avissar, D., Leider, C. N., Bennett, C., and Gailey, R. (2013). “An audio game app using interactive movement sonification for targeted posture control,” in Proceedings of the International Conference on Auditory Display, Łódź, 45–48.
Aziz-Zadeh, L., Iacoboni, M., Zaidel, E., Wilson, S., and Mazziotta, J. (2004). Left hemisphere motor facilitation in response to manual action sounds. Eur. J. Neurosci. 19, 2609–2612. doi: 10.1111/j.0953-816X.2004.03348.x
Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., and Iacoboni, M. (2006). Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Curr. Biol. 16, 1818–1823. doi: 10.1016/j.cub.2006.07.060
Bailey, C. A., Corona, F., Murgia, M., Pili, R., Pau, M., and Côté, J. N. (2018). Electromyographical gait characteristics in Parkinson’s disease: effects of combined physical therapy and rhythmic auditory stimulation. Front. Neurol. 9:211. doi: 10.3389/fneur.2018.00211
Baker, K., Rochester, L., and Nieuwboer, A. (2008). The effect of cues on gait variability-reducing the attentional cost of walking in people with Parkinson’s disease. Parkinsonism. Relat. Disord. 14, 314–320. doi: 10.1016/j.parkreldis.2007.09.008
Bangert, M., Peschel, T., Schlaug, G., Rotte, M., Drescher, D., Hinrichs, H., et al. (2006). Shared networks for auditory and motor processing in professional pianists: Evidence from fMRI conjunction. Neuroimage 30, 917–926. doi: 10.1016/j.neuroimage.2005.10.044
Baram, Y., Aharon-Peretz, J., Badarny, S., Susel, Z., and Schlesinger, I. (2016). Closed-loop auditory feedback for the improvement of gait in patients with Parkinson’s disease. J. Neurol. Sci. 363, 104–106. doi: 10.1016/j.jns.2016.02.021
Barrass, S., Schaffert, N., and Barrass, T. (2010). “Probing preferences between six designs of interactive sonifications for recreational sports, health and fitness,” in Proceedings of the ISon 2010, 3rd Interactive Sonification Workshop (Stockholm: Interactive-Sonification), 23–29.
Batavia, M., Gianutsos, J. G., Vaccaro, A., and Gold, J. T. (2001). A do-it-yourself membrane-activated auditory feedback device for weight bearing and gait training: a case report. Arch. Phys. Med. Rehabil. 82, 541–545. doi: 10.1053/apmr.2001.21931
Baumann, S., Koeneke, S., Schmidt, C. F., Meyer, M., Lutz, K., and Jancke, L. (2007). A network for audio-motor coordination in skilled pianists and non-musicians. Brain Res. 1161, 65–78. doi: 10.1016/j.brainres.2007.05.045
Benoit, C. E., Dalla Bella, S., Farrugia, N., Obrig, H., and Kotz, S. A. (2014). Non-gait related benefits of auditory cueing in Parkinson’s disease. Procedia Soc. Behav. Sci. 126, 210–211. doi: 10.1016/j.sbspro.2014.02.378
Bevilacqua, F., Boyer, E. O., Françoise, J., Houix, O., Susini, P., Roby-Brami, A., et al. (2016). Sensori-motor learning with movement sonification: perspectives from recent interdisciplinary studies. Front. Neurosci. 10:385. doi: 10.3389/fnins.2016.00385
Bianco, R., Novembre, G., Keller, P. E. E., Kim, S.-G. G., Scharf, F., Friederici, A. D. D., et al. (2016). Neural networks for harmonic structure in music perception and action. Neuroimage 142, 454–464. doi: 10.1016/j.neuroimage.2016.08.025
Bidet-Caulet, A., Voisin, J., Bertrand, O., and Fonlupt, P. (2005). Listening to a walking human activates the temporal biological motion area. Neuroimage 28, 132–139. doi: 10.1016/j.neuroimage.2005.06.018
Bloem, B., Hausdorff, J., Visser, J., and Gilaldi, N. (2004). Falls and freezing of gait in Parkinson’s disease: a review of two interconnected, episodic phenomena. Mov. Disord. 19, 871–884. doi: 10.1002/mds.20115
Bood, R. J., Nijssen, M., Van Der Kamp, J., and Roerdink, M. (2013). The power of auditory-motor synchronization in sports: enhancing running performance by coupling cadence with the right beats. PLoS One 8:e70758. doi: 10.1371/journal.pone.0070758
Boyd, J. E., Godbout, A., and Thornton, C. (2012). “In situ motion capture of speed skating: escaping the treadmill,” in Proceedings of the 9th Conference on Computer and Robot Vision, Toronto, 460–467. doi: 10.1109/CRV.2012.68
Braunlich, K., Seger, C. A., Jentink, K. G., Buard, I., Kluger, B. M., and Thaut, M. H. (2018). Rhythmic auditory cues shape neural network recruitment in Parkinson’s disease during repetitive motor behavior. Eur. J. Neurosci. doi: 10.1111/ejn.14227 [Epub ahead of print].
Bresin, R., de Witt, A., Papetti, S., Civolani, M., and Fontana, F. (2010). “Expressive sonification of footstep sounds,” in Proceedings of the Interaction Sonification workshop (ISon) 2010, Stockholm, 51–54.
Bruckner, H. P., Theimer, W., and Blume, H. (2014). “Real-time low latency movement sonification in stroke rehabilitation based on a mobile platform,” in Proceedings of the 2014 IEEE International Conference on Consumer Electronics, Las Vegas, NV, 264–265. doi: 10.1109/ICCE.2014.6775997
Caetano, G., Jousmäki, V., and Hari, R. (2007). Actor’s and observer’s primary motor cortices stabilize similarly after seen or heard motor actions. Proc. Natl. Acad. Sci. U.S.A. 104, 9058–9062. doi: 10.1073/pnas.0702453104
Camponogara, I., Rodger, M., Craig, C., and Cesari, P. (2017). Expert players accurately detect an opponent’s movement intentions through sound alone. J. Exp. Psychol. Hum. Percept. Perform. 43, 348–359. doi: 10.1037/xhp0000316
Cesari, P., Camponogara, I., Papetti, S., Rocchesso, D., and Fontana, F. (2014). Might as well jump: sound affects muscle activation in skateboarding. PLoS One 9:e90156. doi: 10.1371/journal.pone.0090156
Cesarini, D., Hermann, T., and Ungerechts, B. (2014a). “A real-time auditory biofeedback system for sports swimming,” in Proceedings of the 20th International Conference on Auditory Display, New York, NY, 1–3.
Cesarini, D., Schaffert, N., Manganiello, C., and Mattes, K. (2014b). AccrowLive: a multiplatform telemetry and sonification solution for rowing. Procedia Eng. 72, 273–278. doi: 10.1016/j.proeng.2014.06.049
Chen, J. L., Fujii, S., and Schlaug, G. (2016). The use of augmented auditory feedback to improve arm reaching in stroke: a case series. Disabil. Rehabil. 38, 1115–1124. doi: 10.3109/09638288.2015.1076530
Chen, Y., Huang, H., Xu, W., Wallis, R. I., Sundaram, H., Rikakis, T., et al. (2006). “The design of a real-time, multimodal biofeedback system for stroke patient rehabilitation,” in Proceedings of the 14th ACM International Conference on Multimedia, Santa Barbara, CA, 763–772. doi: 10.1145/1180639.1180804
Chiari, L., Dozza, M., Cappello, A., Horak, F. B., Macellari, V., and Giansanti, D. (2005). Audio-biofeedback for balance improvement: an accelerometry-based system. IEEE Trans. Biomed. Eng. 52, 2108–2111. doi: 10.1109/TBME.2005.857673
Chollet, D., Madani, M., and Micallef, J. P. (1992). “Effects of two types of biomechanical bio-feedback on crawl performance,” in Biomechanics and Medicine in Swimming, eds D. MacLaren, T. Reilly, and A. Lees (London: E & FN SPon Press), 57–62.
Chollet, D., Micallef, J. P., and Rabischong, P. (1988). “Biomechanical signals for external biofeedback to improve swimming techniques,” in Swimming Science V. International Series of Sport Sciences, Vol. 18, eds B. Ungerechts, K. Wilke, and K. Reischle (Champaign, IL: Human Kinetics Books), 389–396.
Choo, P. L., Gallagher, H. L., Morris, J., Pomeroy, V. M., and Van Wijck, F. (2015). Correlations between arm motor behavior and brain function following bilateral arm training after stroke: a systematic review. Brain Behav. 5:e00411. doi: 10.1002/brb3.411
Conklyn, D., Stough, D., Novak, E., Paczak, S., Chemali, K., and Bethoux, F. (2010). A home-based walking program using rhythmic auditory stimulation improves gait performance in patients with multiple sclerosis: a pilot study. Neurorehabil. Neural Repair 24, 835–842. doi: 10.1177/1545968310372139
Contreras Lopez, W. O., Higuera, C. A. E., Fonoff, E. T., de Oliveira Souza, C., Albicker, U., and Martinez, J. A. E. (2014). Listenmee® and Listenmee® smartphone application: synchronizing walking to rhythmic auditory cues to improve gait in Parkinson’s disease. Hum. Mov. Sci. 37, 147–156. doi: 10.1016/j.humov.2014.08.001
Crasta, J. E., Thaut, M. H., Anderson, C. W., Davies, P. L., and Gavin, W. J. (2018). Auditory priming improves neural synchronization in auditory-motor entrainment. Neuropsychologia 117, 102–112. doi: 10.1016/j.neuropsychologia.2018.05.017
Dailly, A. I., Sigrist, R., Kim, Y., Wolf, P., Erckens, H., Cerny, J., et al. (2012). “Can simple error sonification in combination with music help improve accuracy in upper limb movements?” in Proceedings of the Fourth IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics, Rome, 1423–1427. doi: 10.1109/BioRob.2012.6290908
Dalla Bella, S., Benoit, C. E., Farrugia, N., Keller, P. E., Obrig, H., Mainka, S., et al. (2017). Gait improvement via rhythmic stimulation in Parkinson’s disease is linked to rhythmic skills. Sci. Rep. 7:42005. doi: 10.1038/srep42005
Dalla Bella, S., Benoit, C. E., Farrugia, N., Schwartze, M., and Kotz, S. A. (2015). Effects of musically cued gait training in Parkinson’s disease: beyond a motor benefit. Ann. N. Y. Acad. Sci. 1337, 77–85. doi: 10.1111/nyas.12651
Danna, J., and Velay, J. L. (2017). On the auditory-proprioception substitution hypothesis: movement sonification in two deafferented subjects learning to write new characters. Front. Neurosci. 11:137. doi: 10.3389/fnins.2017.00137
D’Ausilio, A., Altenmüller, E., Olivetti Belardinelli, M., and Lotze, M. (2006). Cross-modal plasticity of the motor cortex while listening to a rehearsed musical piece. Eur. J. Neurosci. 24, 955–958. doi: 10.1111/j.1460-9568.2006.04960.x
De Dreu, M. J., Van Der Wilk, A. S. D., Poppe, E., Kwakkel, G., and van Wegen, E. E. (2012). Rehabilitation, exercise therapy and music in patients with Parkinson’s disease: a meta-analysis of the effects of music-based movement therapy on walking ability, balance and quality of life. Parkinsonism Relat. Disord. 18, 114–119. doi: 10.1016/S1353-8020(11)70036-0
Del Olmo, M. F., Arias, P., Furio, M. C., Pozo, M. A., and Cudeiro, J. (2006). Evaluation of the effect of training using auditory stimulation on rhythmic movement in Parkinsonian patients - a combined motor and [18F]-FDG PET study. Parkinsonism Relat. Disord. 12, 155–164. doi: 10.1016/j.parkreldis.2005.11.002
Del Olmo, M. F., and Cudeiro, J. (2005). Temporal variability of gait in Parkinson disease: effects of a rehabilitation programme based on rhythmic sound cues. Parkinsonism Relat. Disord. 11, 25–33. doi: 10.1016/j.parkreldis.2004.09.002
Delval, A., Moreau, C., Bleuse, S., Tard, C., Ryckewaert, G., Devos, D., et al. (2014). Auditory cueing of gait initiation in Parkinson’s disease patients with freezing of gait. Clin. Neurophysiol. 125, 1675–1681. doi: 10.1016/j.clinph.2013.12.101
Delwaide, P. J., and Schepens, B. (1995). Auditory startle (audio-spinal) reaction in normal man: EMG responses and H reflex changes in antagonistic lower limb muscles. Electroencephalogr. Clin. Neurophysiol. 97, 416–423. doi: 10.1016/0924-980X(95)00136-9
Dotov, D. G., Bayard, S., de Cock, V. C., Geny, C., Driss, V., Garrigue, G., et al. (2017). Biologically-variable rhythmic auditory cues are superior to isochronous cues in fostering natural gait variability in Parkinson’s disease. Gait Posture 51, 64–69. doi: 10.1016/j.gaitpost.2016.09.020
Dozza, M., Chiari, L., Chan, B., Rocchi, L., Horak, F. B., and Cappello, A. (2005). Influence of a portable audio-biofeedback device on structural properties of postural sway. J. Neuroeng. Rehabil. 2:13.
Dubus, G., and Bresin, R. (2010). “Sonification of sculler movements, development of preliminary methods,” in Proceedings of the ISon 2010 Conference, 3rd Interactive Sonification Workshop, Stockholm, 39–43.
Dyer, J. F., Stapleton, P., and Rodger, M. (2015). Sonification as concurrent augmented feedback for motor skill learning and the importance of mapping design. Open Psychol. J. 8, 192–202. doi: 10.2174/1874350101508010192
Dyer, J. F., Stapleton, P., and Rodger, M. (2017b). Transposing musical skill: sonification of movement as concurrent augmented feedback enhances learning in a bimanual task. Psychol. Res. 81, 850–862. doi: 10.1007/s00426-016-0775-0
Ebersbach, G., Heijmenberg, M., Kindermann, L., Trottenberg, T., Wissel, J., and Poewe, W. (1999). Interference of rhythmic constraint on gait in healthy subjects and patients with early Parkinson’s disease: evidence for impaired locomotor pattern generation in early Parkinson’s disease. Mov. Disord. 14, 619–625. doi: 10.1002/1531-8257(199907)14:4<619::AID-MDS1011>3.0.CO;2-X
Effenberg, A. O., Fehse, U., Schmitz, G., Krueger, B., and Mechling, H. (2016). Movement sonification: effects on motor learning beyond rhythmic adjustments. Front. Neurosci. 10:219. doi: 10.3389/fnins.2016.00219
Effenberg, A. O., and Schmitz, G. (2018). Acceleration and deceleration at constant speed: systematic modulation of motion perception by kinematic sonification. Ann. N. Y. Acad. Sci. 1425, 52–69. doi: 10.1111/nyas.13693
Engel, L. R., Frum, C., Puce, A., Walker, N. A., and Lewis, J. W. (2009). Different categories of living and non-living sound-sources activate distinct cortical networks. Neuroimage 47, 1778–1791. doi: 10.1016/j.neuroimage.2009.05.041
Fadiga, L., Fogassi, L., Pavesi, G., and Rizzolatti, G. (1995). Motor facilitation during action observation: a magnetic stimulation study. J. Neurophysiol. 73, 2608–2611. doi: 10.1152/jn.19220.127.116.1108
Fernández-Miranda, J. C., Wang, Y., Pathak, S., Stefaneau, L., Verstynen, T., and Yeh, F. C. (2015). Asymmetry, connectivity, and segmentation of the arcuate fascicle in the human brain. Brain Struct. Funct. 220, 1665–1680. doi: 10.1007/s00429-014-0751-7
Fischer, T., Kiselka, A., Dlapka, R., Doppler, J., Iber, M., Gradl, C., et al. (2017). “An auditory feedback system in use with people aged +50 Years: compliance and modifications in gait pattern,” in Converging Clinical and Engineering Research on Neurorehabilitation II. Biosystems & Biorobotics, Vol. 15, eds J. Ibáñez, J. González-Vargas, J. Azorín, M. Akay, and J. Pons (Cham: Springer), 881–885.
Franco, C., Fleury, A., Guméry, P. Y., Diot, B., Demongeot, J., and Vuillerme, N. (2013). iBalance-ABF: a smartphone-based audio-biofeedback balance system. IEEE Trans. Biomed. Eng. 60, 211–215. doi: 10.1109/TBME.2012.2222640
Freedland, R. L., Festa, C., Sealy, M., McBean, A., Elghazaly, P., Capan, A., et al. (2002). The effects of pulsed auditory stimulation on various gait measurements in persons with Parkinson’s Disease. NeuroRehabilitation 17, 81–87.
Fujii, S., Lulic, T., and Chen, J. L. (2016). More feedback is better than less: learning a novel upper limb joint coordination pattern with augmented auditory feedback. Front. Neurosci. 10:251. doi: 10.3389/fnins.2016.00251
Fujioka, T., Trainor, L. J., Large, E. W., and Ross, B. (2012). Internalized timing of isochronous sounds is represented in neuromagnetic beta oscillations. J. Neurosci. 32, 1791–1802. doi: 10.1523/JNEUROSCI.4107-11.2012
Ghai, S. (2018). Effects of real-time (sonification) and rhythmic auditory stimuli on recovering arm function post stroke: a systematic review and meta-analysis. Front. Neurol. 9:488. doi: 10.3389/fneur.2018.00488
Ghai, S., and Ghai, I. (2018). Effects of rhythmic auditory cueing in gait rehabilitation for multiple sclerosis: a mini systematic review and meta-analysis. Front. Neurol. 9:386. doi: 10.3389/fneur.2018.00386
Ghai, S., Ghai, I., and Effenberg, A. O. (2018a). Effect of rhythmic auditory cueing on gait in cerebral palsy: a systematic review and meta-analysis. Neuropsychiatr. Dis. Treat. 14, 43–59. doi: 10.2147/NDT.S148053
Ghai, S., Ghai, I., Schmitz, G., and Effenberg, A. O. (2018b). Effect of rhythmic auditory cueing on parkinsonian gait: a systematic review and meta-analysis. Sci. Rep. 8:506. doi: 10.1038/s41598-017-16232-5
Ghai, S., Schmitz, G., Hwang, T. H., and Effenberg, A. O. (2018c). Auditory proprioceptive integration: effects of real-time kinematic auditory feedback on knee proprioception. Front. Neurosci. 12:142. doi: 10.3389/fnins.2018.00142
Gilat, M., de Lima, A. L. S., Bloem, B. R., Shine, J. M., Nonnekes, J., and Lewis, S. J. (2018). Freezing of gait: promising avenues for future treatment. Parkinsonism Relat. Disord. 52, 7–16. doi: 10.1016/j.parkreldis.2018.03.009
Ginis, P., Nackaerts, E., Nieuwboer, A., and Heremans, E. (2017). Cueing for people with Parkinson’s disease with freezing of gait: a narrative review of the state-of-the-art and novel perspectives. Ann. Phys. Rehabil. Med. 61, 407–413. doi: 10.1016/j.rehab.2017.08.002
Godbout, A., Thornton, C., and Boyd, J. E. (2014). “Mobile sonification for athletes: a case study in commercialization of sonification,” in Proceedings of the 20th International Conference on Auditory Display, New York, NY, 1–8.
Gorgas, A. M., Schön, L., Dlapka, R., Doppler, J., Iber, M., Gradl, C., et al. (2017). “Short-term effects of real-time auditory display (sonification) on gait parameters in people with Parkinson’s disease - a pilot study,” in Converging Clinical and Engineering Research on Neurorehabilitation II. Biosystems & Biorobotics, Vol. 15, eds J. Ibáñez, J. González-Vargas, J. Azorín, M. Akay, and J. Pons (Cham: Springer), 855–859.
Grahn, J. A., Henry, M. J., and McAuley, J. D. (2011). FMRI investigation of cross-modal interactions in beat perception: audition primes vision, but not vice versa. Neuroimage 54, 1231–1243. doi: 10.1016/j.neuroimage.2010.09.033
Grahn, J. A., and Rowe, J. B. (2009). Feeling the beat: premotor and striatal interactions in musicians and nonmusicians during beat perception. J. Neurosci. 29, 7540–7548. doi: 10.1523/JNEUROSCI.2018-08.2009
Grahn, J. A., and Rowe, J. B. (2013). Finding and feeling the musical beat: striatal dissociations between detection and prediction of regularity. Cereb. Cortex 23, 913–921. doi: 10.1093/cercor/bhs083
Halsband, U., Ito, N., Tanji, J., and Freund, H. J. (1993). The role of premotor cortex and the supplementary motor area in the temporal control of movement in man. Brain 116, 243–266. doi: 10.1093/brain/116.1.243
Hasegawa, S., Ishijima, S., Kato, F., Mitake, H., and Sato, M. (2012). “Realtime sonification of the center of gravity for skiing,” in Proceedings of the 3rd Augmented Human International Conference, Megève, 11–14. doi: 10.1145/2160125.2160136
Hasegawa, T., Matsuki, K. I., Ueno, T., Maeda, Y., Matsue, Y., Konishi, Y., et al. (2004). Learned audio-visual cross-modal associations in observed piano playing activate the left planum temporale. An fMRI study. Cogn. Brain Res. 20, 510–518. doi: 10.1016/j.cogbrainres.2004.04.005
Hausdorff, J. M., Lowenthal, J., Herman, T., Gruendlinger, L., Peretz, C., and Giladi, N. (2007). Rhythmic auditory stimulation modulates gait variability in Parkinson’s disease. Eur. J. Neurosci. 26, 2369–2375. doi: 10.1111/j.1460-9568.2007.05810.x
Hayden, R., Clair, A. A., Johnson, G., and Otto, D. (2009). The effect of rhythmic auditory stimulation (RAS) on physical therapy outcomes for patients in gait training following stroke: a feasibility study. Int. J. Neurosci. 119, 2183–2195. doi: 10.3109/00207450903152609
Hermann, T., Ungerechts, B., Toussaint, H., and Grote, M. (2012). “Sonification of pressure changes in swimming for analysis and optimization,” in Proceedings of the 18th International Conference on Auditory Display, Atlanta, 60–67.
Hoemberg, V. (2005). “Evidence based medicine in neurological rehabilitation - a critical review,” in Re- Engineering of the Damaged Brain and Spinal Cord, ed. K. von Wild (New York, NY: Springer), 3–14.
Hohmann, T., Troje, N. F., Olmos, A., and Munzert, J. (2011). The influence of motor expertise and motor experience on action and actor recognition. J. Cogn. Psychol. 23, 403–415. doi: 10.1080/20445911.2011.525504
Hollands, K. L., Pelton, T. A., Tyson, S. F., Hollands, M. A., and van Vliet, P. M. (2012). Interventions for coordination of walking following stroke: systematic review. Gait Posture 35, 349–359. doi: 10.1016/j.gaitpost.2011.10.355
Höner, O., Hermann, T., and Grunow, C. (2004). “Sonification of group behavior for analysis and training of sports tactics,” in Proceedings of the International Workshop on Interactive Sonification, Bielefeld, 1–5.
Horsak, B., Dlapka, R., Iber, M., Gorgas, A. M., Kiselka, A., Gradl, C., et al. (2016). SONIGait: a wireless instrumented insole device for real-time sonification of gait. J. Multimodal User Interfaces 10, 195–206. doi: 10.3390/s140101073
Hossner, E. J., Schiebl, F., and Göhner, U. (2015). A functional approach to movement analysis and error identification in sports and physical education. Front. Psychol. 6:1339. doi: 10.3389/fpsyg.2015.01339
Hove, M. J., Suzuki, K., Uchitomi, H., Orimo, S., and Miyake, Y. (2012). Interactive rhythmic auditory stimulation reinstates natural 1/f timing in gait of Parkinson’s patients. PLoS One 7:e32600. doi: 10.1371/journal.pone.0032600
Huffman, R. F., and Henson, O. W. (1990). The descending auditory pathway and acousticomotor systems: connections with the inferior colliculus. Brain Res. Rev. 15, 295–323. doi: 10.1016/0165-0173(90)90005-9
Hummel, J., Hermann, T., Frauenberger, C., and Stockman, T. (2010). “Interactive sonification of German wheel sports,” in Proceedings of the ISon 2010, 3rd Interactive Sonification Workshop, Stockholm, 17–22.
Hurt, C. P., Rice, R. R., McIntosh, G. C., and Thaut, M. H. (1998). Rhythmic auditory stimulation in gait training for patients with traumatic brain injury. J. Music Ther. 35, 228–241. doi: 10.1093/jmt/35.4.228
Hurt-Thaut, C. P. (2014). Rhythmic Auditory Stimulation to Reduce Falls in Healthy Elderly and Patients with Parkinson’s Disease: A Randomized Control Trial. Doctoral dissertation, Colorado State University, Fort Collins, CO.
Immoos, A., Cerny, J., Hertler, B., Sigrist, R., Wolf, P., Kim, Y., et al. (2013). “Repetitive arm training with music and error sonification for therapy following stroke,” in Proceedings of the International Conference on Multisensory Motor Behavior: Impact of Sound, Hannover.
Kennel, C., Hohmann, T., and Raab, M. (2014a). Action perception via auditory information: agent identification and discrimination with complex movement sounds. J. Cogn. Psychol. 26, 157–165. doi: 10.1080/20445911.2013.869226
Kennel, C., Streese, L., Pizzera, A., Justen, C., Hohmann, T., and Raab, M. (2015). Auditory reafferences: the influence of real-time feedback on movement control. Front. Psychol. 6:69. doi: 10.3389/fpsyg.2015.00069
Kim, S. J., Kwak, E. E., Park, E. S., Lee, D. S., Kim, K. J., Song, J. E., et al. (2011). Changes in gait patterns with rhythmic auditory stimulation in adults with cerebral palsy. NeuroRehabilitation 29, 233–241. doi: 10.3233/NRE-2011-0698
Kleiman-Weiner, M., and Berger, J. (2006). “The sound of one arm swinging: a model for multidimensional auditory display of physical motion,” in Proceedings of the 12th International Conference on Auditory Display, London, 278–280.
Ko, B. W., Lee, H. Y., and Song, W. K. (2016). Rhythmic auditory stimulation using a portable smart device: short-term effects on gait in chronic hemiplegic stroke patients. J. Phys. Ther. Sci. 28, 1538–1543. doi: 10.1589/jpts.28.1538
Kobinata, N., Ueno, M., Imanishi, Y., and Yoshikawa, H. (2016). Immediate effects of rhythmic auditory stimulation on gait in stroke patients in relation to the lesion site. J. Phys. Ther. Sci. 28, 2441–2444. doi: 10.1589/jpts.28.2441
Kohler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gallese, V., and Rizzolatti, G. (2002). Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846–848. doi: 10.1126/science.1070311
Konttinen, N., Mononen, K., Viitasalo, J., and Mets, T. (2004). The effects of augmented auditory feedback on psychomotor skill learning in precision shooting. J. Sport Exerc. Psychol. 26, 306–316. doi: 10.1123/jsep.26.2.306
Kos, A., Umek, A., and Tomazic, S. (2015). “Biofeedback in sport: challenges in real-time motion tracking and processing,” in Proceedings of the IEEE 15th International Conference on Bioinformatics and Bioengineering, Belgrade, 1–4. doi: 10.1109/BIBE.2015.7367681
Koshimori, Y., and Thaut, M. H. (2018). Future perspectives on neural mechanisms underlying rhythm and music based neurorehabilitation in Parkinson’s disease. Ageing Res. Rev. 47, 133–139. doi: 10.1016/J.ARR.2018.07.001
Kwakkel, G., Kollen, B. J., and Krebs, H. I. (2008). Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil. Neural Repair 22, 111–121. doi: 10.1177/1545968307305457
Lahav, A., Saltzman, E., and Schlaug, G. (2007). Action representation of sound: audiomotor recognition network while listening to newly acquired actions. J. Neurosci. 27, 308–314. doi: 10.1523/JNEUROSCI.4822-06.2007
Lécuyer, A., Marchal, M., Hamelin, A., Wolinski, D., Fontana, F., Civolani, M., et al. (2011). “Shoes-your-style: changing sound of footsteps to create new walking experiences,” in Proceedings of Workshop on Sound and Music Computing for Human-Computer Interaction (CHItaly), Alghero, 13–16.
Lewis, J. W., Brefczynski, J. A., Phinney, R. E., Janik, J. J., and DeYoe, E. A. (2005). Distinct cortical pathways for processing tool versus animal sounds. J. Neurosci. 25, 5148–5158. doi: 10.1523/JNEUROSCI.0419-05.2005
Lewis, J. W., Talkington, W. J., Puce, A., Engel, L. R., and Frum, C. (2011). Cortical networks representing object categories and high-level attributes of familiar real-world action sounds. J. Cogn. Neurosci. 23, 2079–2101. doi: 10.1162/jocn.2010.21570
Lim, I., van Wegen, E., de Goede, C., Deutekom, M., Nieuwboer, A., Willems, A., et al. (2005). Effects of external rhythmical cueing on gait in patients with Parkinson’s disease: a systematic review. Clin. Rehabil. 19, 695–713. doi: 10.1191/0269215505cr906oa
Lotze, M., Scheler, G., Tan, H. R., Braun, C., and Birbaumer, N. (2003). The musician’s brain: functional imaging of amateurs and professionals during performance and imagery. Neuroimage 20, 1817–1829. doi: 10.1016/j.neuroimage.2003.07.018
Luft, A. R., McCombe-Waller, S., Whitall, J., Forrester, L. W., Macko, R., Sorkin, J. D., et al. (2004). Repetitive bilateral arm training and motor cortex activation in chronic stroke: a randomized controlled trial. JAMA 292, 1853–1861. doi: 10.1001/jama.292.15.1853
Lum, P. S., Burgar, C. G., Shor, P. C., Majmundar, M., and Van der Loos, M. (2002). Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke. Arch. Phys. Med. Rehabil. 83, 952–959. doi: 10.1053/apmr.2001.33101
Maculewicz, J., Erkut, C., and Serafin, S. (2016). An investigation on the impact of auditory and haptic feedback on rhythmic walking interactions. Int. J. Hum. Comput. Stud. 85, 40–46. doi: 10.1016/j.ijhcs.2015.07.003
Mainka, S., Wissel, J., Völler, H., and Evers, S. (2018). The use of rhythmic auditory stimulation to optimize treadmill training for stroke patients: a randomized controlled trial. Front. Neurol. 9:755. doi: 10.3389/fneur.2018.00755
Malcolm, M. P., Massie, C., and Thaut, M. (2009). Rhythmic auditory-motor entrainment improves hemiparetic arm kinematics during reaching movements: a pilot study. Top. Stroke Rehabil. 16, 69–79. doi: 10.1310/tsr1601-69
Manto, M., Bower, J. M., Conforto, A. B., Delgado-García, J. M., Da Guarda, S. N. F., Gerwig, M., et al. (2012). Consensus paper: roles of the cerebellum in motor control-the diversity of ideas on cerebellar involvement in movement. Cerebellum 11, 457–487. doi: 10.1007/s12311-011-0331-9
Marinovic, W., Tresilian, J. R., de Rugy, A., Sidhu, S., and Riek, S. (2014). Corticospinal modulation induced by sounds depends on action preparedness. J. Physiol. 592, 153–169. doi: 10.1113/jphysiol.2013.254581
Matsubara, M., Terasawa, H., Kadone, H., Suzuki, K., and Makino, S. (2012). “Sonification of muscular activity in human movements using the temporal patterns in EMG,” in Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hollywood, CA, 1–5.
McIntosh, G. C., Brown, S. H., Rice, R. R., and Thaut, M. H. (1997). Rhythmic auditory-motor facilitation of gait patterns in patients with Parkinson’s disease. J. Neurol. Neurosurg. Psychiatry 62, 22–26. doi: 10.1136/jnnp.62.1.22
Merchant, H., Grahn, J., Trainor, L., Rohrmeier, M., and Fitch, W. T. (2015). Finding the beat: a neural perspective across humans and non-human primates. Philos. Trans. R. Soc. B Biol. Sci. 370:20140093. doi: 10.1098/rstb.2014.0093
Mezzarobba, S., Grassi, M., Pellegrini, L., Catalan, M., Kruger, B., Furlanis, G., et al. (2018). Action observation plus sonification. A novel therapeutic protocol for Parkinson’s patient with freezing of gait. Front. Neurol. 8:723. doi: 10.3389/fneur.2017.00723
Michael, K. M., Allen, J. K., and Macko, R. F. (2005). Reduced ambulatory activity after stroke: the role of balance, gait, and cardiovascular fitness. Arch. Phys. Med. Rehabil. 6, 1552–1556. doi: 10.1016/j.apmr.2004.12.026
Miller, R. A., Thaut, M. H., McIntosh, G. C., and Rice, R. R. (1996). Components of EMG symmetry and variability in parkinsonian and healthy elderly gait. Electroencephalogr. Clin. Neurophysiol. 101, 1–7.
Morris, M. E., Iansek, R., Matyas, T. A., and Summers, J. J. (1996). Stride length regulation in Parkinson’s disease: normalization strategies and underlying mechanisms. Brain 119, 551–568. doi: 10.1093/brain/119.2.551
Murgia, M., Corona, F., Pili, R., Sors, F., Agostini, T., and Casula, C. (2015). Rhythmic auditory stimulation (RAS) and motor rehabilitation in Parkinson’s disease: new frontiers in assessment and intervention protocols. Open Psychol. J. 8, 220–229. doi: 10.2174/1874350101508010220
Murgia, M., Pili, R., Corona, F., Sors, F., Agostini, T. A., Bernardis, P., et al. (2018). The use of footstep sounds as rhythmic auditory stimulation for gait rehabilitation in Parkinson’s disease: a randomized controlled trial. Front. Neurol. 9:348. doi: 10.3389/fneur.2018.00348
Murgia, M., Prpic, V., McCullagh, P., Santoro, I., Galmonte, A., and Agostini, T. (2017). Modality and perceptual-motor experience influence the detection of temporal deviations in tap dance sequences. Front. Psychol. 8:1340. doi: 10.3389/fpsyg.2017.01340
Murgia, M., Sors, F., Vono, R., Muroni, A. F., Delitalia, L., Di Corrado, D., et al. (2012b). Using auditory stimulation to enhance athletes’ strength: An experimental study in weightlifting. Rev. Psychol. 19, 13–16.
Nascimento, L. R., de Oliveira, C. Q., Ada, L., Michaelsen, S. M., and Teixeira-Salmela, L. F. (2015). Walking training with cueing of cadence improves walking speed and stride length after stroke more than walking training alone: a systematic review. J. Physiother. 61, 10–15. doi: 10.1016/j.jphys.2014.11.015
Newbold, J. W., Bianchi-Berthouze, N., and Gold, N. E. (2017). “Musical expectancy in squat sonification for people who struggle with physical activity,” in Proceedings of the 23rd International Conference on Auditory Display (State College, PA: Pennsylvania State University), 65–72. doi: 10.21785/icad2017.008
Nieuwboer, A., Baker, K., Willems, A. M., Jones, D., Spildooren, J., Lim, I., et al. (2009). The short-term effects of different cueing modalities on turn speed in people with Parkinson’s disease. Neurorehabil. Neural Repair 23, 831–836. doi: 10.1177/1545968309337136
Nieuwboer, A., Kwakkel, G., Rochester, L., Jones, D., van Wegen, E., Willems, A. M., et al. (2007). Cueing training in the home improves gait-related mobility in Parkinson’s disease: the RESCUE trial. J. Neurol. Neurosurg. Psychiatry 78, 134–140. doi: 10.1136/jnnp.200X.097923
Nombela, C., Hughes, L. E., Owen, A. M., and Grahn, J. A. (2013). Into the groove: can rhythm influence Parkinson’s disease? Neurosci. Biobehav. Rev. 37, 2564–2570. doi: 10.1016/j.neubiorev.2013.08.003
Nylander, S., Kent, A., and Tholander, J. (2014). “Swing sound: experiencing the golf swing through sound,” in Proceedings of the CHI’14 Extended Abstracts on Human Factors in Computing Systems, Toronto, 443–446. doi: 10.1145/2559206.2574789
Park, J., Park, S. Y., Kim, Y. W., and Woo, Y. (2015). Comparison between treadmill training with rhythmic auditory stimulation and ground walking with rhythmic auditory stimulation on gait ability in chronic stroke patients: a pilot study. NeuroRehabilitation 37, 193–202. doi: 10.3233/NRE-151252
Pau, M., Corona, F., Pili, R., Casula, C., Sors, F., Agostini, T., et al. (2016). Effects of physical rehabilitation integrated with rhythmic auditory stimulation on spatio-temporal and kinematic parameters of gait in Parkinson’s disease. Front. Neurol. 7:126. doi: 10.3389/fneur.2016.00126
Pennycott, A., Wyss, D., Vallery, H., Klamroth-Marganska, V., and Riener, R. (2012). Towards more effective robotic gait training for stroke rehabilitation: a review. J. Neuroeng. Rehabil. 9:65. doi: 10.1186/1743-0003-9-65
Petter, E. A., Lusk, N. A., Hesslow, G., and Meck, W. H. (2016). Interactive roles of the cerebellum and striatum in sub-second and supra-second timing: support for an initiation, continuation, adjustment, and termination (ICAT) model of temporal processing. Neurosci. Biobehav. Rev. 71, 739–755. doi: 10.1016/j.neubiorev.2016.10.015
Pizzamiglio, L., Aprile, T., Spitoni, G., Pitzalis, S., Bates, E., D’amico, S., et al. (2005). Separate neural systems for processing action-or non-action-related sounds. Neuroimage 24, 852–861. doi: 10.1016/j.neuroimage.2004.09.025
Plotnik, M., Shema, S., Dorfman, M., Gazit, E., Brozgol, M., Giladi, N., et al. (2014). A motor learning-based intervention to ameliorate freezing of gait in subjects with Parkinson’s disease. J. Neurol. 261, 1329–1339. doi: 10.1007/s00415-014-7347-2
Powell, N., and Lumsden, J. (2015). “Exploring novel auditory displays for supporting accelerated skills acquisition and enhanced performance in motorsport,” in Proceedings of the 21st International Conference on Auditory Display, Graz, 173–180.
Prange, G. B., Jannink, M. J., Groothuis-Oudshoorn, C. G., Hermens, H. J., and IJzerman, M. J. (2006). Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. J. Rehabil. Res. Dev. 43, 171–184. doi: 10.1682/JRRD.2005.04.0076
Prassas, S., Thaut, M., McIntosh, G., and Rice, R. (1997). Effect of auditory rhythmic cueing on gait kinematic parameters of stroke patients. Gait Posture 6, 218–223. doi: 10.1016/S0966-6362(97)00010-6
Proverbio, A. M., Calbi, M., Manfredi, M., and Zani, A. (2014). Audio-visuomotor processing in the Musician’s brain: an ERP study on professional violinists and clarinetists. Sci. Rep. 4:5866. doi: 10.1038/srep05866
Ramezanzade, H., Abdoli, B., Farsi, A., and Sanjari, M. A. (2014). The effect of sonification modelling on perception and accuracy of performing jump shot basketball. Int. J. Sport Stud. 4, 1388–1392.
Rao, S. M., Harrington, D. L., Haaland, K. Y., Bobholz, J. A., Cox, R. W., and Binder, J. R. (1997). Distributed neural systems underlying the timing of movements. J. Neurosci. 17, 5528–5535. doi: 10.1523/JNEUROSCI.17-14-05528.1997
Repp, B. H., and Penel, A. (2002). Auditory dominance in temporal processing: new evidence from synchronization with simultaneous visual and auditory sequences. J. Exp. Psychol. 28, 1085–1099. doi: 10.1037/0096-1518.104.22.1685
Roberts, S., Eyckholt, R., and Thaut, M. H. (2000). Search for correlations and evidence for deterministic chaos in rhythmic motor control of the human brain. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 62, 2597–2606.
Robertson, J. V. G., Hoellinger, T., Lindberg, P., Bensmail, D., Hanneton, S., and Roby-Brami, A. (2009). Effect of auditory feedback differs according to side of hemiparesis: a comparative pilot study. J. Neuroeng. Rehabil. 6:45. doi: 10.1186/1743-0003-6-45
Rocha, P. A., Porfírio, G. M., Ferraz, H. B., and Trevisani, V. F. (2014). Effects of external cues on gait parameters of Parkinson’s disease patients: a systematic review. Clin. Neurol. Neurosurg 124, 127–134. doi: 10.1016/j.clineuro.2014.06.026
Rochester, L., Baker, K., Hetherington, V., Jones, D., Willems, A. M., Kwakkel, G., et al. (2010). Evidence for motor learning in Parkinson’s disease: acquisition, automaticity and retention of cued gait performance after training with external rhythmical cues. Brain Res. 1319, 103–111. doi: 10.1016/j.brainres.2010.01.001
Rodger, M. W., and Craig, C. M. (2016). Beyond the metronome: auditory events and music may afford more than just interval durations as gait cues in Parkinson’s disease. Front. Neurosci. 10:272. doi: 10.3389/fnins.2016.00272
Rodger, M. W., Young, W. R., and Craig, C. M. (2014). Synthesis of walking sounds for alleviating gait disturbances in Parkinson’s disease. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 543–548. doi: 10.1109/TNSRE.2013.2285410
Rodríguez-Fornells, A., Rojo, N., Amengual, J. L., Ripollés, P., Altenmüller, E., and Münte, T. F. (2012). The involvement of audio-motor coupling in the music-supported therapy applied to stroke patients. Ann. N. Y. Acad. Sci. 1252, 282–293. doi: 10.1111/j.1749-6632.2011.06425.x
Roerdink, M., Bank, P. J., Peper, C. L. E., and Beek, P. J. (2011). Walking to the beat of different drums: Practical implications for the use of acoustic rhythms in gait rehabilitation. Gait Posture 33, 690–694. doi: 10.1016/j.gaitpost.2011.03.001
Roerdink, M., Lamoth, C. J., Kwakkel, G., Van Wieringen, P. C., and Beek, P. J. (2007). Gait coordination after stroke: benefits of acoustically paced treadmill walking. Phys. Ther. 87, 1009–1022. doi: 10.2522/ptj.20050394
Rosati, G., Rodà, A., Avanzini, F., and Masiero, S. (2013). On the role of auditory feedback in robot-assisted movement training after stroke: review of the literature. Comput. Intell. Neurosci. 2013:586138. doi: 10.1155/2013/586138
Ross, B., Barat, M., and Fujioka, T. (2017). Sound-making actions lead to immediate plastic changes of neuromagnetic evoked responses and induced β-band oscillations during perception. J. Neurosci. 37, 5948–5959. doi: 10.1523/JNEUROSCI.3613-16.2017
Rossignol, S., and Melvill Jones, G. (1976). Audio-spinal influence in man studied by the H-reflex and its possible role on rhythmic movements synchronized to sound. Clin. Neurophysiol. 41, 83–92. doi: 10.1016/0013-4694(76)90217-0
Rubinstein, T. C., Giladi, N., and Hausdorff, J. M. (2002). The power of cueing to circumvent dopamine deficits: a review of physical therapy treatment of gait disturbances in Parkinson’s disease. Mov. Disord. 17, 1148–1160. doi: 10.1002/mds.10259
Sanderson, J., and Hunt, A. (2016). “Using real-time sonification of heart rate data to provide a mobile based training aid for runners,” in Proceedings of the Interactive Audio Systems Symposium (Heslington: University of York), 1–8.
Schaffert, N., Godbout, A., Schlueter, S., and Mattes, K. (2017). Towards an application of interactive sonification for the forces applied on the pedals during cycling on the Wattbike ergometer. Displays 50, 41–48. doi: 10.1016/j.displa.2017.09.004
Schaffert, N., Mattes, K., and Effenberg, A. O. (2010). “Listen to the boat motion: acoustic information for elite rowers,” in Proceedings of the ISon 2010, 3rd Interactive Sonification Workshop, Stockholm, 31–38.
Schaffert, N., and Mattes, K. (2014). Testing immediate and retention effects of acoustic feedback on the boat motion in high-performance rowing. J. Hum. Sport Exerc. 9, 616–628. doi: 10.14198/jhse.2014.92.02
Schaffert, N., Mattes, K., and Effenberg, A. O. (2011). An investigation of online acoustic information for elite rowers in on-water training conditions. J. Hum. Sport Exerc. 6, 392–405. doi: 10.4100/jhse.2011.62.20
Schedel, M., Weymouth, D., Pinkhasov, T., Loomis, J., Morris, I. B., Vasudevan, E., et al. (2016). “Interactive sonification of gait: realtime biofeedback for people with Parkinson’s disease,” in Proceedings of the ISon 2016, Fifth Interactive Sonification Workshop, Bielefeld.
Scheef, L., Boecker, H., Daamen, M., Fehse, U., Landsberg, M. W., Granath, D. O., et al. (2009). Multimodal motion processing in area V5/MT: evidence from an artificial class of audio-visual events. Brain Res. 1252, 94–104. doi: 10.1016/j.brainres.2008.10.067
Schmitz, G., Kroeger, D., and Effenberg, A. O. (2014). “A mobile sonification system for stroke rehabilitation,” in Proceedings of the 20th International Conference on Auditory Display, New York, NY, 1–7.
Schmitz, G., Mohammadi, B., Hammer, A., Heldmann, M., Samii, A., Münte, T. F., et al. (2013). Observation of sonified movements engages a basal ganglia frontocortical network. BMC Neurosci. 14:32. doi: 10.1186/1471-2202-14-32
Schneider, S., Münte, T., Rodriguez-Fornells, A., Sailer, M., and Altenmüller, E. (2010). Music-supported training is more efficient than functional motor training for recovery of fine motor skills in stroke patients. Music Percept. 27, 271–280. doi: 10.1525/mp.2010.27.4.271
Schneider, S., Schönle, P. W., Altenmüller, E., and Münte, T. F. (2007). Using musical instruments to improve motor skill recovery following a stroke. J. Neurol. 254, 1339–1346. doi: 10.1007/s00415-006-0523-2
Scholz, D. S., Rhode, S., Großbach, M., Rollnik, J., and Altenmüller, E. (2015). Moving with music for stroke rehabilitation: a sonification feasibility study. Ann. N. Y. Acad. Sci. 1337, 69–76. doi: 10.1111/nyas.12691
Scholz, D. S., Rohde, S., Nikmaram, N., Brückner, H. P., Großbach, M., Rollnik, J. D., et al. (2016). Sonification of arm movements in stroke rehabilitation–a novel approach in neurologic music therapy. Front. Neurol. 7:106. doi: 10.3389/fneur.2016.00106
Secoli, R., Milot, M.-H., Rosati, G., and Reinkensmeyer, D. J. (2011). Effect of visual distraction and auditory feedback on patient effort during robot assisted movement training after stroke. J. Neuroeng. Rehabil. 8:21. doi: 10.1186/1743-0003-8-21
Shahraki, M., Sohrabi, M., Torbati, H. T., Nikkhah, K., and NaeimiKia, M. (2017). Effect of rhythmic auditory stimulation on gait kinematic parameters of patients with multiple sclerosis. J. Med. Life 10, 33–37.
Sigrist, R., Rauter, G., Marchal-Crespo, L., Riener, R., and Wolf, P. (2015). Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Exp. Brain Res. 233, 909–925. doi: 10.1007/s00221-014-4167-7
Sigrist, R., Rauter, G., Riener, R., and Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20, 21–53. doi: 10.3758/s13423-012-0333-8
Sihvonen, A. J., Särkämö, T., Leo, V., Tervaniemi, M., Altenmüller, E., and Soinila, S. (2017). Music-based interventions in neurological rehabilitation. Lancet Neurol. 16, 648–660. doi: 10.1016/S1474-4422(17)30168-0
Skodda, S., Flasskamp, A., and Schlegel, U. (2010). Instability of syllable repetition as a model for impaired motor processing: is Parkinson’s disease a “rhythm disorder”? J. Neural Transm. 117, 605–612. doi: 10.1007/s00702-010-0390-y
Song, J. H., Zhou, P. Y., Cao, Z. H., Ding, Z. G., Chen, H. X., and Zhang, G. B. (2015). Rhythmic auditory stimulation with visual stimuli on motor and balance function of patients with Parkinson’s disease. Eur. Rev. Med. Pharmacol. Sci. 19, 2001–2007.
Sors, F., Murgia, M., Santoro, I., Prpic, V., Galmonte, A., and Agostini, T. (2017). The contribution of early auditory and visual information to the discrimination of shot power in ball sports. Psychol. Sport Exerc. 31, 44–51. doi: 10.1016/j.psychsport.2017.04.005
Sors, F., Prpic, V., Santoro, I., Galmonte, A., Agostini, T., and Murgia, M. (2018a). Loudness, but not shot power, influences simple reaction times to soccer penalty sounds. Psihologija 51, 127–141. doi: 10.2298/PSI170926016S
Sors, F., Lath, F., Bader, A., Santoro, I., Galmonte, A., Agostini, T., et al. (2018b). Predicting the length of volleyball serves: the role of early auditory and visual information. PLoS One 13:e0208174. doi: 10.1371/journal.pone.0208174
Spaulding, S. J., Barber, B., Colby, M., Cormack, B., Mick, T., and Jenkins, M. E. (2013). Cueing and gait improvement among people with Parkinson’s disease: a meta-analysis. Arch. Phys. Med. Rehabil. 94, 562–570. doi: 10.1016/j.apmr.2012.10.026
Stauffer, C. C., Haldemann, J., Troche, S. J., and Rammsayer, T. H. (2012). Auditory and visual temporal sensitivity: evidence for a hierarchical structure of modality-specific and modality-independent levels of temporal information processing. Psychol. Res. 76, 20–31. doi: 10.1007/s00426-011-0333-8
Stienstra, J., Overbeeke, K., and Wensveen, S. (2011). “Embodying complexity through movement sonification: case study on empowering the speed-skater,” in Proceedings of the 9th ACM SIGCHI Italian Chapter International Conference on Computer-Human Interaction: Facing Complexity, Alghero, 39–44. doi: 10.1145/2037296.2037310
Stupacher, J., Hove, M. J., Novembre, G., Schütz-Bosbach, S., and Keller, P. E. (2013). Musical groove modulates motor cortex excitability: a TMS investigation. Brain Cogn. 82, 127–136. doi: 10.1016/j.bandc.2013.03.003
Suh, J. H., Han, S. J., Jeon, S. Y., Kim, H. J., Lee, J. E., Yoon, T. S., et al. (2014). Effect of rhythmic auditory stimulation on gait and balance in hemiplegic stroke patients. NeuroRehabilitation 34, 193–199. doi: 10.3233/NRE-131008
Tarnas, J., and Schaffert, N. (2017). “Sonifikacja w żeglarstwie regatowym, Sonification in sailing regatta,” in Diagnostics in Sport, Handbook for Modern Coaches, eds K. Kusego and J. Zienlinskiego (Poznań: Akademia Wychowania Fizycznego w Poznaniu), 213–227.
Teki, S., Grube, M., and Griffiths, T. D. (2012). A unified model of time perception accounts for duration-based and beat-based timing mechanisms. Front. Integr. Neurosci. 5:90. doi: 10.3389/fnint.2011.00090
Thaut, M. H., Miller, R. A., and Schauer, L. M. (1998b). Multiple synchronization strategies in rhythmic sensorimotor tasks: phase vs period correction. Biol. Cybern. 79, 241–250. doi: 10.1007/s004220050474
Thaut, M. H., and Kenyon, G. P. (2003). Rapid motor adaptations to subliminal frequency shifts in syncopated rhythmic sensorimotor synchronization. Hum. Mov. Sci. 22, 321–338. doi: 10.1016/S0167-9457(03)00048-4
Thaut, M. H., Kenyon, G. P., Hurt, C. P., McIntosh, G. C., and Hoemberg, V. (2002a). Kinematic optimization of spatiotemporal patterns in paretic arm training with stroke patients. Neuropsychologia 40, 1073–1081.
Thaut, M. H., Leins, A. K., Rice, R. R., Argstatter, H., Kenyon, G. P., McIntosh, G. C., et al. (2007). Rhythmic auditory stimulation improves gait more than NDT/Bobath training in near-ambulatory patients early poststroke: a single-blind, randomized trial. Neurorehabil. Neural Repair 21, 455–459. doi: 10.1177/1545968307300523
Thaut, M. H., McIntosh, G. C., and Hoemberg, V. (2015). Neurobiological foundations of neurologic music therapy: rhythmic entrainment and the motor system. Front. Psychol. 6:1185. doi: 10.3389/fpsyg.2015.01185
Thaut, M. H., McIntosh, G. C., Prassas, S. G., and Rice, R. R. (1992). Effect of rhythmic auditory cuing on temporal stride parameters and EMG patterns in normal gait. J. Neurol. Rehabil. 6, 185–190. doi: 10.1177/136140969200600403
Thaut, M. H., McIntosh, G. C., Prassas, S. G., and Rice, R. R. (1993). Effect of rhythmic auditory cuing on temporal stride parameters and EMG. Patterns in hemiparetic gait of stroke patients. J. Neurol. Rehabil. 7, 9–16. doi: 10.1177/136140969300700103
Thaut, M. H., McIntosh, G. C., Rice, R. R., Miller, R. A., Rathbun, J., and Brault, J. M. (1996). Rhythmic auditory stimulation in gait training for Parkinson’s disease patients. Mov. Disord. 11, 193–200. doi: 10.1002/mds.870110213
Thaut, M. H., McIntosh, K. W., McIntosh, G. C., and Hoemberg, V. (2001). Auditory rhythmicity enhances movement and speech motor control in patients with Parkinson’s disease. Funct. Neurol. 16, 163–172.
Thaut, M. H., Rice, R. R., Braun Janzen, T., Hurt-Thaut, C., and McIntosh, G. C. (2018). Rhythmic auditory stimulation for reduction of falls in Parkinson’s disease: a randomized controlled study. Clin. Rehabil. doi: 10.1177/0269215518788615 [Epub ahead of print].
Thaut, M. H., Stephan, K. M., Wunderlich, G., Schicks, W., Tellmann, L., Herzog, H., et al. (2009). Distinct cortico-cerebellar activations in rhythmic auditory motor synchronization. Cortex 45, 44–53. doi: 10.1016/j.cortex.2007.09.009
Ticini, L. F., Schütz-Bosbach, S., Weiss, C., Casile, A., and Waszak, F. (2012). When sounds become actions: higher-order representation of newly learned action sounds in the human motor system. J. Cogn. Neurosci. 24, 464–474. doi: 10.1162/jocn_a_00134
Tinazzi, M., Fiaschi, A., Frasson, E., Fiorio, M., Cortese, F., and Aglioti, S. M. (2002). Deficits of temporal discrimination in dystonia are independent from the spatial distance between the loci of tactile stimulation. Mov. Disord. 17, 333–338. doi: 10.1002/mds.10019
Tissberger, J. P., and Wersenyi, G. (2011). “Sonification solutions for body movements in rehabilitation of locomotor disorders,” in Proceedings of the 17th International Conference on Auditory Display, Budapest, 1–6.
Tomeo, E., Cesari, P., Aglioti, S. M., and Urgesi, C. (2012). Fooling the kickers but not the goalkeepers: behavioral and neurophysiological correlates of fake action detection in soccer. Cereb. Cortex 23, 2765–2778. doi: 10.1093/cercor/bhs279
Torres, A. V., Kluckner, V., and Franinovic, K. (2013). “). Development of a sonification method to enhance gait rehabilitation,” in Proceedings of the ISon 2013, 4th Interactive Sonification Workshop, Erlangen, 37–43.
Uchitomi, H., Ota, L., Ogawa, K. I., Orimo, S., and Miyake, Y. (2013). Interactive rhythmic cue facilitates gait relearning in patients with Parkinson’s disease. PLoS One 8:e72176. doi: 10.1371/journal.pone.0072176
van der Zwan, R., MacHatch, C., Kozlowski, D., Troje, N. F., Blanke, O., and Brooks, A. (2009). Gender bending: auditory cues affect visual judgements of gender in biological motion displays. Exp. Brain Res. 198, 373–382. doi: 10.1007/s00221-009-1800-y
Van Vugt, F. T., Ritter, J., Rollnik, J. D., and Altenmüller, E. (2014). Music-supported motor training after stroke reveals no superiority of synchronization in group therapy. Front. Hum. Neurosci. 8:315. doi: 10.3389/fnhum.2014.00315
Vogt, K., Pirrò, D., Kobenz, I., Höldrich, R., and Eckel, G. (2010). “PhysioSonic - evaluated movement sonification as auditory feedback in physiotherapy,” in Auditory Display. CMMR 2009, ICAD 2009. Lecture Notes in Computer Science, Vol. 5954, eds S. Ystad, M. Aramaki, R. Kronland-Martinet, and K. Jensen (Berlin: Springer), 103–120.
Wallis, I., Ingalls, T., Rikakis, T., Olsen, L., Chen, Y., Xu, W., et al. (2007). “Real-time sonification of movement for an immersive stroke rehabilitation environment,” in Proceedings of the 13th International Conference on Auditory Display, Montreal, 26–29.
Wang, X., Pathak, S., Stefaneanu, L., Yeh, F. C., Li, S., and Fernandez-Miranda, J. C. (2016). Subcomponents and connectivity of the superior longitudinal fasciculus in the human brain. Brain Struct. Funct. 221, 2075–2092. doi: 10.1007/s00429-015-1028-5
Whitall, J., Waller, S. M., Silver, K. H., and Macko, R. F. (2000). Repetitive bilateral arm training with rhythmic auditory cueing improves motor function in chronic hemiparetic stroke. Stroke 31, 2390–2395. doi: 10.1161/01.STR.31.10.2390
Willems, A. M., Leuven, K. U., Nieuwboer, A., Chavret, F., Desloovere, K., Dom, R., et al. (2006). The use of rhythmic auditory cues to influence gait in patients with Parkinson’s disease, the differential effect for freezers and non-freezers, an explorative study. Disabil. Rehabil. 28, 721–728. doi: 10.1080/09638280500386569
Witt, S. T., Laird, A. R., and Meyerand, M. E. (2008). Functional neuroimaging correlates of finger-tapping task variations: an ALE meta-analysis. Neuroimage 42, 343–356. doi: 10.1016/j.neuroimage.2008.04.025
Wittwer, J. E., Webster, K. E., and Hill, K. (2013). Rhythmic auditory cueing to improve walking in patients with neurological conditions other than Parkinson’s disease–what is the evidence?. Disabil. Rehabil. 35, 164–176. doi: 10.3109/09638288.2012.690495
Wolf, A., Scheiderer, R., Napolitan, N., Belden, C., Shaub, L., and Whitford, M. (2014). Efficacy and task structure of bimanual training post stroke: a systematic review. Top. Stroke Rehabil. 21, 181–196. doi: 10.1310/tsr2103-181
Woods, E. A., Hernandez, A. E., Wagner, V. E., and Beilock, S. L. (2014). Expert athletes activate somatosensory and motor planning regions of the brain when passively listening to familiar sports sounds. Brain Cogn. 87, 122–133. doi: 10.1016/j.bandc.2014.03.007
Wright, R. L., Bevins, J. W., Pratt, D., Sackley, C. M., and Wing, A. M. (2016). Metronome cueing of walking reduces gait variability after a cerebellar stroke. Front. Neurol. 7:84. doi: 10.3389/fneur.2016.00084
Wright, R. L., Brownless, S. B., Pratt, D., Sackley, C. M., and Wing, A. M. (2017). Stepping to the beat: feasibility and potential efficacy of a home-based auditory-cued step training program in chronic stroke. Front. Neurol. 8:412. doi: 10.3389/fneur.2017.00412
Yang, J., and Hunt, A. (2013). “Sonic trainer: real-time sonification of muscular activity and limb positions in general physical exercise,” in Proceedings of the ISon 2013, 4th Interactive Sonification Workshop, Erlangen, 44–51.
Yang, J., and Hunt, A. (2015). “Real-time sonification of biceps curl exercise using muscular activity and kinematics,” in Proceedings of the 21st International Conference on Auditory Display, Graz, 289–293.
Yoon, S. K., and Kang, S. H. (2016). Effects of inclined treadmill walking training with rhythmic auditory stimulation on balance and gait in stroke patients. J. Phys. Ther. Sci. 28, 3367–3370. doi: 10.1589/jpts.28.3367
Young, W. R., Rodger, M. W., and Craig, C. M. (2014). Auditory observation of stepping actions can cue both spatial and temporal components of gait in Parkinson’s disease patients. Neuropsychologia 57, 140–153. doi: 10.1016/j.neuropsychologia.2014.03.009
Young, W. R., Shreve, L., Quinn, E. J., Craig, C., and Bronte-Stewart, H. (2016). Auditory cueing in Parkinson’s patients with freezing of gait. What matters most: action-relevance or cue-continuity? Neuropsychologia 87, 54–62. doi: 10.1016/j.neuropsychologia.2016.04.034
Keywords: acoustic feedback, movement sonification, rhythmic auditory stimulation, sports, motor rehabilitation, Parkinson’s disease, stroke
Citation: Schaffert N, Braun Janzen T, Mattes K and Thaut MH (2019) A Review on the Relationship Between Sound and Movement in Sports and Rehabilitation. Front. Psychol. 10:244. doi: 10.3389/fpsyg.2019.00244
Received: 09 July 2018; Accepted: 24 January 2019;
Published: 12 February 2019.
Edited by:Mauro Murgia, University of Trieste, Italy
Reviewed by:Gerd Schmitz, Leibniz University Hannover, Germany
Ivan Camponogara, New York University Abu Dhabi, United Arab Emirates
Copyright © 2019 Schaffert, Braun Janzen, Mattes and Thaut. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Nina Schaffert, firstname.lastname@example.org