Systematic Review ARTICLE
Neuromuscular Adaptations to Multimodal Injury Prevention Programs in Youth Sports: A Systematic Review with Meta-Analysis of Randomized Controlled Trials
- 1Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
- 2Department of Public and Occupational Health & Amsterdam Movement Sciences, VU University Medical Center, Amsterdam, Netherlands
- 3College of Human Medicine, Michigan State University, East Lansing, MI, United States
- 4Institute of Training and Computer Science in Sport, German Sport University Cologne, Köln, Germany
Objective: Neuromuscular injury prevention programs (IPP) can reduce injury rate by about 40% in youth sport. Multimodal IPP include, for instance, balance, strength, power, and agility exercises. Our systematic review and meta-analysis aimed to evaluate the effects of multimodal IPP on neuromuscular performance in youth sports.
Methods: We conducted a systematic literature search including selected search terms related to youth sports, injury prevention, and neuromuscular performance. Inclusion criteria were: (i) the study was a (cluster-)randomized controlled trial (RCT), and (ii) investigated healthy participants, up to 20 years of age and involved in organized sport, (iii) an intervention arm performing a multimodal IPP was compared to a control arm following a common training regime, and (iv) neuromuscular performance parameters (e.g., balance, power, strength, sprint) were assessed. Furthermore, we evaluated IPP effects on sport-specific skills.
Results: Fourteen RCTs (comprising 704 participants) were analyzed. Eight studies included only males, and five only females. Seventy-one percent of all studies investigated soccer players with basketball, field hockey, futsal, Gaelic football, and hurling being the remaining sports. The average age of the participants ranged from 10 years up to 19 years and the level of play from recreational to professional. Intervention durations ranged from 4 weeks to 4.5 months with a total of 12 to 57 training sessions. We observed a small overall effect in favor of IPP for balance/stability (Hedges' g = 0.37; 95%CI 0.17, 0.58), leg power (g = 0.22; 95%CI 0.07, 0.38), and isokinetic hamstring and quadriceps strength as well as hamstrings-to-quadriceps ratio (g = 0.38; 95%CI 0.21, 0.55). We found a large overall effect for sprint abilities (g = 0.80; 95%CI 0.50, 1.09) and sport-specific skills (g = 0.83; 95%CI 0.34, 1.32). Subgroup analyses revealed larger effects in high-level (g = 0.34–1.18) compared to low-level athletes (g = 0.22–0.75), in boys (g = 0.27–1.02) compared to girls (g = 0.09–0.38), in older (g = 0.32–1.16) compared to younger athletes (g = 0.18–0.51), and in studies with high (g = 0.35–1.16) compared to low (g = 0.12–0.38) overall number of training sessions.
Conclusion: Multimodal IPP beneficially affect neuromuscular performance. These improvements may substantiate the preventative efficacy of IPP and may support the wide-spread implementation and dissemination of IPP. The study has been a priori registered in PROSPERO (CRD42016053407).
Physical inactivity is a major public health burden and an independent risk factor for non-communicable diseases as well as increased mortality (Blair, 2009; Kohl et al., 2012). Already in childhood, inappropriate physical activity levels can cause considerable health problems on individual as well as society level (Janssen and Leblanc, 2010; Andersen et al., 2011). Consequently, the World Health Organization recommends at least 60 min of moderate to vigorous physical activity on top of everyday physical activity to counter harmful cardiovascular, neuromuscular, and metabolic developments (WHO, 2010). As a side-effect, however, sport and high levels of physical activity are associated with a high prevalence of injuries (Caine et al., 2006; Emery, 2010). For instance, there is evidence from several countries (Switzerland, United States, Canada, France, the Netherlands, the United Kingdom and Sweden) that (organized and non-organized) sports is the main cause of injury in children and adolescents with more than 50% of all injuries caused by sports (Bijur et al., 1995; Mummery et al., 1998; Belechri et al., 2001; Michaud et al., 2001; Hedstrom et al., 2012).
In order to reduce injury incidence while being physically active and, therefore, supporting the beneficial health effects of sport and physical activity, injury prevention programs (IPP) have been developed and evaluated. There is convincing evidence that exercise-based prevention programs can reduce the overall injury rate by about 40% in child and adolescent sport (Rössler et al., 2014). IPP are usually designed as multimodal exercise interventions targeting potential deficits in neuromuscular abilities, such as, leg muscle strength and power or postural stability. Neuromuscular performance can be regarded as the ability of the neuromuscular system to functionally control and drive movements by an appropriate use and coordination of muscular strength and endurance, muscle recruitment pattern, proprioceptive feedback, and reflex activity (Huston and Wojtys, 1996; Zech et al., 2010). Neuromuscular deficits may potentially increase the risk of injury, although evidence in this regard is not conclusive to date (Bahr and Holme, 2003; Emery, 2003; Meeuwisse et al., 2007; Lehr et al., 2017). Successful prevention programs usually include exercises targeting static and dynamic balance, plyometrics, as well as lower limb strength and power (Mandelbaum et al., 2005; Abernethy and Bleakley, 2007; Soligard et al., 2008; Kiani et al., 2010). There are studies, which analyzed the extent to which potential neuromuscular risk factors for injuries were affected by such programs. Some of these studies, however, suffer from low sample sizes, the results appear heterogeneous and definitive conclusions on the expectable effect sizes of potential adaptations are not possible. A systematic analyses of the existing scientific literature in this regard is missing to date.
From a practical perspective, improvements of neuromuscular performance are relevant regarding sport-specific performance (Lesinski et al., 2015; Granacher et al., 2016). In consequence, performance improvements can be a relevant argument—next to the injury prevention perspective—to convince coaches and athletes to implement IPP in their training routine. As deficits in neuromuscular control are considered relevant risk factors for injuries to the lower limbs (Bahr and Holme, 2003; Meeuwisse et al., 2007; Alentorn-Geli et al., 2009; Frisch et al., 2009; Myer et al., 2011), data on adaptations in neuromuscular performance may additionally provide insights in mechanisms underlying the preventative efficacy of multimodal IPP. It can be further argued that the adaptive potential of athletes is limited at higher performance levels, as physical capacities are already well-developed. Therefore, it is of particular interest whether adaptations to prevention programs designed for a large mass of mainly recreational-level children and adolescents can be transferred to high-level youth athletes. Moreover, there is consistent evidence that age and sex affect injury risk (Emery, 2003; Frisch et al., 2009; Faude et al., 2013) and that training dose determines the size of training adaptations to neuromuscular training programs (Lesinski et al., 2015, 2016).
The aims of our systematic review with meta-analysis were: (a) To summarize the scientific literature on neuromuscular performance adaptations resulting from multimodal IPP in organized child and adolescent sport, (b) to quantify the effect sizes of adaptations in neuromuscular performance measures, and (c) to perform sub-group analyses in order to evaluate potential influences of performance level, sex, age-group, training volume, and potential differences between specific IPP. We hypothesized that multimodal, neuromuscular IPP can improve several neuromuscular performance parameters and that these effects were greater in low-level as compared to high-level youth athletes.
We conducted and reported this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009). The study has been a priori registered in PROSPERO (CRD42016053407). In contrast to the primary registration, we extended the age-range of includable youth athletes from 18 to a maximum of 20 years of age, as we noticed during the literature search process that a part of the athletes, particularly at the highest performance level, competing in the oldest youth age groups are older than 18 years and we aimed to present a comprehensive overview in organized youth sports.
A systematic literature search was conducted independently by two researchers (OF, LD) until May 8th, 2017 in the following electronic databases: PubMed, Web of Science, EMBASE, CINAHL, and SCOPUS. The search strategy was adopted using the PICOS (population, intervention, comparison, outcome, study design) approach. Selected search terms related to youth sports, injury prevention, and neuromuscular performance were combined in Boolean logic. No restriction with regard to publication date was applied. Only full-text articles written in English language were considered. Our search strategy is described in detail in Appendix 1 (Supplementary Material). In addition, we screened the reference lists of the selected articles as well as the authors' own bibliographies. Finally, we searched Google Scholar with the same search terms in order to control for potentially overseen relevant articles.
Eligibility Criteria and Study Selection
Inclusion criteria (according to the PICOS approach) were: (i) healthy participants, up to 20 years of age, who were involved in organized sport (club, high school, college, sports associations) (P), (ii) the study included an intervention arm performing a multimodal neuromuscular training program focusing on injury prevention (I) as well as a control arm following a common training or a sham treatment without a specific neuromuscular focus (C), (iii) at least one neuromuscular performance parameter (static or dynamic balance/stability, power, strength, sprint ability) was assessed (O), and (i) the study was designed as a (cluster-) RCT (S).
Exclusion criteria were: (i) any intervention outside organized sport (for instance, unorganized recreational or leisure time sport, physical education classes), (ii) any neuromuscular intervention without a specific injury prevention focus (e.g., general strength and conditioning programs focusing on performance enhancement), (iii) the control group performing another structured neuromuscular training program outside common training routine, and (iv) athletes suffering from systemic neurological or neuromuscular disease or disability or being injured or undergoing rehabilitation after injury. Studies were independently selected by two investigators (OF, LD). A final decision on eligibility was achieved by consensus.
Data Extraction and Outcome Parameters
Data extraction was independently performed by two investigators (OF, RoR). In case of discrepancies a third researcher (LD) was consulted. Relevant study information regarding author, year, number of participants, intervention (weeks, frequency, duration per session) and passive control condition were extracted and transferred to an excel spread sheet.
Main outcome parameters for this analysis were measures of neuromuscular performance, which have been associated with injury risk and/or which have been assumed to be associated with sport-specific performance. We analyzed parameters indicating balance/stability (sub-categories static and dynamic balance as well as dynamic stability, i.e., the ability to stabilize the center-of-pressure after a dynamic movement, e.g., a jump), leg power (basic and reactive vertical as well as horizontal jump performance), and strength (isokinetic hamstrings and quadriceps strength as well as strength ratios) as these measures might be related to injury risk (Lehr et al., 2017). In addition, we analyzed parameters related to sprint ability (basic straight sprint performance, acceleration, and change-in-direction speed) and sport-specific skills (here: soccer-specific skills like slalom dribbling and the wall-volley test). All analyzed parameters, the particular sub-categories as well as the specific tests, which were integrated in each sub-category, are presented in Appendix 2 (Supplementary Material). We purposely did not analyze potential anatomical or biomechanical risk factors for injuries, which are not directly related to sports performance (e.g., lower limb joint angles or moments or ground reaction forces). If more than one potential parameter for a particular sub-category was reported in a single study, we chose the following procedure for statistical analyses: (i) in case of two parameters (e.g., single leg stance with opened and closed eyes as an indicator of static balance performance) we used the one showing the smaller effect in order to arrive at a conservative estimate; (ii) in case of three parameters (e.g., if only the three reaching directions, but no average or composite score was reported in the Y-balance test) we used the reaching direction showing the medium effect for further analysis.
Means and standard deviations (SD) of pre- and post-tests were available in most studies and par for par extracted. In two studies (Vescovi and VanHeest, 2010; Reis et al., 2013), data were only available as graphs. In these two studies, means and SD were independently extracted from the figures by two investigators (OF, RoR) and the average value was used for further analysis. In three studies (Steffen et al., 2008, 2013; Zech et al., 2014), only pre-test values and change scores were available. In these cases, we calculated the post-test mean by adding the change score to the pre-test mean and used the pre-test SD for statistical analysis.
Risk of Bias Assessment
The methodological quality of the included RCTs was rated using the PEDro scale. This scale comprises 11 dichotomous items (either yes or no). Studies were rated by two reviewers independently (LD and OF). After completing the evaluation, both examiners came to a consensus on every item. The raters were not blinded to study authors, place of publication, and results.
In order to examine a potential publication bias, we performed a risk-of-bias related sensitivity analysis between “weak” (score 5 and 6 on PEDro scale) and “strong” (score 7 on PEDro scale) studies for all main outcome parameters. Furthermore, we conducted a qualitative funnel plot evaluation.
We conducted a quantitative synthesis of the included studies. We calculated the sample size adjusted standardized mean differences [Hedges' g with 95% confidence intervals (CI)] from pre- to post-test for each variable and for each study arm. The difference of the target outcome between the intervention and the respective control group including the pooled standard deviations was computed. Negative effects in favor of the control arm were symbolized with a minus sign. As we analyzed studies, which were basically different in many ways (e.g., regarding participants, interventions, researchers, etc.), data were analyzed using an inverse-variance model with random effects (Borenstein et al., 2010; Deeks and Higgins, 2010). We used the Cochrane Review Manager Software (RevMan 5.3, Cochrane Collaboration, Oxford, UK) for statistical analyses. Forest plots with 95% CI were created. The magnitude of g was classified according to the following scale: 0–0.19 = negligible effect, 0.20–0.49 = small effect, 0.50–0.79 = moderate effect and ≥0.80 = large effect (Cohen, 1988).
Subgroup and Exploratory Analyses
We conducted a subgroup analysis with regard to different performance levels of participants. Thereby, performance level was classified as high, if it was explicitly stated that players competed on the highest youth level in the corresponding age group or on national level and that athletes trained three or more times per week with additional competitions at the weekend. Performance level was classified as low, if most players competed on a sub-elite level or that they trained maximally two times per week. Further exploratory analyses referred to potential differences related to sex, the effects of a particular IPP, age-group, and the number of overall training sessions. Age-group was analyzed by comparing studies, which investigated participants being 14 years or younger, with studies investigating participants being older than 14 years of age. This cut-off was chosen based on the available studies in order to arrive at two distinct age-groups. Studies with overlapping age ranges were not considered in this analyses. For the analysis regarding the overall training sessions, we divided the study sample according to the median split technique. All meta-analytical subgroup and exploratory analyses were only conducted, if at least two data points were available (Valentine et al., 2010). In all exploratory analyses, we carefully rated effects being different as indicated by a qualitative analysis of the change magnitude (i.e., effects being small, moderate, or large).
In total, 12,113 potentially relevant articles were initially found (Figure 1). After removing duplicates, 9,976 article titles and abstracts were carefully screened for relevance. The full-texts of the remaining 100 potentially relevant articles were thoroughly studied and 82 papers were excluded as not meeting the inclusion or fulfilling the exclusion criteria.
In total, data from 14 different RCTs (comprising 704 participants) published in 18 different scientific articles were finally included in the quantitative meta-analysis (Table 1). All studies were published in 2008 or later. Seventy-one percent of all comparisons analyzed soccer players with basketball, field hockey, futsal, Gaelic football, and hurling being the remaining sports. The average age of the study populations ranged from 10 years up to 19 years and the level of play from low amateur to professional. Intervention duration ranged from 4 weeks to 4.5 months with a total of 12 to maximal 57 training sessions.
Four studies were classified as high-level (Steffen et al., 2008; Daneshjoo et al., 2012a,b, 2013; Heleno et al., 2016; Ayala et al., 2017), eight as low-level (Kilding et al., 2008; DiStefano et al., 2010; Vescovi and VanHeest, 2010; Lindblom et al., 2012; Reis et al., 2013; Steffen et al., 2013; Zech et al., 2014; Rössler et al., 2016) and in two studies (Lim et al., 2009; O'Malley et al., 2017) the performance level could not be definitely estimated and, thus, these studies were not included in this particular sub-analysis.
Eight studies reported data on boys only (Kilding et al., 2008; Daneshjoo et al., 2012a,b, 2013; Reis et al., 2013; Zech et al., 2014; Heleno et al., 2016; Rössler et al., 2016; Ayala et al., 2017; O'Malley et al., 2017) and five studies on girls only (Steffen et al., 2008, 2013; Lim et al., 2009; Vescovi and VanHeest, 2010; Lindblom et al., 2012).
The following IPP were analyzed in at least two studies (Valentine et al., 2010): “11+” (Daneshjoo et al., 2012a,b, 2013; Reis et al., 2013; Steffen et al., 2013; Ayala et al., 2017), “The 11” (Kilding et al., 2008; Steffen et al., 2008), “HarmoKnee” (Daneshjoo et al., 2012a,b, 2013; Ayala et al., 2017) or the “Prevent Injury Enhance Performance” program (PEP or a modified version; Lim et al., 2009; DiStefano et al., 2010; Vescovi and VanHeest, 2010). Further programs used either a combination of exercises extracted from these established programs (Zech et al., 2014; O'Malley et al., 2017), adapted versions for younger children (DiStefano et al., 2010; Rössler et al., 2014) or applied sensorimotor training combined with plyometrics (Heleno et al., 2016).
Three studies analyzed participants being 14 years or younger (Kilding et al., 2008; DiStefano et al., 2010; Rössler et al., 2016) and in six studies athletes were 15 years or older (Steffen et al., 2008; Lim et al., 2009; Daneshjoo et al., 2012a,b, 2013; Reis et al., 2013; Ayala et al., 2017; O'Malley et al., 2017).
Seven studies had a large (>23 sessions; Kilding et al., 2008; Steffen et al., 2008, 2013; DiStefano et al., 2010; Vescovi and VanHeest, 2010; Daneshjoo et al., 2012a,b, 2013; Reis et al., 2013) and the remaining studies a low (<23 sessions; Lindblom et al., 2012; Zech et al., 2014; Heleno et al., 2016; Rössler et al., 2016; Ayala et al., 2017; O'Malley et al., 2017) number of training sessions during the study period.
Risk of Bias
The funnel plot evaluation (Appendix 3 in Supplementary Material) showed no obvious risk of bias in balance/stability, leg power, and strength measures. In sprint abilities and sport-specific tests a slight overrepresentation of small studies with large effects seems apparent.
The results of the study quality assessment are presented in Appendix 4 (Supplementary Material). Seven studies obtained the PEDro score 7 (Steffen et al., 2008; DiStefano et al., 2010; Zech et al., 2014; Heleno et al., 2016; Rössler et al., 2016; Ayala et al., 2017; O'Malley et al., 2017), six studies a score of 6 (Kilding et al., 2008; Lim et al., 2009; Vescovi and VanHeest, 2010; Daneshjoo et al., 2012a,b, 2013; Lindblom et al., 2012; Reis et al., 2013) and one study was rated as PEDro 5 (Steffen et al., 2013).
When comparing studies, which were rated as PEDro 7, with all other studies, we observed no relevant differences between “strong” and “weak” studies in balance/stability, leg power, and sprint abilities, whereas effects were larger in the “weak” studies for isokinetic strength and sport-specific tests (Appendix 5 in Supplementary Material). However, the latter analyses were based on merely two “strong” studies (Steffen et al., 2008; Rössler et al., 2016).
Main Analysis—Intervention Effects
We observed a small overall effect in favor of IPP for balance/stability outcomes including static and dynamic balance measures [g = 0.37 (95%CI 0.17, 0.58); Figure 2]. For dynamic stability measures we found a moderate effect (g = 0.72), but CI overlap zero. Similarly, a small overall effect was present for leg power outcomes [g = 0.22 (95%CI 0.07, 0.38); Figure 3], particularly for basic (g = 0.31) and reactive power (g = 0.29) parameters, but not for horizontal power (g = 0.04). Further, we found small to moderate effects for isokinetic hamstrings (g = 0.56) and quadriceps strength (g = 0.49) as well as for hamstrings-to-quadriceps ratio (g = 0.40) at low movement speed (60° per second), but not at fast movement speed (240° per second; g = 0.13 to 0.31; Figure 4).
Figure 2. Standardized mean effects of injury prevention programs (IPP) on balance and stability parameters as compared to a control (CON) group. Data are separately presented for static and dynamic balance as well as dynamic stability measures. SE, standard error; IV, inverse variance model; CI, confidence interval; SMT, sensorimotor training; NMT, neuromuscular training.
Figure 3. Standardized mean effects of injury prevention programs (IPP) on leg power parameters as compared to a control (CON) group. Data are separately presented for basic, reactive and horizontal power measures. SE, standard error; IV, inverse variance model; CI, confidence interval; PEP, Prevent Injury Enhance Performance.
Figure 4. Standardized mean effects of injury prevention programs (IPP) on leg isokinetic strength as compared to a control (CON) group. Data are separately presented for hamstring (H) and quadriceps (Q) strength as well as H/Q ratios at movement velocities of 60 and 240°/s. SE, standard error; IV, inverse variance model; CI, confidence interval.
We observed a large overall effect for sprint abilities [g = 0.80 (95%CI 0.50, 1.09); Figure 5], which was particularly present in acceleration (g = 0.92) and change-in-direction speed (g = 0.88). Basic speed abilities were moderately improved (g = 0.66). With regard to sport-specific skills (here: soccer-specific) we found a moderate effect for slalom dribbling (g = 0.54) and a large effect for the wall-volley test (g = 1.46) with CI including the zero (Figure 6).
Figure 5. Standardized mean effects of injury prevention programs (IPP) on sprint abilities as compared to a control (CON) group. Data are separately presented for basic speed, acceleration and change-in-direction speed. SE, standard error; IV, inverse variance model; CI, confidence interval; PEP, Prevent Injury Enhance Performance.
Figure 6. Standardized mean effects of injury prevention programs (IPP) on sport-specific skills as compared to a control (CON) group. Data are separately presented for slalom dribbling and the wall-volley test. SE, standard error; IV, inverse variance model; CI, confidence interval.
Subgroup and Exploratory Analyses
Table 2 displays the effects for the different levels of play. We found consistently larger effects in the high-level group (g = 0.34–1.18) compared to the low level group (g = 0.22–0.75), with large effects in sprint abilities and sport-specific tests and moderate effects in balance/stability measures. The low-level group showed small to moderate effects in all categories. However, CI largely overlapped.
Table 2. Training adaptations (pooled standardized mean differences with 95% confidence intervals; qualitative assessment of effect magnitude) for high- vs. low-level players.
Boys showed small to moderate effects for balance/stability, leg strength and power, whereas girls showed negligible to small effects (Table 3). Particularly for sprint abilities, we found large effects in boys. With regard to the different IPP, we found small to moderate effects for “11+” and “HarmoKnee” in balance/stability as well as leg power and strength parameters and large effects for sprint abilities and sport-specific skills for these IPP (Table 4). PEP showed moderate effects for leg power and sprint abilities. The effects for “The 11” were negligible to small. While we observed similar moderate effects in balance/stability measures in both the younger and older age-group, the older athletes showed a large effect in sprint abilities (Table 5). The studies with <23 training sessions showed in all categories negligible to small effects, whereas those studies with more than 23 sessions showed a small effect in leg power, a moderate effect in balance/stability and large effects in sprint abilities and sport-specific skills (Table 6).
Table 3. Training adaptations (pooled standardized mean differences with 95% confidence intervals; qualitative assessment of effect magnitude) between sexes.
Table 4. Training adaptations (pooled standardized mean differences with 95% confidence intervals; qualitative assessment of effect magnitude) for the different injury prevention programs.
Table 5. Training adaptations (pooled standardized mean differences with 95% confidence intervals; qualitative assessment of effect magnitude) between young and old players.
Table 6. Training adaptations (pooled standardized mean differences with 95% confidence intervals; qualitative assessment of effect magnitude) relative to the total number of training sessions.
The main aim of present meta-analysis was to summarize the scientific literature on neuromuscular performance adaptations resulting from multimodal IPP in organized child and adolescent sport and to quantify the effect sizes of adaptations in various neuromuscular performance measures. Furthermore, we performed subgroup analyses regarding potential differences in adaptations between performance levels, sex, age-groups, specific IPP, and number of training sessions.
With regard to our main study question we found that multimodal IPP improved several neuromuscular performance measures. We observed small effects for balance/stability measures as well as leg power and a medium effect for isokinetic leg strength at low movement velocities. For sprint abilities and sport-specific tests we found large effects.
Regarding subgroup and exploratory analyses, we obtained slightly larger effects in athletes of a higher performance level. There were differences in adaptations between different IPP, greater adaptations in boys, older players and in studies with higher number of training sessions during the study period.
Overall Interpretation and Generalizability
Neuromuscular deficits, for instance regarding balance, stability, leg power, and leg strength, are considered potential intrinsic risk factors for injuries (Meeuwisse, 1994; Bahr and Holme, 2003; Myer et al., 2011; Lehr et al., 2017). These risk factors are modifiable by appropriate neuromuscular training regimens (Alentorn-Geli et al., 2009; Myer et al., 2011; Bizzini and Dvorak, 2015). IPP, which have been shown to reduce injuries, target these risk factors within a multimodal training approach (Mandelbaum et al., 2005; Soligard et al., 2008; Kiani et al., 2010; Walden et al., 2012). Our analysis showed that parameters in relevant neuromuscular domains, such as, balance, postural stability, or leg strength and power, notably benefit from IPP. Effects were, however, small to moderate. It can be speculated, that small effects may be sufficient to relevantly reduce the risk for injury, particularly, as small effects in different domains (e.g., power, strength, balance) may act synergistically (Myer et al., 2005). For instance, an improvement of neuromuscular joint control, e.g., resulting from slightly improved balance and an increased strength of thigh musculature, may reduce forces and moments on muscles and ligaments in situations with high biomechanical loads (e.g., cutting maneuvers) sufficiently in order to avoid traumatic events. Interestingly, time-to-stabilization after single leg landing as an indicator of the ability to stabilize the body in a dynamic situation showed a large effect. As time-to-stabilization was assessed in only two studies (DiStefano et al., 2010; Zech et al., 2014), the confidence interval slightly overlapped the zero and the reliable assessment is methodologically and technically challenging, this result should be cautiously interpreted. However, it might be regarded as an interesting parameter for future injury prevention studies as exercises including one leg standing and landing situations are a main part of IPP. Similarly, we also found large improvements in agility tests. Such tests aim at rapid de- and accelerations while changing movement direction, i.e., the ability to stabilize the joints under large biomechanical loads. Such an ability may allow for an efficient transfer of forces and moments and, consequently, has high relevance for performance and likely also for the prevention of injuries.
In addition, we also observed large improvements in straight sprinting speed as well as in the ability to accelerate rapidly. Whereas, it is questionable whether straight sprinting is relevant from an injury prevention perspective, it is generally considered one of the most important physical abilities in team sports from a performance perspective. For instance, Faude et al. (2012) showed that straight sprinting is the most important powerful action preceding goal situations in professional soccer. Particularly, the “11+” program revealed large effects on sprint performance. This may be due to exercises such as, the Nordic hamstrings, plyometrics, or the bounding jumps. Evidence suggests that these exercises and, particularly, combinations of it can effectively improve sprinting abilities (Rumpf et al., 2016). As many coaches in team sports accentuate the development of sprinting speed, our findings may contribute to convincing coaches to implement IPP in their training routine.
Improvements in sport-specific tests like slalom dribbling or the wall-volley test may have a similar effect on coaches' willingness to implement IPP. The underlying reasons why athletes enhance their sport-specific skills through IPP is unclear. One might speculate that improved neuromuscular control during sport-specific skills may enable athletes to better and faster process the ball and, thus, have more attentional capacity to control movements. In this regard it is interesting to note that inappropriately developed sport-specific skills are also considered a potential intrinsic risk factor (Meeuwisse, 1994; Bahr and Holme, 2003). Therefore, the observed large improvements in sport-specific skills may contribute to a decreased injury risk.
IPP are frequently designed as warm-up programs lasting about 15–20 min. Bizzini et al. (2013) showed, for instance, that the “11+” fulfills the requirements of a warm-up program in soccer players. Furthermore, there is a large body of evidence that the “11+” program can reduce injury rate considerably (Barengo et al., 2014; Bizzini and Dvorak, 2015; Thorborg et al., 2017). Taken this evidence together, appropriately designed IPP lasting only 15–20 min per session can serve as appropriate warm-up programs. Thereby, these programs are able to improve potential risk factors for injury as well as performance indicators and, simultaneously, to reduce injury rate. This might be a strong point toward a broad implementation of such programs in sports practice.
An interesting finding was that high-level players showed slightly larger effects compared to low-level players. Intuitively, one would assume that the adaptive potential in low-level players is greater and, consequently, adaptations in these players should have been larger. In high-level players a ceiling effect in training adaptability seems reasonable as the stimuli provided by IPP may not be appropriate to induce further improvements. Our results are contradictory to these assumptions. We cannot definitely conclude on the underlying reasons for this finding. It might be speculated that in the high-level teams the coaching staff is better qualified and, thus, the training stimulus was applied in a more appropriate or suitable manner. In contrast, coaches on lower levels of play are frequently not appropriately qualified and, hence, it might be more difficult for them to instruct a correct execution of exercises for the sake of an adequate training stimulus. However, the results should be carefully interpreted as CI were wide and overlapping. Therefore, future research seems necessary.
We found also larger effects in older players as compared to their younger counterparts. This result might be related to the differences in adaptations with different playing levels as the high-level players were older (14–20 years) than the low-level players (7–18 years). Therefore, we cannot definitely distinguish between both factors. Age has been consistently shown to be an important non-modifiable risk factor for injuries (Emery, 2003). Thus, the observed adaptations in the older players might be particularly relevant from an injury prevention perspective.
We obtained larger effects in boys than in girls, particularly, in isokinetic leg strength and sprint abilities. The performance effects in girls were negligible to small. A recent meta-analysis (Rössler et al., 2014) has shown that IPP are efficacious in organized youth sports in both sexes. Based on our results we cannot conclude on possible mechanisms for this finding. Faude et al. (2013) reported that joint-ligament injuries (sprains), particularly knee sprains, are more frequent in girls compared to boys in youth soccer. Thus, injury prevention studies in female youth sports often focus on knee injuries, whereas this is not the case in studies on boys (Rössler et al., 2014). One might speculate that anatomical and biomechanical risk factors (leg alignment, knee valgus moments, knee internal rotation, cutting task biomechanics, etc.) contribute more to the risk for knee injuries and, consequently, are more important regarding injury prevention in females. There is evidence that such biomechanical risk factors are also modifiable by IPP (Pappas et al., 2015). Future research regarding sex-specific training adaptations is warranted.
When comparing the different IPP it is obvious that the performance effects were comparable between the “11+,” the “HarmoKnee” and the PEP program. The effects of the “The 11,” the predecessor of the “11+,” were considerably smaller. This finding is in line with meta-analytical data showing that the application of “11+” resulted in large reductions in injury incidence, whereas the preventative efficacy of “The 11” could not be substantiated (Al Attar et al., 2016; Thorborg et al., 2017).
Regarding the total number of training sessions our results clearly demonstrate that the effect is larger as with a greater total training volume. Thus, it is advisable to conduct IPP over a longer period of time in order to increase efficacy. This finding strengthens the results of Sugimoto et al. (2014) showing that there is a relationship between dosage and ACL injury reduction in female athletes. Similarly, Lesinski et al. (2015) showed that strength training adaptations in youth athletes are larger when training duration exceeded 23 weeks emphasizing the relevance of longer training durations. There is also evidence that training compliance had a relevant effect on program efficacy (Sugimoto et al., 2012). Similarly, Steffen et al. (2013) showed that a high compliance to the “11+” program resulted in improvements in functional balance and a reduced injury risk. Currently, it is recommended to perform the program at least 1.5 times per week in order to optimize training effectiveness (Barengo et al., 2014). These recommendations were exceeded by all studies included in our analyses. Information on the compliance in the included studies was limited. Thus, we cannot conclude on the influence of compliance on our results. It has to be mentioned that the evidence on the appropriate training frequency is currently limited.
Within the present meta-analysis, we focused on (cluster-) RCTs published in peer-reviewed scientific journals. All studies were of sufficient to high methodological quality (PEDro score ≥ 5). Thus, the present results are based on high-quality research. We did not include non-randomized studies or gray literature, thus, accepting the potential for a publication bias. However, a risk of bias analysis did only show a slight risk for the sprint and sport-specific test categories. We analyzed studies, which used evidence-based IPP or blends of such programs. Studies applying other neuromuscular or strength and conditioning programs focusing on performance improvements and not on injury prevention were not analyzed, although such programs also have the potential to reduce injury risk in youth sport (Myer et al., 2011; Granacher et al., 2016). Most studies in our analysis were conducted in soccer. Evidence in other sports and the transferability of our results is, therefore, currently limited. Finally, it has to be mentioned that the statistical power of our analyses was low for some subgroup and exploratory analyses. As CI were large and overlapping, these results should be carefully interpreted. Also, the separation of age-groups in being older and younger than 14 years of age was due to practicality. It would have been better to apply a measure of maturational status, but such information was not available.
Conclusions, Practical Implications and Perspectivess
Multimodal IPP, which have been shown to reduce the risk of injury, improve several neuromuscular performance parameters, which have been associated with injury risk. Although effects were partly small, this may give an explanation for the preventative efficacy of these programs. The general improvements in neuromuscular performance may also support the wide-spread implementation and dissemination of IPP as performance improvements are a strong argument for coaches to use particular training programs. The implementation of IPP as a routine 15–20-min warm-up can effectively prepare for the following training session, can improve performance and reduce the risk of injury. The optimal IPP may fulfill the following requirements: (i) It should be effective with regard to injury prevention and performance enhancement; (ii) it should be efficient with regard to time and resources needed to apply the program; (iii) it should be feasible and practical; (iv) it should be specific, i.e., targeted with regard to the specific sport, age, sex, and performance level.
Future research is needed to analyze which parts of multimodal programs are most effective. This may allow for optimizing existing programs or making them more efficacious and time efficient. Further, it may allow for adapting programs to other sports and settings. Furthermore, it seems warranted to adapt programs particularly for girls in order to improve performance parameters also in female youth athletes. Studies in younger children are underrepresented. Injury incidence in the youngest children is lower as compared to older athletes and injury characteristics partly differ (Faude et al., 2013). Only two studies used a program, which was specifically adapted for the youngest athletes. This is of particular interest as Myer et al. (2013) suggested to introduce prevention programs before the onset of neuromuscular deficits. Early preventive measures might be important as children make up a large part of the active population, as it may be of particular importance to prevent or at least delay the first injury and as it may support the sensitization of young athletes with injuries and appropriate preventive means.
OF had the idea, designed and conducted this meta-analysis. RoR, EP, RaR, LZ, and LD assisted in the design of this study. OF, LD, and RoR developed the search strategy. LD and OF conducted the literature search and the study quality assessment. RoR and OF extracted study information and outcome data. OF performed the statistical analyses and wrote the first paper draft. All authors revised the manuscript for important intellectual content and approved the final version of the article.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphys.2017.00791/full#supplementary-material
Al Attar, W. S., Soomro, N., Pappas, E., Sinclair, P. J., and Sanders, R. H. (2016). How effective are F-MARC injury prevention programs for soccer players? a systematic review and meta-analysis. Sports Med. 46, 205–217. doi: 10.1007/s40279-015-0404-x
Alentorn-Geli, E., Myer, G. D., Silvers, H. J., Samitier, G., Romero, D., Cugat, R., et al. (2009). Prevention of non-contact anterior cruciate ligament injuries in soccer players. Part 1: mechanisms of injury and underlying risk factors. Knee Surg. Sports Traumatol. Arthrosc. 17, 705–729. doi: 10.1007/s00167-009-0813-1
Ayala, F., Pomares-Noguera, C., Robles-Palazon, F. J., Del Pilar Garcia-Vaquero, M., Ruiz-Perez, I., Hernandez-Sanchez, S., et al. (2017). Training effects of the FIFA 11+ and harmoknee on several neuromuscular parameters of physical performance measures. Int. J. Sports Med. 38, 278–289. doi: 10.1055/s-0042-121260
Barengo, N. C., Meneses-Echavez, J. F., Ramirez-Velez, R., Cohen, D. D., Tovar, G., and Bautista, J. E. (2014). The impact of the FIFA 11+ training program on injury prevention in football players: a systematic review. Int. J. Environ. Res. Public Health 11, 11986–12000. doi: 10.3390/ijerph111111986
Belechri, M., Petridou, E., Kedikoglou, S., Trichopoulos, D., and Sports Injuries European Union, G. (2001). Sports injuries among children in six European union countries. Eur. J. Epidemiol. 17, 1005–1012. doi: 10.1023/A:1020078522493
Bijur, P. E., Trumble, A., Harel, Y., Overpeck, M. D., Jones, D., and Scheidt, P. C. (1995). Sports and recreation injuries in US children and adolescents. Arch. Pediatr. Adolesc. Med. 149, 1009–1016. doi: 10.1001/archpedi.1995.02170220075010
Bizzini, M., and Dvorak, J. (2015). FIFA 11+: an effective programme to prevent football injuries in various player groups worldwide-a narrative review. Br. J. Sports Med. 49, 577–579. doi: 10.1136/bjsports-2015-094765
Bizzini, M., Impellizzeri, F. M., Dvorak, J., Bortolan, L., Schena, F., Junge, A., et al. (2013). Physiological and performance responses to the “FIFA 11+” (part 1): is it an appropriate warm-up? J. Sports Sci. 31, 1481–1490. doi: 10.1080/02640414.2013.802922
Borenstein, M., Hedges, L. V., Higgins, J. P., and Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Res. Synth. Methods 1, 97–111. doi: 10.1002/jrsm.12
Daneshjoo, A., Mokhtar, A. H., Rahnama, N., and Yusof, A. (2012a). The effects of comprehensive warm-up programs on proprioception, static and dynamic balance on male soccer players. PLoS ONE 7:e51568. doi: 10.1371/journal.pone.0051568
Daneshjoo, A., Mokhtar, A. H., Rahnama, N., and Yusof, A. (2012b). The effects of injury preventive warm-up programs on knee strength ratio in young male professional soccer players. PLoS ONE 7:e50979. doi: 10.1371/journal.pone.0050979
Daneshjoo, A., Mokhtar, A. H., Rahnama, N., and Yusof, A. (2013). Effects of the 11+and harmoknee warm-up programs on physical performance measures in professional soccer players. J. Sports Sci. Med. 12, 489–496.
Deeks, J. J., and Higgins, J. P. (2010). Statistical Algorithms in Review Manager 5. Statistical Methods Group of The Cochrane Collab. 1–11. Available online at: http://ims.cochrane.org/revman/documentation/Statistical-methods-in-RevMan-5.pdf
DiStefano, L. J., Padua, D. A., Blackburn, J. T., Garrett, W. E., Guskiewicz, K. M., and Marshall, S. W. (2010). Integrated injury prevention program improves balance and vertical jump height in children. J. Strength Cond. Res. 24, 332–342. doi: 10.1519/JSC.0b013e3181cc2225
Granacher, U., Lesinski, M., Busch, D., Muehlbauer, T., Prieske, O., Puta, C., et al. (2016). Effects of resistance training in youth athletes on muscular fitness and athletic performance: a conceptual model for long-term athlete development. Front. Physiol. 7:164. doi: 10.3389/fphys.2016.00164
Hedstrom, E. M., Bergstrom, U., and Michno, P. (2012). Injuries in children and adolescents–analysis of 41,330 injury related visits to an emergency department in northern Sweden. Injury 43, 1403–1408. doi: 10.1016/j.injury.2011.01.027
Heleno, L. R., da Silva, R. A., Shigaki, L., Araujo, C. G., Coelho Candido, C. R., Okazaki, V. H., et al. (2016). Five-week sensory motor training program improves functional performance and postural control in young male soccer players - a blind randomized clinical trial. Inj. Prev. 22, 74–80. doi: 10.1016/j.ptsp.2016.05.004
Janssen, I., and Leblanc, A. G. (2010). Systematic review of the health benefits of physical activity and fitness in school-aged children and youth. Int. J. Behav. Nutr. Phys. Act. 7:40. doi: 10.1186/1479-5868-7-40
Kiani, A., Hellquist, E., Ahlqvist, K., Gedeborg, R., Michaelsson, K., and Byberg, L. (2010). Prevention of soccer-related knee injuries in teenaged girls. Arch. Intern. Med. 170, 43–49. doi: 10.1001/archinternmed.2009.289
Kohl, H. W. III, Craig, C. L., Lambert, E. V., Inoue, S., Alkandari, J. R., Leetongin, G., et al. (2012). The pandemic of physical inactivity: global action for public health. Lancet 380, 294–305. doi: 10.1016/S0140-6736(12)60898-8
Lehr, M. E., Kime, D., Onks, C., Silvis, M., and Streisel, M. (2017). Development of a preliminary evidence-based neuromusculoskeletal exercise guideline to reduce injury risk in the lower limb. Phys. Ther. Sport 25, 76–83. doi: 10.1016/j.ptsp.2016.08.012
Lesinski, M., Hortobagyi, T., Muehlbauer, T., Gollhofer, A., and Granacher, U. (2015). Dose-response relationships of balance training in healthy young adults: a systematic review and meta-analysis. Sports Med. 45, 557–576. doi: 10.1007/s40279-014-0284-5
Lesinski, M., Prieske, O., and Granacher, U. (2016). Effects and dose-response relationships of resistance training on physical performance in youth athletes: a systematic review and meta-analysis. Br. J. Sports Med. 50, 781–795. doi: 10.1136/bjsports-2015-095497
Lim, B. O., Lee, Y. S., Kim, J. G., An, K. O., Yoo, J., and Kwon, Y. H. (2009). Effects of sports injury prevention training on the biomechanical risk factors of anterior cruciate ligament injury in high school female basketball players. Am. J. Sports Med. 37, 1728–1734. doi: 10.1177/0363546509334220
Lindblom, H., Walden, M., and Hagglund, M. (2012). No effect on performance tests from a neuromuscular warm-up programme in youth female football: a randomised controlled trial. Knee Surg. Sports Traumatol. Arthrosc. 20, 2116–2123. doi: 10.1007/s00167-011-1846-9
Mandelbaum, B. R., Silvers, H. J., Watanabe, D. S., Knarr, J. F., Thomas, S. D., Griffin, L. Y., et al. (2005). Effectiveness of a neuromuscular and proprioceptive training program in preventing anterior cruciate ligament injuries in female athletes: 2-year follow-up. Am. J. Sports Med. 33, 1003–1010. doi: 10.1177/0363546504272261
Meeuwisse, W. H., Tyreman, H., Hagel, B., and Emery, C. (2007). A dynamic model of etiology in sport injury: the recursive nature of risk and causation. Clin. J. Sport Med. 17, 215–219. doi: 10.1097/JSM.0b013e3180592a48
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:b2535. doi: 10.1136/bmj.b2535
Mummery, W. K., Spence, J. C., Vincenten, J. A., and Voaklander, D. C. (1998). A descriptive epidemiology of sport and recreation injuries in a population-based sample: results from the Alberta Sport and recreation injury survey (ASRIS). Can. J. Public Health 89, 53–56.
Myer, G. D., Faigenbaum, A. D., Ford, K. R., Best, T. M., Bergeron, M. F., and Hewett, T. E. (2011). When to initiate integrative neuromuscular training to reduce sports-related injuries and enhance health in youth? Curr. Sports Med. Rep. 10, 155–166. doi: 10.1249/JSR.0b013e31821b1442
Myer, G. D., Ford, K. R., Palumbo, J. P., and Hewett, T. E. (2005). Neuromuscular training improves performance and lower-extremity biomechanics in female athletes. J. Strength Cond. Res. 19, 51–60. doi: 10.1519/13643.1
Myer, G. D., Sugimoto, D., Thomas, S., and Hewett, T. E. (2013). The influence of age on the effectiveness of neuromuscular training to reduce anterior cruciate ligament injury in female athletes: a meta-analysis. Am. J. Sports Med. 41, 203–215. doi: 10.1177/0363546512460637
O'Malley, E., Murphy, J. C., McCarthy Persson, U., Gissane, C., and Blake, C. (2017). The effects of the Gaelic Athletic Association 15 training program on neuromuscular outcomes in Gaelic football and hurling players: a randomized cluster trial. J. Strength Cond. Res. 31, 2119–2130. doi: 10.1519/JSC.0000000000001564
Pappas, E., Nightingale, E. J., Simic, M., Ford, K. R., Hewett, T. E., and Myer, G. D. (2015). Do exercises used in injury prevention programmes modify cutting task biomechanics? A systematic review with meta-analysis. Br. J. Sports Med. 49, 673–680. doi: 10.1136/bjsports-2014-093796
Reis, I., Rebelo, A., Krustrup, P., and Brito, J. (2013). Performance enhancement effects of Federation Internationale de Football Association's “The 11+” injury prevention training program in youth futsal players. Clin. J. Sport Med. 23, 318–320. doi: 10.1097/JSM.0b013e318285630e
Rössler, R., Donath, L., Bizzini, M., and Faude, O. (2016). A new injury prevention programme for children's football–FIFA 11+ kids–can improve motor performance: a cluster-randomised controlled trial. J. Sports Sci. 34, 549–556. doi: 10.1080/02640414.2015.1099715
Rössler, R., Donath, L., Verhagen, E., Junge, A., Schweizer, T., and Faude, O. (2014). Exercise-based injury prevention in child and adolescent sport: a systematic review and meta-analysis. Sports Med. 44, 1733–1748. doi: 10.1007/s40279-014-0234-2
Rumpf, M. C., Lockie, R. G., Cronin, J. B., and Jalilvand, F. (2016). Effect of different sprint training methods on sprint performance over various distances: a brief review. J. Strength Cond. Res. 30, 1767–1785. doi: 10.1519/JSC.0000000000001245
Soligard, T., Myklebust, G., Steffen, K., Holme, I., Silvers, H., Bizzini, M., et al. (2008). Comprehensive warm-up programme to prevent injuries in young female footballers: cluster randomised controlled trial. BMJ 337:a2469. doi: 10.1136/bmj.a2469
Steffen, K., Bakka, H. M., Myklebust, G., and Bahr, R. (2008). Performance aspects of an injury prevention program: a ten-week intervention in adolescent female football players. Scand. J. Med. Sci. Sports 18, 596–604. doi: 10.1111/j.1600-0838.2007.00708.x
Steffen, K., Emery, C. A., Romiti, M., Kang, J., Bizzini, M., Dvorak, J., et al. (2013). High adherence to a neuromuscular injury prevention programme (FIFA 11+) improves functional balance and reduces injury risk in Canadian youth female football players: a cluster randomised trial. Br. J. Sports Med. 47, 794–802. doi: 10.1136/bjsports-2012-091886
Sugimoto, D., Myer, G. D., Bush, H. M., Klugman, M. F., Medina McKeon, J. M., and Hewett, T. E. (2012). Compliance with neuromuscular training and anterior cruciate ligament injury risk reduction in female athletes: a meta-analysis. J. Athl. Train. 47, 714–723. doi: 10.4085/1062-6050-47.6.10
Sugimoto, D., Myer, G. D., Foss, K. D., and Hewett, T. E. (2014). Dosage effects of neuromuscular training intervention to reduce anterior cruciate ligament injuries in female athletes: meta- and sub-group analyses. Sports Med. 44, 551–562. doi: 10.1007/s40279-013-0135-9
Thorborg, K., Krommes, K. K., Esteve, E., Clausen, M. B., Bartels, E. M., and Rathleff, M. S. (2017). Effect of specific exercise-based football injury prevention programmes on the overall injury rate in football: a systematic review and meta-analysis of the FIFA 11 and 11+ programmes. Br. J. Sports Med. 51, 562–571. doi: 10.1136/bjsports-2016-097066
Valentine, J. C., Pigott, T. D., and Rothstein, H. R. (2010). How many studies do you need? a primer on statistical power for meta-analysis. J. Educ. Behav. Stat. 35, 215–247. doi: 10.3102/1076998609346961
Vescovi, J. D., and VanHeest, J. L. (2010). Effects of an anterior cruciate ligament injury prevention program on performance in adolescent female soccer players. Scand. J. Med. Sci. Sports 20, 394–402. doi: 10.1111/j.1600-0838.2009.00963.x
Walden, M., Atroshi, I., Magnusson, H., Wagner, P., and Hagglund, M. (2012). Prevention of acute knee injuries in adolescent female football players: cluster randomised controlled trial. BMJ 344:e3042. doi: 10.1136/bmj.e3042
Zech, A., Hubscher, M., Vogt, L., Banzer, W., Hansel, F., and Pfeifer, K. (2010). Balance training for neuromuscular control and performance enhancement: a systematic review. J. Athl. Train. 45, 392–403. doi: 10.4085/1062-6050-45.4.392
Zech, A., Klahn, P., Hoeft, J., zu Eulenburg, C., and Steib, S. (2014). Time course and dimensions of postural control changes following neuromuscular training in youth field hockey athletes. Eur. J. Appl. Physiol. 114, 395–403. doi: 10.1007/s00421-013-2786-5
Keywords: exercise training, sensorimotor, leg strength, balance, power, efficacy, risk factor, team sport
Citation: Faude O, Rössler R, Petushek EJ, Roth R, Zahner L and Donath L (2017) Neuromuscular Adaptations to Multimodal Injury Prevention Programs in Youth Sports: A Systematic Review with Meta-Analysis of Randomized Controlled Trials. Front. Physiol. 8:791. doi: 10.3389/fphys.2017.00791
Received: 17 July 2017; Accepted: 26 September 2017;
Published: 12 October 2017.
Edited by:Urs Granacher, University of Potsdam, Germany
Reviewed by:Jon Oliver, Cardiff Metropolitan University, United Kingdom
Simon Steib, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
Copyright © 2017 Faude, Rössler, Petushek, Roth, Zahner and Donath. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Oliver Faude, firstname.lastname@example.org