This study utilized a cohort of two different NCAA Division I football programs (n = 177): Team A was a Football Bowl Subdivision (FBS) team (n = 102); Team B was a Football Championship Subdivision (FCS) program (n = 75). Each player was tested prior to the beginning of pre-season camp in 2019 and 2020, for WBRA. Testing was part of the evaluation process utilized by the strength and conditioning staff at each university. In cases where football players were tested for both seasons, a player's initial assessment was used in the analysis and corresponding injury data for that season were used.

Each player was assessed for WBRA utilizing two lateral movement tests and two diagonal-backward movement tests (TRAZER^® Sports Stimulator; Traq Global Ltd, Westlake, OH). The TRAZER^® has been found to be reliable and valid for assessing WBRA metrics (Hogg et al., 2021). The four tests used were a standard screening protocol within the unit. The order of the four tests were set by the manufacturer and were the same for each player. The player stood 2.7 m from a 48 × 86 cm television screen in the middle of an area ~1.75 × 1.75 m (Hogg et al., 2021; Figure 1). An avatar of the player was projected onto the screen and they would respond to virtual targets (Figure 2). For the lateral agility and diagonal agility tests, the athlete would respond to eight randomly appearing targets (four on each side), side-shuffling 1.8 meters to the right or left for Lateral Agility or moving backward and laterally at a 45° angle 2.7 meters for the Diagonal Agility test (Figures 1, 2). Both Lateral Agility and Diagonal Agility are defined as single-task activities. The other two tests were dual-task activities incorporating the Flanker Test, a reliable measure of executive function (Paap and Sawi, 2016). The Flanker Test is a discrimination RT type test, involving a series of five arrows appearing at the top of the screen for 500 ms, and are in one of four configurations: congruent < < < < < or > > > > >; or incongruent < < > < < or > > < > > (Eriksen and Eriksen, 1974; Paap and Sawi, 2016; Wilkerson et al., 2021b). The player was instructed to go to the target that corresponded in the direction of the middle arrow (Lateral Flanker, Diagonal Flanker). Because the athlete must first find the middle arrow, interpret the direction and then move to the corresponding target defines this as a dual-task activity. There were eight randomly appearing patterns, four repetitions each to the right and left, with two of each of the congruent patterns and two of each of the incongruent patterns. Body movement of 1.8 meters was necessary to deactivate the lateral targets, or 2.5 meters of movement to deactivate the diagonal targets (Wilkerson et al., 2018, 2020). No practice trials were permitted, but the players were allowed to watch their teammates perform the tests. The test session took each player < 6 min to complete all four tests.

Reaction time was determined by “the time elapsed from target appearance to 0.2 m of body core displacement” in the correct direction (Wilkerson et al., 2020). These times were averaged over the eight trials, four in each direction, to provide an average RT. Additionally, speed, acceleration, and deceleration () were assessed simultaneously during the same movement patterns and averages for each of these metrics were also provided. For the Flanker tests, the proportion of correct responses out of the eight trials were also produced (Wilkerson et al., 2020).

The athletic training staffs at the schools used in this study were permitted to utilize their usual and customary procedures for injury tracking and record keeping. At the conclusion of the season a record review of the injury tracking software was performed. An injury was operationally defined as any sprains or strains suffered as a result of intercollegiate athletic participation, were evaluated by a certified athletic trainer or a physician, required a change in participant level (< 100%), for at least 1 day and were recorded on a coaches' report or daily injury report. To focus our analysis we included only those injuries resulting from insufficient neuromuscular response to dynamic loads; therefore, we excluded concussions, contusions, fractures, open wounds, and overuse disorders from the study.

The record review included which players had suffered an injury, how many injuries they suffered throughout the season, and the region of the body injured. Regions included: 1. core (trunk and spine), 2. lower extremity, 3. upper extremity, 4. concussion. Core and lower extremity injuries had the greatest frequency of injuries combined, but not separately, thus we chose core and lower extremity injuries (CLEI) as our dependent variable. Additionally, a review of the athletic trainers' daily injury reports provided to the coaches was performed to ensure all injuries had been accounted for and recorded. Any discrepancies or questions regarding a player's injury were taken to the athletic trainer in charge of football for consultation.

Each of the four test modes (Lateral Agility, Lateral Flanker, Diagonal Agility, and Diagonal Flanker), generated data for four metrics (RT, Speed, Acceleration, and Deceleration) creating 16 variables. Imputation for the missing values was accomplished using the group mean for the variable. There were only six scores that needed to be imputed [6/(16 variables × 177 players) = 2,832 total variables = 0.212%]. To examine for outliers, the raw values of the 16 variables were transformed into z-scores. Because a smaller value for RT corresponds to superior performance, the RT z-scores were multiplied by −1 to represent better performance with a higher z-score value. A review of the data was performed searching for outliers, which were operationally defined as any z-score greater than three standard deviations above or below the mean. In such cases, the outlier raw scores were replaced with a value corresponding to three standard deviations above or below the variable-specific mean value. Only 25 scores [25/(16 × 177 = 2,832) = 0.883%], were determined to be outliers (Tabachnick et al., 2007).

Asymmetry (Asym) was defined as the absolute difference between test metric values for the two directions to the better of the two performance values, yielding a proportion of the difference for each of the four test's metrics (RT Asym, Speed Asym, Acceleration Asym, and Deceleration Asym), producing an additional 16 variables (Wilkerson et al., 2020). The average of the four asymmetry scores for each of the four test modes was calculated (Asym Avg). An additional metric titled “Efficiency Index” was calculated by dividing the Avg RT by the Correct Proportion of Flanker Test responses. The smaller the Efficiency Index, the better the score. The mean scores and the asymmetries for each of the four metrics for each of the four test modes produced a total of 32 variables. The four Avg Asym plus both EI scores produced an additional six variables for a total of 38 variables to analyze (Halliday et al., 2018).

Multicollinearity analysis was performed to eliminate any variables that were highly correlated with other variables. Multicollinearity is present when the variance inflation factor (VIF) is >10.0 (Mertler and Vannetta, 2005; Field, 2009). Variables with high VIF values were eliminated from further consideration.

Receiver Operator Characteristic (ROC) analyses were used to assess each of the remaining variables. Our goal was to determine the best possible set of predictors and not to screen variables for statistical significance; therefore we operationally determined an ROC curve with an Area under the Curve (AUC) of ≥0.550 or a p-value of ≤0.20 for continuous variables, to advance to multivariable analysis (Kuijpers et al., 2006; Teyhen et al., 2007; Alba et al., 2017). The remaining continuous variables were entered into a backward logistic regression analysis to determine the best set of predictors. Predicted probabilities were calculated by SPSS for each player in the analysis for both the continuous and binary versions of the predictor variables where the natural log odds values for each participant were converted to predicted probabilities. Another ROC analysis was performed to assess the model for the probability of a CLEI occurring using the model. Further analysis of the prediction model was made utilizing the Area Under the Curve. Variables which the logistic regression identified as having the strongest association with CLEI were converted to binary variables on the basis of Youden's Index from the univariable ROC analyses (Youden's Index is the difference between the sensitivity and 1-specificity). The largest difference provided the cut-point for binary classification of the predictor variables (Youden, 1950; Teyhen et al., 2020). The logistic regression was repeated for the binary form of the surviving variables with predicted probabilities also calculated through SPSS. Receiver Operator Characteristic analysis was performed on this set of predicted probabilities and a cut-point was established using Youden's Index from this analysis (Teyhen et al., 2020).

Based on the binary variables, participants were coded with a “1” if associated with a CLEI history or a “0” if not associated with a CLEI. The coded variables were then summed, and a ROC analysis was repeated to determine the optimal number of predictors. Participants' whose summed total of positive predictor variables was at or above this cut point were coded with a “1,” those with a sum below the identified cut point were coded with a “0.” Two-by-two cross-tabulation analysis was used to calculate Sensitivity, Specificity, Positive Predictive Value, Negative Predictive Value, Odds Ratio, and Relative Risk, with associated 95% confident intervals (CI) for the individual predictors and combinations of predictors.

The predicted probabilities were calculated as part of the logistic regression analysis for the continuous variables. A ROC analysis of the predicted probabilities for the continuous variables was produced and had an AUC of 0.670 (p ≤ 0.001) (Figure 4). The prognostic accuracy was determined utilizing Youden's Index (Youden, 1950). The Youden's Index cut-point provided 71.9% Sensitivity, 60.0% Specificity, and an Odd Ratio of 3.84 (95% CI: 1.94, 7.61). The Positive Predictive Value of a football player suffering a CLEI was 46.1% and Negative Predictive Value was 81.8% (Figure 4).

The logistic regression analysis was repeated with the binary form of the final three variables and the predicted probabilities were calculated. A ROC analysis of the predicted probabilities for the binary variables was produced and had an AUC of 0.709 (p > 0.001) (Figure 4). The prognostic accuracy was determined using the Youden's Index (Youden, 1950). The Youden's Index cut-point provided 63.2% Sensitivity, 76.3% Specificity, Odd Ratio of 5.38 (95% CI: 2.72, 10.63). The Positive Predictive Value of a football player suffering a CLEI was 56.2% and Negative Predictive Value was 81.1% (Figure 4; Table 6).

Table 7 provides the injury incidence by the number of risk-factors a player possessed. Players with ≥2 factors suffered the majority of the injuries with 55.4% of those players with two or more predictors becoming injured. Only 18.8% of the players with <2 factors suffered a CLEI.

Our intent was to identify the WBRA assessment metrics that help to identify those Division I football players might be classified as high-risk for CLEI. Initially, we used ROC analyses to determine the individual strength of the predictor variable using a liberal threshold for the AUC of ≥0.550 or a p-value of ≤ 0.20 for variables to advance to multivariable analysis (Kuijpers et al., 2006; Teyhen et al., 2007; Alba et al., 2017). We decided that these criteria would prevent us from excluding potentially important predictors. There were five predictors that were retained by this process and we created our model using logistic regression with the continuous representations of these variables. The logistic regression analysis retained three predictors (Lateral Agility Acceleration Asym, Lateral Flanker Deceleration Asym, and Diagonal Agility RT Avg), as the strongest set. To assess the potential for a simplification of the prediction model, univariable analyses were performed to create binary cut-points. Binary logistic regression is a commonly used method to create multivariable clinical prediction models (Hosmer and Lemeshow, 2000; Moons et al., 2015). The “events per variable” (EPV) is the ratio of events (i.e., CLEI occurrence), to the number of predictor variables, which has been recommended to be no <10:1 (Moons et al., 2014; Pavlou et al., 2016; Austin and Steyerberg, 2017). For our final model, we had a ratio of 19 injuries per predictor variable (57 injuries/3 predictor variables).

We also acknowledge a common criticism of binary prediction models in that some participants may be misclassified because they are just above or below the established cut-point. It is understood the cut-points we established may not be exactly the same cut-points for similar samples; however, we are confident that our predictor variables would be similar to a similar sample. We provided the reader with the accuracy for the predictors for our model and for the combination of predictors with 2 × 2 cross-tabulation. The accuracy of the individual predictors ranged from 57.4 to 65.9% (Table 5), and was 72.0% for players having ≥ 2 factors (Table 6). We further understand the potential for a non-linear relationship between the predictors and CLEI. Further study is warranted to account for a possible non-linear relationship, thus potentially strengthening our prediction model.

A common criticism of prediction modeling is the lack of adequate sample size. Our sample of 177 football players was found to produce a statistical power level of 99.6% for avoidance of a Type II error (Soe et al., 2005). Stratification to explore interaction effects could potentially lower the cell counts to single digits (Prieto-Marañón et al., 2012).

A key problem with many prediction models is lack of external validation. Future research should determine the generalizability of our model to a validation cohort. Large multi-site data collection may lead to model refinement that will provide strong external validity.

The ROC curve, specifically the AUC, relates to classification accuracy for this cohort. For the predicted probabilities for the continuous variables the ROC curve had an AUC of 0.670 (95% CI: 0.584, 0.755). For the predicted probabilities for the binary version of the variables, the ROC curve was even stronger with an AUC of 0.709 (95% CI: 0.625, 0.792). According to Alba et al. (2017), ROC curve AUC of 0.60–0.75 are interpreted as “possibly helpful discrimination.”

Clinically, when examining prediction models, examining the predicted probabilities might be a logical way to compare not only results across groups, but it also provides each individual with a probability of his or her likelihood of sustaining the specified outcome event.

We operationally defined RT has the time between the appearance of the target until the body's core moved 0.2 m in the correct direction (Wilkerson et al., 2020). Several studies have demonstrated the importance of reaction time to athletic success (Caccese et al., 2021; Lempke et al., 2020). Reaction time is also affected when someone suffers a concussion (Gorus et al., 2006; Caccese et al., 2021; Lempke et al., 2020; Wilkerson et al., 2020). We found Diagonal Agility RT Avg to be predictor of CLEI. Dual-task processing tests such as the Lateral Flanker, would have slowed RTs from having to process the arrow sequence and decide the direction in which to move to the proper target.

Optimal movement is the ability to perform a functional task equally well to either side as the task demands. Asymmetry between extremities or movement directions may be present following musculoskeletal injury, surgery, or a concussion, which may constrain an athlete's movement repertoire during participation in demanding activities (Hughes et al., 2020; Wilkerson et al., 2020). A deficiency in one's ability to move in either direction equally well can potentially elevate injury risk. Several studies have demonstrated asymmetry as a risk factor in injury prediction (Chalmers et al., 2017; Eagle et al., 2019; Hughes et al., 2020; Wilkerson et al., 2020; King et al., 2021). Although not part of the current study, others have speculated an imbalance in the distribution of excitatory and inhibitory moderators between the hemispheres of the brain may be a contributing factor in movement asymmetries. One's ability to adjust and accommodate to ever changing environments is often the difference between success and failure (Serrien et al., 2006; Takeuchi et al., 2012; Garrett et al., 2013; Grady and Garrett, 2018; Wilkerson et al., 2020). If not adequately addressed, asymmetries can persist long after the athlete has returned to play from an injury. However, some clinicians fail to either recognize the existence of the asymmetry or neglect to assess for the asymmetry. This can cause a clinician to believe the athlete has returned to “normal,” when in reality the asymmetry problem that was not assessed may manifest itself at a later time (Wilkerson et al., 2020, 2021a; King et al., 2021).

What has not been universally accepted is the threshold for these differences? In a study of professional soccer players, Read et al., attempted to do quantify this threshold with a variety of tests, including hopping and jumping. They concluded that they were unable to find a single asymmetry threshold, stating that it was their belief that the thresholds were sport and age specific (Read et al., 2021). Others have stated that an asymmetry of >10% to be problematic (Schiltz et al., 2009; Lambert et al., 2020). The biggest challenge is a lack of prospective studies to identify the asymmetries from a set of baseline measures. The data used for our study was gathered at baseline, prior to the start of pre-season practice, when both teams were nearing the end of their pre-season strength and conditioning programs with the common goal of being in excellent physical condition for the upcoming season. Our model identified two asymmetries as predictors of CLEI: Lateral Agility Acceleration Asym (≥14.7%), and Lateral Flanker Deceleration Asym (≥8.1%). The univariable ORs for these predictors of CLEI were 2.75 (95% CI: 1.38, 5.49), and 2.42 (95% CI: 1.22, 4.80), respectively. Although these cut-points may not provide equivalent discrimination in similar cohorts, it does not diminish the potential importance of asymmetrical movement as a factor that may elevate CLEI risk.

Each sport has its unique acceleration and deceleration demands. Acceleration, one's ability to speed up, has a greater metabolic cost. Deceleration, one's ability to slow down, has a greater mechanical burden, due to greater peak loads and loading rate (Lieberman et al., 2015; Harper et al., 2019, 2020). Deceleration forces include horizontal and vertical forces to include braking power and braking impulse (Lieberman et al., 2015; Harper et al., 2019, 2020), but it is the horizontal forces which are costlier than the vertical forces. The horizontal forces contribute more than a third of the total cost of running (Lieberman et al., 2015). Just like in an automobile where braking puts more stress and strain on the vehicle's systems than acceleration, braking of the body creates increased forces and cause more structural damage than acceleration (Harper et al., 2019). High-intensity accelerations and decelerations have been studied in soccer players. In a typical soccer match, a player may perform from 6 to 8 high-intensity acceleration and deceleration movement per minute of game time (Harper et al., 2020). In college football, defensive backs displayed higher acceleration and deceleration values than other position groups. As a unit, defensive players experienced great acceleration-deceleration values (Wellman et al., 2017). Our prediction model found one acceleration factor and one deceleration factor as indicators of elevated CLEI risk.

In a meta-analysis by Harper et al., they examined high and very high intensities for acceleration and deceleration. Eleven of the studies established 3.0 m/s⁻² as the high intensity threshold while six studies established 3.5 m/s⁻² as the very-high intensity threshold (Harper et al., 2019). We applied the Harper et al., thresholds to assess the possible relevance of their findings to our data of the Lateral Agility Acceleration Avg and the Lateral Flanker Deceleration Avg. The largest percentage of players at a particular intensity was for the high intensity threshold – Lateral Agility Acceleration Avg with 50% of the players at ≥3.0 m/s⁻². There was about one-third or fewer of the players at or above for the other three metrics, very high intensity threshold – Lateral Agility Acceleration Avg, very high intensity threshold – Lateral Flanker Acceleration Avg, and high intensity threshold – Lateral Flanker Acceleration Avg. An examination of the ORs found all four metrics to have ORs < 2.0 and all of the 95% CIs lower limits were below the 1.0 threshold. For our data set, these thresholds do not have a meaningful impact.

The ability to proficiently process a variety of neural inputs and produce a correct physical response is the essence of athletic movements (Nuzzo, 2015; Tarara et al., 2016; Araújo et al., 2019). Athletic-related movements by their nature are dual-task activities. Rarely do athletic activities occur as single-task actions. Movement happens with some degree of cognitive processing in order for the proper response to occur. Research has demonstrated those with a history of concussion or anterior cruciate ligament reconstruction have slowed dual-task performance (Marques et al., 2020; Ness et al., 2020). This slowing of dual-task neural processing may lead to differences in performance capabilities which may also lead to an elevated risk for musculoskeletal injury (Wilkerson et al., 2017, 2020). Several authors have presented evidence of a possible residual, low-level inflammatory response in the brain following concussion, which causes altered neurocognitive processing (Ezza and Khadrawyb, 2014; Churchill et al., 2017, 2019). If true, then not only would cognitive function be impaired, but so would dual-task WBRA actions. The degree to which dual-task processing is integral to functional movement makes it remarkable that clinicians do not spend more time assessing this phenomenon. Further study is warranted to examine more closely the effects of neurocognitive dysfunction and its effect on physical performance which could yield benefits for all levels of competition.

The Flanker Test adds a cognitive component to the movements as the player must first discern the middle arrow, then interpret that arrow's direction, and finally move its direction (Wilkerson et al., 2021a). As expected, the metrics for the lateral flanker test were worse than the metrics on the Lateral Agility. Surprisingly, this was not the case for the Diagonal Flanker test vs. the Diagonal Agility test as the Diagonal Flanker variables did not demonstrate substantial discriminatory power. One possible reason for this phenomenon is that the diagonal movements required for the WBRA tests mimic those that offensive lineman perform in a football game or practice, using their pass protection skills. Anecdotally, it appeared that offensive linemen and defensive backs did better on the diagonal tests than defensive linemen and wide receivers. Unfortunately, the data set did not contain specific position data of each player, only their binary classification as a lineman or a skill position player. Further study examining these specific positions groups would be beneficial to further understand the role biomechanics play in these assessments.

Although the order of the specific repetitions within each test mode was randomized, the order of the test modes was not randomized as the protocol used is pre-set by system software. The order in which the WBRA tests were administered was the same for each participant: Lateral Agility, Lateral Flanker, Diagonal Agility, and Diagonal Flanker. It cannot be ruled out the role of doing the agility test prior to the Flanker test may have had an impact on the Flanker exam outcome. The players were able to watch their teammates perform the test battery. It cannot be determined how a player watching his teammates perform the test prior to their turn may have had on their performance when they completed the test battery.

Another limitation to our study, was the effort given by the players. It would be expected with full effort by the players there would be a statistically significant differences between the Football Bowl Subdivision team and the Football Championship Subdivision team; however, this was not necessarily the case. Analysis of the means of test metrics found the players from the Football Championship Subdivision team outperformed the players from the Football Bowl Subdivision team on several of the test metrics. Finally, as a means to limit the time each player spent doing the test battery, only eight repetitions were performed for each of the WBRA test modes. Four repetitions to either side may not have been sufficient to ensure reliable measurements.

Caution should be exercised applying our results to other college football teams. Further examination utilizing players from other levels of intercollegiate football would be beneficial. Ideally, large multi-site data collection may lead to model refinement that will provide strong external validity.