Pros and Cons of 19 Sport-Related Concussion Educational Resources in Canada: Avenues for Better Care and Prevention

Objective: The goal of this research was to assess the effectiveness of available concussion educational resources in Canada, the means used to disseminate this knowledge and the impact of these educational resources on players' concussion prevention knowledge. Methods: We assessed concussion knowledge before and after exposure to one or more of 19 resources introduced through a national program aimed to increase awareness and knowledge of concussion. The effectiveness of the mode of delivery was measured by changes in concussion knowledge scores (CKS) between pre and pro scores. Measures: Concussion knowledge scores (CKS) were calculated for pre- and post- exposure to concussion educational resources and used as a measure of both, the effectiveness of each resource as well as the effectiveness of the delivery method. The effectiveness of each educational resource was also measured by the respondents' rating of each concussion educational resource. Results: Respondents in post-survey had higher CKS than those in pre-survey. Two out of the 19 newly developed concussion educational resources were effective in improving the resource users' CKS. Linear regression showed that using more resources further increased CKS. Four out of six modes of delivery enhanced respondents' concussion knowledge. Conclusion: Our findings demonstrate that the newly developed Canadian concussion educational resources were effective at improving users' concussion knowledge. Our data demonstrates that using three or more resources further enhanced the users' concussion knowledge. Future research, however, is critical to assess whether concussion prevention knowledge is sufficient to reduce injuries and factors influencing it.


INTRODUCTION
Traumatic Brain Injury (TBI) is a significant public health concern (1) and a "silent epidemic" (2) that affects approximately 1.7 million people in the United States each year from which about 275,000 are hospitalized and 52,000 die (3). In 2010 alone, it was estimated that the cost of TBIs in the United States was about $76.5 billion and $10 billion in Canada (4)(5)(6). Despite the tremendous healthcare and economic burden of TBI, a lack of knowledge in recognizing TBI or concussion is quite common and worrisome. For example, in one study, it was observed that more than half of the study's participants underestimated the prevalence of concussion and more than 30% were unaware of any return-to-play protocols (7). Other studies have identified several gaps in concussion awareness and knowledge (8,9) including the lack of awareness of the effects of concussion on psychosocial functioning post injury, and the lack of care management post-concussion among athletes (10). However, researchers have also proposed that through educational interventions or knowledge translation activities, it may be possible to improve concussion knowledge (11)(12)(13). In a recent systematic review focusing on the effectiveness of interventions in preventing injuries in hockey (14) it was observed that rule changes could lead to a reduction in injury rate. However, only 11 studies provided conclusive evidence supporting the effectiveness of increasing concussion knowledge or awareness by educational intervention. Delivery of this knowledge was achieved through mobile app, or pamphlet (15), website (16), information sheet or seminar/presentation (17)(18)(19), physical education class (20), poster or other formats (21)(22)(23)(24)(25). In spite of some attempts to increase players' education, a recent study showed that over the last 20 years, old misconceptions about concussion continue to persist among school educational professionals surveyed between 1997 and 2017 (26). Although a multimodal educational intervention has been shown to positively influence primary care providers to adhere to the consensus-based guidelines in their concussion management practice (27), even after the legislation that mandated concussion education for student athletes, athletes fail to recognize concussion symptoms (28). Little is known whether or not the format of educational intervention (e.g., video vs. text) can play a role and whether or not the combination of different formats of educational interventions would be more effective in changing the behavior of nonmedical professionals. Furthermore, overall little information exists to evaluate concussion educational resources with the ultimate goal to better translate concussion knowledge to the general public. To this end, the Public Health Agency of Canada (PHAC) engaged four Partners consisting of Hockey Canada (HC), Coaching Association of Canada (CAC), Canadian Centre for Ethics in Sport (CCES), and Think First (now a part of Parachute) to develop various concussion educational resources and survey thousands of Canadians about these resources. Our research group has reported the survey results and respondents' concussion knowledge previously (29,30). The analyses we present here, however, extend previous reports by evaluating the effectiveness of the newly developed concussion resources individually to deliver concussion education, their added value to concussion knowledge, as well as evaluating the effectiveness of their different formats of delivery. We hypothesized that exposure to concussion educational resources would improve participants' knowledge and understanding of concussion.

Ethics
The study received approval from the Research Ethics Board of St. Michael's Hospital. Participants provided consent prior to completion of the survey.

National Surveys and Concussion Educational Resources
This study reports on the results of two Canadian crosssectional surveys that were administered before and after the respondents' exposure to concussion educational resources. Presurvey responses were completed by 7,258 respondents whereas post-survey were completed by 6,031 respondents, respectively. Both surveys were described previously (29,30). The 19 concussion educational resources evaluated are described in Table 1. A cross-sectional, anonymous, voluntary, open survey with a convenience sample of the stakeholders in the sports community in Canada, was conducted in two waves as described previously.

Study Sample
As the surveys were conducted anonymously, the computer's IP address was used as a proxy for user's identification. Based on the survey data, 1,186 unique IP addresses were identified from which 503 were used in both the pre-and post-survey and 180 were used only in the pre-survey. The analysis reported here was conducted using the data set from the 503 survey respondents who completed the pre and post survey. The sample size obtained for each resource is indicated in Table 1.

Demographic and Social Characteristics
Language, age, sex, ethnicity, education, annual household income, geographic location, and the non-medical community role the survey respondent most strongly identified with in their community were collected.
Changes in concussion knowledge scores (CKS) between pre-and post-survey testing as a measure of the effectiveness of concussion resources. As described previously (29,30). CKS were calculated by summing the correct answers to 27 items that surveyed concussion knowledge related to physical/cognitive/mental health symptoms, concussion treatment modality, return-to-sport following a concussion, or diagnosis of concussion post head injury. The 27 items were evaluated using a dichotomous scoring process. Correct responses were given a score of one and incorrect responses were given a score of zero. Higher CKS scores indicated better knowledge of concussion. The CKS changes pre-and postsurvey between the resource users and non-users as well as all respondents was used as a measure of resource effectiveness.

350)
A webpage that provides current information on concussion approaches in injury prevention, and a variety of material on roles and responsibilities for all stakeholders.
Easy to access and use; can be made widely available.
The information provided is subjective to the understanding of its readers.
Concussion Toolkit (n =200) Included concussion management handouts for coaches, trainers, athletes, parents and medical personnel. It also included personal concussion records for each player, and an easy to follow guide for coaches and trainers.
A large amount of information tailed to specific stakeholders. It is not interactive or dynamic.
Content may need to be updated overtime. Information provided is limited.
Concussion education sessions (n = 256) A webinar series on the brain, case studies, injury prevention strategies, signs and symptoms, myth busters, and potential short and long term effects of a concussion, and resources to assist with concussion management.
Information provided is comprehensive. Content is more for adults than youths/kids.

248)
A 20" × 28" "I am not invincible" visual poster featuring two athletes and a short summary of their personal experiences.
Touching stories; simple format for all ages. The information provided doesn't cover all aspects of concussion.
Concussion cards (n = 359) Two page-length cards including general information designed to aid with assessment of concussions. It included information on visible cues, signs and symptoms of a concussion, memory functions and potential red flags.
Useful and easy to make; also useful on field.
Focus on assessment and the decision to remove a player from a game may be subjective to athletes' or coaches' attitude toward concussion. Ratings were based on participants' self-reports on the level of effectiveness (not effective, neutral, or extremely effective) for each concussion educational resource.

Mode of Delivery and its Effectiveness
Each concussion resource was classified into one of six forms of delivery: (1) webpage (e.g., a website from Parachute); (2) written guidelines (e.g., active and safe self-assessment tool for sport organizations); (3) quizzes (e.g., concussion IQ); (4) videos (e.g., the concussion stories video); (5) phone apps (e.g., the children and adult's concussion awareness mobile apps); and (6) in-person resources (e.g., 2012 Petro Canada sport leadership conference concussion session). Educational mode of delivery effectiveness was evaluated by examining CKS changes between pre-and postscores for each form of delivery.

DATA ANALYSES
Statistical analyses were performed using SAS 9.4 4 (SAS Institute, Inc., Cary, NC, USA) and STATA 13.0 (StataCorp LLC, College Station, TX, USA). Descriptive statistics of language, sex, age, education, household income, and survey respondent location were used to describe the demographic characteristics of the survey respondents. Pearson's correlation coefficient and Wald's F test were used to compare the effectiveness of each concussion educational resource measured by changes on CKS. Student t-tests were performed to compare CKS between preand post-surveillance between the resource users and nonusers in the post-survey group. An ordinary linear regression was performed with CKS as the dependent variable and age, sex, level of education, household income, use (or not use) the 19 educational resources as the independent variables. An alpha value of 0.05 or less was considered statistically significant, however, Bonferroni adjustments were applied for significance assessments to correct for experiment-wise error.

Characteristics of Survey Respondents
In both pre-and post-survey, the proportion of respondents falling into each age category and the male to female ratio (2:1) remained primarily unchanged. Roughly 78% of respondents wereas between the age of 35 and 54 while less than 4% fell into the younger (<25 years) or older respondent (>64 years) categories (n = 1,186, Table 2). Almost two thirds of respondents resided in Ontario, Bristish Columbia, or Alberta and reported a post secondary education at a college/university level (about 84%) with a household income greater than $20,000 ( Table 2).

Effectiveness of Concussion Resources Measured by CKS Change
Group comparisons were performed and results showed that both resource users and non-users in the post-survey had significantly higher CKS than at pre-survey while users' CKS were significantly higher than that of non-users ( Table 3). Linear regression analyses showed that level of education affected the respondents' CKS. A higher level of education was associated with higher CKS overall. However, age, sex, and income were not statistically significantly associated with changes in the respondents' CKSs from pre to post test. Results also indicated that the use of Concussion IQ and Concussion Cards was the only form of concusion educational resource that  statistically significantly increased respondents' CKS from pre to post survey (Table 4). Importantly, using three or more resources was predictive of a statistically significant increase in CKS than using a single resource (Figure 1).

Effectiveness of Concussion Resources Related to Mode of Delivery
The mode of delivery for all concussion educational resources was analyzed to determine the most effective method of  Figure  (1.2). Percentage of concussion educational resources used by respondents.
distribution of concussion knowledge and awareness resources. Based on the mode of delivery, CKS of respondents who used any of the concussion resources in post-survey were compared to that of those who were not exposed to such resources in pre-survey (Figure 2) or those who did not use such resources in post-survey (Figure 2).
When comparing resource users with non-users in the postsurvey, we found that 4 out of 6 modes of delivery significantly increased the users' CKS. Quizzes and videos were the only two delivery methods that failed to impact CKS (Figure 2). When compared, there was no significant difference found among different educational format modalities (p > 0.05).

Effectiveness of Concussion Resources Measured by Respondents' Rating
Respondents were asked to rate the effectiveness of each concussion resource (Figure 3). Overall, most resources were rated as "extremely effective" (Figure 3). However, a regression analysis on the correlation between respondents' ratings and their CKS did not reveal a statistically significant associations (p > 0.05).

Main Findings
The results presented illustrate that educational resources delivered at a national level to athletic communities can improve knowledge of concussion. We found that 2 out of 19 resources and 4 out of 6 modes of concussion education delivery were successful in achieving this goal. The webpage educational resource was the most effective mode of delivery concussion knowledge. In addition, our results showed that those who used three or more educational resources had more significant improvements in their concussion knowledge.

Implications
Studies in concussion education have tried various approaches to promote autonomy in online learners (31)(32)(33) from written  guidelines for athletes, coaches, educators and parents, brochures, quizzes, videos, and websites (34) to tailoring educational interventions to specific needs, skills, preferences, and the culture of the consumer in order to enhance the effectiveness of such educational resources (35). Indeed, autonomous self-regulated learning is emphasized in Piaget's constructivist learning and teaching theory (36) and has been identified as one of the most effective ways to promote behavioral change and active learning, long term (37)(38)(39). Yet much remains to be done to change engrained cultural perceptions of concussions and their effects on the individuals, their family and community teaching over time. It takes a community to make changes such as these and stakeholders at all levels that exert an influence on behavior, from the individual, to sport teams, schools, parents, coaches, teams, cities, provinces, and the government of the country in which players reside. Together these bodies of influence could impact greater change and more rapidly than any one level of influence operating alone. Governments through the ministries of education and health, could exert influence through legislation, could collaborate with physician groups to improve levels of education among physicians and other health professionals, school teachers athletic lead trainers. Parents could facilitate collaborations and knowledge translation by becoming engaged at multiple levels of athlete education-schools, clubs, leagues, and provincial and national sports organizations.
The present study supports the need to tailor concussion educational resources for the intended audience but many questions remain unanswered and could be addressed with a larger sample size. For the studied populations, age did not affect their CKS, regardless of the educational resources or mode of delivery used. This may not be the case if very young athletes (e.g., 15 years old or younger) are the targeted population as their knowledge base and reading comprehension are quite different (16). Similarly, income was not a significant predicting factor of CKS for the participants in this study as the group with very low income (<$20,000) was very small (0.9%).
The finding that respondents were able to improve their concussion knowledge in a short period of time suggests these participants may have been motivated learners. According to the health belief model, they may be motivated by (1) the desire to avoid illness, or get well, if they already have had a concussion, and (2) the belief that knowing more about concussion will help prevent or cure it (40). With the learning paradigm shifted from 3 R's (reading, 'riting, 'rithmetic) to 4 Es (exposing knowledge, employing information, expressing ideas compellingly, and ethics-right and wrong on the information highway) in the digital era (41), online concussion educational resources are fashioned with several benefits pertaining to eHealth such as avoiding unnecessary delays in receiving the resources and getting advice/new knowledge from authorities (e.g., concussion experts' opinion and up-to-date concussion guidelines) (42) and reaching a large audience (43). Unfortunately, the knowledge gained from such educational interventions may not last for a long time (44), suggesting that such educational efforts should be made on a regular basis or ongoing, and based on our results, combining different formats of delivery can achieve more knowledge gains. Nevertheless, it is important to note that the effectiveness of concussion educational resources doesn't necessarily translate to attitude or behavioral changes (especially in coaches, results not shown), which has been reported in several studies (45)(46)(47).

Limitations
Although we had a large sample size, our desire to do a paired analysis restricted that sample size to just over 1,100 responses before and after educational resource exposure. Readers must be cautions interpreting these results as using a voluntary sampling method may exclude certain demographic or ethical groups and using IP addresses as a proxy for the same respondent may overestimate the number of unique respondents. While the results may not be representative of the entire country, Ontario, British Columbia, and Alberta constitute over half of the Canadian population rendering our results some breadth. Since the distribution of concussion education resources was disseminated via online and mobile applications, the targeted audience was limited by the availability and accessibility of survey respondents to the Internet. Another limitation is the possibility that a respondent's improved knowledge regarding concussion education happened on their own accord and did not stem from the use of the concussion resources we presented. This may have posed a maturational threat of internal validity and potentially impacted the outcome of the study. Further, data were not collected for other events that might have influenced levels of knowledge and awareness. Lastly our results could have been influenced by events such as media reporting of concussion or other education initiatives made available throughout the time period of this survey.

CONCLUSIONS
Despite these limitations the results presented here have merit and warrant follow up investigations. Our findings demonstrated that the concussion educational resources we provided, based on respondents' results, were able to improve user concussion knowledge. Using three or more educational resources appeared to be more effective than single resources. Future research is needed to evaluate whether implementing these concussion educational resources translate into fewer injuries short and long term.

ETHICS STATEMENT
This study was carried out in accordance with the recommendations of Tri-Council Policy Statement guidelines, Research Ethics Board at St. Michael's Hospital with implied informed consent from all subjects. All subjects gave informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Board at St. Michael's Hospital.

AUTHOR CONTRIBUTIONS
MC was responsible for conception and design as well as analysis and interpretation of the data and revising the manuscript. JT-V, SZ, RJ, AG and GI contributed to the analysis and interpretation of the data as well as revising of the manuscript.