The Stability of Retrospective Pre-injury Symptom Ratings Following Pediatric Concussion

Objective: To determine the stability of children's retrospective ratings of pre-injury levels of symptoms over time following concussion. Methods: Children and adolescents (n = 3,063) between the ages of 5–17 diagnosed with a concussion by their treating pediatric emergency department (PED) physician within 48 h of injury completed the Post-Concussion Symptom Inventory (PCSI) at the PED and at 1, 2, 4, 8, and 12-weeks post-injury. At each time point, participants retrospectively recalled their pre-injury levels of post-injury symptoms. The PCSI has three age-appropriate versions for children aged 5–7 (PCSI-SR5), 8–12 (PCSI-SR8), and 13–18 (PCSI-SR13). Total scale, subscales (physical, cognitive, emotional, and sleep), and individual items from the PCSI were analyzed for stability using Gini's mean difference (GMD). Results: The mean GMD for total score was 0.31 (95% CI = 0.28, 0.34) for the PCSI-SR5, 0.19 (95% CI = 0.18, 0.20) for the PCSI-SR8, and 0.17 (95% CI = 0.16, 0.18) for the PCSI-SR13. Subscales ranged from mean GMD 0.18 (physical) to 0.31 (emotional) for the PCSI-SR8 and 0.16 (physical) to 0.31 (fatigue) for the PCSI-SR13. At the item-level, mean GMD ranged from 0.13 to 0.60 on the PCSI-SR5, 0.08 to 0.59 on the PCSI-SR8, and 0.11 to 0.41 on the PCSI-SR13. Conclusions: Children and adolescents recall their retrospective pre-injury symptom ratings with good-to-perfect stability over the first 3-months following their concussion. Although some individual items underperformed, variability was reduced as items were combined at the subscale and full-scale level. There is limited benefit gained from collecting multiple pre-injury symptom queries. Clinical Trial Registration: Clinicaltrials.gov through the US National Institute of Health/National Library of Medicine. (NCT01873287; http://clinicaltrials.gov/ct2/show/NCT01873287).


INTRODUCTION
Concussions are a prevalent injury among children and adolescents throughout North America. In the US, concussions account for one in 220 PED visits, accumulating over 700,000 PED visits annually (1)(2)(3). Rates of reporting to the emergency department are even higher in Canada, where concussions are responsible for one in every 70 PED visits (4). Currently, an objective biomarker for concussion diagnosis and subsequent recovery is elusive. Instead, health care providers must rely upon a clinical examination in combination with symptom, balance, and cognitive assessments for concussion diagnosis and management (5,6). This can be particularly difficult when evaluating children and adolescents, especially elementary-aged children, as current clinical assessments are developed for use with adults and are often not validated or developmentally appropriate for young patients (7).
Concussions can result in a number of somatic, emotional, cognitive, or sleep-related symptoms (8), with headache, imbalance/dizziness, and fatigue the most commonly reported post-concussion symptoms (8,9). Most clinicians do not have access to a concussion symptom checklist completed by the child pre-injury to serve as a baseline level of normal functioning. One alternative approach is to rely on the injured child to retrospectively report pre-injury levels of their current postinjury symptoms. As concussion management requires serial assessment, and potentially oversight from multiple health care providers (10), it may require children to provide retrospective pre-injury ratings multiple times throughout recovery.
The Post-Concussion Symptom Inventory (PCSI) is a commonly used pediatric symptom checklist that is sensitive to concussion (8). The PCSI has three, developmentally appropriate versions for children aged 5-7, 8-12, and 13-18 years old. Several psychometric properties of the PCSI have been previously reported, including its factor structure, internal consistency, rater concordance, and convergent validity (8,11,12). The test-retest reliability of the PCSI has been assessed in healthy (i.e., noninjured) children and interclass correlation coefficients ranged from moderate-to-high (0.65 < ICC < 0.89) (8). However, the presence of concussion may alter the stability of this assessment, particularly when retrospectively recalling pre-injury ratings. Adults with concussion are suspected to engage in a "good-old-days" bias, whereby they idealize retrospective preinjury levels compared to uninjured controls (13,14). Recall bias in self-reported symptom outcomes is also highlighted in the scientific literature, with patient age and time since event named as important factors (15,16). Therefore, it is critical to formally evaluate the stability of retrospective pre-injury ratings throughout pediatric concussion recovery, since pre-injury status can have important implications on clinical decision making following injury.
The primary purpose of this study is to determine the stability of retrospectively provided pre-injury ratings from children and adolescents over the first 3-months following a diagnosed concussion. Due to the relatively short follow-up period (12-weeks), we anticipate the effect of a "good-old-days" or recall bias will be relatively minimal and hypothesize that retrospective pre-injury ratings will be highly stable overall. However, we further hypothesize that individuals who report higher retrospective pre-injury symptom ratings will be less stable than individual who report few or no pre-injury symptoms.

Participants
Data collection methods have been previously described in detail in other studies (17,18). Briefly, children presenting to nine PEDs across Canada were eligible if they were between 5 and 17 years old, were diagnosed with a concussion using the definition reported by the 2012 Zurich consensus statement (19), presented to the PED within 48 h of injury, and were proficient in either English or French. Children were excluded if they had: (1) A Glasgow Coma Scale score ≤ 13; (2) Any abnormal findings on standard neuroimaging (if neuroimaging was clinically indicated); (3) Neurosurgical operative intervention, intubation, or intensive care stay, (4) Multisystem injuries with treatment requiring hospital stay; (5) Severe developmental delay resulting in communication difficulties; (6) Intoxication at time of PED presentation; (7) No clear primary mechanism of trauma (e.g., sports-related injury, motor vehicle accident, fall) for the current head injury; or (8) Previously enrolled in the same study. Research assistants were present at each PED from 12:00 to 22:00 h to screen for potential participants, assess eligibility, and explain study procedures to eligible patients. This study was carried out in accordance with the recommendations of the Research Ethics Board of each participating institution with written informed consent from all subjects. Written consent was obtained from all eligible and willing parents as well as children and adolescents capable of consenting on their own behalf in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Board of each participating institution.

Initial PED Visit
Once enrolled, a research assistant collected data from both the parent and child with an electronic survey in their primary language (English or French) on a portable computer tablet to gather demographic information. Along with demographic data, the parent and the child were then asked to complete the age-appropriate version PCSI prior to leaving the PED. Symptoms on the PCSI were rated once scoring the child's current level of post-injury symptoms and once retrospectively recalling the child's pre-injury levels of those same symptoms. In addition to data collection, each treating physician provided a routine clinical assessment of the child, provided discharge instructions, and gave follow-up information as per the normal standard-of-care procedures of that PED site. Prior to leaving the PED, contact information was obtained for all families.

Study Follow-Up
Parents chose to complete follow up measures either online using a web-based platform (REDCap) or over the phone (Email: n =

Post-concussion Symptom Inventory (PCSI)
The PCSI is a developmentally-appropriate concussion symptom scale that was made for children and adolescents. The PCSI has three age-based versions (PCSI-SR5: Ages 5-7, 13-items, 3-point Guttman Scale; PCSI-SR8: Ages 8-12, 25-items, 3point Guttman Scale; PCSI-SR13: Ages 13-18, 26-items, 7-point Guttman Scale). As the PCSI uses different scaling based on the age-appropriate version, the PCSI-SR5, and the PCSI-SR8 were transformed by multiplying each item by three to make the scale range equivalent to the PCSI-SR13 which used a 7-pt Guttman scale most commonly found in the literature. The PCSI was chosen because it is one of only two symptom scales with age-appropriate version for younger children, with appropriate psychometric data published in the literature (8,11,12). Additionally the PCSI-SR8 and PCSI-SR13 can be evaluated as physical, cognitive, emotional, and sleep subscales (8). The list of symptoms and which subscales they contribute to can be found in Table 1.

Statistical Analysis
All analyses were completed in R version 3.3.2 (Vienna, Austria) (20). Stability was evaluated by quantifying the mean absolute discrepancy of the retrospective pre-injury ratings for each patient, a quantity also known as the Gini's Mean Difference (GMD), and post-injury symptom ratings were not analyzed in this study. GMD is a descriptive outcome calculated by taking the average of the absolute value for all pair-wise comparisons between pre-injury ratings at all study time points (i.e., pre-injury rating at the PED and 1, 2, 4, 8, and 12-weeks follow-ups). GMD was chosen because no reference time-point must be declared (i.e., no single time-point was assumed to be more accurate than others) and it is free from distributional assumptions (i.e., remain valid under non-normal distributions). With six total time points throughout this study, a maximum of 15 pairwise comparisons were possible for each individual item. To be included in the analysis, individuals were required to complete the retrospective pre-injury ratings on the PCSI at least twice. Individuals who only completed the PCSI at one time point were excluded as GMD could not be calculated. Additional information about the Age, median (IQR) 6.7 (5.9, 7.3) 6.6 (5.9, 7.2) 6.6 (5.8, 7.2) 0.30 10.8 (9.7, 11.9) 10.7 (9.5, 11.9) 10.6 (9.5, 11.9) 0.95 15 (   Frontiers in Neurology | www.frontiersin.org amount of missing data present throughout this study can be found in the Supplementary Material. A GMD value was first calculated using the rating provided for each individual item listed on the PCSI. At the subscale and fullscale level, the GMD value was calculated by taking the average of all constituent items involved (i.e., if the subscale had three items, the GMD values calculated for those three individual-items were averaged to create a GMD value for the entire subscale). This approach allowed all GMD values reported in this study to range from 0 (perfect stability) to 6 (worst possible stability), providing consistency between all levels of analysis (full scale, subscale, and individual-items) and more appropriate comparisons for situations where all individual items were not completed at all time points. To calculate 95% confidence intervals for the GMD values, estimates were provided by percentile bootstrap based on 100 repetitions. As the number of items differs on each version of the PCSI, we chose to report GMD values separately for the PCSI-SR5, PCSI-SR8, and the PCSI-SR13. There is no literature available to define what level of stability is needed in clinical use, but some level of variability is to be expected. We defined an acceptable level of stability to be good-to-perfect to be a GMD < 1, which is 1/6th (16.7%) of the possible value given the PCSI range.
Previous concussion studies have shown that ∼40% of individuals report no pre-injury level of symptoms (21). Individuals who consistently report no pre-injury levels of problems across all study time-points would have perfect stability (GMD = 0); a large proportion of individuals with perfect stability may mask clinically important findings. Therefore, we completed a subgroup analysis by removing all individuals who reported no pre-injury symptoms at all six time points studied and re-analyzed stability only with individuals reporting some retrospective pre-injury symptom presence throughout the study (+ Pre-Injury Ratings). Stability results for the entire sample and for the + Pre-Injury Ratings subgroup are presented throughout the study.

RESULTS
A total of 3,063 children aged 5-17 were recruited through the Predicting and Preventing Postconcussive Problems in Pediatrics (5P) study. Of those 3,063 children, 2,866 (93.6%) completed the retrospective pre-injury PCSI rating during at least two time points and were included into the analysis (Ages 5-7: N = 499, Ages 8-12: N = 1,203, Ages 13-17: N = 1,164). In our sample, 207 (41.5%) participants aged 5-7, 343 (28.5%) participants aged 8-12, and 252 (21.6%) participants aged 13-17 retrospectively rated all pre-injury levels as zero throughout the entire study period (GMD=0) and were not included in the subgroup analysis. Descriptive statistics for the study sample can be found in Table 2.

Full Sample
The full scale for the PCSI-SR5 was highly stable, with 94% of children displaying good-to-perfect stability. The mean GMD for

Pre-Injury Ratings subgroup only. No subscales exist in the PCSI-SR5.
Frontiers in Neurology | www.frontiersin.org the full 5-7-year-old sample ranged from 0.13 to 0.60 at the itemlevel. Dizziness was the most stable individual item and difficulty concentrating showed the lowest stability.

+ Pre-injury Ratings
When looking only at individuals who retrospectively reported at least one non-zero pre-injury rating throughout the study period, the overall stability was reduced but trends were similar to the full sample. The full scale remained highly stable for the + Pre-Injury sample, with 89% of children continuing to show good-toperfect. Balance problems and difficulty concentrating remained the most and least stable individual items, respectively. Overall results for the PCSI-SR5 can be found in Table 3.

Full Sample
Overall, the full scale for the PCSI-SR8 was highly stable (GMD = 0.19, 95% CI: 0.18, 0.20). The physical subscale showed the highest relative stability, with moving slowly and vision problems encompassing the most stable individual items. The emotional subscale showed the lowest stability among the four subscales. Headache had the lowest relative stability at the item-level, followed by difficulty concentrating.

+ Pre-injury Ratings
The PCSI-SR8 remained highly stable for the + Pre-Injury sample, but different trends emerged in this group. The physical domain continued to be the most stable subscale, but nausea and headache had the highest relative stability of individual-items. In disagreement with the full sample, fatigue (subscale), and vision problems (item-level) were the least stable relative to other outcomes. Full results for the PCSI-SR8 can be found in Table 4.

Full Sample
The full scale for the PCSI-SR13 had high overall stability.
Vision problems, answer slowly, and nausea were the most stable individual items and the physical domain was the most stable subscale. Conversely, fatigue, headache, and the fatigue subscale had the lowest relative stability.

+ Pre-injury Ratings
The overall stability for the full scale remained high for the + Pre-Injury sample. The physical subscale and dizziness (item) had the highest relative stability. On the low end, fatigue (subscale) and headache (item) had the lowest stability. Overall results for the PCSI-SR13 can be found in Table 5.

DISCUSSION
The results of this study confirm our hypothesis that the retrospective recall of pre-injury ratings is stable in children and adolescents. At the full-scale level, all mean GMD values were <5% of the scale range, suggesting little dispersion, and high stability in the retrospective recall of pre-injury levels in the 3-months post-concussion. Averaging over all items, including those with very low mean GMDs, and non-consistent retrospective reporting in opposing direction (i.e., retrospectively rating one symptom 1-point higher than previous reports while rating a different symptom 1-point lower than previous reports) likely contribute to the high stability seen across both the full scale and sub-scale level. The wider 95% confidence intervals and smaller percentage of individuals displaying good-to-perfect stability indicate that the youngest children (ages 5-7) may have slightly less stability in their retrospective symptom recall, in agreement with previous literature suggesting young children have the least reliable recall (15). Our results support the high overall stability of retrospective pre-injury ratings despite large, non-linear changes in post-injury symptoms ratings, which have been shown previously in this sample of concussed children and adolescents (22).
Every individual item on each version of the PCSI had a mean GMD ≤ 1/10th of the scale range and a median GMD of 0, indicating little dispersion. Headache, fatigue, and difficulty concentrating were among the least stable individual items across all versions of the PCSI. These particular symptoms are not specific to concussion and are among the most commonly reported baseline symptoms in healthy children and adolescents for their respective age groups (8,23). Additionally, the severity at which these problems occur if present often fluctuates over time, which can account for the higher variability (less stability) seen in these items. This rationale is supported by previous research in children with headache history, which shows that children can accurately recall headache frequency but struggle with recalling headache intensity and duration (16). Conversely, the items with the highest individual stability (vision, balance/dizziness, and move/answer slowly) are among the least frequently endorsed baseline symptoms in our study, which agrees with previous literature (8).
The PCSI was highly stable as hypothesized. Previous studies show that the "good-old-days" and recall biases can elevate preinjury symptom reports, which would theoretically decrease the stability of the assessment. However, reports of these biases are traditionally found over longer periods of time than evaluated in this study and in individuals with on-going symptoms (13,15). The main objective of this study was to describe the overall stability of retrospective symptoms rating on the PCSI. Therefore, stability was examined only as a collective whole over the followup period (i.e., not specific to time intervals within the followup period) and with pre-injury symptom ratings [post-injury symptom scores and their progression over the follow-up period in this sample have been previously reported (22)]. As such, our methodology does not allow for a conclusive determine regarding the presence of the "good-old-days" in our sample, which should be evaluated further in future studies.
The primary strength of this study lies in the large, diverse sample. Our pediatric participants comprised a wide age range (5-17), included various mechanisms of injury, and embraced patients with behavioral, learning, and psychological problems [more specific information about our sample can be found in Zemek et al. (18)]. Our broad inclusion criteria increase the generalizability of findings. The use of a validated concussion symptom checklist, which has three, developmentally appropriate versions for children and adolescents, is an additional strength. Lastly, the analytical method of GMD  is considered a strength due its ability to tolerate nonnormally distributed data and handle missing data (only two retrospectively PCSI rating were needed for inclusion). Even though missing data may mean different time intervals between assessments, there is no natural bias toward stability when this is the case which allows the inclusion of more patients and increases generalizability of our results.
Although the psychometric properties of the PCSI are a strength, the number of items and rating scales for the three different versions are not equivalent and represent a limitation. All reported pre-injury rating responses (0-2 scale) on the PCSI-SR5 and PCSI-SR8 were multiplied by three to make all response outcomes range of GMD 0-6. However, this induces more instability into the PCSI-SR5 and PCSI-SR8 and assumes a 1-point discrepancy on the 0-2 scale is equal to a 3-point discrepancy on a 0-6 scale. The multiple rating scales utilized in the PCSI complicates comparisons across age groups; however, rescaling prevents an inaccurate misinterpretation that younger children have higher stability because the scale range is smaller.
Original GMDs values for the PCSI-SR5 and PCSI-SR8 can be found by dividing values reported in this manuscript by three. IN addition, GMD values reported are group averages. Therefore, some individuals may have had large changes (low stability) in their retrospective pre-injury recall. Clinicians should be aware of this when working on the individual patient-level.
Symptom reporting is limited by both floor and ceiling effects, floor effects being a larger concern in our study evaluating preinjury ratings. The GMD looks at the absolute value of differences in pre-injury reporting over multiple time points. Patients who consistently report no symptoms would have a GMD of zero, thus increasing the stability of the group analysis. However, even patients at the floor (total score of zero) can have a large GMD if they report higher pre-injury ratings at subsequent assessments. Recovery status is not considered in this study. Some participants will have reached full concussion recovery during the follow-up period while other were still experiencing post-concussion symptoms and deficits. How recovery status may have affected retrospective symptom reporting is unknown. Nearly all patients preferred email follow-ups, but a minority of patients (n = 276) requested phone follow-ups. Similarly, very young children may need their parent to help read or understand the symptom checklist. It is possible that interacting with a research assistant via phone follow-ups or parent may influence the patient's response.
While some items underperformed relative to other and some minor differences in stability between the full sample and the + Pre-Injury Ratings reporters were apparent at the item-level, the stability for both groups was high at both the subscale and full-scale level. Based on the results of this study, there appears to be no additional clinical benefit to having patients retrospectively recall their pre-injury symptom levels multiple times throughout the recovery period and this can reduce burden on the healthcare provider and the injured child. If multiple retrospective pre-injury ratings are needed (e.g., seeing multiple providers), clinicians can be assured that children and adolescent will retrospectively rate pre-injury levels with relatively high stability. As long as clinical decisions are based on the subscale or the full-scale level, multiple pre-injury ratings will not have meaningful differences and provide equivalent results.
Children and adolescents retrospectively rate pre-injury levels of post-concussion symptoms with good-to-perfect stability over the first 3-months following their injury. Although some individual items underperformed, variability was reduced as items were combined at the subscale and full-scale level. There is limited benefit to multiple pre-injury symptom queries.

DATA AVAILABILITY
The dataset for this study can be requested through Ontario Brain Institute (OBI) BRAIN-CODE after registering using the following link: https://www.braincode.ca/user/register

ETHICS STATEMENT
This study was carried out in accordance with the recommendations of the Research Ethics Board of each participating institution with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Board of each participating institution.