The Preschool Physical Literacy Assessment Tool: Testing a New Physical Literacy Tool for the Early Years

Abstract

Background: Physical literacy is essential to physical activity across the lifespan. While there is an emerging body of research on physical literacy in school-aged children, the preschool years have largely been ignored. We tested the psychometric properties of the new tool, the Preschool Physical Literacy Assessment Tool (Pre-PLAy) designed to address this gap.

Methods: We recruted 78 children (aged 19–49 months) across 5 childcare centers in Hamilton, Ontario. Two Early Childhood Educators (ECE) completed the Pre-PLAy for each child at two points in time to assess inter-rater reliability and test-retest reliability. We assessed the agreement between the Pre-PLAy tool with gross motor skills and the ability of the PPLAy to predict physical activity.

Results: Results indicated Pre-PLAy is related to gross motor skills and predictive of physical activity for females, but not males. Inter-rater and intra-rater reliability was at least adequate for all but the coordinated movements items and scale for females, but ECEs showed poor agreement for males.

Conclusions: These results suggest initial support for the Pre-PLAy tool as a measure of physical literacy during the early years. However, some modification to the items and training are required to address the gender-specific effects found in this sample.

Introduction

Physical literacy (PL) encompasses the knowledge, skills, motivation, and feelings related to physical activity (PA) and movement (1). There is a growing interest regarding the definition and utility of PL by academics and professional organizations [e.g., (1–3)]. Despite this interest in PL, the preschool years have largely been ignored. This is an important gap as PL has been proposed as foundational to engagement in PA (2) and PA behaviors are established early in life (4). Therefore, supporting PL in the early years has the potential to increase PA. Understanding and supporting PL in the early years is important considering the concern regarding the low activity levels among young children (5) and the prevalence of obesity (6).

A PL assessment instrument for the preschool years is needed to address the PL research gap at this age. Existing PL measures were developed for school-aged children (3, 7). Without a PL tool for the early years, researchers have relied on measures of motor skill to assess PL. This approach fails to assess the multi-dimensional and lifelong aspects of PL and can contribute to confusion regarding the construct (8, 9). We developed the Preschool Physical Literacy Assessment Tool (Pre-PLAy) to be completed by Early Childhood Educators (ECEs) for children ages 18 months to kindergarten entry (about age 4). Pre-PLAy was designed to measure PL that assess the following domains of PL, consistent with the literature: Movement competences, coordinated movements, motivation, and enjoyment (2, 10). Knowledge of the importance of PA has also been identified as a domain of PL (1, 2). However, we did not include it in Pre-PLAy as we did not consider it developmentally appropriate for preschool children. Preschool children will neither have acquired nor be able to demonstrate knowledge regarding the importance of PA.

All items are scored based on ECE observations of the child across structured, free play, and self-care (e.g., dressing) activities. The movement competences items measure competence in specific skill domains (e.g., object control skills) and coordinated movements items assess coordinated executions of multiple skills simultaneously (e.g., running and striking). In motivation and enjoyment domains ECEs are instructed to complete items related to motivation and enjoyment based on behaviors, reactivity, and enjoyment while active. This approach is necessary because children at this age are not able to reliably self-report their perceived competence in specific areas (11).

The purpose of the study was to assess the reliability and validity of Pre-PLAy. To assess validity we examined the agreement between Pre-PLAy with a standardized test of gross motor skills and whether the Pre-PLAy predicted objectively measured PA. We expected a moderate correlation between the total Pre-PLAy and gross motor skills given that PL includes movement competences, but is also broader than movement competencies. For the subscales, we expect a stronger association between gross motor skills with the movement competencies and coordinated movements scales than motivation. Gross motor skills are conceptually related to movement competencies, though they are distinct (12).

We also hypothesized that Pre-PLAy would moderately predict PA. We posit a moderate relationship as the conceptual work of PL has positioned PL as foundational to PA [e.g., (2, 9, 13)], but previous research has reported low correlations between motor skill and PA during this developmental period (14). We anticipate the inclusion of motivation and enjoyment in the PL construct, will demonstrate a stronger relationship than motor skill alone has shown.

We will also examine gender differences, considering past research has demonstrated sex differences in PL and related constructs. For example, Longmuir (3) reported gender differences in the physical competence domain of PL measured by the Canadian Assessment of Physical Literacy in a sample of school aged children. Sex differences were also found in overall PL and other subdomains, but only for children older than age 10. Other research has reported gender differences in motivation and enjoyment of PA [e.g., (15)].

Materials and methods

Sample

We recruited 78 children and 26 female ECEs across 5 childcare centers in Hamilton, Ontario. Children who were under 18 months, had a diagnosis of developmental delay (e.g., autism, cerebral palsy), attended the center for <2 full days each week, or were leaving the class or center before the study period finished were excluded from the study. Research staff also attended the center during pick-up time to speak with parents, answer any questions, and obtain consent for eligible participants.

Measures

Physical literacy

The Pre-PLAy tool was used to assess PL. Research staff trained ECEs on the Pre-PLAy tool in a 45 min session at the center. Training consisted of a review of the tool, and asking ECEs to complete the tool by selecting a child in their classroom and reflecting on their experience with them. Pre-PLAy is comprised of 19 items: 10 items that assess simple skill movement competencies (e.g., sending, transporting), 4 items that assess co-ordinated movements, 4 items that assess motivation and enjoyment, and 1 global item assessing overall PL (see Table 1). For movement competencies, ECEs are asked to rate the skill level for the child considering children of a similar age. This is to ensure that developmentally appropriate comparisons are made. Responses are measured on a modified visual analog scale. ECEs are first asked to select the skill level of the child from the following options: does not display skill, displays skills with instruction, displays skill without instruction, displays with other skills, and creatively displays skills. A rubric is included to describe each skill level in more detail. These are displayed as horizontal connected boxes (see Figure 1 for an example of one item from the scale). Once the level of skill is selected, ECEs are asked to treat the box as a visual analog scale and draw a line vertically in the box to indicate the child's proficiency within the skill level. The further to the right a line placed, the higher the level of PL. The bar was 15 cm long and a ruler was used to measure where the ECE drew the line and determine the child's score for the item from 0 to 15. Co-ordinated movement responses are scored on a 4-point adjectival scale from never to always. Motivation and enjoyment items instruct ECEs to rate their agreement with statements on a 5-point adjectival scale. The final item asks ECEs to rate the child's overall PL using a visual analog scale that is 15 cm long. The Pre-PLAy score was calculated by summing all items in the tool. Scale scores were computed by summing all items in a scale. Item 17 (“When the opportunity to participate in new active games and play that use a variety of movement competencies, the child seems cautious/hesitant.”) is reverse scored.

Figure 1

Physical activity

Objective PA was measured using ActiGraph GT3XP hip-worn accelerometers. ECEs were instructed to have the children wear the accelerometer for the entire time the child was at the day care center, including nap times. Accelerometers were worn at the center for up to 2-weeks (weekdays only). This allowed children who attended the center part time the opportunity to meet the minimum wear time criteria. ECEs recorded the time the child's monitor was put on in the morning, taken off when they left, and other instances when the belt was removed. Nap time was also recorded. Minimum wear time was defined as 3 days of wear time. In order for a day to be counted, the child needed for have worn the accelerometer for at least 4 h of wear time. The 4 h wear time criteria per day meant each day included pre and post nap time programming. Sixty minutes of consecutive zeros was the cut off for a time period to be considered a non-wear period, however, no participant met this cut off. The accelerometer data was cleaned and analyzed using ActiLife Version 6.13.3. Pate, Almeida, McIver, Pfeiffer, and Dowda (16) cut-points were used to define total activity, and moderate to vigorous physical activity (MVPA). We examined minutes of total activity per hour (light, moderate, and vigorous) and minutes of MVPA to define PA.

Gross motor skills

The Peabody Developmental Motor Scale [PDMS-2; (17)] was used to assess gross motor skills. The PDMS-2 includes three subscales specifically designed to assess gross motor skills in children aged 1–5 years old: Stationary, Locomotion, and Object Manipulation. Each PDMS-2 assessment was completed by two trained research staff. The assessment took place within 2 weeks of when ECEs completed the first Pre-PLAy for the child. Each child was assessed individually and the assessments took place in areas away from the classroom where distractions were limited (gyms, separate rooms, hallways/corridors). One researcher administered the test (provided instruction and demonstration) and the other scored each item. Scoring was discussed at the end of each assessment and agreed upon by both research staff.

Procedure

Participating children were matched with two ECEs in each class to assess inter-rater reliability. In classes with three ECEs, random allocation was used to match children to two ECEs. Re-allocation following randomization was required for 8 children to allow ECEs to be matched with children they would have more opportunity to observe due to the structure of the classroom. ECEs were asked to observe children for 2-weeks prior to completing the P-PLAY tool. A second 2-week observation began the day after the first Pre-PLAy tool was completed. This allowed us to test the 2-week test re-test reliability of the tool. Research staff attended the center and completed the PDMS-2 for each child within 2-weeks of the first Pre-PLAy. Research staff completing the PDMS-2 were blinded to Pre-PLAy scores, and ECEs were blinded to the PDMS-2 when completing the P-PLAY.

Analysis

To assess initial Pre-PLAy item results, test-retest agreement, and validity analysis, a Pre-PLAy from each child was randomly selected when there were two valid scores. Inter-rater and test retest reliability were assessed using a two-way random, consistency, single-measure intraclass correlation (ICC). Criteria outlined by (17) were used to evaluate inter-rater and intra-rater reliability: poor agreement (below 0.4), fair agreement (0.40–0.59), good agreement (0.60–0.74), and excellent agreement (>0.75).

Nesting of the sample occurred at two levels. First, children were nested in classrooms, which were nested in centers. The design effect was calculated using the ICC and cluster size to assess the impact of nesting on the data. A design effect greater than or equal to 2 requires analysis that addresses the nested nature of the data. Minutes of MVPA had a design effects of 3.03. Mixed effects models were employed using R package nlme version 3.1–117 (18) to assess the ability of Pre-PLAy to predict PA. Models were run with classroom nested within center included as a random effect. To probe interaction effects, Preacher, Curran, and Bauer's (19) simple slopes calculator was used.

Results

Description of sample

Four children were excluded because they did not have a completed Pre-PLAy or were missing both the PDMS-2 and accelerometer data. This resulted in a sample of 74 children (51% female). Table 2 includes detailed sample characteristics. Average wear time for accelerometers was 7 days across the sample. There were no gender differences in wear time. Boys engaged in more PA than females across all measures. ECEs also rated boys significantly higher on the Pre-PLAy. In contrast, there were no gender differences on the PDMS-2.

Pre-PLAy item results

Absences of ECEs and children resulted in three ECEs unable to complete Pre-PLAys on seven children at time 1. Item level missing data was present for 18 children. Item 12 (Uses playground equipment) had a high proportion of missing data (16%) due to the limited availability of playground equipment in some centers: three of the five centers did not have outdoor playground equipment. In those centers, some ECEs felt they could complete the item based on how children played in the natural environment of the playground (e.g., climbing on tree stumps), whereas others did not feel they were able to complete this item. Item 19 was printed on the backside of the last page of the tool. As a result, the item was missed by some ECEs when completing the tool (11%). Due to the high proportion of missing data on these items, the Pre-PLAy total score excluded these items in the current analysis.

Item level distributions showed observed scores were similar to potential scores, indicating ECEs used most or all of the response options when observing children. One exception is that no child was rated over 12 in the movement competencies item. A rating of above 12 would reflect an exceptional level of skill in the area (see Table 3).

Reliability

Internal consistency

Internal consistency, corrected item-total correlations, and inter-item correlations were assessed to examine the degree that items on each scale appeared to measure a consistent construct. Correlations for sending items (upper and lower body) and receiving items (upper and lower body) were near or above 0.90. Alphas for the movement competencies, and motivation and enjoyment scale showed good internal consistency (0.941 and 0.841). In contrast, the alpha for coordinated movements was poor (0.623). Item 11 “Uses a variety of moving vehicles” showed poor item-total and inter-item correlations (<0.25). Deleting this item would increase alpha to 0.901.

Inter-rater and test-retest agreement

Forty-nine children (66%) had valid Pre-PLAy scores from two ECEs at time 1 and were included in the inter-rater reliability analysis. Sixty-five children (88%) had valid Pre-PLAy scores from an ECE 2 weeks apart. Detailed results for inter-rater and test retest agreement are shown in Table 4. Rater agreement varied by sex, for females, agreement between ECEs ranged from fair to excellent on most items. The exception were the coordinated movements scale score and items which showed poor agreement. ECE's ratings were more divergent for males, with agreement on eight items below the acceptable range. Of the remaining items for boys, only the two sending items showed good agreement. Test-retest agreement across a 2-week period for items was fair to good across all items for both females and males. Again, the co-ordinated movements scale had the lowest reliability value for both females and males.

Convergent validity

The relationships between the Pre-PLAy tool and PDMS-2 varied across gender. For females, correlations were low to moderate range between Pre-PLAy total with the total gross motor scale of the PDMS-2, stationary, and locomotion subscales. The same pattern of results was found for the movement competencies subscale of the Pre-PLAy. No significant correlations were found with the coordinated movements and motivation and enjoyment domains. No significant associations between the Pre-PLAy and PDMS-2 were found for males (see Table 5).

The relationship between the Pre-PLAy and PA was moderated by gender. Gender moderated the ability of the Pre-PLAy to predict minutes of activity per hour, except coordinated movements (see Table 6). Gender also moderated the relationship between motivation and minutes of MVPA per hour. Simple slopes analyses were conducted to probe the interactions. Results for males and females were plotted at 1 standard deviation (SD) below and above the mean in Figures 2, 3. Interactions showed the Pre-PLAy predicted minutes per hour of PA for girls (B = 0.06, SE = 0.03, z = 2.27, p = 0. 02) and was not significant for boys (B = −0.03, SE = 0.03, z = −0.85, p = 0.40). The movement competency domain of the Pre-PLAy approached significance for females (B = 0.06, SE = 0.03, z = 1.76, p = 0.08), and the motivation domain significantly predicted Pre-PLAy scores for females (B = 0.73, SE = 0.18, z = 4.00, p = 0.0001). Neither domain significantly predicted Pre-PLAy in males (B = −0.04, SE = 0.04, z = −0.99, p = 0.32; B = 0.05, SE = 0.18, z = 0.26, p = 0.80). A similar pattern was found between motivation and MVPA with significant results for females (B = 0.32, SE = 0.09, z = 3.39, p = 0.0007) and non-significant results for males (B = 0.04, SE = 0.09 z = 0.48, p = 0.63).

Figure 2

Figure 3

Discussion

This is the first study that has tested a PL assessment instrument in preschool children. Results of the study suggest promise for the Pre-PLAy tool, but also areas in need of modification. In terms of promise, for females, the tool demonstrated adequate reliability, and there was evidence of validity in agreement with a standardized measure of gross motor skills and objectively measured PA. In contrast, results for males were poor, with many items demonstrating poor reliability, and no significant agreement with gross motor skills and PA.

The discrepant results for boys and girls indicate a potential sex bias in ECE ratings. While there is a dearth of research on this in the early years setting, prior research with teachers ratings of school-aged children have shown sex biases related to motor competence. For example, Hay and Donnelly (20) found teachers overestimated boys motor abilities and underestimated girls. There is also evidence that the sex of the teacher can impact ratings of child development (21), however the reason for these sex differences has not been examined. Therefore, our results might suggest ECEs hold different beliefs regarding the motor skills of males and females in this developmental period. These beliefs might vary based on cultural differences, years in the field, and whether the ECE is male or female. In the current study, all ECEs were female so rater bias attributable to sex of the evaluator is not an issue. Unfortunately, we did not collect additional information about years of training or experience of ECEs in our sample. The beliefs and perceptions of ECEs about motor skill is an important, and understudied area of research.

Another factor that can help explain the lack of agreement for males is the presence of disruptive behaviors. Rivard et al. (22) found the presence of disruptive behaviors suppressed teacher concerns about motor skill in school aged children. Disruptive behaviors are most prevalent, and are in fact normative during the preschool years, and higher among males compared to females (23, 24). Therefore, ECEs might be less accurate in assessing a child's PL if the child engages in more disruptive behaviors, and this is more likely to occur among male preschool children.

These results suggest the need for further training with ECEs to understand and address sex differences in Pre-PLAy ratings. Training on the tool focused on ensuring ECEs understood the instrument, and discussion around different opportunities to observe children during regular programming to complete the tool. Ensuring there are opportunities to use the tool with both male and female children, and to discuss and address any discrepancies will be an important area of development for training on the tool.

Item level results suggest additional revisions could improve the tool. For example, when assessing movement competencies, ECEs did not rate any child in the creatively combines skills category. If this is replicated in future studies with larger samples, it could be beneficial to remove this category. The high proportion of missing data on the uses playground equipment item suggests modifying the tool to either revise the item to include other active play equipment in centers or making the item optional in scoring the tool would be useful. The high correlations between upper and lower body skills suggests ECEs did not distinguish between the two items. Specific training focusing on upper and lower body sending and receiving might help ECEs better distinguish between these items, though this needs to be tested in future research.

Internal consistency, corrected item-total, and inter-item correlations were supportive of the multidimensional structure of the Pre-PLAy. Most items fit well on their respective scales with the exception of item 11 (uses a variety of moving vehicles), which item did not fit with the other coordinated movement items based on item total correlations and inter-item correlations. Removal of this item would also improve the alpha of the scale. Results suggest either modifying or removal of the coordinated movements scale might be warranted. The coordinated movements scale demonstrated inadequate inter-rater agreement for males and females and the lowest intra-rater agreement of all the Pre-PLAy scales. It also did not appear to contribute to the validity of the tool with no significant relationship with the PDMS-2 or PA. However, considering limitations of the current study (including potential gender bias in the study and the small sample size), further testing of the scale in a larger sample might be warranted before removal.

We conducted a real-world test of the Pre-PLAy tool with ECEs completing the Pre-PLAy tool alongside their regular programming and responsibilities. Results of the study are important for both future work in preschool PL, and also for research in the early years more broadly. While there is a body of research on a number of observational instruments for teachers and school-aged children, there has been little examination of similar types of tool for ECEs and preschool children. Results suggest further research on the Pre-PLAy tool is warranted.

References

• 1.

  DudleyDA. A conceptual model of observed physical literacy. J Phys Educ. (2015) 72:236–60. 10.18666/TPE-2015-V72-I5-6020

  • CrossRef
  • Google Scholar

• 2.

  EdwardsLCBryantASKeeganRJMorganKJonesAM. Definitions, foundations and associations of physical literacy: a systematic review. J Sports Med. (2017) 47:113–26. 10.1007/s40279-016-0560-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  LongmuirPEBoyerCLloydMYangYBoiarskaiaEZhuWTremblayMS. The Canadian assessment of physical literacy: methods for children in grades 4 to 6 (8 to 12 years). BMC Public Health (2015) 15:767. 10.1186/s12889-015-2106-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  MooreLLGaoDBradleeMLCupplesLASundarajan-RamamurtiAProctorMHet al. Does early physical activity predict body fat change throughout childhood?Prev Med. (2003) 37:10–7. 10.1016/S0091-7435(03)00048-3

  • CrossRef
  • Google Scholar

• 5.

  ReillyJJJacksonDMMontgomeryCKellyLASlaterCGrantS. Total energy expenditure and physical activity in young Scottish children: mixed longitudinal study. Lancet (2004) 363:211–2. 10.1016/S0140-6736(03)15331-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  OgdenCLCarrollMDLawmanHGFryarCDKruszon-MoranDKitBKFlegalKM. Trends in obesity prevalence among children and adolescents in the United States, 1988-1994 through 2013-2014. JAMA. (2016) 315:2292–9. 10.1001/jama.2016.6361

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  CanadianSport for Life. Physical Literacy Assessment for Youth. Canadian Sport Institute. Victoria, BC (2013).

  • Google Scholar

• 8.

  CairneyJBedardCDudleyDKriellaarsD. Towards a physical literacy framework to guide the design, implementation and evaluation of early childhood movement-based interventions targeting cognitive development. Ann Sports Med Res. (2016) 3:10731–5.

  • Google Scholar

• 9.

  WhiteheadM. The concept of physical literacy. Eur Phys Educ Rev. (2001) 6:127–38. 10.1080/1740898010060205

  • CrossRef
  • Google Scholar

• 10.

  WhiteheadMMurdochE. Physical literacy and physical education: conceptual mapping. Phys. Educ. Matters (2006) 1:6–9.

  • Google Scholar

• 11.

  HarterS. Causes, correlates, and the functional role of global self-worth: a life-span perspective. In: SternbergRJKolligianJ editors. Competence Considered.New Haven, CT: Yale University Press (1990), 67–97.

  • Google Scholar

• 12.

  GiblinSCollinsDButtonC. Physical literacy: Importance, assessment and future discussions. Sports Med. (2014) 44:1177–84. 10.1007/s40279-014-0205-7

  • CrossRef
  • Google Scholar

• 13.

  DudleyDCairneyJWainwrightNKriellaarsDMitchellD. Critical considerations for physical literacy policy in public health, recreation, sport, and education agencies. Quest (2017) 69:436–52. 10.1080/00336297.2016.1268967

  • CrossRef
  • Google Scholar

• 14.

  FisherAReillyJJKellyLAMontgomeryCWilliamsonAPatonJY. Fundamental movement skills and habitual physical activity in young children. Med Sci Sports Exerc. (2005) 37:684–8. 10.1249/01.MSS.0000159138.48107.7D

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 15.

  FigueroaRAnR. Motor skill competence and physical activity in preschoolers: a review. Matern Child Health J. (2017) 21:136–46. 10.1007/s10995-016-2102-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  PateRRAlmeidaMJMcIverKLPfeifferKADowdaM. Validation and calibration of an accelerometer in preschool children. Obesity (2006) 14:2000–6. 10.1038/oby.2006.234

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  CicchettiDV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. (1994) 6:284. 10.1037/1040-3590.6.4.284

  • CrossRef
  • Google Scholar

• 18.

  PinheiroJBatesDDebRoySSarkarDHeisterkampSVan WilligenB. Linear and Nonlinear Mixed Effects Models. R package version 3.1-117. Available online at http://CRAN.R-project.org/package=nlme (Accessed June 10, 2017).

  • Google Scholar

• 19.

  PreacherKJCurranPJBauerDJ. Computational tools for probing interaction effects in multiple linear regression, multilevel modeling, and latent curve analysis. J Educ Behav Stat. (2006) 31:437–448. 10.3102/10769986031004437

  • CrossRef
  • Google Scholar

• 20.

  HayJDonnellyP. Sorting out the boys from the girls: teacher and student perceptions of student physical ability. Avante (1996) 2:36–52.

  • Google Scholar

• 21.

  SideridisGDAntoniouFPadeliaduS. Teacher biases in the identification of learning disabilities: an application of the logistic multilevel model. Learn Disabil Q. (2008) 31:199–209.

  • Google Scholar

• 22.

  RivardLMMissiunaCHannaSWishartL. Understanding teachers' perceptions of the motor difficulties of children with developmental coordination disorder (DCD). Br J Educ Psychol. (2007) 77:633–48. 10.1348/000709906X159879

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  BaillargeonRHZoccolilloMKeenanKCôtéSPérusseDWuHX. Gender differences in physical aggression: A prospective population-based survey of children before and after 2 years of age. Dev Psychol. (2007) 43:13. 10.1037/0012-1649.43.1.13

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  TremblayRE. Developmental origins of disruptive behaviour problems: the ‘original sin’ hypothesis, epigenetics and their consequences for prevention. J Child Psychol Psychiatry (2010) 51:341–67. 10.1111/j.1469-7610.2010.02211.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar