“A Piece of Sanity in the Midst of Insane Times”: Girls on the Run Programming to Promote Physical Activity and Psychosocial Well-Being During the COVID-19 Pandemic

Afterschool programs have the potential to promote social, emotional, and physical health outcomes among youth participants. The positive youth development (PYD) framework argues that acquiring desirable attitudes and behaviors occurs when skill-building opportunities are explicitly provided within a safe and supportive climate guided by caring, competent, and compassionate instructors. Girls on the Run (GOTR) is a PYD program that uses running, motor skills, and other physical activities as a platform for promoting positive psychosocial outcomes and life skills learning among elementary- and middle school-aged girls. The onset of the COVID-19 pandemic challenged GOTR to modify lessons, coach training, and program delivery (in-person, virtual, or hybrid) to accommodate public health guidelines. The purpose of this study was to assess caregivers' and coaches' perceptions of program effectiveness in light of these changes. Following the Fall 2020 season, caregivers (n = 1,617) and coaches (n = 991) from 1,077 teams and 39 councils completed an online survey about program experiences. Both stakeholder groups positively rated program impact regardless of delivery mode, although in-person mode was rated higher for satisfaction with the end-of-season event. Thematic analysis of open-ended responses revealed that caregivers and coaches identified increased physical activity opportunities and life skills learning as well as improved social, psychological, and emotional development as a result of participating. Both stakeholders noted GOTR provided a sense of normalcy during this time of great need. Findings using mixed methods provide evidence of program effectiveness and recommendations for youth programming during challenging times.


INTRODUCTION
Girls on the Run (www.girlsontherun.org) is a physical activitybased positive youth development (PA-PYD) program using running, motor skills, and physical activities as a platform for promoting psychosocial development and life skills among girls 8-11 (grades 3-5) and 12-14 years (grades [6][7][8]. The program typically serves 200,000 girls annually and engages 50,000 coaches (98% female). The intentional curriculum and systematic coach training are aligned with best practices for PYD programs, including opportunities for skill building, a safe physical and psychological space, appropriate structure, supportive relationships, and feelings of belonging (1,2).
The program adopts Lerner's Five Cs framework to guide lessons comprising the life skills curriculum (3). Lessons are designed to help girls develop social, emotional, and physical competence, feel confident in who they are, create positive connections with peers and adults, develop strength of character, and respond to others and self with care and compassion. A sixth C reflects contribution to the greater good through implementing a community service project. The curriculum for 3rd−5th grade girls focuses on three themes: identity (self-care and selfawareness), connectedness (selecting and maintaining healthy relationships), and empowerment (celebrating and sharing one's strengths), while the 6th−8th grade curriculum focuses on lessons related to self, team, and community. Sessions also prepare girls to complete the culminating 5 K, by setting goals, regulating progress, and participating in a practice 5 K to build confidence. To attain program goals, girls and coaches meet twice a week for 75-90 min over a 10-week season. Coaches are trained to deliver the curriculum with fidelity, along with an emphasis on building relationships, creating a positive, inclusive environment, and emphasizing a mastery climate. More information about GOTR is available in other sources [e.g., (4)(5)(6)].
Evidence of program effectiveness shows GOTR's impact on physical activity, life skills, and holistic health [e.g., (7)(8)(9)(10)]. Pre-to post-season improvements emerged for activity levels, self-esteem, physical competence, and peer support; these were retained 3 months after season's end. Life skills transfer for managing emotions, helping others, resolving conflicts, and decision-making favorably compared to girls in organized sport and school physical education. Qualitative data from girls, caregivers, coaches, and school personnel indicated that girls improved in social and emotional behaviors and physical activity motivation as a result of participating in GOTR.
The onset of the COVID-19 pandemic challenged GOTR to modify program delivery to accommodate public health guidelines, while continuing to promote PYD in a time of great need. The curriculum was shortened from 20 to 16 lessons and delivery options included in-person, virtual, or hybrid. Health and safety policies were adopted utilizing CDC guidance for all modes. When in person, coaches delivered pre-COVID lessons with modifications for physical distancing (e.g., "air" high fives, shadow tag) and the use of new journals outlined in a curriculum addendum. Girls also used the journals in the virtual space. If a team was 100% virtual or needed to transition to the virtual space (i.e., hybrid), coaches used the newly developed Virtual Lesson Curriculum, which contained lessons that mirrored the in-person learning objectives. Virtual lessons were shortened to 45-60 min plus a 20-30-min independent workout (modified from 75-90 min in-person lessons), and a variety of physical activities were provided to be inclusive and allow for space constraints (modified from in-person workouts that primarily included running and other locomotor skills). The workout typically completed in person was transitioned to a "separate but together workout" that girls started together at the end of the lesson and continued on their own. Coach training, typically delivered in-person (4.5 h) and online (1 h), was moved to completely online. Three training modules were added: impact of COVID-19, coaching virtually, and coaching for social inclusion. The end-of-season 5 K, typically a large community celebration, became a "K Your Way" event.
Consistent with GOTR's commitment to ongoing evaluation for improving curricula and program delivery, the organization developed an online survey to seek caregivers' and coaches' perceptions of program effectiveness in light of COVID-19 protocol changes. Based on utilization-focused evaluation, Patton (11) accentuates bridging empirical research and practical implications of evaluation findings. He suggests that evaluations provide answers to three questions: (1) What information emerges about attitudes, skills, and behaviors? (2) So What do findings imply about program effectiveness? (3) Now What recommendations can be made for making program improvements? The purpose of this study was to answer these questions based on stakeholders' perceptions of COVID-19 safety protocol changes to program delivery.

METHOD
Following the Fall 2020 season, all GOTR councils (∼200) were invited to evaluate the curricular and delivery modifications due to COVID-19 restrictions. Thirty-nine councils elected to participate, consisting of 1,077 teams in all regions of the U.S. GOTR national headquarters distributed an online survey to all caregivers and coaches within the 39 councils.

Survey Questions
For caregivers, 5 statements probed level of agreement on the GOTR experience (Table 1), ranging from strongly disagree (1) to strongly agree (5). Another question sought degree of satisfaction with the end-of-season event, ranging from completely dissatisfied (1) to completely satisfied (5). They also rated "how true" a statement was ("my girl feels lonely", "my girl is confident", "my girl is physically active") before and after participation in GOTR, ranging from not true at all (1) to really true (5). Coaches responded to 8 statements probing level of agreement about their GOTR experience and 3 questions about how satisfied they were with the COVID-19 safety protocols, end-of-season event, and communication from GOTR ( Table 2). One open ended question for each stakeholder was selected for qualitative analysis based on relevance to the COVID-19 protocol changes. For caregivers, "Share one way that your girl has been positively impacted by her Girls on the Run experience, such as something she learned, a favorite moment or activity, or a lasting takeaway." For coaches, "This is a challenging time for many girls. What do you think was the greatest need met through the program during this time?"

Data Analysis
Quantitative responses were analyzed using analysis of variance to determine delivery mode differences. We applied a Bonferroni adjustment to avoid Type 1 errors (12). For caregivers, statistical significance was determined using p < 0.006 (0.05/9 items) and for coaches using p < 0.005 (0.05/11 items). Student-Newman-Keuls post-hoc tests were used in the event of statistically significant differences. Effect size (ES) was calculated for statistically significant differences using Cohen's d (13): d ≥ 0.20 = small, ≥0.50 = medium, ≥0.80 = large.
An inductive content analysis was adopted for the qualitative responses (14). One researcher initially coded data units (words, phrases, sentences) and lower-order themes for half the sample of caregivers (n = 780) and coaches (n = 458) due to early saturation of responses. A second researcher randomly selected 100 responses for each stakeholder and independently coded data units and lower-order themes. The researchers then met to discuss convergence and divergence of findings and came to consensus. Together they derived a set of higher-order themes that emerged from the lower-order themes and data units (14).

Caregivers: Quantitative Responses
Responses ( Table 1) fell between agree and strongly agree regardless of delivery mode (virtual, in-person, hybrid). Caregivers favorably viewed the COVID-19 safety precautions in the in-person and hybrid modes. Analysis revealed no differences for: (a) gained skills for handling stress, (b) participating led to at least one conversation between caregiver and girl about an important topic and (c) COVID-19 precautions gave confidence that GOTR was striving to create Caregivers reported the location of their end-of-season event as in-person at site (72.3%), virtual (22.0%) or other (5.7%). Satisfaction ratings with the event fell between somewhat and completely satisfied for all delivery modes, but caregivers of girls who participated in virtual sessions scored significantly lower than girls who experienced in-person or hybrid lessons. ES's were small.
For the before and after questions -"my girl felt/feels lonely", "I would describe my girl as confident", and "my girl was/is physically active"-scores improved for all three delivery modes. Caregivers reported girls as less lonely, more confident, and more physically active. No differences in change scores emerged by delivery mode.

Coaches: Quantitative Responses
Responses ( Table 2) fell between agree and strongly agree regardless of program delivery. Coaches reported positive experiences across the board, including high ratings for "coach support was available to me throughout the season" and "I had the support I needed during COVID times." Coaches favorably viewed COVID-19 safety protocols for in-person and hybrid modes. No differences emerged for: (a) received sufficient training to effectively implement the program (b) as a GOTR coach, I felt like I was making a difference in girls' lives, and (c) because of participating in Girls on the Run, girls on my team are more confident.
In-person delivery received more favorable scores than virtual delivery for: I formed positive relationships with girls on my team and girls developed positive relationships with teammates; these items were also higher for hybrid vs. virtual. All ES's were small. When asked if coaching GOTR has been a valuable experience, coaches in the hybrid mode scored significantly lower than virtual and in-person modes, but ES's were small.
Coaches reported location of the end-of-season event as in-person at site (62.5%), virtual (28.7%) or other (8.9%). Satisfaction ratings with the event were significantly higher for coaches who delivered the program in-person or hybrid than for virtual mode. ES's were small (virtual vs. hybrid) and medium (virtual vs. in-person). No differences emerged between delivery modes for satisfaction with safety protocols and support and communication from GOTR.

Caregivers: Qualitative Themes
Coding responses to, "Share one way that your girl has been positively impacted by her GOTR experiences . . . " resulted in 9 higher-order themes derived from 15 lower-order themes and hundreds of data units. Table 3 depicts the thematic analysis.
Improved Physical Activity Motivation and Behavior was evident in data units revealing opportunities for being physically active with teammates and looking forward to running and training for the 5 K. As one parent said, "My daughter has grown to love running and wants to run with me now. She sees herself as a runner and this is a positive identity that she will carry with her into the future." Another parent shared, "My daughter did not like to do any physical activity prior to this program. After GOTR, she would come home in such a positive mood and feeling better about herself. Now she is pushing me to incorporate exercise into our lives and strives to be a healthy person." Social Development was characterized by opportunities to develop and strengthen friendships, socially interact with peers, and experience group belongingness-various ways that allowed girls to share common bonds with others their age. One parent offered, "This was a wonderful way for Sadie 1 to get reconnected with kids her age and have some meaningful experiences in the midst of this COVID chaos. Thank you for providing a safe and supportive way for her to be with other kids and continue working on her running skills." This parent gave a more vivid example, "My daughter became motivated and dedicated to completing her own physical fitness routine because of the positive energy and camaraderie generated by her GOTR coaches and team. She felt she was part of something important, a little bit more connected to others and appreciated as a person." Psychological Growth was defined by themes of achieving goals and gaining confidence. Caregivers attested to girls setting goals and developing strategies to successfully complete them (e.g., 5 K). Others highlighted how confidence was enhanced through activities and lessons in GOTR. One parent shared, "My daughter was really struggling and feeling alone during COVID. She lost her motivation and happy spirit . . . After just one practice she was happy again. She was eager to set goals and achieve them. As she achieved goals her confidence grew. This program was even more important this year!" Emotional Growth entailed themes of being able to express feelings, adopting a positive outlook, and feeling proud of accomplishments. Responses alluded to becoming more open with communicating, engaging in positive talk, and feeling a sense of pride in completing the 5 K. A parent shared, "She was really struggling with COVID and not being able to be with her friends . . . when she started GOTR back up she was excited, she looked forward to each session, it brought her back to life and her outgoing self again." Another offered, "GOTR gave her peers to interact with . . . she was able to share her thoughts on the lesson without hesitation. GOTR has always been a positive, uplifting experience where she feels accepted." Life Skills Development was prominent, characterized by learning life lessons and helping others. Caregivers confirmed that girls acquired specific "tools" (Take a Breather) and learned important skills (e.g., choosing good friends) that transferred to 1 All names are pseudonyms.    Having a social and active outlet during pandemic GOTR experience helped her feel much more connected to the school community at a time of stress and loneliness many areas of girls' lives. A parent shared, "The lessons have been so helpful for Pria. We have gone back to lessons again and again and generalized for everyday life. Positive self-talk and Take a Breather helped us get through things like the flu shot and a medical procedure that was pretty scary. As a child with anxiety . . . having those tools in her toolkit helped us more than I can express in words." Helping Others was a theme reinforcing lessons that emphasize standing up for others (being a "Stand-Byer") and giving back to community and society. Developing empathy and a sense of civic engagement is accentuated in the curriculum through completion of a community service project. Two higher-order themes revolved around meaningful adult relationships, Positive Coaching Influence and Closer Family Connections. Caregivers were uniformly praiseworthy of their girls' coaches as positive role models, sources of encouragement, and fostering a safe and inclusive climate. In addition, several caregivers commented on how girls' involvement in GOTR fostered opportunities for being physically active as a family and having conversations related to lesson themes. Having Fun was a theme that reflected the sheer enjoyment of running, being with peers, and experiencing the overall program.
A Sense of Normalcy during the Pandemic emerged as directly relevant to the COVID-19 pandemic. Caregivers praised the value of the program in light of virtual school and other social activities being canceled. One parent summarized concisely, "She appreciated the time to do something NORMAL in this notnormal year." Another shared, "My daughter was new to her school and with it being virtual, she hadn't met anyone. Her GOTR experience helped her feel much more connected to the school community at a time of stress and loneliness."

Coaches: Qualitative Themes
Coding responses to, ". . . What do you think was the greatest need met through the program during this time" resulted in 7 higher-order themes derived from 9 lower-order themes and hundreds of data units. Table 4 depicts the thematic analysis.
Some themes were similar to those of caregivers, revolving around opportunities to socialize with peers, be physically active, build confidence, learn life lessons, be surrounded by a positive social environment, and feel a sense of normalcy. One coach nicely summarized how GOTR met girls' needs during this challenging time: "Giving girls the opportunity to get out of the house and develop strong habits/practices in exercise, mental health, confidence, and relationships." Numerous responses exemplified the theme, Increased Physical Activity, including opportunities to be physically active by getting outside, participating in sports and activities with friends, and as one coach implored, "Encouraged them to get up and MOVE!" Another shared, "Many parents told us that before GOTR, their girls were not getting any physical activity. Not only were they active during the lesson, but it continued at home too." As a PA-PYD program, GOTR is unique in emphasizing physical activity as a healthy behavior through opportunities to enjoyably experience running, motor skills, and movement activities that are continued outside of the structured program.
Social Development entailed themes of socializing with peers and feeling part of a group. Coaches highlighted meaningful interactions and relationships as a crucial need that was achieved through activities promoting teamwork, unity, connectedness, and companionship. One coach shared, "The program provided the girls an opportunity to meet and interact with other girls and school staff (coaches). This gave the girls a much-needed opportunity to socialize with peers/adults during the difficult time caused by COVID (school closures, social distancing)." Another added, "I think social interaction was the greatest need. Girls had a platform to share in their joys, excitement, triumphs and hardships. My girls made valuable connections and even exchanged numbers to keep in touch with one another." Psychological Growth included lower-order themes of coping with stress and building confidence. Coaches elaborated on how GOTR met these needs, "It gave the girls a sense of calm in all this chaos", "Having lessons on emotions bridges gaps with how they are feeling and how they can express it", and "They just needed someone besides people in their immediate households to talk with and to explore the world around them with different people so that they weren't stuck stressing about school. They got the opportunity to put some stress aside to have fun and to become a stronger and more confident girl."   Higher-order theme: sense of normalcy during the pandemic Sense of normalcy during the pandemic A piece of sanity in the midst of insane times Feeling a sense of normalcy and belonging Surrounded by their peers gave them some normalcy Gave the girls a sense of calm in all this chaos Focusing on growth and positivity during such a limiting and negative time was wonderful for them Great way to get girls moving during this pandemic where you have to spend a majority of time indoors Giving them something to look forward to during this time For many, this was their first time getting together with people outside of their family since before COVID Ability to have a routine Provided girls with stability during uncertain times Chance to be with other girls and stay active during this period of isolation Giving the girls something to look forward to twice a week Consistency-allow girls to continue participating in a meaningful program they enjoy and look forward to. Giving the girl something positive to do during this hard time The girls were able to be "normal" during our time together. Getting back to something somewhat normal! A feeling of normalcy-when so much has been canceled including almost all clubs, GOTR was there. A distraction from the pandemic Life Skills Development was characterized by learning life lessons, such as adopting a positive mindset, managing emotions, developing social skills, and helping others. One coach explained, ". . . the opportunity to develop social skills (including sharing, cooperation, overcoming shyness, productive disagreement and conflict resolution), given that many aren't in school." Another said, "Many of the girls developed skills to help them see their strengths, push themselves to reach their goal, and how to support each other." One elaborated: "The girls were able to build a toolbox of skills through the lessons that helped them in this current odd environment . . . The girls learned that other girls felt the same way about life/the world. There was safety in knowing they were not alone." Positive Social Environment was exemplified by having a safe and supportive space for engaging in activities and interacting with peers. A coach stated, "GOTR gave the girls something to look forward to and people to interact with. I think most of them were lonely and hadn't seen any peers in a while. GOTR gave them a space to talk about their lives and how they've been feeling." Other coaches stated, "This made connection possible outside of the household. Positive, meaningful, and real connection that kept safety the top priority", and "The program was able to help with providing a safe place for girls to connect while they are experiencing isolation due to the pandemic . . . the content was encouraging and helped to keep them positive and growing during a challenging time." The higher-order theme, Having Fun, also reflected a positive and supportive climateenjoying activities with peers and fun games to take their mind off things (i.e., COVID).
Sense of Normalcy during the Pandemic depicted a theme specific to meeting girls' needs during COVID-19. One coach succinctly characterized GOTR's role, "A piece of sanity in the midst of insane times." Another exclaimed, "Getting back to something somewhat normal! For the time we were together, besides masks, distancing and sanitizer, it felt good to do something other than be on the computer at home all day." Another stated, "Normalcy. So many of our girls had things interrupted over the last 8 months and to bring the program back at our school was so helpful and nice to see them involved in an after-school activity".

DISCUSSION
Utilization-focused evaluation research is a means for bridging evidence-based findings with practical implications (11). It begins with the premise, ". . . evaluations should be judged by their utility and actual use . . . how real people in the real world apply evaluation findings" (p. 4). Patton states that evaluation research should answer three questions: What . . . changes in attitudes, skills, and behaviors occur in participants? So What . . . do the findings imply about the degree to which the program is considered a success? Now What . . . recommendations flow from the findings? We systematically address study findings relative to these questions based on caregivers' and coaches' perceptions of safety protocol changes to curricula and delivery mode.
What changes in girls' attitudes, skills, and behaviors occurred by participating in GOTR? Quantitative responses by caregivers and coaches were favorable (e.g., my girl learned skills to handle stress associated with the pandemic), regardless of the curriculum delivered in-person, virtually, or hybrid. Caregivers reported girls as less lonely, more confident, and more physically active at season's end-for all modes. Qualitative themes provided detail for how girls were impacted by GOTR (caregivers) and what greatest need was met by participating during this challenging time (coaches). Based on these data, GOTR provided a means of sustaining physical activity, strengthening friendships, achieving goals, and building confidence. Girls acquired ability to express feelings, maintain optimism, and take pride in completing the 5 K. Evidence of season-long improvement in physical and psychosocial outcomes align with the mission, vision, and core values of the program (5) as well as longitudinal findings during "normal" program delivery (9,10).
Qualitative data also revealed that learning life skills and feeling a sense of normalcy were central to countering the challenges girls faced, by acquiring skills to cope with stress, think positively, and develop relationships (6). Stakeholder groups credited GOTR with teaching life lessons that informed intentional decision-making (positive self-talk, choosing friends), managing negative emotions, and adopting healthy behaviors to navigate many areas of girls' lives during the pandemic. Ability to generalize lessons learned in GOTR to domains outside the program-school, family, peers-is a signature of PYD programs (2,6).
So What . . . do findings imply about the degree to which GOTR was effective? Mixed methods revealed improvement in girls' physical activity and psychosocial attitudes and behaviors that infer program success using varied delivery modes and safety precautions. These promising outcomes occurred within a safe and supportive social environment that included encouraging coaches and meaningful peer interactions. "Social distancing" was not equivalent to reduced opportunities for socializing with peers, developing social skills, and experiencing a sense of community-innovative ways of delivering the program were successful in promoting PYD. Peer acceptance and close friendships are critical needs for youth in childhood and adolescence (15,16), which was achieved through GOTR's quality programming and coach training. Building positive relationships is a key element of coach training, attained through activities that highlight cooperation, inclusion, support, and a mastery climate (5,17). As a PA-PYD program challenged with modifying curricular lessons to accommodate safety precautions, GOTR was successful in providing an environment that made a positive impact on girls' physical activity and psychosocial and emotional growth and development.
Now What . . . recommendations can be made for program improvements? While favorable ratings emerged for all delivery modes, the virtual experience was rated lower for a few items (albeit small effect sizes). This included, "GOTR has been a valuable experience for my girl", "I formed positive relationships with girls on my team", and "girls on my team developed positive relationships with teammates." Satisfaction with a virtual end-of-season event was rated lowest. GOTR used the findings to recommend areas for improving the Spring 2021 season. First, councils were encouraged to conduct the end-of-season 5 K in person and at site-based locations if possible. Second, to optimize developing positive relationships, virtual session length was extended from 45 to 60 min and ideas were added to the virtual curriculum for coaches to make more informal connections. Third, a team workout option was added to the virtual curriculum (girls and coaches complete it together), which increased time for connection and motivation for physical activity.
Despite strengths of this study (e.g., mixed methods, multiple stakeholders), we note some limitations. First, 39 councils agreed to participate and, although large samples of caregivers and coaches from over 1,000 teams completed the survey, it is unknown how those in other councils viewed the program experience. Second, the sample was not as diverse as the makeup of the broader organization. Survey responses represented 25% girls of color, whereas the network composition for Fall 2020 was 37%. GOTR remains committed to gathering insights from individuals of diverse sociocultural backgrounds and is pursuing strategies to ensure representation. Third, to maximize return rate the survey length was kept reasonable, which did not allow for more quantitative items. The open-ended questions provided rich data to complement ratings and lent perspective on areas of holistic health and well-being attained through participating.
In conclusion, GOTR resourcefully and effectively applied COVID-19 safety precautions with in-person, virtual, and hybrid modes. All modes were received favorably and open-ended narrative revealed the breadth and depth of program impact. Evaluation findings provide GOTR and other youth programs with critical information for improving curricula, coach training, and program delivery while maintaining high safety protocols during a challenging health crisis.

DATA AVAILABILITY STATEMENT
The datasets presented in this article are not readily available because the dataset is proprietary to Girls on the Run International. Requests about the data should be directed to Maureen Weiss, mrweiss@umn.edu.

ETHICS STATEMENT
The project was reviewed by Research Integrity and Compliance (RIC), Texas State University, San Marcos, TX, USA. Because the study exclusively involved the examination of anonymous, secondary data, the research is not regulated by RIC, and written informed consent for participation was not required.

AUTHOR CONTRIBUTIONS
MW and AR conceived the study. AR collected the data. MW and LK drafted the earlier versions of the manuscript and conducted the qualitative data analysis. LK conducted the quantitative data analysis. All authors contributed to revising the manuscript and approved the final submitted version.