The potential usefulness of standardized assessments to measure participant outcomes of adaptive/therapeutic horseback riding: a survey study

Adaptive or therapeutic riding (A/TR) is a recreational activity which provides mounted and ground-based horsemanship opportunities adapted to the abilities of the participants. A/TR provides physical and psychological benefits to participants with diverse disabilities, including physical, developmental, cognitive, and age-related disabilities, promoting higher quality of life. A/TR professionals may be limited in their capacity to implement outcome assessments and report the benefits of their community-based A/TR services to a broad audience. The purpose of this study was to identify whether and how A/TR professionals currently measure participant outcomes; benefits and barriers to implementing standardized assessments in A/TR; and characteristics which would make assessments useful in the community-based A/TR environment. To address this purpose, we conducted a survey among A/TR professionals. We found that while A/TR professionals measure outcomes among their participants, they typically do not use standardized assessments. Survey respondents believed benefits of implementing standardized assessments included bolstering the A/TR profession, acquiring funding, and communicating about A/TR services to a broad audience. Respondents also identified several barriers to implementing standardized assessments including time, systemic, and expertise constraints. Respondents reported that useful standardized assessments would be relevant to all age groups and populations who receive A/TR services. Finally, respondents shared that for standardized assessments to be useful, they would need to be low-cost, require less than 10–20 min, and available in either paper or computer format. This study revealed that standardized assessments may be a strong support to the A/TR profession; however, assessments must meet the unique needs of A/TR professionals.


Introduction
Adaptive riding, or therapeutic riding (A/TR), is a recreational activity in which horseback riding is adapted to the needs of participants with diverse abilities and diagnoses (1).A/TR typically occurs in a group setting, with a therapeutic riding instructor teaching skills that address horsemanship goals, allowing diverse populations to access the natural benefits of horsemanship (1).A/TR provides an engaging recreational activity for people with diverse disabilities, such as individuals with autism spectrum disorder or cerebral palsy, veterans with post-traumatic stress disorder, and older adults with dementia (2).Furthermore, A/TR has been proposed to improve a variety of outcomes, including: self-confidence, motivation, courage, social involvement, self-perceived physical competency, and gross motor function (3)(4)(5)(6).There are different levels of certifications available to therapeutic riding instructors; certified therapeutic riding instructors (CTRIs) are certified for entry-level riding instruction, while advanced (ATRIs) and master (MTRIs) instructors can provide intermediate/advanced instruction and program management, and MTRIs have even further expertise in equine-assisted services broadly.
Many researchers have identified broad benefits of participating in A/TR (3)(4)(5)(6); however, therapeutic riding instructors and A/TR centers are limited in their capacity to gather and report the benefits of their community-based services.The field of human-animal interaction research has often identified the need for increased use of valid and reliable outcomes assessments that are used consistently across studies, in order to advance the field (7).This same need for valid, reliable, and consistent outcomes measures exists in community-based practice, but the needs of community-based professionals differ from the needs of researchers.The assessments often used in A/TR research may not be useful in a community-based A/TR setting.For example, many assessments used in research require administrators to have advanced training in standardized assessments (8), which is not required to become a CTRI/ATRI/MTRI.Additionally, some assessments used in A/TR research are validated for a specific age range or diagnosis, whereas A/TR centers often serve many different populations concurrently.Finally, many of the assessments used in research are costly and may not be sustainable for long-term use in the A/TR context.Due to these limitations, A/TR professionals currently do not have standardized assessments that can be feasibly used in their community-based context.Therefore, there is a need to identify high-quality standardized assessments which A/TR professionals can use to measure participant outcomes of their services; identifying such assessments could bolster the profession's credibility and facilitate improved communication about the benefit of A/TR services to participants, funders, and the community.
A key consideration when identifying standardized assessments to be used in community-based services is the clinical usefulness of the assessment (9).Clinical usefulness is defined as whether an assessment improves the quality of services, is acceptable to administrators and participants, can improve the quality of services, and is worth the cost of its use.The purpose of the current study was to gather feedback from A/TR professionals pertaining to elements of standardized assessments which make them clinically useful to measure participant outcomes of community-based A/TR.To achieve this aim, we asked the following research questions: 1. How are CTRIs/ATRIs/MTRIs currently measuring participant outcomes, if at all? 2. Do A/TR interested parties believe it is important to identify standardized assessments to measure participant outcomes of A/TR?If so, what participant outcomes do A/TR interested parties believe are most important to assess and most likely to change as a result of A/TR participation? 3. What do A/TR interested parties believe would be benefits of, and barriers to, implementing standardized assessments to measure participant outcomes of TR? 4. What qualities of standardized assessments would be most useful in the community-based A/TR setting (e.g., for what populations is the assessment validated, frequency of assessment, assessment length and cost, etc.)?

Study design and participants
The Temple Grandin Equine Center (TGEC) at Colorado State University (CSU) and the Professional Association of Therapeutic Horsemanship, International (PATH, Intl.) formed a working group of A/TR interested parties.The working group was developed and met prior to this study.The ongoing work group consists of 11 members, selected because they represent different perspectives related to A/TR, including CTRIs, leadership from PATH, Intl.centers, researchers, and representatives from: TGEC, PATH Intl, the American Hippotherapy Association Inc., and the Horses and Humans Research Foundation.This work group provided feedback that greatly influenced the methods presented below.
This study was approved by the CSU Institutional Review Board (#3229) and a study survey was distributed to A/TR interested parties.To be eligible to complete the survey, respondents were required to be age 18 or older, understand English, and self-identify as one of the following A/TR interested parties: CTRI, center leadership, A/TR volunteer, A/TR participant, or caregiver of an A/TR participant.To be included in analyses, survey respondents had to complete at least 50% of the survey.Survey responses were submitted anonymously.

Data collection
The survey was created in Research Electronic Data Capture (REDCap), a secure, web-based software platform (10, 11).The survey was distributed to a national mailing list of A/TR interested parties maintained by PATH, Intl., the educational and credentialing body for A/TR professionals and centers.The email invitation included details of the study, including that participation was voluntary and 1 respondent would be randomly selected to receive a $100 gift card; clicking the survey link indicated consent.The survey was open for 8 weeks.Survey questions fell into five categories: (1) general information about the respondent; (2) if and how the respondent currently measures A/TR outcomes (for CTRIs and PATH., Intl leadership only); (3) the perceived importance and likelihood to change of possible participant outcome constructs (e.g., horsemanship skills, social skills, physical improvements); (4) benefits of and barriers to implementing standardized outcome assessments in community-based A/TR; and (5) considerations that may affect the usefulness of standardized assessments in the A/TR context (e.g., time and cost, etc.).The survey consisted of Likert-scale questions, "Select all that apply" questions, "Yes or No" questions, and open-ended short-answer questions (e.g., "Please describe, " or "Please list…").The survey included 107 distinct questions, and operated using branching logic, such that each respondent answered a set of questions based on their previous answers; for example, respondents who self-identified as a CTRI were presented with different questions than those who selfidentified as an A/TR participant.Depending on their answers, respondents were presented with an average of 50 questions.The survey allowed respondents to save their responses and return at a later time.

Data analysis
Survey data were exported from REDCap and downloaded to Microsoft Excel for analysis.We calculated descriptive statistics such as frequencies, percentages, medians, and interquartile ranges (IQR).
For "Select all that apply" questions, total percentages across all response options summed greater than 100%, as respondents were allowed to select multiple responses.For short-answer questions, we category-coded responses into pre-existing answer options or created new codes derived from participant answers; we report category counts to summarize the short-answer responses (12).

Respondent characteristics
Three hundred forty-seven total respondents completed portions of the survey.Two hundred seventy-seven participants completed at least 50% of the survey and therefore, were included in further analyses.Two hundred sixteen (78%) respondents identified as A/TR instructors, 99 (36%) identified as PATH, Intl.Center staff, 23 (8%) identified as A/ TR volunteers, and 8 (3%) identified as either an A/TR participant or a caregiver of an A/TR participant.Due to the dearth of volunteer, participant, and caregiver respondents, we chose to focus all further analyses on respondents who identified as either A/TR instructors or PATH, Intl.center staff.A total of 233 A/TR instructors or PATH, Intl.center staff completed at least 50% of the survey, and 221 completed the entire survey.The remainder of the manuscript provides the results from these 233 A/TR instructors or PATH, Intl Center leadership.
Of the 216 respondents who identified as A/TR instructors, 200 (93%) identified as CTRIs; 27 (13%) identified as Advanced Therapeutic Riding Instructors (ATRIs), and 11 (5%) identified as Master Therapeutic Riding Instructors (MTRIs).These instructor types were combined for further analysis and will be hereafter identified as CTRIs.One hundred forty-one (65%) respondents reported being employed at a PATH, Intl.A/TR center, 37 (17%) reported working at a non-PATH, Intl.A/TR center, 22 (10%) reported contracting with PATH, Intl.centers, and 22 (10%) reported not currently working with an A/TR center.

Current assessments
One hundred and ninety-one respondents (82%) reported they currently measure participant outcomes of A/TR.Table 1 includes participant outcome constructs currently assessed by survey respondents.
Among the 71 respondents using unstandardized assessments, respondents most commonly implemented the assessment before and after a session, which was defined as a period of consecutive weeks or months during which A/TR lessons are provided (e.g., a 10 weeks session of A/TR lessons; n = 42; 59%).The next most common time to implement the unstandardized assessment was after a lesson, defined as a single A/TR lesson (n = 23; 32%).Similarly, of the 24 respondents using standardized assessments, they most commonly implemented the assessment before and Respondents identified utilizing several different types of unstandardized assessments, including participant and caregiver surveys (n = 18, 25%), "in-house" assessments created by A/TR centers (n = 9, 13%), modified standardized assessments (n = 2, 2.5%), and surveys completed by A/TR instructors (n = 1, 1.4%).The most common standardized assessments were Goal Attainment Scaling [GAS; n = 9, 38%, (13)] and the Rider Instruction, Development, and Evaluation System [RIDES, n = 3, 13%, ( 14 • Insomnia Severity Index [ISI, n = 1, 4%, ( 22)], and • Recreation Therapy Assessment (n = 1, 4%).

Importance of identifying a standardized outcome assessment
Figure 1 illustrates the extent to which CTRIs and PATH, Intl.staff believed it is important to identify a standardized assessment to measure participant outcomes of A/TR.While a notable minority (29%) found it "unimportant" or "very unimportant", most respondents (64%) reported it was "important" or "very important" to identify standardized assessments to measure participant outcomes of A/TR.
Figure 2 illustrates the participant populations for whom respondents believed it would be important to measure outcomes.Over 90% of respondents reported that it would be important to measure A/TR outcomes in all participant age and diagnosis groups.
Table 2 illustrates respondents' perception of the importance and likelihood to change of several different potential participant A/TR outcomes that could be measured.Twenty-six survey respondents reported "other" outcomes which they believed were important or likely to change.Their responses were coded as: outcome importance depends on individual goals and abilities (n = 4, 15%), connection with the horse (n = 4, 15%), self-regulation (n = 3, 12%), sensory integration (n = 2, 8%), relationship building (in a family or between peers; n = 2, 8%), academic performance (n = 2, 8%), and self-confidence and leadership skills (n = 2, 8%).
Given the perceived benefits of and barriers to implementing outcome assessments, respondents were asked to report how likely they would be to use standardized outcome assessments if they were recommended by PATH, Intl.Most respondents reported they were likely (46%, n = 108) or very likely (39%, n = 90) to implement recommended standardized assessments, while fewer reported they were unlikely (12%, n = 28) or very unlikely (3%, n = 7).

Usefulness of implementing A/TR outcome assessments
Most survey respondents (n = 178, 76%) reported that CTRIs would be the best individuals to report on A/TR participant outcomes, followed by A/TR participant caregivers (n = 134, 58%), A/TR participants (n = 118, 51%), and "other" (n = 31, 13%).Respondents indicated that they would be willing for CTRIs to complete assessments that require a median length of 20 min (IQR = 10-22.5),and that they would feel comfortable asking A/TR participants to complete assessments lasting a median length of 10 min (IQR = 10-20).Additionally, of the 221 total survey respondents, 97 respondents (44%) indicated that they would prefer assessments via a computer/ iPad, 94 respondents (43%) identified that they would implement assessments in either computer/iPad or pencil/paper format, and 30 respondents (14%) stated that they would prefer pencil/paper Reported importance of identifying a standardized assessment to measure participant outcomes of A/TR.assessments.Finally, most respondents identified that that they would only use assessments that are either free (n = 106, 48%) or cost $1-2 per use (n = 79, 36%).Fewer respondents stated that they would use assessments costing $3-5 per use (n = 33, 15%) or $6-10 per use (n = 3, 1.4%).

Discussion
The current study gathered feedback from CTRIs and PATH, Intl center leadership pertaining to the current use of and attitudes towards standardized assessments to measure participant outcomes of community-based A/TR.As further discussed below, results substantiated there is an existing need across the A/TR industry to identify standardized assessments that can be used for this purpose, and elucidated qualities of standardized assessments that would be useful in the community-based A/TR setting.

Substantiating the need for standardized assessments in the A/TR industry
Most respondents reported currently measuring participant outcomes of A/TR participation, primarily through progress notes, interviews, or un-standardized assessments.While these commonly reported methods of recording participant information may be useful to track progress or communicate about participants within centers, they do not easily facilitate communication across centers, between professionals, and to the public.Outward Bound, an organization that provides outdoor education programs, provides an example of how a community-based recreation and learning program can benefit from a standardized assessment to measure participant outcomes.The Outward Bound Outcomes Survey "allows all 11 regional Outward Bound schools to collect consistent data on how students are impacted by their Outward Bound experiences in key areas of social-emotional development" (23).Therefore, this comparable program that provides recreational access to both general and vulnerable populations may serve as a model for how the A/TR profession may begin using standardized assessments to measure participant outcomes, facilitating communication between centers and to the public.
While the majority of respondents (64%) reported that it is important to identify standardized assessments to measure outcomes of A/TR participation, a notable minority (29%) of respondents did not think it was important.The rationale for those respondents who reported that it is "not important" to identify a standardized outcome assessment is not clear, as respondents did not have the opportunity to describe why they selected their answers.Other authors have reported that community-based service professionals feel that standardized assessments may not reflect the unique characteristics of their community, programs, and the skills and resources of their participants (24).Furthermore, education on standardized assessments is not included in CTRI training materials (25); therefore, the "unimportant" responses could result from a lack of knowledge about standardized assessments.Despite this disagreement among respondents on the importance of standardized assessments, a large majority of respondents stated they would be either likely or very likely to implement standardized assessments recommended by PATH, Intl.Therefore, results of this survey suggest that if standardized assessments were available and recommended for A/TR professionals, they would be likely to use them.However, the lack of standardized assessment availability is indicated by the fact that only 13% of respondents reported currently using standardized assessments.Therefore, this study substantiated that there is currently an unmet need to identify or develop standardized assessments to measure participant outcomes of community-based A/TR.

Current assessments
The current survey collected information from the few respondents who are using standardized assessments, which may provide helpful insights for identifying ideal assessments to be used in the A/TR context.The most frequently used standardized assessments measure improvements in horsemanship skills, the primary focus of A/ TR services.The Therapeutic Riding Assessment Impact Network [TRAIN; (26)] designed a TR-specific goal-attainment scaling process that was implemented by 9 respondents; this process involves the development and rating of goals to measure changes in horsemanship skills as a result of participating in A/TR.The next most prevalent standardized assessment, the RIDES tool, was developed by an A/TR center to assess horsemanship skills, develop goals, and track goal progress (14).The RIDES and TRAIN assessment tools could be valuable resources for A/TR professionals interested in measuring improved horsemanship abilities.However, they do not measure the natural benefits of horsemanship in participants' everyday lives outside of the equine context (e.g., physical outcomes and cognitive outcomes, etc.), which were more often identified by respondents as important constructs to measure (see Table 2).
There was much less consensus among survey respondents regarding standardized assessments currently used to measure the natural health or wellbeing benefits of horsemanship, outside the equine context.This reflects the diverse populations who participate in A/TR and the myriad of benefits which engaging with A/TR provides (2).However, survey results also highlight the difficulty of communicating A/TR outcomes across different centers, and further substantiates the need to identify or develop standardized assessments that can be used across several diverse PATH, Intl centers.

Benefits of and barriers to implementing standardized outcome assessments
The benefits of implementing standardized A/TR assessments reported by survey respondents mirror national trends for program development and evaluation.In recent decades, there has been a national emphasis on evidence-based policies and programs, including community-based programs.Program funders, such as policymakers and funding agency leaders, often require programs to demonstrate their efficacy through research or program evaluation (27).Additionally, for programs to receive national attention and support, they must be evaluated at regular intervals (28).One such method of program evaluation involves using standardized assessments to measure the outcomes which programs claim to address among participants.Standardized assessments are used to measure whether participants are eligible for programs/services, to ensure high quality intervention, and to communicate with internal and external interested parties.This aligns with respondents' perceived benefits of using standardized assessments in the current survey, which included demonstrating the value of A/TR, communicating with funders and other professionals, advocating for insurance coverage and funding, and bolstering the A/TR profession's credibility.These responses are similar to benefits of standardized assessments reported in other professions (29)(30)(31).Furthermore, consistent use of the same standardized assessments across centers could facilitate consistency in future A/TR research, a stated priority for increasing rigor in humananimal interaction research (7,32).Overall, the A/TR profession has an opportunity to strengthen its national credibility by implementing standardized assessments to effectively communicate the broad benefits of A/TR participation.
Despite these potential benefits, there are significant barriers to A/TR programs implementing standardized assessments.A/TR programs are not required to implement standardized measures, and training for CTRIs does not include education about how to implement standardized outcome assessments (25).The PATH, Intl course which potential CTRIs are required to take includes a section on participant evaluation and progress notes, but it does not provide education on implementing standardized outcome assessments to track participant progress or measure program effectiveness.Most barriers identified by survey respondents reflect restraints in the A/TR context, including the time, financial, and systemic restraints inherent in A/TR practice.Other professionals have reported similar barriers to those identified by survey respondents, including time, participant burden, and a lack of resources (29)(30)(31).These reported barriers should be considered when identifying potentially useful assessments to implement in A/ TR settings.

Usefulness of standardized outcome assessments
Given the barriers to using standardized assessments discussed above, it is critical that standardized assessments identified to measure participant outcomes of community-based A/TR are useful in the A/TR context.Usefulness is particularly important for A/TR professionals, as they often serve diverse populations with varying needs and abilities and are constrained by the limited time they spend with their participants, limited financial resources, and credentialing restrictions.Over 90% of respondents reported that a useful standardized outcome assessment would be appropriate for use with A/TR participants of all ages and diagnoses/life experiences.Additionally, respondents consistently reported that a wide variety of outcome constructs (i.e., cognitive outcomes, communication outcomes, emotion regulation, and physical outcomes, etc.) are important to measure in A/TR practice.The outcome constructs identified by respondents are consistent with research findings in the A/TR literature; specifically, research has demonstrated that A/ TR can improve mental functions (33), social functioning (34), emotional regulation skills (35), communication skills (34), physical function (36,37), quality of life (38, 39), and community integration (40).Given these findings, a battery of standardized outcome assessments proposed for use in the A/TR setting should measure a wide variety of outcomes in A/TR participants from a variety of age groups, diagnoses, and life experiences.Such a battery of assessments would then likely need to be accompanied by a decision-tool and training to guide A/TR professionals in selecting and implementing the assessment(s) most pertinent to the participants they serve.
Regarding the logistics of implementing standardized assessments, there was not consensus among survey respondents about who should provide information for the assessment (i.e., CTRI-administered assessment vs. participant or caregiver-completed questionnaire), or how the assessment should be delivered (i.e., computer/iPad, paperpencil, or both).The best standardized assessment format likely depends on individual characteristics of A/TR facilities so it would be beneficial for standardized assessments to be available in both virtual and paper/pencil forms.Most respondents agreed that assessments should not take longer than 10-20 min to administer and should be freely available or low-cost.Therefore, if multiple assessments are identified, they should require no more than

Limitations
This study includes several limitations.Firstly, respondents were not asked their geographical location or demographic information.These data would have allowed us to understand if trends existed in different areas across the country or based on gender, age, race, or ethnicity.Such demographic data should be included in future studies.Additionally, some sections of the survey did not include a shortanswer section, forcing respondents to identify a pre-written response and limiting our capacity to understand the nuances of their answers.Finally, this study is limited by the absence of responses from A/TR participants and their caregivers, whose opinions should be considered in relation to if assessment is important to them, what outcomes should be assessed, and how much time they would be willing to dedicate to participating in an assessment.

Future research directions and conclusions
To our knowledge, this is the first study to gather information from A/TR professionals about the potential usefulness of standardized assessments in community-based A/TR services.Future research should identify or develop standardized assessments that may be useful.To this end, a Delphi study of A/TR experts could aid in developing a prioritized list of potential standardized assessments to measure participant outcomes of A/TR.Once assessments are identified, they should then be piloted in community-based A/TR to understand whether they are feasible in actual A/TR environments.Future directions may also include bolstering community-academic partnerships, to support the implementation of standardized assessments in community-based A/TR.Overall, implementation of standardized assessments in A/TR could enhance the credibility of the profession and provide a means for communicating the vast benefits of community-based A/TR to a variety of audiences.

FIGURE 2 (
FIGURE2(A) Participant age groups identified as important to measure A/TR outcomes in, and (B) populations identified as important to measure A/TR outcomes in.NC, neurodevelopmental conditions; MH, mental health conditions; ND, neurological disorders.
min total, and the cost for administering assessments should remain minimal.

TABLE 1 A
/TR outcome constructs currently assessed by A/TR professionals.
"Other" outcomes included transference of skills to daily life, independence, and mental health.

TABLE 2
Importance and likelihood to improve of participant outcome constructs.