The Implementation Process of Two Evidence-Based Protocols: A Spatial Neglect Network Initiative

Introduction Spatial neglect, a neurocognitive disorder of lateralized spatial attention, is prevalent among stroke survivors especially in inpatient rehabilitation facilities (IRFs). The ultimate goal of the project was to improve spatial neglect care in inpatient rehabilitation and trained as many OTs as possible using both tools in their regular practices as the means to achieve our overall objective. Therefore, we conducted a project aimed at implementing two evidence-based protocols, one for assessment (KF-NAP®) and the other for treatment (KF-PAT®), and share the implementation process, which included barriers and facilitators identified during and after the process, and implementation outcomes. Methods Sixteen IRFs were involved. The Knowledge-To-Action Cycle was used to describe the process of knowledge inquiry (training), translating knowledge (implementation) and evaluating the use of knowledge in clinical practice (outcomes). Barriers and strategies were reported using the Consolidated Framework for Implementation Research and identified through a survey, after the study concluded. Results Thirty-two therapists at the participating sites were trained to some level of the KF-NAP and KF-PAT. Throughout the project and also once after it finished, different barriers were identified by researchers and clinicians, who then determined together actions to eliminate or minimize the barriers. For example, multiple sites reported: “not having time to train other staff at their hospital due to high patient volume and other responsibilities.” Discussion The project shared our implementation process which demonstrated the importance of using implementation methods and incorporating a researcher-clinician partnership, not only for knowledge generation but also knowledge translation. Frequent communications and exchanging information with stakeholders at different levels, may be determinant to the success of each implementation phase. Further research is needed.


INTRODUCTION
Spatial neglect is a neurocognitive disorder that is characterized by the inability to attend to, perceive, and orient to the space that is contralateral to the injured or damaged cerebral hemisphere (1). It affects 20-40% of stroke survivors (2) and individuals with other acquired brain injuries (3). Since the 1980 s, devastating impacts of spatial neglect on rehabilitation progress, functional recovery, community reintegration, and caregiver burden have been demonstrated in various studies conducted by independent research groups around the world (4)(5)(6)(7)(8)(9)(10). Furthermore, spatial neglect prolongs inpatient rehabilitation and increases the risks of falls and injuries (5,8,10,11). Treatments and assessments have been developed, examined, and recommended as guidelines, by organizations such as the American Heart/Stroke Association (12), Canadian Stroke Association (13), and the Australian Stroke Foundation (14). Nonetheless, in practice, it has been arbitrary whether individuals with spatial neglect are provided the recommended evidencebased treatment and assessment services. The problem may be related to the hurdles to achieve knowledge translation, dissemination, and implementation (15)(16)(17)(18).
The ultimate goal of the wider research project was to improve spatial neglect care. To achieve this wider goal, we sought to implement two evidence-based standardized protocols, one for assessment and the other for treatment. The assessment protocol was the Kessler Foundation Neglect Assessment Process (KF-NAP R ), and the treatment protocol was the Kessler Foundation Prism Adaptation Treatment (KF-PAT R ). In this article, we aim to report the implementation process, which included barriers and facilitators identified during and after the process, and share implementation outcomes using quantitative and qualitative information.
In order to report the implementation process systematically, we incorporated methods from the implementation science literature (19). First, we followed the Knowledge-to-Action (KTA) cycle (20) to describe the progression from knowledge dissemination, protocol implementation, to outcome evaluation. The KTA cycle provides a "map" for how to move knowledge into action and encouraged revisiting phases of the action cycle as many times as necessary (Figure 1). The KTA cycle has been widely used in practice because it captures the complexities of real-world application and encourages transformation of knowledge that has been generated in research settings to promote use of evidence-based practices in the clinic (21).
Second, we used the Consolidated Framework for Implementation Research (CFIR), a framework of constructs related to implementation (22), to report barriers identified during the implementation process and the subsequent strategies used to address each barrier (23,24). The CFIR is organized into five domains (intervention characteristics, outer setting, inner setting, characteristics of the individuals involved and the process of implementation) which provides organization and specificity to evaluate the project's impact (25). There are multiple examples of pragmatic research projects that use the KTA and the CFIR in a rehabilitation setting. Studies suggest that using the CFIR may increase the replicability and generalizability of study findings (26)(27)(28), and the KTA cycle can contribute to positive changes in stroke rehabilitation practices (21).
This project implemented the KF-NAP and the KF-PAT. The KF-NAP is a standardized method to administer and score the 10-item Catherine Bergego Scale (CBS) during daily activities (29). The items include gaze orientation, limb awareness, auditory attention, personal belongings, dressing, grooming, navigation, collisions, meals, and cleaning after meals. The scoring uses a scale between 0 and 3 for each item, with the total score ranging between 0 and 30 (the higher the number, the more severe the neglect is). The KF-PAT is a standardized protocol to deliver prism adaption treatment (PAT) (30). PAT is one of the treatment approaches recommended for stroke rehabilitation by the latest guidelines of the American Heart/Stroke Association (12). Both the KF-NAP and KF-PAT, and related materials such as clinician-oriented manuals and equipment, were developed through clinical research and trials over the past decade by our research team (31)(32)(33)(34)(35) and thus were the preferred choices in the present implementation project.
The discipline that participated in this project was occupational therapy (OT). Conventionally, in the United States, OT is the discipline in neurorehabilitation providing care related to visuospatial deficits, and is the discipline known to document the observable symptoms of spatial neglect, such as head and eye deviation. Thus, the present project was focused on integrating the two evidence-based protocols into inpatient OT (31)(32)(33)(34)(35). In addition, both the KF-NAP and KF-PAT were developed in and for the inpatient rehabilitation care through studies and clinical trials with much involvement of occupational therapists (OTs) (31)(32)(33)(34)(35). In the present project, we trained as many OTs as possible using both tools in their regular practices as the means to achieve our overall objective (i.e., to implement both tools) and move closer to our ultimate goal (i.e., to improve spatial neglect care).

Participants and Setting
Sixteen inpatient rehabilitation facilities (IRFs) across 11 different states in the United States participated in this assessment and treatment implementation project through an agreement with the research center in New Jersey. The agreement included OT training and de-identified clinical information sharing. Twelve sites were on the East Coast (New Jersey, Ohio, Florida, Pennsylvania, Georgia, Maryland, New York), one on the West Coast (California), two in the Southwest (Texas, Arizona) and one in the Midwest (Missouri). The project was approved by the research center's Institutional Review Board (IRB) and the local IRB of each hospital that had a research infrastructure. IRFs without a research infrastructure were attached to the research center's IRB protocol through a federal assurance agreement. Directors of rehabilitation at each site nominated one or two lead OTs (i.e., implementation champions) for project participation. A total of 32 champions were trained to use the KF-NAP and KF-PAT throughout the project. They implemented the protocols in their practice, participated in monthly calls, tracked information regarding implementation, provided feedback to the research center, and were encouraged to train their peers. The implementation information being tracked included de-identified patient clinical records. Patient outcomes were reported separately (36,37).
The project was initiated in June 2017, and the therapist user feedback completed in March 2021 marking the end of the project. OTs at three sites that had participated in our previous research (31)(32)(33)(34)(35) and KF-NAP and KF-PAT development were more familiar with either tools than OTs at other sites. OTs at sites that joined the project earlier might be more experienced with either tool than OTs at sites that joined the project later over the years. While the project was not designed as a research study, lead OTs served as "study participants, " representing their sites, in the user feedback survey at the end of project, and the consent form was waived. Evaluation of the implementation outcomes was based on responses to the survey and information shared about de-identified clinical records.

Procedures
Following the KTA cycle (Figure 1), the research team led and was actively involved with hospital management leaders and clinicians in the knowledge dissemination (center of the KTA cycle), protocol implementation (Phase 1 to Phase 5), and outcome evaluation (Phase 6). Knowledge here refers to the two protocols of the KF-NAP and the KF-PAT. Phase 6 required the research team to step back and conduct outcome evaluations. This project did not move into Phase 7.

Knowledge Dissemination
Knowledge dissemination involved the lead OTs participating in a formal 2-day training. The training took place at either the research center, the IRF where the OTs worked, or a specified IRF that would host a few groups of OTs at the same time. Sixty percentage of the lead OTs received the training at their sites, and the remaining 40% traveled to the location where the training was provided. Completing the training would enable them to reach Level 2 of competency on both protocols (Table 1), and they were instructed and encouraged to fulfill the requirements for the highest level of competency (Level 3) on their own. Level 3 competency would qualify the therapists to be able to teach their colleagues how to administer the KF-NAP or KF-PAT. This entailed using Table 1's criteria to guide the training process. For example, the trainer scheduled time for each therapist to first observe them completing each protocol, and then had the trainees perform the assessment and treatment protocols under supervision.
The 2-day training was taught by the same members of the research team, who developed the protocols and have therefore extensive knowledge on the topic of spatial neglect. The curriculum started with didactics about spatial neglect and then the two protocols, hands-on practice using both protocols under the trainer's supervision and ended with discussions on implementation. The lead OTs were provided training materials including the Manuals, lecture handouts, and pre-recorded video tutorials. Fidelity of both the assessment and intervention were discussed when these materials were being distributed. This discussion included that the Manuals were mandatory to use, the training of other staff members must include the lecture handouts, video tutorials should be reviewed prior to the inperson supervision and hands-on-training, and competency forms should be used. The lead OTs were also instructed about how the ways in which to communicate with the research team. In addition, the research team offered lectures to other disciplines of therapists, medical staff, case managers, and trainees during new employee trainings, regular staff meetings, and schedule meetings outside of working hours. The lectures focused on the mechanisms, clinical presentations, and consequences of spatial neglect. These knowledge dissemination activities were developed to help coworkers of the OTs understand why the KF-NAP and KF-PAT were being implemented in their facilities. Thus, consistent with "culture, " a construct of the inner setting domain within the CFIR framework, an environment friendly for spatial neglect care was cultivated, potentially beneficial for the implementation project.

Daily Practice and Frequent Communications
After the training, the lead OTs returned to their daily clinical activities and started to gain experience by completing the protocols independently with their own patient case load. They were instructed to assess all neurological patients for spatial neglect using the KF-NAP within 4 days after admission, regardless of whether patients' symptoms were clearly observable or not. In addition, the lead OTs were also instructed to treat patients using the KF-PAT Portable Kit 1 when spatial neglect was confirmed through the KF-NAP, and assess these patients using 1 KF-PAT Portable Kit, Stoelting, 620 Wheat Lane, Wood Dale, IL, USA 60191. the KF-NAP again after completing 10 sessions of PAT or before IRF discharge.
In addition to implementing both the KF-NAP and the KF-PAT in their clinical practice, the lead OTs were asked to document why the assessment and treatment sessions were not performed or performed in a way deviated from the standardized manuals. The research team provided a spreadsheet template to the lead OTs and required that it be filled out with de-identified clinical records of patients who were assessed using the KF-NAP and who were treated using the KF-PAT. The spreadsheet was submitted to the research team every quarter. This spreadsheet was one way that the barriers to implementation and strategies trialed were communicated. The other way was during the monthly calls.
The research team hosted monthly 1-h conference calls with all participating IRFs, represented by the lead OTs. This is an example of the KTA cycle being used to monitor outcomes. A total of 37 conference calls were conducted from June 2017 to September 2020, and they were recorded to document the meeting and later used to create meeting minutes. These minutes were shared via email to all attendees and the lead OTs who were not able to attend. During the calls, the OTs and the research team had the opportunity to share progress related to the training and implementation. The OTs shared any facilitators or barriers implementing either protocol that they may have listed in their spreadsheet or that were new and therefore being reported for the first time. For example, a facilitator was that OTs exchanged experiences about obtaining leadership support when integrating both protocols into their clinical practice. Barriers often shared were related to managing time and resources. When a barrier was shared, other OTs on the call would offer suggestions that worked or did not work for them, and the research team would help determine a resolution. However, if no resolution was suggested during the call or if the solution suggested was not agreed upon, the research team would follow up with the clinicians and sometimes their supervisors after the call. The OTs were encouraged to contact the research team via email at any time for further questions and comments. The calls also allowed the research team to clarify details in the assessment or treatment protocols that might have been forgotten from the training. In addition, starting at the June 2018 meeting, the research team would ask an OT to share a patient case to the group. The cases presented usually included a short medical history and details of either protocol used with that patient. The attendees could offer their comments and asked questions. The sharing of cases facilitated engagement during the meetings as well as encouraged discussions and comradery.

Engaging Other Stakeholders
While the project was focused on integrating the evidencebased protocols into the OT's clinical practice, there were other stakeholders who played significant roles. Because the OTs were involved in the implementation project, there were increased discussions about spatial neglect during care team meetings attended by all disciplines that provided medical and therapy services. In addition, other disciplines were able to easily and informally observe how the KF-NAP and KF-PAT were administered in spaces shared by all disciplines. Also, at several participating sites, the OTs conducted in-services with physicians, nurses, and nutrition staff to share information about this implementation project. This involvement of multiple disciplines could be one action that facilitated the implementation progress, especially in Phase 3 and Phase 4 of the KTA cycle ( Figure 1). Also, the IRF administrative leadership were engaged in discussions with the Chief Executive Officers and supported the implementation of both protocols after reviewing implementation goals of the project. Directors of Rehabilitation were informed with the project progress and were encouraged to offer comments and suggestions throughout the project.

Addressing Barriers
The implementation procedures evolved and adapted to different hospital contexts as we continuously learned from OTs' experience using either KF-NAP or KF-PAT protocol with patients and interacting with their colleagues, and from administrators' guidance on hospital regulatory and operational standards. From time to time, a situation would occur that was not expected, or a solution was not immediately available, and we classified the situation as a barrier. When barriers were identified, via verbal or written report, then strategies were carefully customized based on the needs of the setting, and then implemented. A strategy and related actions were decided and executed based on the available resources and contextual situations at the time, usually right after the barrier was identified, rather than based on a pre-determined decisionmaking roadmap. We followed the CFIR framework (22) to organize the identified barriers and summarize strategies and actions taken to eliminate the barriers ( Table 2). Barriers were identified in 4 of the 5 CFIR domains. In these domains, there were no identified barriers in some constructs, but a number of barriers in other constructs. Throughout the process, two limiting factors-time and staff-emerged frequently across different constructs and domains. Time was limited against the administration of the KF-NAP and KF-PAT. Shortage of trained staff trained on the two protocols was a common barrier to assessing all patients with neurological conditions and treating patients with spatial neglect. Collaborating with clinicians and their managers, we offered potential solutions. Some of the barriers identified were resolved shortly such as those identified in the Intervention Characteristics domain (Domain I, Table 2), some barriers ultimately required multiple strategies to fully address the problem such as those identified in the Inner Setting domain (Domain III), and others remained challenging such as several barriers identified in the Process domain (Domain V).

Implementation Outcomes
We evaluated the implementation outcomes using quantitative and qualitative information that was collected during the 4year period. The study also included collecting patient outcomes, which was reported separately (36,37).

Numbers of Therapists Trained and Patients Receiving the Care
We used the number of OTs trained to indicate the success of knowledge dissemination and the number of patients receiving spatial neglect care through either the KF-NAP or KF-PAT to indicate the success of knowledge implementation. In April 2020, the research team asked the lead OTs to report the number of therapists they trained on both protocols, and what competency level they reached. In November 2020, we completed the collection of de-identified clinical records shared by participating sites. Note that patient outcomes (i.e., improvement in spatial neglect and rehabilitation outcomes) were reported in separate articles (36,37).
Lead OTs were interviewed to provide context of the number of therapists trained and the number of patients receiving spatial neglect care. Responses to the interviews were summarized in writing with no audio records. Interview responses were reviewed by Authors PC and CGS.

User Feedback Survey
After the last monthly call (September 2020), a survey was sent to the lead OTs via an online platform, Survey Gizmo. The OTs were asked to report on the IRF that they represented, based on their own experience. See the survey in Appendix 1.
The goal was to identify any sustaining barriers and additional facilitators to implementing either protocol now that the research team stepped back. Categorical responses were summarized and described in percentages. The qualitative answers were reviewed by Authors KH, PC, and CGS to enable the identification of emerged categories. We each separately read the data to determine reoccurring information, discussed and compared notes via conference calls and came to an agreement on how to code the data into similar groups or categories. Table 3 reports that overall, 169 OTs at the participating sites were trained to some level of the KF-NAP, and among them, 81 (47.9%) reached the highest level (level 3) of competency. One hundred forty-one OTs were trained for the KF-PAT, and 110 (78.0%) reached the highest level of competency. Site 9 reported that they only implemented the KF-NAP at the beginning of Domain I: Intervention characteristics (Key attributes of interventions, i.e., KF-NAP ® and KF-PAT ® , that influence the success of implementation). No barrier was identified in four constructs including intervention source (stakeholder perception about whether the intervention is externally or internally developed), evidence strengthen and quality (stakeholder perception of the quality and validity of evidence supporting the belief that the intervention will have desired outcomes), relative advantage (stakeholder perception of the advantage of implementing the intervention vs. an alternative solution), and trialability (the ability to test the intervention on a small scale in the organization, and to be able to reverse course if warranted).

Construct (definition)
Barrier Strategy and actions to remove the barrier Adaptability (The degree to which an intervention can be adapted, tailored, refined, or reinvented to meet local needs) • KF-NAP assessment: • Time limitation against adding a new assessment protocol • Determining best time to complete the assessment during the admission and before discharge • KF-PAT treatment: • Set-up time • Patients with varied medical, physical, cognitive, and neuropsychological conditions (also see Domain II: Outer Setting, the construct of Patient Needs and Resources) • KF-NAP assessment: • Suggestions made to integrate KF-NAP with conventional ADL assessment in the morning to increase the efficiency of time allocation • Instruction added to complete the assessment by Day 4 from admission date (giving patients time to be acclimated to the facility) and give enough time for treatment to be complete • Suggestions made to prioritize KF-NAP assessment after KF-PAT completion or the day before discharge • KF-PAT treatment: • Solutions provided to reduce the set-up time, such as allowing assistance of therapy aides to set up the equipment and laminating the stimulus sheets. • Instructions refined and clarified in the manuals for how to provide commands and when to skip a task and move on. • Suggestion to consult with optometry if the person was prescribed with optical lenses. Reading glasses can be used under the prism lens, if necessary.
Complexity (Perceived difficulty of implementation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy, and number of steps required to implement) • KF-NAP assessment: • Time limitation against completing all 10 items. • KF-PAT treatment: • Hesitance of some neuro-optometrists who were unfamiliar with the intervention. • Time limitation against fitting the treatment into an OT therapy session • KF-NAP assessment: • In-person instructor-guided assessment practice to demonstrate how to complete all 10 items in one session. • Suggested actions to take in order to decrease assessment time based on the facilities' unique needs. • KF-PAT treatment: • Invitations to neuro-optometrists to discuss the treatment mechanisms and why the KF-PAT is within the scope of OT • In-person instructor-guided treatment practice to demonstrate how to fit the treatment into regular OT sessions Design quality and packaging (Perceived excellence in how the intervention is bundled, presented, and assembled) • KF-NAP assessment: • The booklet of the manual was not easy to carry around when administering the assessment. • KF-PAT treatment: • Equipment assembly not always intuitive.
• Device not fit on patients with a much smaller or larger body size. • Frequent wear and tear of the equipment • KF-NAP assessment: • A two-page double-side-printed shortened 'cheat sheet' was developed. • KF-PAT treatment: • Added pictures to the manual and production of short video clips showing how to set up the equipment. • The visual field occluder (the wearable shelf) was modified to accommodate a wider range of body sizes. • Device repairs and replacement were provided. Cost (Costs of the intervention, and costs associated with implementing the intervention, including investment, supply and opportunity costs) • KF-NAP assessment: • Limited budget for staff training • KF-PAT treatment: • Did not have the equipment • KF-NAP assessment: • Provision of 100% discount to the online tutorial • Free email and phone consultation • Support from the management to provide travel funds for therapists to attend in-person hands-on trainings • KF-PAT treatment: • The equipment was loaned to the sites by the research team, as part of the research agreement Domain II: Outer setting (The economic, political, and social context within which an organization resides).
No barrier was identified in three constructs including cosmopolitanism (the degree to which an organization is networked with other external organizations), peer pressure (mimetic or competitive pressure to implement an intervention), and external policies and incentives (a broad construct that includes external strategies to spread interventions).

Construct (definition) Barrier
Strategy/action used to remove the barrier Patient needs and resources (The extent to which patient needs, as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization) • KF-NAP assessment: • Not all 10 items were scored in certain patients due to physical disabilities or cognitive impairment.
• KF-NAP assessment: • Solutions and in-person demonstrations regarding how to build a rapport with patients, observe neglect symptoms, and score as many items as possible Frontiers in Health Services | www.frontiersin.org • Unable to follow commands due to severe neglect symptoms • Unable to follow commands due to language barriers in non-English-speaking patients • Unable to use certain equipment components as intended due to physical disabilities • Unable to tolerate the prism goggles (e.g., feeling dizzy or seeing doubles due to optical shifts, too much physical pressure on the head by the goggles for patients who wear a helmet) • Simplified directions and gesturing to work with non-Englishspeaking patients • Device modifications and alternative ways of putting on goggles to allow most patients to participate in the treatment Domain III: Inner setting (Features of structural, political, and cultural contexts through which the implementation process will proceed).
No barrier was identified in one construct, which was structural characteristics (the social architecture, age, maturity, and size of an organization). Implementation climate (The absorptive capacity for change, shared receptivity of involved individuals to an intervention, and the extent to which the use of the intervention will be rewarded, supported, expected within their organization).

Construct (definition)
No barrier was identified in three sub-constructs including compatibility (the degree of tangible fit between meaning and values attached to the intervention by involved individuals, how those align with individuals' own norms, values, and perceived risks and needs, and how the intervention fits with existing workflows and systems), goals and feedback (the degree to which goals are clearly communicated, acted upon, and fed back to staff, and alignment of that feedback with goals), and learning climate (a climate in which: leaders express their own fallibility and need for team members' assistance and input; team members feel that they are essential, valued, and knowledgeable partners in the change process; individuals feel psychologically safe to try new methods; and there is sufficient time and space for reflective thinking and evaluation).

Sub-construct (definition) Barrier
Strategy/action used to remove the barrier Tension for change (The degree to which stakeholders perceive the current situation as intolerable or needing change) • Competing demands in the therapy departments, self-initiative of the therapists to use the protocols consistently • Frequent communications with the IRF management highlighting that these protocols would provide guidance for staff, and the potential impact of the implementation on quality of care Relative priority (Individuals' shared perception of the importance of the implementation within the organization) • KF-NAP assessment: • Some therapists prioritized the administration of KF-NAP in patients who already showed neglect symptoms • KF-PAT treatment: • Some therapists did not provide KF-PAT to patients with "very mild" neglect • KF-NAP assessment: • Frequent reminders during staff training and monthly calls that it was of great importance to assess all patients with neurological conditions because certain symptoms were not apparent. It was also important to confirm the absence of spatial neglect.

(Continued)
Frontiers in Health Services | www.frontiersin.org • Some therapists prioritized other treatment than KF-PAT in patients whose length of stay was pre-determined to be shorter than 10 days • KF-PAT treatment: • Frequently discussed during monthly calls regarding the factors to be considered in initiating and completing the treatment.
Organizational incentives and rewards (Extrinsic incentives such as goal-sharing awards, performance reviews, promotions, and raises in salary, and less tangible incentives such as increased stature or respect) • No competency measure existed as well as no specific incentive to participate in the research project • Free meal if training lectures were offered during the lunch hour • Development of competency certification processes for both KF-NAP and KF-PAT such that certificates could be added to therapists' profiles, which may help promotion • Participation in the research was an approved task that counted toward clinical promotion Readiness for implementation (Tangible and immediate indicators of organizational commitment to its decision to implement an intervention).
No barrier was identified in one sub-construct, which was access to knowledge & information (ease of access to digestible information and knowledge about the intervention and how to incorporate it into work tasks). No barrier was identified in all five constructs including knowledge and beliefs about the intervention (individuals' attitudes toward and value placed on the intervention as well as familiarity with facts, truths, and principles related to the intervention), self-efficacy (individual belief in their own capabilities to execute courses of action to achieve implementation goals), individual stage of change (characterization of the phase an individual is in, as he or she progresses toward skilled, enthusiastic, and sustained use of the intervention), individual identification with organization (a broad construct related to how individuals perceive the organization, and their relationship and degree of commitment with that organization), and other personal attributes (a broad construct to include other personal traits such as intellectual ability, motivation, values, competence, capacity and learning style).
Domain V: Process (Essential activities of implementation process).
No barrier was identified in one construct, which was planning (the degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance, and the quality of those schemes or methods).

Construct (definition) Barrier
Strategy/action used to remove the barrier Executing (Carrying out or accomplishing the implementation according to plan) • Initial lead OTs left the position or were on medical leave • Not all OTs were trained to administer the KF-NAP or KF-PAT • Collaboration between researchers and IRF leaders to improve the staff training capacity • Training experienced and enthusiastic therapists to become lead OTs and trainers • Developing remote education modules Reflecting and evaluating (Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience) • Infrequent feedback provided from therapists who did the frontline work of implementation • Centralizing the information related to the progress and outcomes of the KF-NAP and KF-PAT implementation at the research team, who summarized and shared the information periodically to lead therapists and their supervisors of all campuses Engaging (Attracting and involving appropriate individuals in the implementation and use of the intervention through a combined strategy of social marketing, education, role modeling, training, and other similar activities).
No barrier was identified in two sub-constructs including formally appointed internal implementation leaders (individuals from within the organization who have been formally appointed with responsibility for implementing an intervention as coordinator, project manager, team leader, or other similar roles) and champions (individuals Frontiers in Health Services | www.frontiersin.org Sub-construct (definition) Barrier Strategy/action used to remove the barrier who dedicate themselves to supporting, marketing, and driving through an implementation, overcoming indifference or resistance that the intervention may provoke in an organization).
Opinion leaders (Individuals in an organization who have formal or informal influence on the attitudes and beliefs of their colleagues with respect to implementing the intervention) • KF-NAP assessment: No barrier identified • KF-PAT treatment: • Hesitance of some neuro-optometrists who were unfamiliar with the intervention.
• KF-NAP assessment: Not applicable • KF-PAT treatment: Invitations to neuro-optometrists for discussing the treatment mechanisms External change agents (Individuals who are affiliated with an outside entity who formally influence or facilitate intervention decisions in a desirable direction) • Length of stay may be shorter than anticipated • Therapists were instructed to start the treatment very soon after admission, to be able to have as much treatment as possible provided This qualitative analysis was based on verbal reports and informal observations during (rather than after) the implementation process. Each identified barrier may represent one therapist's experience, a few participating site's situation, or a general observation of almost all participating sites. ADL, activities of daily living; IRF, inpatient rehabilitation facility; OT, occupational therapist.
the project and decided not to continue because an ongoing study was using the CBS following the original non-standardized questionnaire format (38). Nonetheless, Site 9 continued using the KF-PAT in their care.

Number of Patients Receiving Care
OTs assessed a total of 4,454 patients for spatial neglect using the KF-NAP, and 2,491 (56%) of them had the syndrome. 1,078 (43%) of the patients with spatial neglect were treated using the KF-PAT for at least one session.
To understand why more than half of the patients with spatial neglect did not receive PAT, we had discussions with the lead OTs during the conference calls and reviewed the OT's field notes in the de-identified clinical records in order to determine categories. Also, while the COVID pandemic led to a long pause in 2020 of no treatment using the KF-PAT Portable Kits across all participating sites, there were various factors that contributed to different implementation rates of either protocol in different sites, and therefore We classified the 15 participating sites (excluding Site 9) into three categories: early adopters, additional trained staff needed, and developing facilities.
Five sites (Sites 1, 2, 3, 4, and 6) were the first sites to be trained (aka early adopters). Therefore, many staff OTs, in addition to the implementation champions (lead OTs), were trained during the project. Although the rate of lead OT changes was 50% among these first trained sites, the other OTs were able to step up and take on the lead OT roles. These five sites collectively assessed 3,698 (83%) of all assessed patients and treated 783 (73%) of all treated patients across 16 sites. Based on the shared clinical records, the median CBS scores of PAT-untreated patients at these sites ranged from 2.5 to 3.75, which is a mild level of severity. In the de-identified clinical records and field notes, OTs reported that other deficits such as upper extremity impairment, rather than spatial neglect, was prioritized in OT sessions because the neglect was not severe.
Eight sites (Sites 5, 7, 8, 10, 12, 13, 15, and 16) were classified as "additional trained staff needed." The median CBS scores of PAT-untreated patients at these sites ranged from 7.5 to 17.38, which is a wide range covering all levels of neglect severity. These sites relied on their lead OTs to implement both protocols while other OTs may or may not have integrated either protocol in their practice. Thus, only patients under lead OTs' care were assessed for spatial neglect, and other OTs who were not trained on the KF-NAP themselves, had to refer patients with neglect signs on their caseloads to the lead OTs for KF-NAP assessment. This indicated that patients whose neglect symptoms were less apparent may have not been identified and therefore not referred. Furthermore, not all patients who were confirmed with spatial neglect received PAT because the work load was too high for the lead OTs. Sites reported in field notes: "we do not have time to train other therapists in these protocols because of other responsibilities." In addition, when a lead OT left, there was a lag in time during the transition and thus the average 45% change rate of implementation champions among these sites became a significant barrier. These eight sites planned to have additional staff trained early in Year 2020, which however was soon on pause due to the COVID pandemic. The pandemic caused particular challenges for Sites 15 and 16 who had joined the project a few months prior to the shutdown of research activities.
Lastly, four sites fell into the category of "developing facilities." The management team was at the early stage of development in Sites 15 and 16 (also classified as "additional trained staff needed") and Sites 11 and 14. These sites were having a hospitalwide staffing call, in order to recruit more therapist to work at their hospitals. Therefore, the combination of situations resulted in low numbers of identified individuals with neglect. That is, the infra-structure was not ready to fully support the implementation project.

User Feedback Survey
Fifteen sites (93.75%) responded to the User Feedback survey. The quantitative aspect of the survey results was summarized in Figure 2.
Multiple responses to open-ended questions were able to be categorized. First, the top three barriers related to the KF-NAP not being administered 100% of the time were: (1) lack of time to train the rest of staff on unit to perform the assessment, (2) patients being discharged earlier than expected, and (3) patients requiring other considerations at discharge such as extensive family training. It was also reported by multiple individuals that patients at multiple sites did not always receive

DISCUSSION
There is still work to be done related to translating research into practice and decreasing the research-to-practice gap in the rehabilitation settings despite many efforts being made by many teams (26,28). The shorter time it takes for the latest evidence to be applied to clinical practice, the greater chance for patients to receive better care (39). This manuscript highlights an implementation project that used the KTA cycle as the process model to assist the transfer of scientific knowledge into clinical practice. Specifically, the KTA cycle helped to guide the project development, report the results and provide specific information for future reproducibility. The CFIR was another implementation tool that was used in this project to assist with the categorization of barriers identified as well as provided a way to organize strategies that were trialed. Similar to others conducting rehabilitation implementation research (26), we found the use of the frameworks to be a strength of the study because the researcher-clinician team had a "road map" to guide implementation of the protocols as well as evaluate outcomes of the implementation. The close collaboration between researchers and clinicians was key to achieve knowledge translation. The research team and the OTs had frequent communications through the de-identified clinical records and the monthly conference calls. This suggests that rather than a one-way, top-down instruction provision from the research team to the OTs, participants worked together and modified certain aspects on how to administer the KF-NAP and KF-PAT at specific sites. This is one example of how we used the KTA cycle and made an adaptation to fit the local context. However, it was important to the researchers that the core elements of both protocols remained unchanged, in other words, fidelity was maintained. For instance, regarding the KF-NAP protocol, as long as the therapist assessed tasks that relate to skin care or hair care (including facial) then they could be creative to what they ask the patient to complete (e.g., applying makeup instead of washing their face, which is the task suggested in the manual).
Most barriers identified during the implementation process were aligned with previous studies that offered reasons for the difficulties in knowledge translation and evidence-based practice (EBP) implementation in stroke care and rehabilitation (16,40). One barrier, however, was unexpected. This was when consulting neuro-optometrists questioned whether administering PAT was within the scope of OT practice. The strategy addressing this barrier was to be collaborative, transparent, and be open to inter-professional learning. More specifically, the research team-initiated discussions with the neuro-optometrists about the mechanisms of prism adaptation and offered treatment demonstrations to share the procedures of the KF-PAT. Both KF-NAP and KF-PAT protocols were in use as part of clinical practice by the end of the implementation period. This suggests the tools were accepted by the OTs and adopted into the standard of care (41).
Another interesting finding was that focusing on a single discipline has the potential to change the overall quality of care in the multidisciplinary inpatient rehabilitation system. Shown in a prior study, only 31% of spatial neglect cases were mentioned in care team meetings, which potentially impeded provision of comprehensive care to all patients with spatial neglect (42). In the present project, the change of OT practice via the implementation of both spatial neglect assessment and treatment protocols increased the awareness of spatial neglect care among other disciplines. The CBS scores via the KF-NAP (indicating severity of spatial neglect) and improvements observed after PAT were discussed during care team meetings attended by all disciplines that provided medical and therapy services. Other disciplines could observe the procedures of both protocols easily as OTs worked in the same space with them. Thus, a new vocabulary was created and understood by all the care team members. This became an inner-setting facilitator (22) that emerged during the implementation process, different from facilitators provided by the research team and hospital administrative leaderships. Thus, the researcher-clinician collaboration is critical to initiate knowledge translation, and clinician buy-in and subsequent spontaneous inter-disciplinary communications are essential to strengthen the translation.
The therapists' time to complete the assessment and treatment was a limitation and was the most reported barrier. For the KF-NAP, the difficulty was related to the second (i.e., "post-treatment") assessment, which is essential in order to measure changes in spatial neglect severity from before to after treatment. The second assessment which was to occur before the patients' discharge was difficult to administer when scheduling conflicts occurred more often at discharge than at admission. Time limitation with the assessment also affected treatment delivery. As recommended by the KF-PAT protocol, patients should complete the full treatment course which includes 10 sessions. In the present project, OTs at several sites shared that they prioritized other therapy activities over PAT when knowing that there was insufficient time to provide the recommended 10 sessions of PAT. This is due to the fact that the length of stay is usually estimated and predetermined by insurers. Extending approved length of stay in an inpatient rehab solely for the purpose of completing the KF-NAP protocol may be applicable for a small percentage of patients with private insurers. For patients covered by Medicare's prospective payment plan, the hospital may elect to extent a patient's length of stay if the interdisciplinary team feels the additional days to complete the protocol would outweigh benefits of the projected discharge date. Among 2019 Medicare beneficiaries, for example, the average length of stay is 17, 16, and 15 days for patients with stroke, traumatic brain injury, and non-traumatic brain injury, respectively. Even if patients are assessed using the KF-NAP within the first few days of admission, there may be <10 full treatment days for a therapist to provide the recommended 10 oncedaily PAT sessions. This factor in addition to many other factors may contribute to a decision not to provide PAT to certain patients.
The limited number of trained OTs in either protocol was another major barrier. This seems inconsistent with the fact that about 170 therapists were trained to integrate the KF-NAP and about 140 therapists were trained to use the KF-PAT in their practice across 16 IRFs. However, 10 participating sites primarily relied on lead OTs in administering the protocols, Who were unable to train other OTs due to time constraints, other clinical duties, and factors related to readiness of certain hospitals. The overall 45% change rate of lead OTs created lags of researcher-clinician communication and further slowed down the implementation progress. Regarding user feedback, 46.7% of survey respondents reported that it would be beneficial to have meetings or calls with their individual leadership, so they could share any continued barriers and determine strategies together. Almost 90% of the survey respondents mentioned that they would be interested in a yearly refresher course, on both protocols. This may help sustain implementation and also ensure fidelity (43). Thus, after collectively providing spatial neglect care to more than 4,500 patients in the context of this implementation project, OTs saw the need of continuing implementing KF-NAP and KF-PAT in their practice.

Study Limitations
The project was initially driven by researchers and fueled by a collaborative effort shared by researchers and clinicians (including hospital administrative leaders). This is a strength but also the limitation of the project such that the outcomes may not be generalizable to facilities that have little access to researchers, especially researchers knowledgeable about implementation science. Another limitation was the inability to evaluate which strategies that were used toward eliminating a given barrier during the implementation process, had the best success rates vs. other strategies. It was not our priority to determine the best strategy but to offer solutions at the time when a barrier was present. Therefore, we cannot comment on the recommended strategies or which strategies should be trialed first. Further investigations formally testing outcomes such as feasibility, adoption and acceptability of delivering the protocols are needed to identify all the challenges to maintaining the implementation, and to determine how to overcome those challenges (43).

CONCLUSION
The project demonstrated a researcher-clinician partnership in not only knowledge generation but also knowledge translation (e.g., dissemination and implementation of knowledge to be applied clinically). Evidence-based protocols can be implemented through multiple, tireless iterations of barrier reduction and problem solving with active participation of practitioners and practical support from leaders. There were no unintended consequences of the implementation efforts. Frequent communications and exchanging information with stakeholders at different levels, may be determinant to the success of each implementation phase. The results of the present project appeared promising in EBP implementation for spatial neglect care. However, further efforts are needed to promote the persistent inclusion of EBP for spatial neglect as the standard of care in inpatient rehabilitation. We also suggest the following future implementation efforts: (1) a pre-trial consultation with organizational leadership could ensure that sufficient clinician time can be blocked out, and (2) enabling all staff to receive training and deliver the intervention with higher levels of fidelity.

DATA AVAILABILITY STATEMENT
The data is not available to be shared.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Kessler Foundation. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

AUTHOR CONTRIBUTIONS
KH and PC designed the study, analyzed and interpreted the data, and then wrote the manuscript. PC, KH, and CG-S interpreted the qualitative component of the study. JM contributed to editing the manuscript and was essential in data collection efforts. CG-S was essential to study member recruitment and edits to the entire manuscript. AB was key to study design and edited the entire manuscript. RG edited the manuscript. All authors read and approved the final manuscript.

FUNDING
This work was supported by the Wallerstein Foundation for Geriatric Improvement and the Charles and Ann Serraino Foundation. Contents in this paper do not represent the policy of the funding agencies.