Locally-tailored vs. centrally-administered strategies for implementation of primary human papillomavirus (HPV) screening in an integrated healthcare system: a qualitative research study

Introduction: Primary human papillomavirus (HPV) testing is recommended for cervical cancer screening for women aged 30–65 years without a history of abnormal results. However, there is little clear guidance regarding effective strategies for implementing primary HPV screening. As part of an ongoing randomized trial comparing implementation strategies for primary HPV testing (a centrally administered + usual care strategy vs. centrally administered + locally tailored strategy), we evaluated clinician experiences and perceptions of large-scale implementation of primary HPV screening in an integrated healthcare system, Kaiser Permanente Southern California.

Materials and methods: We conducted qualitative interviews with internal medicine, family medicine and obstetrics/gynecology clinicians to gain insight into fidelity to the interventions and implementation strategies, barriers and facilitators to implementation, and recommendations. Participants from both arms of the trial were recruited. Interview guides were developed with the Consolidated Framework for Implementation Research (CFIR). We recruited physicians, licensed vocational nurses, and medical assistants after primary HPV screening had been implemented. Interviews were recorded and transcribed. Using a team coding approach, we developed an initial coding structure refined during iterative analysis; data were subsequently organized thematically into domains, key themes, and sub-themes using thematic analysis, followed by framework analysis informed by CFIR.

Results: Thirty-two interviews were conducted. Participants in both arms of the trial noted high awareness, preparedness, buy-in, and fidelity to the new screening process. Initial barriers concerned specimen collection, proper ordering, and lab delays. An unanticipated barrier was the length of time needed to return lab results for reflexive cytology tests after a positive HPV result which reportedly increased patient anxiety. Participants in both arms reported fidelity to the centralized strategy (e.g., attending webinars, leadership announcements). In the local-tailored arm, few participants recalled the local-tailored resources.

Discussion: The centralized strategy was perceived as highly acceptable and feasible, and fidelity to the associated interventions appear to be facilitators of practice change. Recommendations for improving implementation included patient education, outreach and ongoing clinician training. Findings can be applied to other health systems and settings considering primary HPV screening implementation, particularly those within the U.S. or with a similar health care model.

Trial Registration: ClinicalTrials.gov, identifier #NCT04371887

Introduction

Within the United States, primary human papillomavirus (HPV) testing is now recommended for cervical cancer screening for women aged 30–65 years with recommendations from the United States Preventive Services Task Force (USPSTF) (1) and other professional societies (2–4). These recommendations are based on studies demonstrating that primary HPV testing is superior to cytology alone (Papanicolaou testing, or Pap) (5) and is as effective as co-testing (Pap plus HPV test) (6–8). Many healthcare organizations are preparing to switch from the previously recommended strategies of co-testing or Pap alone and initiate primary HPV testing, as has been done in several other countries including the Netherlands, Turkey, Canada, England, and Australia (9–12).

Previous research has investigated providers' perceptions and knowledge of the HPV test and primary HPV screening. Studies have found that a variety of clinicians in varying roles viewed primary HPV testing positively (13, 14), knowledge-level among clinicians predicts willingness to accept primary HPV testing (15), and strategies are needed to increase provider knowledge regarding primary HPV screening test effectiveness and guideline-recommended screening intervals (16). Patients' perceptions of primary HPV testing are often less accepting (17–19), with research suggesting that increasing women's knowledge about HPV and HPV testing may facilitate the acceptability of primary HPV testing (20), including the development of patient-directed communication which highlights the effectiveness of primary HPV screening (19).

Based on these experiences and modeling studies, the transition to primary HPV testing is expected to result in improved cost effectiveness and simplified clinical care processes for those converting from co-testing to primary HPV testing (e.g., one sample taken instead of two at point of care) (21, 22). However, there is little clear guidance regarding effective strategies for achieving optimal implementation and stakeholder-centered outcomes. Within the United States, the decision to transition to primary HPV screening is mainly left to individual health systems rather than implemented on a national or state level. In 2020, Kaiser Permanente Southern California (KPSC), a large integrated health system, enacted a system-level transition from HPV co-testing to primary HPV testing for women aged 30–65 in response to the updated USPSTF guidelines.

The overarching goal of the main trial was to compare two implementation strategies for primary HPV testing: a “centrally-administered + usual care” strategy (hereafter referred to as “centrally-administered”) vs. “centrally-administered + locally-tailored” strategy (hereafter referred to as “locally-tailored”). The centrally-administered strategy that all sites received included clinician-level interventions such as education (e.g., training webinars, announcements from local and regional leadership) as well as system-level interventions including infrastructure changes (e.g., laboratory preparedness, electronic medical record alerts and cervical cancer screening order redesign). The locally-tailored strategy added adaptable interventions such as additional clinician education and patient educational materials for clinician-patient communication tailored to local clinic needs and context. We were guided by the core function/form framework as the basis for complex health intervention adaptation and tailoring (23–26). Clinic teams at clinics in the locally-tailored arm of the trial were convened to rank needs/barriers followed by review and discussion of rankings and selection of locally-tailored intervention options. A detailed description of the clinical trial protocol has been previously published (27) as well as the main findings (28).

This manuscript presents the post-implementation qualitative research design, recruitment efforts, materials and methods, and qualitative thematic findings.

Materials and methods

Study design

As part of a cluster randomized trial evaluating implementation strategies to facilitate primary HPV testing, we conducted a series of post-implementation semi-structured qualitative interviews with KPSC clinicians to evaluate perceptions and experiences with the implementation strategies and interventions. Our findings are reported using the criteria for reporting qualitative research (COREQ) (29). The checklist can be found in Additional File 1.

Study setting and participants

KPSC is an integrated healthcare delivery system serving over 4.4 million members. Members are racially, ethnically, and socioeconomically diverse and broadly representative of the underlying Southern California population (30). The transition to primary HPV testing was implemented in mid/late 2020, with educational interventions concluding in late fall 2020. We conducted interviews with participants from the centrally-administered arm of the trial starting early 2021. Clinicians from the tailored arm were recruited in early 2022, after the tailoring activities and tailored-intervention delivery were complete (summer-fall 2021). We invited clinicians practicing in settings affected by the practice change in internal medicine, family medicine, and obstetrics/gynecology clinics, including physicians, licensed vocational nurses (LVNs), and medical assistants (MAs). We obtained verbal consent from all participants, which was approved by the KPSC Institutional Review Board [IRB #12015].

Interview guide and conceptual model

We conducted interviews both prior to the practice change (pre-implementation) to gain insight into perceived barriers and recommendations and after the implementation period had concluded (post-implementation). Details on the baseline interviews have been previously published (14). Here we present our post-implementation qualitative findings from clinician interviews, which included stakeholders from both arms of the trial. The interviews focused on reflection and evaluation of their care transition experience, following guidance from the Consolidated Framework for Implementation Research (CFIR), specifically the “Implementation Processes' domain (46).

Interviews were facilitated using semi-structured interview guides. The overall goal of the interviews was to obtain reflections, evaluation, and experiences with the primary HPV screening implementation over time, after the initial implementation period had concluded. We also assessed awareness of and fidelity to the implementation strategies. For this study, we refer to the term “fidelity” to describe reported stakeholder receipt, awareness, and use of the implementation strategies (e.g., messaging and materials provided by either the centrally-administered or locally-administered strategies in preparation of the transition to primary HPV screening), as well as fidelity to conducting primary HPV screening (e.g., did they transition away from co-testing to adoption of primary HPV testing).

The guides included open-ended questions based on central research questions and probing questions, which allowed for deep exploration of participant responses and probes for emergent themes. Our iterative interview guide development was informed by the CFIR (31). The CFIR is widely recognized to provide a comprehensive overview of relevant constructs applicable to implementation research and clinical practice change (32). We developed potential interview questions based on the HPV testing and implementation science literature and mapped the question domains to relevant CFIR constructs. The interview guide was further discussed and refined by the study team after the initial 2–3 interviews.

Domains in the interview guides included the following: (1) pre-implementation resources and training (e.g., “Thinking back, what primary HPV testing educational and/or training resources were you provided before the change to HPV primary testing?”, “To what extent do you believe you were sufficiently trained and prepared to personally help implement this practice change?” (2) intervention implementation—fidelity and team feedback/engagement (e.g., “Has your team fully adopted the change from co-testing to HPV primary testing?” (3) barriers and facilitators to implementation (e.g., “Overall, how easy or difficult was changing from co-testing to HPV primary testing?”, “How have patients reacted to this practice change?” (4) adaptations to recommended care process required to change from co-testing to HPV primary testing (e.g., “Did your team make any changes or adaptations to the recommended care processes required to change from co-testing to HPV primary testing?”). The domains and questions in the interview guides for both the tailored and usual care arm were the same. The interview guide is available in Additional File 2.

Recruitment

To attain theoretical data saturation, we planned 25–35 total interviews. We emailed invitations (including a study information sheet describing the research team's interest in the research topic and interview format) and three reminders to approximately 1,300 clinicians assuming anywhere from a 2%–10% positive response and, of those, 20%–50% who would schedule, and complete interviews based on our prior experience in conducting qualitative research with these 7–8 stakeholder groups (33–35). No prior relationships existed between the interviewers (CMP, NTC, CH) and participants prior to study commencement. Participants were informed about the interviewers' role as qualitative researchers on the study team, who had an interest in improving cervical cancer screening practices.

Interview procedures

We conducted one-time individual semi-structured interviews. Interviews were conducted over the telephone or via Microsoft TEAMS by CMP, NTC, and CH. Interviews lasted approximately 30–60 min and were digitally recorded and transcribed verbatim into written transcripts as preparation for coding and analysis. We halted further interviews once we were confident that we reached thematic saturation with our interview sample whereby we were no longer eliciting new pertinent information or themes from additional interviews (36–39). Transcription was conducted by an institutional approved vendor. We collected gender and profession data from KPSC's clinician database.

Coding and analysis

Three research team members with experience in qualitative research (CMP, CH, EEH) coded, organized and analyzed the resulting qualitative data. Analysis followed the table-based matrix approaches as described by Miles and Huberman (40), which allows for pattern analysis of responses. This facilitated identification of items related to fidelity, barriers and facilitators to implementation, use of adaptations, and specific recommendations for refinement. All analysis was conducted using NVivo qualitative software (© QSR International 2020).

We identified a lead coder (CMP) with over 25 years' experience in qualitative data collection and analyses and two secondary coders (EEH and CH) to conduct team coding and analysis. We developed an initial coding hierarchy, which was reviewed and modified iteratively throughout the analysis. All coding development steps were tracked and reported in a codebook, which contributed to the rigor and transparency of the process (41). Coding team members independently coded a random sample of 3 transcripts from both the centralized and the locally-tailored sites. Coders then met several times to compare the independently coded transcripts and analytical memos, work through divergent views of the textual analysis, and achieve consensus. From there, the lead coder coded the remaining transcripts and once that step was complete, the coding team reconvened to finalize the analysis, producing a number of summary documents categorized into domains and primary and secondary themes (i.e., classification trees, quotes-by-theme tables, etc.). Our team did capture brief notes to share with the study team in real-time, these notes were not included as part of the formal analysis, and we did not share transcripts or summary data with the interviewees.

Results

Participant characteristics

Thirty-two clinicians completed interviews. Table 1 provides a breakdown of the roles and characteristics. Most participants (88%) were female, 50% were physicians, 44% LVNs, and 6% MAs. Of the thirty-two individuals interviewed, 56% were from the centralized sites and 44% from the locally-tailored sites (Table 1).

Table 1

Table 1. Clinician participant characteristics (n = 32).

Thematic findings

Participants reflected on the feasibility and acceptability of the practice change, fidelity to the intervention and implementation strategies, barriers to implementation, and use of adaptations (see detailed quotes included in Table 2). Participant recommendations for improving implementation are summarized in Table 3.

Table 2

Table 2. Providers' post-implementation perspectives about the transition to primary HPV cervical cancer screening for patients with a normal screening history, aged 30–65 years.

Table 3

Table 3. Participant recommendations to improve implementation of primary HPV cervical cancer screening.

Feasibility and acceptability of the practice change

Participants from both arms of the trial noted awareness, preparedness, and buy-in regarding the transition to primary HPV screening was high and that primary HPV screening was quicker and easier. Most clinicians understood the change was coming due to communication from the internal KPSC HPV Cervical Cancer Screening Task Force, and overall, the implementation reportedly “…seemed to go okay.” The general impression of the practice change was that it improved efficiency because teams would be collecting only one specimen as opposed to two (“…only have to touch the cervix once”) and was evidenced based (“I don't remember anyone having any major concerns about this at all. Scientifically, I think we all kind of accepted that this made sense.”). In hindsight, clinicians perceived it as an easier practice change than previous transitions at KP, given that most changes to practice add to the clinical workload, rather than reduced effort (“And I don't know, I felt they did a better job than other times when changes have occurred.”).

Fidelity to the practice change and implementation strategies

Fidelity to the primary HPV screening practice change.

Fidelity to the new HPV primary screening process was high and clinicians reported quickly shifting to the new process and using Pap reflexively according to guidelines (i.e., after a positive HPV result). Many participants reported that the new screening process was quicker and easier to conduct (“It was pretty easy just to, “Okay, you're going to do one sample and one specimen container.” And that's it.”). In addition, access to the clinician education (emails, webinars) and provision of correct supplies (i.e., exam tray set-up) were key facilitators for MAs and LVNs to conduct the screening:

“I know they gave a flyer because I saw the flyer on my desk. That was very self-explanatory..But that was enough…[to explain]…it's going to be the 2-in-1 in one vial and then showing us a little vial with a little scrape, a little brush…Instructions…”

Fidelity to the centrally-administered strategy.

Regarding the centrally-administered implementation, which included the core educational elements (e.g., emails to department chiefs to share with clinical teams, webinars, materials), participants reported general awareness and use of the implementation strategies, indicating a degree of fidelity to the centrally-administered strategy:

“So, I think the first, initial way that I found out about [the transition to primary HPV screening] was via an email…I think it even had a video to it and I didn't look at the video for a really long time, I don't think. I think I just let it sit in my inbox, but then I did look at it. And if you went through the video, it was super self-explanatory…I know that the actual ADAs [assistant department administrators] didn't really explain it to us and go over it until it was almost implemented.”

The dose of the implementation strategies was perceived as variable, with some participants remembering multiple strategies and others one or none [“They were just told that here is the date we're switching to just doing one (specimen).”] Overall, the strategies as delivered had an impact and allowed the clinical teams to make the transition smoothly (“Actually, I think it went pretty smooth in our department. They did talk to us about it… we had a whole little meeting about it, maybe about two weeks prior to the big change.”). However, the majority of participants were unsure of all of the details of the strategies (“Yeah, emails, and I think it was mentioned in a meeting. Like, I don't remember the exact setting of…who talked about it and what they talked about, but I remember we had some education about it.”), and some did not recall any at all. Some participants did note the timing of the implementation during COVID [“I feel like (leadership) were more hands-on with the whole COVID and what's happening, and I felt like that's what they were focused on.”].

Fidelity to the locally-tailored strategy

Within the local-tailored arm, where teams received additional resources that were tailored to their reported implementation needs, most participants could not recall receiving the locally tailored, patient- and clinician-facing materials: “I don't recall them really touching the subject [HPV primary screening] again. But again, it was the pandemic, and there were other things they wanted to discuss.” While few remembered receiving locally tailored material, participants agreed that getting follow-up and locally tailored materials would be a good reinforcement strategy for ongoing education.

Barriers to implementation

Initial barriers experienced immediately after the implementation were around specimen collection, which tests to order, and lab delays. One member of the nursing team explained:

“[In the beginning]…it was just hard for the nurses…they kept both orders in our system for some reason, and so the nurses for the longest time didn't know which order to put in.”

Another participant referenced the initial challenges some clinical teams experienced pending orders:

“Like I said, because me and my doctor, it took us a little while to finally figure out the correct way to order when it was a person who had an abnormal history vs. a person who had a normal history.”

There were several key barriers that persisted over time. The unanticipated increased length of time it took for laboratory to provide results for reflexive cytology tests (Pap) in cases where the primary HPV screening was positive increased patient anxiety and confusion. This in turn resulted in a reported uptick of patient email messaging and phone calls to clinicians and staff:

“Unfortunately, we're in this world where everything gets resulted and available to the patient instantly, which in this case is bad because to know you have a positive HPV and now have been told, ‘We need to wait for the Pap results,’ that's where the calls and confusion come in. So, we either stop giving them HPV results, but we can't, or you preempt by telling them, ‘If it's positive, we have to wait for the Pap. Then we'll let you know how we're going to triage you.’”

In addition, some participants feared missing cancer when transitioning from co-testing to primary HPV testing. This resolved for some members of the team after the practice change, but lingered for others, even months after the initial implementation period. One participant noted:

“I share an office with somebody. The two of us I remember were discussing like, ‘Oh, occasionally we'll have somebody who is HPV-negative but still have something on their cervical pathology.’ So, there was maybe a little concern that something would be missed… Honestly speaking, a lot of the LVNs don't know the answer to that…And I'll be honest, I still don't know the answer to that.”

On the other hand, patient pushback regarding the practice change was reportedly minimal. Clinicians largely attributed this to limited patient knowledge about the practice change as clinicians had switched their wording to “cervical cancer screening' rather than “Pap smear' when describing the screening service, and there had been little patient outreach about the practice change. As a result, many patients were unaware or unconcerned about the change unless their clinicians mentioned it (“I do a lot of messages for my clinic providers and there are a lot of patients requesting their Pap results that I am unable to give and (were) not aware of the change until a provider mentioned it without explanation.”). One MA remarked, “It would help if every woman eligible for routine screening got an email or letter that explains the change automatically.”

Use of adaptations across sites

The majority of participants did not report any specific adaptations to the practice change, rolling it out as the region requested (“I think we did it exactly as prescribed..We didn't make any adaptations.”). One Ob-Gyn LVN explained how her team worked to approach the practice change cohesively and by design, “…the team was pretty good about, obviously, if they weren't getting it right, then they would just come and ask questions, which was good.”

Recommended refinement to the practice change approach

Participants offered recommendations for improving implementation focused on suggestions for patient education and clinician training. For instance, better education for patients about the sequencing of testing and the expected resulting timeframe was recommended to alleviate patient anxiety while waiting for reflexive cytology results. Other clinicians, such as the Ob-Gyn physician below, suggested it would be better to provide education materials to patients directly (e.g., through messages via the patient portal) rather than relying on clinicians to offer them to patients as clinicians have limited time and educational materials often get lost in message overload:

“I don't know what is being shared with patients…maybe if they sent this to the patient [in a] patient friendly format. This is how we're doing this now, and 99.7% is caused by HPV, and explained it in patient language. I think that would help the providers. I don't know if that was done.”

MAs and LVNs recommended standardized communication/training for nursing staff, perceiving they did not receive the same extent/quality of training as physicians (“As screening nurses we learn about the process on collecting the specimen not why it is recommended. Only the doctors go to these seminars to learn the why's.”). One LVN suggested that when it comes to training for a practice change like this, treat the nurses and physicians like a team:

“I would think you would just need a team. A team that's going to educate everybody. Not separate teams. Not a physician rolling it out and explaining it to the physicians and then another nurse rolling it out to the staff. Because obviously, they're not on the same page as to explaining or giving the information…say you had a team that was running…the HPV rollout and that same team explained it to the doctors and that same team explained it to the staff because then we know we're all getting the same message.”

Discussion

Overall, participants reported smooth implementation and a high degree of fidelity to the new HPV primary screening process. The ease of implementation was often credited to the simple practice change of going from two clinical specimens to one. Initial barriers included getting comfortable with the new collection process and entering proper cervical cancer testing orders for patients with both routine and abnormal history. Ongoing challenges included some clinicians continued fear of missing cancers without co-testing, and patient lack of awareness regarding the practice change prior to implementation. An important but unanticipated barrier was the delay in results for the reflexing cytology (Pap) after a positive primary HPV test, resulting in patient anxiety about the positive HPV test while waiting for the reflexive cytology result. This may be a significant challenge for other health systems making the transition to primary HPV cervical cancer screening. This finding echoes those from a recent study from Australia, where national primary HPV screening was implemented starting in 2017. In the Australian study, key stakeholders in the implementation and use of primary HPV screening (clinicians, administrators, policy makers, pathology laboratories) were evenly divided on whether the implementation was successful (42). While overall support for program implementation was high, significant barriers were identified with a particular emphasis on feasibility. Barriers to feasibility included lack of knowledge regarding clinical guidelines and testing procedures at the clinician- and lab-level, errors in correspondence regarding results, and increased need for colposcopy. Although the overall uptake and acceptability of the primary HPV screening program was high in the KPSC setting in our study, the concerns regarding communication and understanding of reflexive cytology results was a significant issue.

Regarding fidelity to the implementation strategies, there was reported fidelity to the centralized educational strategy for clinicians (e.g., attending webinars, reading emails from department leadership, etc.) and these interventions appear to have been a strong facilitator of practice change. However, some participants reported low or no engagement with (or memory of) the educational resources provided via the centralized approach. In the local-tailored arm, few remembered the local-tailored resources provided, and it was difficult for participants to determine what was centralized and what was tailored at the local-tailored sites. Additionally, the few who could recall receiving specific materials generally didn't remember using them in their clinic. Interestingly, the data showed some contradiction regarding the local-tailored materials: participants agreed that receiving tailored follow-up materials and resources is a good approach that can help with process change (e.g., reinforcement); however, few remembered receiving or using them. The COVID pandemic likely affected recall, with several participants remarking that the pandemic was front and center such that educational efforts focused on other issues were not attended to or remembered. Additionally, the tailoring activities occurred after the centralized approach from the health system, potentially blurring the activities together in participant memory. It is also possible that the “dose' of the locally-tailored strategy (1–2 meetings with the local team, lack of in-person meetings for discussion and dissemination) was not enough to produce meaningful differences in fidelity or uptake/memory of tailored intervention materials.

Participant recommendations to improve implementation included educating patients prior to implementation about primary HPV screening and what patients should expect regarding the sequence of results in the event of a positive primary HPV test result. Within the context of other recent studies conducted in the U.S. and internationally, patient-facing education about primary HPV screening and interpreting HPV results should be a key element of implementation. In a recent qualitative study conducted in England, where a national primary HPV screening strategy was implemented, women reported confusion and multiple unanswered questions about both the HPV screening protocol and the meaning of the HPV result (43). This was particularly acute for women with lower educational attainment. Another study from England using a cross-sectional survey design found that women who received their first positive HPV result had high anxiety about cervical cancer (44). A 2015 survey study of U.S. women found that HPV primary screening was the least acceptable cervical cancer screening option (17). Increasing patients' knowledge about primary HPV testing for routine cervical cancer screening and interpretation of results—including the need for reflexive cytology for those with positive HPV tests—in advance of widespread implementation may help to reduce patient anxiety and decrease the burden on clinicians. Patient education about cervical cancer screening modalities has been shown to be effective; for example, a short video about preferred method and interval of screening was found to increase acceptance of screening modalities other than annual Pap smear in screening-eligible women (45).

We also found that nursing staff, including MAs and LVNs, strongly recommended standardized pre-implementation communication and training for all clinical staff; they perceived that they did not receive the same extent and quality of training as physicians/advanced practice providers. Training across clinical teams may be a key element of primary HPV screening implementation. This finding is echoed in a recent study of physicians, advanced practice providers, and MAs based in Indiana (U.S.) where cross-sectional survey findings illuminated limited acceptance and uptake of primary HPV screening, with only 3% of participants reporting using primary HPV testing and 50% willing to adopt it (15). In-depth interviews conducted as part of the study found high levels of uncertainty, knowledge gaps, and perceived limitations of primary HPV screening (45). Within the U.S., there is the added layer of complexity that U.S. guidelines currently recommend a suite of screening strategies with the new addition of primary HPV as an option [screening every 3 years with cytology alone, every 5 years with HPV testing alone or every 5 years with HPV testing in combination with cytology (co-testing) in women 30–65 years, based on risk level], European guidelines unequivocally recommend primary HPV screening (9). Recommending multiple options for screening may compound the difficulties of implementation in the U.S. context given the multiple options recommended in current clinical practice guidelines.

Limitations

The primary limitations related to the interview results are potential selection bias and imperfect recall. Selection bias may have occurred because we were only able to interview a few members of any clinical team across sites and the perspectives of those interviewed may not be representative of the views of other team members who did not participate. Clinicians who completed interviews may be somewhat different compared to clinicians who did not respond. Imperfect recall likely occurred, as there was indication that some participants were unclear on what materials and resources they received from the centralized vs. the locally tailored strategy. In addition, all participants were from a single large integrated healthcare system (KPSC) which may limit the overall generalizability of results. However, other studies conducted both within the U.S. and internationally have identified similar themes, particularly around the need for coordinated education for both patients and clinicians.

In conclusion, despite challenges, the overall implementation of primary HPV screening had high acceptability, feasibility, and fidelity, and the centralized strategy appears to have been an effective implementation strategy for this practice change. Initial challenges to the rollout included specimen collection and primary HPV ordering issues within the electronic medical record. Other barriers included fear of missed cancer, variations in communication and training for nurses vs. physicians, and increased patient anxiety due to delays in results for reflexive cytology when the primary HPV test was positive. Our findings can be applied to other health systems and settings considering implementation, particularly those within the U.S. or with a similar health care model.

Data availability statement

The datasets presented in this article are not readily available because Individual-level data reported in this study involving human research participants are not publicly shared due to potentially identifying or sensitive patient information. Upon request, and subject to review, KPSC may provide the deidentified aggregate-level data that support the findings of this study. Anonymized data (deidentified data including participant data as applicable) that support the findings of this study may be made available from the investigative team in the following conditions: (1) agreement to collaborate with the study team on all publications, (2) provision of external funding for administrative and investigator time necessary for this collaboration, (3) demonstration that the external investigative team is qualified and has documented evidence of training for human subjects protections, and (4) agreement to abide by the terms outlined in data use agreements between institutions. Requests to access the datasets should be directed toQ2h1bi5SLkNoYW9Aa3Aub3Jn.

Ethics statement

The studies involving humans were approved by Kaiser Permanente Southern California Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The ethics committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants' legal guardians/next of kin because the interviews were with clinicians. The research poses minimal risk, is impractical to conduct otherwise, and won't jeopardize clinicians employment or healthcare.

Author contributions

EH: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. CM-P: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. CH: Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. NC: Funding acquisition, Project administration, Resources, Writing – original draft, Writing – review & editing. QN-M: Methodology, Writing – original draft, Writing – review & editing. MG: Methodology, Writing – original draft, Writing – review & editing. BM: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. MH: Methodology, Writing – original draft, Writing – review & editing. DT: Conceptualization, Methodology, Writing – original draft, Writing – review & editing. CC: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. Research reported in this manuscript was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (CDR-2018C1-10987).

Acknowledgments

We would like to thank our interview participants for giving their time for our study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2025.1595934/full#supplementary-material

References

1. Curry SJ, Krist AH, Owens DK, Barry MJ, Caughey AB, Davidson KW, et al. Screening for cervical cancer: US preventive services task force recommendation statement. JAMA. (2018) 320(7):674–86. doi: 10.1001/jama.2018.10897

PubMed Abstract | Crossref Full Text | Google Scholar

2. Practice bulletin No. 168: cervical cancer screening and prevention. Obstet Gynecol. (2016) 128(4):e111–e30. doi: 10.1097/AOG.0000000000001708

PubMed Abstract | Crossref Full Text | Google Scholar

3. Fontham ETH, Wolf AMD, Church TR, Etzioni R, Flowers CR, Herzig A, et al. Cervical cancer screening for individuals at average risk: 2020 guideline update from the American cancer society. CA Cancer J Clin. (2020) 70(5):321–46. doi: 10.3322/caac.21628

PubMed Abstract | Crossref Full Text | Google Scholar

4. Huh WK, Ault KA, Chelmow D, Davey DD, Goulart RA, Garcia FAR, et al. Use of primary high-risk human papillomavirus testing for cervical cancer screening: interim clinical guidance. Obstet Gynecol. (2015) 125(2):330–7. doi: 10.1097/AOG.0000000000000669

PubMed Abstract | Crossref Full Text | Google Scholar

5. Ronco G, Dillner J, Elfström KM, Tunesi S, Snijders PJ, Arbyn M, et al. Efficacy of HPV-based screening for prevention of invasive cervical cancer: follow-up of four European randomised controlled trials. Lancet. (2014) 383(9916):524–32. doi: 10.1016/S0140-6736(13)62218-7

PubMed Abstract | Crossref Full Text | Google Scholar

6. Moyer VA. Screening for cervical cancer: u.S. Preventive services task force recommendation statement. Ann Intern Med. (2012) 156(12):880–91. w312. doi: 10.7326/0003-4819-156-12-201206190-00424

PubMed Abstract | Crossref Full Text | Google Scholar

7. Massad LS, Einstein MH, Huh WK, Katki HA, Kinney WK, Schiffman M, et al. 2012 Updated consensus guidelines for the management of abnormal cervical cancer screening tests and cancer precursors. Obstet Gynecol. (2013) 121(4):829–46. doi: 10.1097/AOG.0b013e3182883a34

PubMed Abstract | Crossref Full Text | Google Scholar

8. Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain J, et al. American Cancer society, American society for colposcopy and cervical pathology, and American society for clinical pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin. (2012) 62(3):147–72. doi: 10.3322/caac.21139

PubMed Abstract | Crossref Full Text | Google Scholar

9. Maver PJ, Poljak M. Primary HPV-based cervical cancer screening in Europe: implementation status, challenges, and future plans. Clin Microbiol Infect. (2020) 26(5):579–83. doi: 10.1016/j.cmi.2019.09.006

PubMed Abstract | Crossref Full Text | Google Scholar

10. Machalek DA, Roberts JM, Garland SM, Thurloe J, Richards A, Chambers I, et al. Routine cervical screening by primary HPV testing: early findings in the renewed national cervical screening program. Med J Aust. (2019) 211(3):113–9. doi: 10.5694/mja2.50223

PubMed Abstract | Crossref Full Text | Google Scholar

11. Polman NJ, Snijders PJF, Kenter GG, Berkhof J, Meijer C. HPV-based cervical screening: rationale, expectations and future perspectives of the new Dutch screening programme. Prev Med. (2019) 119:108–17. doi: 10.1016/j.ypmed.2018.12.021

PubMed Abstract | Crossref Full Text | Google Scholar

12. Delpero E, Selk A. Shifting from cytology to HPV testing for cervical cancer screening in Canada. CMAJ. (2022) 194(17):E613–e5. doi: 10.1503/cmaj.211568

PubMed Abstract | Crossref Full Text | Google Scholar

13. Cooper CP, Saraiya M. Perceived effectiveness of HPV test as a primary screening modality among US providers. Prev Med. (2015) 78:33–7. doi: 10.1016/j.ypmed.2015.06.007

PubMed Abstract | Crossref Full Text | Google Scholar

14. Hahn EE, Munoz-Plaza C, Altman DE, Hsu C, Cannizzaro NT, Ngo-Metzger Q, et al. De-implementation and substitution of clinical care processes: stakeholder perspectives on the transition to primary human papillomavirus (HPV) testing for cervical cancer screening. Implement Sci Commun. (2021) 2(1):108. doi: 10.1186/s43058-021-00211-z

PubMed Abstract | Crossref Full Text | Google Scholar

15. Rodriguez NM, Brennan LP, Claure L, Balian LN, Kasting ML, Champion VL, et al. Clinician practices, knowledge, and attitudes regarding primary human papillomavirus testing for cervical cancer screening: a mixed-methods study in Indiana. Prev Med Rep. (2023) 31:102070. doi: 10.1016/j.pmedr.2022.102070

PubMed Abstract | Crossref Full Text | Google Scholar

16. Kruse GR, Lykken JM, Kim EJ, Haas JS, Higashi RT, Atlas SJ, et al. Provider beliefs in effectiveness and recommendations for primary HPV testing in 3 health-care systems. JNCI Cancer Spectrum. (2023) 7(1):pkac086. doi: 10.1093/jncics/pkac086

PubMed Abstract | Crossref Full Text | Google Scholar

17. Saraiya M, Kwan A, Cooper CP. Primary HPV testing: U.S. Women’s awareness and acceptance of an emerging screening modality. Prev Med. (2018) 108:111–4. doi: 10.1016/j.ypmed.2017.12.007

PubMed Abstract | Crossref Full Text | Google Scholar

18. Smith LW, Racey CS, Gondara L, Krajden M, Lee M, Martin RE, et al. Women’s acceptability of and experience with primary human papillomavirus testing for cervix screening: HPV FOCAL trial cross-sectional online survey results. BMJ Open. (2021) 11(10):e052084. doi: 10.1136/bmjopen-2021-052084

PubMed Abstract | Crossref Full Text | Google Scholar

19. Ogden SN, Glanz K, Schapira MM, Bocage C, Eriksen W, Rendle KA. Patient perspectives on primary HPV testing for routine cervical cancer screening. J Gen Intern Med. (2020) 35(3):956–8. doi: 10.1007/s11606-019-05175-4

PubMed Abstract | Crossref Full Text | Google Scholar

20. Tatar O, Thompson E, Naz A, Perez S, Shapiro GK, Wade K, et al. Factors associated with human papillomavirus (HPV) test acceptability in primary screening for cervical cancer: a mixed methods research synthesis. Prev Med. (2018) 116:40–50. doi: 10.1016/j.ypmed.2018.08.034

PubMed Abstract | Crossref Full Text | Google Scholar

21. Jansen E, Naber SK, Aitken CA, de Koning HJ, van Ballegooijen M, de Kok I. Cost-effectiveness of HPV-based cervical screening based on first year results in The Netherlands: a modelling study. BJOG. (2021) 128(3):573–82. doi: 10.1111/1471-0528.16400

PubMed Abstract | Crossref Full Text | Google Scholar

22. Smith MA, Canfell K. Projected impact of HPV vaccination and primary HPV screening on cervical adenocarcinoma: example from Australia. Papillomavirus Res. (2017) 3:134–41. doi: 10.1016/j.pvr.2017.04.003

PubMed Abstract | Crossref Full Text | Google Scholar

23. Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. (2019) 34(6):1032–8. doi: 10.1007/s11606-018-4818-7

PubMed Abstract | Crossref Full Text | Google Scholar

24. Esmail LC, Barasky R, Mittman BS, Hickam DH. Improving comparative effectiveness research of complex health interventions: standards from the patient-centered outcomes research institute (PCORI). J Gen Intern Med. (2020) 35(Suppl 2):875–81. doi: 10.1007/s11606-020-06093-6

PubMed Abstract | Crossref Full Text | Google Scholar

25. Jolles MP, Lengnick-Hall R, Mittman BS. Correction to: core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med. (2019) 34(6):1032–8. doi: 10.1007/s11606-018-4818-7

PubMed Abstract | Crossref Full Text | Google Scholar

26. PCORI Methodology Standards. Available at: https://www.pcori.org/research-related-projects/about-our-research/research-methodology/pcori-methodology-standards#Complex (Accessed June 11, 2025).

Google Scholar

27. Chao CR, Cannizzaro NT, Hahn EE, Tewari D, Ngo-Metzger Q, Hsu C, et al. A study protocol for a cluster randomized pragmatic trial for comparing strategies for implementing primary HPV testing for routine cervical cancer screening in a large health care system. Contemp Clin Trials. (2023) 124:106994. doi: 10.1016/j.cct.2022.106994

PubMed Abstract | Crossref Full Text | Google Scholar

28. Chao CR, Cannizzaro N, Hahn EE, Shen E, Hsu C, Ngo-Metzger Q, et al. A pragmatic randomized trial to compare strategies for implementing primary HPV testing for routine cervical cancer screening in a large healthcare system. Implement Sci. (2025) 20(1):21. doi: 10.1186/s13012-025-01432-9

PubMed Abstract | Crossref Full Text | Google Scholar

29. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. (2007) 19(6):349–57. doi: 10.1093/intqhc/mzm042

PubMed Abstract | Crossref Full Text | Google Scholar

30. Koebnick C, Langer-Gould AM, Gould MK, Chao CR, Iyer RL, Smith N, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US census bureau data. Perm J. (2012) 16(3):37–41. doi: 10.7812/TPP/12-031

PubMed Abstract | Crossref Full Text | Google Scholar

31. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | Crossref Full Text | Google Scholar

32. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Implement Sci. (2016) 11(1):72. doi: 10.1186/s13012-016-0437-z

PubMed Abstract | Crossref Full Text | Google Scholar

33. Hahn EE, Munoz-Plaza C, Wang J, Garcia Delgadillo J, Schottinger JE, Mittman BS, et al. Anxiety, culture, and expectations: oncologist-perceived factors associated with use of nonrecommended Serum tumor marker tests for surveillance of early-stage breast cancer. J Oncol Pract. (2017) 13(1):e77–90. doi: 10.1200/JOP.2016.014076

PubMed Abstract | Crossref Full Text | Google Scholar

34. Hahn EE, Munoz-Plaza CE, Schottinger JE, Brasfield FM, Gould MK, Parry C. Developing innovative models of care for cancer survivors: use of implementation science to guide evaluation of appropriateness and feasibility. Support Care Cancer. (2019) 27(5):1737–45. doi: 10.1007/s00520-018-4425-5

PubMed Abstract | Crossref Full Text | Google Scholar

35. Munoz-Plaza CE, Parry C, Hahn EE, Tang T, Nguyen HQ, Gould MK, et al. Integrating qualitative research methods into care improvement efforts within a learning health system: addressing antibiotic overuse. Health Res Policy Syst. (2016) 14(1):63. doi: 10.1186/s12961-016-0122-3

PubMed Abstract | Crossref Full Text | Google Scholar

36. Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. (2010) 25(10):1229–45. doi: 10.1080/08870440903194015

PubMed Abstract | Crossref Full Text | Google Scholar

37. Guest G, Bunce A, Johnson L. How many interviews are enough?: an experiment with data saturation and variability. Field Methods. (2006) 18(1):59–82. doi: 10.1177/1525822X05279903

Crossref Full Text | Google Scholar

38. Hennink MM, Kaiser BN, Weber MB. What influences saturation? Estimating sample sizes in focus group research. Qual Health Res. (2019) 29(10):1483–96. doi: 10.1177/1049732318821692

PubMed Abstract | Crossref Full Text | Google Scholar

39. Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. (2018) 52(4):1893–907. doi: 10.1007/s11135-017-0574-8

PubMed Abstract | Crossref Full Text | Google Scholar

40. Miles MB, Huberman AM, Saldana J. Qualitative Data Analysis: A Methods Sourcebook. London: Sage (2014).

Google Scholar

41. MacQueen KM, McLellan E, Kay K, Milstein B. Codebook development for team-based qualitative analysis. CAM J. (1998) 10(2):31–6. doi: 10.1177/1525822X980100020301

Crossref Full Text | Google Scholar

42. Brotherton JML, McDermott T, Smith MA, Machalek DA, Shilling H, Prang K-H, et al. Implementation of Australia’s primary human papillomavirus (HPV) cervical screening program: the STakeholders opinions of renewal implementation and experiences study. Prev Med Rep. (2023) 33:102213. doi: 10.1016/j.pmedr.2023.102213

PubMed Abstract | Crossref Full Text | Google Scholar

43. Mulcahy Symmons S, Waller J, McBride E. Testing positive for human papillomavirus (HPV) at primary HPV cervical screening: a qualitative exploration of women’s information needs and preferences for communication of results. Prev Med Rep. (2021) 24:101529. doi: 10.1016/j.pmedr.2021.101529

PubMed Abstract | Crossref Full Text | Google Scholar

44. McBride E, Marlow LAV, Forster AS, Ridout D, Kitchener H, Patnick J, et al. Anxiety and distress following receipt of results from routine HPV primary testing in cervical screening: the psychological impact of primary screening (PIPS) study. Int J Cancer. (2020) 146(8):2113–21. doi: 10.1002/ijc.32540

PubMed Abstract | Crossref Full Text | Google Scholar

45. Schrier E, Holt HK, Kuppermann M, Sawaya GF. Changing preferences for a cervical cancer screening strategy: moving away from annual testing. Women’s Health Rep. (2022) 3(1):709–17. doi: 10.1089/whr.2022.0007

PubMed Abstract | Crossref Full Text | Google Scholar

46. Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated consolidated framework for implementation research based on user feedback. Implement Sci. (2022) 17(1):75. doi: 10.1186/s13012-022-01245-0

PubMed Abstract | Crossref Full Text | Google Scholar