“Treatment is both psychological and medical”: applying mixed methods to the understanding of mental and behavioral health in pediatric specialty medicine

Introduction: Mental and behavioral health (MBH) access varies in pediatric specialty medicine and rural healthcare. This study aimed to characterize healthcare professionals' perceptions and experience with MBH services and readiness for change.

Methods: Thirty rural healthcare professionals completed validated questionnaires and semistructured interviews designed to meet study objectives. A convergent mixed methods design with a comparison method of interpretation was followed.

Results: Thematic analysis yielded three major themes highlighting the prevalence of MBH concerns in practice, global perceptions of MBH care, and related barriers/facilitators to implementation. Current levels of MBH integration varied though overall readiness for change was high.

Discussion: Findings highlight stakeholder perceptions about the role of MBH in pediatric specialty medicine. Key considerations for those who utilize or plan to adopt integrated MBH care at various levels are discussed, including readiness to change and other structural (e.g., organizational) or individual factors serving as facilitators and barriers to implementation.

Introduction

Pediatric mental health is a public health crisis and a leading cause of global disease burden (1, 2). Typical onset of mental health conditions is prior to 18 years of age and symptoms often persist into adulthood (1, 3). One in eight pediatric patients have a mental or behavioral health (MBH) condition warranting intervention (i.e., impairing development of well-being); yet many are not receiving care (1). The COVID-19 pandemic exacerbated MBH concerns globally (4, 5); however, pre-pandemic, MBH concerns and reduced health-related quality of life were well-documented for youth with complex medical needs (6). As risk and symptoms are likely to persist over time, youth with chronic health conditions warrant particular attention (7).

Areas of pediatric specialty medicine have recognized the need for MBH care standards, including routinely screening/assessing, preventing, and intervening upon MBH symptoms (8–11). For example, US News and World Report rankings are dependent upon access and integration of MBH services, including the implementation of MBH screening. Pediatric primary care has also established frameworks detailing strategies for how medical teams can integrate MBH specialists into standard care (12). A Standard Framework for Levels of Integrated Healthcare (13), for example, illustrates MBH integration as a continuum, ranging from minimal coordination (coordinated care) to full collaboration (integrated care) where roles and operations of medical and MBH care are part of a single, merged practice.

Despite literature supporting the efficacy of various care models, psychosocial screening tools, and MBH interventions, few studies have examined specific barriers and facilitators in these settings or ways to enhance implementation of MBH into pediatric specialty care and even fewer studies have examined these factors in the context of rural healthcare systems. A contributing factor to this void is the lack of partnership between healthcare professionals and administrators in developing the models they are expected to implement and/or utilize. For example, recent reports and commentaries describe how growth in healthcare administration has outpaced growth of physicians. This has facilitated a gap such that physicians and the teams they manage may feel burdened by increased scrutiny, workload demands, and regulatory requirements, contributing to physician burnout and fatigue (14, 15). This perpetuates a science-to-practice gap and misalignment between research, administrative regulations and practice operations, and the lived experiences of barriers and facilitators of healthcare professionals providing care to patients. Further, other recent research has discussed barriers experienced by healthcare professionals, including physicians, with the integration of psychosocial and MBH screening. A recent qualitative study in oncology, for example, described despite expectations to implement MBH screening, those expected to implement often perceived limited institutional support and commitment and often felt burdened by these practices without sufficient resources to implement or act upon screening results (16).

In academic medical settings, including smaller, under resourced, and rural healthcare systems, healthcare professionals such as physicians hold clinical care appointments in addition to clinic/division and hospital administrative roles. This provides a unique opportunity to evaluate preferences, facilitators, barriers, and other key considerations related to acceptability and feasibility of MBH integration in pediatric specialty care. The current study thus brings various healthcare professional and administrator perspectives together for this purpose.

Aims and objective

Using a convergent mixed methods (QUAL-quant) design, the study aimed to (1) identify key themes from semi-structured interviews about perceptions of MBH care and applicability to current practice; (2) examine levels of current MBH coordination/integration and readiness for change domains and (3) integrate findings to identify and explore relations between qualitative and quantitative indicators of lived experiences with MBH in everyday practice. While aims were exploratory, it was anticipated that (1) thematic analysis of semi-structured interviews would result in the identification of varied perceptions and experiences with MBH services within and across clinical care teams (i.e., pediatric specialty areas) and (2) that relations could be drawn between qualitative thematic data, current levels of MBH coordination/integration among specialty areas, and quantitatively-collected readiness for change factors.

Methods

Participants

Thirty-two healthcare professionals in pediatric specialty care were recruited from a hospital in the Appalachian region of the U.S. Per specialty area represented in our data, we attempted to recruit one healthcare professional with hospital or department-level administrative responsibilities plus at least two team members. Inclusion criteria for all participants were as follows: (1) healthcare professionals (e.g., physicians, nurse practitioners, registered nurses, dieticians, social workers, psychologists) who practice in a pediatric medical specialty (i.e., provide direct patient care or clinical teaching/supervision of individuals providing direct patient care, has direct interactions with other healthcare professionals and patients/families as part of scope of practice); (2) at least 1 year of practice/experience in specialty area; and (3) employed by the recruitment site for at least 6 months. Administrator-specific criteria also included intimate knowledge/experience with specialty practice operations as determined by 1) formal administrative designation or title (e.g., Director, Chief, Chair, Vice Chair) and (2) leadership role(s) and responsibilities within the specialty area or hospital (i.e., leads faculty and staff; involved in the development and/or implementation of hospital/specialty area regulatory procedures and policies; facilitates research, educational, clinical services, strategic planning, development, resource management). Students/trainees were excluded.

Participants were 30–62 years of age, with an average age of 42.5 years (SD = 9.9) for non-administrative health professionals and 50.7 years (SD = 9.1) for administrators. Most participants had 5 or more years of experience in their specialty area (n = 20, 86.9% of non-administrative healthcare professionals; n = 7, 100.0% of administrators). Most non-administrative healthcare professionals were female (n = 19, 87.0%) and White (n = 22, 95.7%). Most had an advanced degree (n = 17; 73.9%) and were trained in nursing (n = 12, 52.2%), medicine (n = 4, 17.4%), or social work (n = 4, 17.4%). Most administrators were also female (n = 4, 57.1%) and White (n = 5, 71.4%). All (n = 7, 100%) held an advanced (doctoral) degree in medicine. Additional details, including roles held on the clinical team, are described in Table 1.

Table 1

Table 1. Sample characteristics.

Procedure

Procedures approved by an Institutional Review Board (Protocol #1809263858). Reporting on this mixed-methods study follows the Journal Article Reporting Standards for Qualitative Research in Psychology, specifically the Mixed Methods Article Reporting Standards (17).

Recruitment and engagement

Participants were recruited across specialties and training backgrounds to capture breadth and depth in perspectives. Given potential barriers to recruiting and retaining healthcare professionals in research (e.g., time, fear of evaluation), evidence-based recruitment and engagement strategies (i.e., email, recruitment letters/flyers, in-person meetings, physician champion) were implemented (18, 19). Two authors (DNW and JK) conducted recruitment and engagement procedures and served as qualitative interviewers for the study.

While 32 healthcare professionals were enrolled, data from 30 participants were available for analysis. One healthcare professional was withdrawn due to lack of response from other healthcare professionals in their specialty area. Another participant withdrew due to an employment-related research restriction. Of the 11 pediatric specialty teams contacted, nine participated: Gastroenterology/Hepatology/Nutrition, Cystic Fibrosis, Neurology, Cardiology; Endocrinology; Hematology/Oncology, Nephrology/Hypertension; and the Pediatric Infusion Center. Psychology/Behavioral Medicine was also represented as a team and included MBH clinicians who deliver these services within one or more of the enrolled medical specialties. Of note, the Pediatric Infusion Center serves patients from multiple medical teams, including several represented in the study. Nevertheless, services provided in the Infusion Center were generally independent from those in the specialty clinic settings leading to its denotation as a separate specialty team. An exception was Hematology/Oncology given significant collaboration/coordination of care. Thus, comments made by Infusion Center participants specific to Hematology/Oncology services (and clarified as such during the interview) were described as Hematology/Oncology team findings. To maintain confidentiality due to the small sample sizes of each team (n ≤ 4 each), all team-specific data reported henceforth will use a randomly assigned team number (Teams 1–9). Among teams represented in the study, enrollment ranged from 66.7% to 100.0% and refusal ranged from 0.0% to 17.0%. According to sampling procedures, not all members of a team were approached; thus, enrollment and refusal rates were calculated based on the total number of individuals approached per team.

Sampling techniques

Consistent with the convergent mixed methods design, qualitative and quantitative data collection and analysis occurred concurrently (20). Purposive and theoretical sampling fostered representation of diverse professional backgrounds, roles, and experiences within/across settings and strategic recruitment of individuals/specialty areas with varied MBH experiences and care models. The first author (DNW) maintained a log to track recruitment decisions and use of these sampling strategies. For example, during a few healthcare professional interviews, it was recommended that the research team talk to an individual who holds a specific role within their team to more comprehensively answer some of our questions. This often led to our team attempting to recruit these individuals to ensure inclusiveness as these perspectives could influence resulting themes. Analytically, these approaches informed the determination of codebook stability, or the inability to identify new codes or themes not categorized within the existing codebook (21–24).

Measures

Qualitative semi-structured interview guide

Participants completed a semi-structured interview (60 minutes, sometimes split into two sessions). Interviews were conducted by phone or in-person with a trained interviewer (DNW or JK) and were audio-recorded and transcribed. The interview guide was developed in consultation with an expert in integrated primary care and piloted with ineligible healthcare professionals. Participants were compensated $50 (gift card).

Quantitative measures

Participants completed electronic study questionnaires via the Research Electronic Data Capture (REDCap) platform (25).

Healthcare professional information form (PIF)

The PIF is a study-specific measure of participant demographics, work environment, and role(s) within specialty teams.

Readiness for change questionnaire (RCQ)

To assess readiness for organizational change, participants completed the 25-item RCQ which has been previously validated with healthcare professionals (26, 27). Items were rated on a 7-point, Likert-type scale (“Strongly Disagree” to “Strongly Agree”) and correspond to four readiness for change domains (Appropriateness, Management Support, Change Efficacy, and Personally Beneficial) conceptually consistent with Holt and colleagues' (2010) framework. With permission, RCQ language was slightly modified (e.g., “management” changed to “director/administrator,” “organization” changed to “healthcare setting”). The RCQ demonstrated acceptable reliability across subscales (α = .77-.88).

Integrated practice assessment tool, version 2.0 (IPAT)

At their scheduled interview, administrators were administered the 8-item IPAT to determine their team's current level of MBH coordination/integration (28). Employing a decision-tree model and series of yes/no questions, the IPAT categorized each specialty medical team into either: Coordinated Care (Level 1, Minimal Collaboration and Level 2, Basic Collaboration at a Distance); Co-Located Care (Level 3, Basic Collaboration Onsite and Level 4, Close Collaboration Onsite with Some Systems Integration); or Integrated Care (Level 5, Close Collaboration Approaching an Integrated Practice and Level 6, Full Collaboration in a Transformed/Merged Integrated Practice.) For two specialty teams, an administrator was not required to complete the IPAT per eligibility criteria and team structure. IPAT responses for the remaining seven teams were obtained and audio-recorded for confirmation of scoring accuracy.

Though intentions were to calculate one IPAT score per team, four teams had unique circumstances warranting calculation of a second IPAT score. A secondary IPAT score was calculated when the specialty team had (1) a recent change in their level of MBH support (i.e., loss of contract/funding) or (2) differing levels of MBH support within their specialty (i.e., one clinic with a higher level of MBH integration than most clinics). These circumstances resulted in pre- and post-change IPAT scores or an overall and clinic specific IPAT score, respectively.

Both scores were reported descriptively; however, the score reflective of most clinics/care provided within the specialty area or current (post-change) IPAT levels was used in between-group analyses.

Sample size and data analysis

Qualitative and quantitative analyses were conducted using NVivo 12 plus and IBM

SPSS Statistics (Version 26), respectively. Sample size was determined by extant literature (23, 29). Qualitative thematic analysis was led by the first author (DNW), with significant involvement of the second author (JK), both of whom served as interviewers for the current study. Additional coders (n = 2) included trained undergraduate research assistants not involved with study procedures. Each transcript was reviewed by 3 coders, with at least one coder not involved in study conduct. Remaining authors (DMD and CLD) held supervisory/mentorship roles in study conduct and data analysis. Given inherent differences in sample sizes across groups (i.e., fewer administrators as compared to healthcare professionals), group differences were quantitatively assessed using nonparametric statistics, specifically using Mann–Whitney, Kruskal–Wallis tests. There were no missing data.

Aim 1

The first aim examined perceptions regarding the role and feasibility of implementing MBH care in pediatric specialty care settings. The four trained coders followed an iterative thematic analytic process involving five stages (30–34). First, coders became familiarized with the data via independent review without a priori hypotheses. Each coder then compiled an initial list of codes, or patterns of conversation, among interviews. During the second stage, coders collectively discussed independently generated codes, allowing for codes to be grouped together, refined, and solidified into a condensed number of codes. This process employed both deductive and inductive coding methods and incorporated reliability checks to create a finalized codebook (Stage 3). Specifically, some codes pulled from the data were influenced by questions included in the semi-structured interview (deductive) and others emerged as new, related topics were brought by participants and observed during independent reviews and team discussions (an inductive process). The fourth stage involved a second independent review using the codebook, with the addition of a new coder not involved with the first round of review for reliability purposes. A similar discussion post-review occurred as well as calculation of inter-rater reliability during the fifth and sixth levels of analysis. These stages allowed for further refinement of the codebook, resolving discrepancy to 100 percent consensus, and establishing a finalized coding frame including thematic clusters or categories of codes and intersections/relations among codes (i.e., creation of sub-themes).

Aim 2

The second aim identified current levels of readiness to change and levels of coordination/integration across medical teams. Descriptive statistics were calculated for the RCQ (Appropriateness, Management Support, Change Efficacy, and Personally Beneficial scales) and IPAT by healthcare professional type (administrator vs. non-administrative healthcare professional). A secondary goal was to characterize and explore group differences in readiness for change factors. Mann–Whitney and Kruskal–Wallis nonparametric tests were conducted to examine potential differences in RCQ scores by demographic factors and the level of coordination/integration for each specialty area.

Aim 3

The third aim integrated qualitative (Aim 1) and quantitative (Aim 2) findings according to a convergent design and integration through connecting and merging approaches (35). As the present study involved quantitative measures and qualitative interviews were conducted concurrently, connecting occurred through sampling (i.e., participants completed both procedures) and the convergent study design. This process involved comparing two sets of data to draw a more complete conceptual picture, determine how results inform or expand upon each other, describe different aspects of the research question, and visually integrate data utilizing a mixed methods matrix (20, 36).

Results

Qualitative findings

Thematic analysis of interview data revealed three overarching themes, each with corresponding sub-themes (see Table 2).

Table 2

Table 2. Thematic analytic findings.

Theme 1: prevalence of MBH concerns and relevance to pediatric specialty medicine

This theme broadly captured references to the prevalence and relevance of MBH concerns to routine subspecialty care. Six sub-themes were identified.

Overlap between psychosocial concerns and physical health, medical complexities, and/or medical treatment

This subtheme included statements about the frequency and relevance of psychosocial concerns to the care of patients and families. Healthcare professionals also discussed that psychosocial concerns are not always related to medical concerns but could exacerbate or contribute to chronic illness management.

Healthcare Professional (Team 4): “[MBH concerns] may or may not actually be related to what they have going on medically…the treatment is both psychological and medical and you have to do both…and if you don’t do both parts, they don’t get better.”

Specific diagnoses or MBH concerns

This subtheme included references to common MBH concerns observed in practice, including anxiety; depression/suicidal ideation, externalizing and risk-taking behaviors; attention-deficit/hyperactivity disorder; and autism spectrum disorder and other developmental or intellectual disabilities. Healthcare professionals also noted differential risk or prevalence of MBH challenges among specific subgroups within their populations (e.g., based on developmental, chronicity, and/or disease severity considerations).

Administrator (Team 5): “…Anxiety, depression, infantilization…there’s suicidal ideation…a lot of fear…there’s so many bad habits, bad behavioral responses to things that happen to kids…[and] parents in the hospital..they just result from all these horrible things that people aren’t coping with or are talking about or aren’t working through….”

Current or prospective roles of MBH clinicians

This subtheme captured references to current or prospective roles and support provided by MBH clinicians to medical teams. Examples include providing referrals or resources; clinical intervention and/or assessment; and serving a consultative role to patients, families, and staff. Healthcare professionals and administrators also discussed “gaps” MBH clinicians fill in routine practice and how psychosocial concerns were/were not addressed without MBH support.

Healthcare Professional (Team 4): “[The MBH clinician] is usually asked to help with kids who are having psychological contributors that are impacting their medical problems or…that the team has gotten a sense of and feels like it’s not being addressed.”

Other psychosocial concerns impacting medical care or patient functioning

This subtheme referenced other non-diagnostic psychosocial concerns and patient/familial considerations raised by participants (e.g., disease burnout, treatment engagement, treatment responsibility, adherence, and self-management behaviors). These concerns were described as impacting prognosis, treatment effectiveness, or overall patient functioning.

Healthcare Professional (Team 3): “[Our MBH clinician is helpful] with…trying to come up with ways to encourage excitement with trying new things and problem solving [around barriers].”

MBH referrals and related processes

This subtheme captured references to internal or external MBH referral processes and follow-up, if applicable. Participants with limited MBH support noted barriers to follow-up on the referrals provided to patients and families. In these instances, families navigate the referral process more independently, unless members of the team are specifically knowledgeable about resources available. Further, healthcare professionals also discussed limited resources available in rural Appalachia, even when interested in providing referrals.

Administrator (Team 5): “[I refer more] older kids…because I can find them places. Younger kids, I can’t find them anywhere to go…it’s not for lack of [trying], there’s nowhere they can go, so there’s nowhere to send them..”

Guidelines/policies around MBH

This subtheme emphasized references to any guidelines or policies around MBH for the area of pediatric specialty. In most instances, healthcare professionals were unaware of any specific guidelines or policies and instead described variation in how/if MBH concerns were addressed by healthcare professionals within the team. For healthcare professionals aware of recommendations, the guidelines were often described as either vague or not feasible due to adoption/implementation barriers. Specific barriers noted were included within the “Perceived Feasibility and Acceptability of Coordination/Integration MBH Care” theme.

Administrator (Team 8): “…The [organization name omitted] actually lays out some [recommendations] for disorders… [but] they leave the mental health component fairly vague…it’s usually [the same general] recommendations, pretty much for everyone…A lot of it comes [from research]…but there’s not anyone in the field who has come out and said, ‘This is the exact model that [MBH care] should be provided in’.”

Theme 2: perceptions of coordination/integration MBH care services and models

This theme, incorporating two subthemes, captured participant preferences around coordination/integration MBH services and models.

Perceived need for increased MBH support

Participants frequently reported interest in coordination/integration MBH due to patients and family needs. Healthcare professionals described MBH clinicians as having nuanced skillsets and training helpful to both patients/families and the medical team, including allowing healthcare professionals to meet visit goals and objectives more efficiently. Healthcare professionals and administrators often described how their hospital system has historically undervalued/underemphasized MBH integration and the role services play in prevention of expensive, complex, and/or debilitating comorbidities.

Administrator (Team 1): “…[Patients] think they have come for medication and even if I asked [about MBH concerns], they would tell me something superficial. I think [the MBH clinician will] be better…she has the right tools to get that information from them.”

Thoughts around models of coordination/integration care and any evidence of services meeting the definition of coordinated care, co-located care, or integrated care

This subtheme highlighted discussion around the specific models of coordination/integration care according to the Standard Framework for Levels of Integrated Healthcare. Healthcare professionals discussed their preferences and perspectives around these models as well as articulated understanding and/or confusion about the terminology. They provided specific examples from within their practice and beyond (e.g., prior work experiences) with these varied models and highlighted general pros and cons of each approach.

Administrator (Team 6): “I would love to have a psychologist integrated right in our team. And he’s a person who sees every single patient in our clinic, maybe not every visit, but gets to know families and patients and…help them…[with] whatever they need. It’s a reliable, trusted, very qualified person who is not going to change…”

Theme 3: perceived feasibility and acceptability of coordination/integration MBH care

This theme highlighted perceived facilitators and barriers to the implementation of coordination/integration MBH Care at the organizational, professional, and patient levels.

Organizational considerations

This subtheme reflected thoughts, actions, behaviors, or reasons that discouraged the implementation of coordination/integration care models or increase of MBH services within their specialty area or organization more broadly. Barriers were discussed across the larger organization level as well as at the level of individual specialties and clinics. Examples include insurance and healthcare structure considerations; use of electronic medical records; healthcare growth/expansion; and the availability of staffing, time, space, and other resources.

Healthcare Professional (Team 4): “I think co-located care is really important…I'm a huge proponent of integrated care, but there are times where co-located makes the most sense, particularly when we are trying to ease way into changes within the system and/or when we do need psychological staff [MBH clinicians] to have more time and space than is appropriate for an integrated care setting.”

Administrator (Team 9): “It’s all about money,…having enough…[MBH clinicians] having enough physical space in the clinic, and finding a way to get those services reimbursed. Some of that is not sustainable, but….we have clinic patient rooms, exam rooms, and we have Healthcare Professional workstations. And the idea would be to have the [MBH clinician] have a workstation in the same physician work area…with the rest of the team and just available on an ad-hoc basis to say, ‘Hey, I just got done with this family. I think they could really benefit from some time with you.’ And then giving a warm hand-off.”

Healthcare professional-level considerations

Healthcare professionals and administrators also discussed barriers and facilitators at the individual Healthcare Professional level, including personal barriers around communication and interest or comfort with MBH topics. For example, healthcare professionals discussed difficulties in verifying follow-through with referrals provided, making coordinated care more difficult to navigate effectively and systematically. They also discussed concerns within the electronic medical record system, including stressors associated with “breaking the glass” to read MBH notes for coordination of care purposes.

Administrator (Team 3): “I don’t know how many people don’t end up going to their referrals and I think [obtaining] that feedback…could be improved…I personally don’t like breaking the glass [to enter a MBH note]….and some of the concerns obviously can’t be out there in the medical chart at large, I just don’t know how to better coordinate…”

Healthcare Professional (Team 7): “I would love to see a [MBH clinician] dedicated to our team…We can discuss the patient and see if there are any things that I can improve from the patient point of view, the [medical] issues. And in the meantime, I can transfer my thinking and my thought about the overall situation to the [MBH clinician] and we discuss what would be best for the patient. But definitely, I [would] encourage her or him, the [MBH clinician], to see the patient on his own. To take his time and to assess him as he feels like.”

Patient population-level considerations

Several perceived barriers for patients and families were mentioned by participants, such as patient disinterest (though described as rare), the large catchment area served by their hospital, patient population growth, and other demographic contributory factors, including social determinants of health, historical trauma (e.g., opioid crisis) and systemic barriers to healthcare heightened in rural, Appalachian, and other marginalized communities.

Healthcare Professional (Team 4): “I think coordinated care is challenging…First, being that we know that patients do not routinely follow through with recommendations from their medical [team] to go see a [MBH clinician]. So, if you see a patient, they say they have depression, you tell them that you need them to go to [practice name omitted] or another outpatient mental health facility, there’s about a 50% chance that they're actually going to go and follow up with that [for numerous reasons].”

Administrator (Team 7): “When someone calls me, finally admits [substance abuse] and says I need help…particularly as [their specialty care team] for so long, they think of us as their primary care doctor. So, I can’t say, ‘Yeah, that’s great you’re admitting it, go see your pediatrician or your whatever about it.’ Wouldn’t it be nice if I could’ve got [this patient] hooked at least on the phone with somebody who knew what to do, because I have no idea what to do with that.”

Quantitative findings

Differences in RCQ scores

Non-parametric independent samples tests (i.e., MannWhitney, Kruskal–Wallis) assessed mean differences in RCQ scores based on demographics (Table 3). Significant differences were observed for the Personally Beneficial subscale, such that the scores of administrators (Mdn = 7.00) were higher than those of non-administrative healthcare professionals (Mdn = 6.33), U(Nhealthcare professionals = 23, Nadministrators = 7,) = 123.50, z = 2.21, p = .033. No significant differences were found for other subscales (i.e., Appropriateness, Management Support, Change Efficacy) based on healthcare professional type or other demographic variables (i.e., age, gender, years of experience, education level, area of specialty).

Table 3

Table 3. Mean and Median Differences for RCQ Scores by Healthcare Professional Type.

IPAT scores

Seven primary and four secondary IPAT scores were calculated. Among primary IPAT scores, minimal variation was observed, and all seven IPAT scores identified teams as Co-Located Care (Level 3, n = 2, 28.6%; Level 4, n = 5, 71.4%). Regarding within-specialty differences, secondary IPAT scores revealed increased variability, with higher levels of integrated care represented (Co-Located Care: Level 3, n = 2, 18.2% and Level 4, n = 6, 54.5%. Integrated care: Level 5, n = 2, 18.2% and Level 6, n = 1, 9.1%). No teams were categorized as Coordinated Care (Levels 1 or 2). Results are incorporated into the mixed-methods matrix described below. There were no significant differences found between IPAT and RCQ scores.

Integration of qualitative and quantitative findings

Integration of qualitative and quantitative findings followed a convergent mixed methods design using a comparison method of analysis (Figure 1). Comparing the two sets of data helped elucidate how results informed or expanded upon each other and described different aspects of the research question. We examined overlap between IPAT scores and readiness to change factors (RCQ scores) within and across teams and in relation to identified themes, specifically to better understand context of individual and team familiarity and experience working with MBH providers and addressing MBH concerns in their work (Theme 1), perceptions of care models (Theme 2) and specific facilitators and barriers (Theme 3), which may impact perceptions of MBH services or readiness for change. Outputs from this integrative process are represented in a mixed methods matrix (Table 4).

Figure 1

Figure 1. Convergent mixed methods design.

Table 4

Table 4. Mixed methods matrix.

Overall, healthcare professionals reported an ardent desire for increased MBH support and collaboration with MBH clinicians in their routine practice regardless of current levels of MBH coordination/integration. Healthcare professionals and administrators described benefits of MBH services across organizational, healthcare professional, and patient/family levels, though recognized important barriers to adoption across the spectrum of care. Consistently, quantitative findings suggested high readiness for change. Qualitatively and quantitatively, teams reported prior experience with MBH services typically at the Co-Located level, as consistent with IPAT scores. Some teams experienced varied access to MBH clinicians over time or across settings, such as varied access based on the day or clinic within a specialty area or changes in funding for this role over time (i.e., a grant-funded position). While only clinic within a team reached the highest level of integrated care, most participants qualitatively described integrated care as the “ideal” model of care.

Findings suggest participants perceived MBH coordination/integration or increased access to these services as worthwhile, with a legitimate rationale. Healthcare professionals described MBH services as appropriate and relevant in the context of pediatric subspecialty medicine and beneficial to the larger organization, specialty team, themselves as individual healthcare professionals, and the patients and families they serve. Many participants, though especially those identifying as MBH clinicians or healthcare professionals with experience with integrated care models, discussed benefits of MBH access such as improvement in medical and psychosocial outcomes among patients and families, the promotion of positive health behaviors (e.g., adherence, self-management), and successful transition to adult care. Participants also highlighted how access to MBH care can decrease risk for worsening MBH symptoms and medical complications, the need for higher-level intervention in the future, and overall reduction in healthcare costs.

High levels of confidence around a team's ability to make a change in the model of MBH care) were also observed quantitatively and qualitatively. Higher RCQ (Change Efficacy) scores were consistent with reported interest and willingness to problem-solve and consider practice changes to support increased MBH access. However, administrators often expressed concern that organizational support may hinder these efforts, consistent with Management Support scores being the lowest domain for both non-administrative healthcare professionals and administrators. At the same time, some healthcare professionals and administrators also described the value for continuing to advocate for increased access to services to higher levels of hospital administration and recognized some potential concerns of inconsistent access over time (i.e., establishing psychosocial screening procedures and then losing the support to address identified needs sufficiently). A few participants who made these statements described how inclusion of MBH clinicians on teams supports the efficiency of the team, allowing each healthcare professional to act within their scope of practice. These participants noted they are often not sufficiently trained to address MBH concerns (e.g., suicidality, substance use), though these concerns routinely arise in their care of patients and families. Relatedly, significant differences between healthcare professionals and administrators regarding the Personally Beneficial subscale of the RCQ were observed in qualitative data, supplementing administrators' qualitative responses of interest and empathy for patients and families about the need for increased access to MBH services and advantages of implementation. This result is clinically significant in that administrators had overall higher levels of experience upon which to base their opinions and perspectives.

Interestingly, when presented with a verbal description and definitions of each care model, healthcare professionals who were not MBH clinicians often had difficulty identifying the current care model of their practice. This confusion resulted in discrepancy between administrator IPAT scores and qualitative reports by healthcare professionals (vs. administrators). For example, it was observed that those on Level 3 Co-Located Care teams and a few healthcare professionals on Level 4 CoLocated Care teams (based on IPAT scores) were often likely to describe their teams as subscribing to a coordinated care model (i.e., Level 2 Coordination, Basic Collaboration at a Distance). Based on further review of these discrepancies, it is also possible that teams had difficulty distinguishing between the two when operating in a capacity that uses multiple models. For example, several teams described different clinic days or times in which MBH clinicians were present and other times in which they were not. On the days without a MBH clinician, teams were more likely to refer patients to community services or internally to the MBH department (more coordinated, focused on written referral-based communication). Conversely, for clinics with an MBH clinician present, communication was more regular and face-to-face, but the MBH clinician held more of a consultative role for the team. Healthcare professionals often described the MBH clinician as an important part of their team, though called upon as needed. Participants also described MBH clinicians as helpful in responding to a team member's thoughts, ideas, or question, in addition to being someone who could address MBH concerns separately while the remainder of the team focused more on medical needs. As these examples demonstrate, while these healthcare professionals often initially identified their teams as having a more coordinated model of MBH care, they often described all key elements of co-located care, consistent with their team's IPAT scores.

Additionally, while integrated care was often described as an “ideal” scenario, it was more frequently identified as unrealistic in participants' setting due to perceived organizational constraints (e.g., time, money, space, number of available MBH clinicians, other resources). This often led to conversations about perceived benefits of lower care models or a preference for colocated care.

Discussion

The current study examined pediatric specialty healthcare professional and administrator perceptions and experiences with MBH coordination/integration and readiness to change. Extant research on coordinated/integrated MBH is robust, though often focuses on primary care or specific pediatric or adult populations (37–40). Further, while more routine in implementation science and primary care research (41), fewer specialty-medicine-focused studies have used a rigorous mixed methods approach to investigate these factors; included a wide range of healthcare professionals and administrators from various specialty areas, training backgrounds, and experiences with MBH services; and targeted a rural, Appalachia-serving hospital system. Despite this, several qualitative and quantitative studies, trials, systematic reviews, and metaanalyses have explored the benefits of and barriers and facilitators to implementing MBH services into medical contexts (40). Findings suggest the inclusion of MBH support on medical teams improves patient outcomes and reduces healthcare costs (37, 42, 43). Research has also demonstrated how strategies, such as cost-effectiveness analyses, can be used by pediatric psychologists and other MBH clinicians to demonstrate the beneficial economic impacts of their work (44). Still, widespread integration MBH services in pediatric subspecialty medicine is far from reality for many healthcare settings, particularly in rural settings where patients are more likely to experience mental health concerns, less likely to receive treatment, and experience higher rates of suicide-related deaths (45).

While an investment in expansion of MBH services has demonstrated reduced overall healthcare costs and preventive impacts on healthcare utilization (46), participants in this study also emphasized finances as a key barrier to the adoption, implementation, and sustainability of MBH integration. Increased awareness of the benefits of integrated care on a financial level, particularly in rural settings like Appalachia, appears to be an area of future need particularly as comments such as little to no reimbursement for MBH visits were noted by several healthcare professionals/administrators. While some specialties/teams developed unique strategies to circumvent financial barriers (i.e., soliciting external grant funding, using trainees from local graduate programs as MBH clinicians), these approaches pose sustainability barriers. This was highlighted in both qualitative and quantitative data obtained from Team 6, whose level of coordination/integration changed following the loss of an external MBH grant. As such, it is likely that conversations, collaboration, and advocacy at the organizational, payor, and policy level are warranted, detailing the different options and approaches to coordination/integration care possible as well as the best strategies for both implementation and sustainability. Further, given known disparities in MBH access, particularly for rural, Appalachian contexts, facilitating incorporation of MBH access in other, ongoing health equity efforts is paramount.

During analysis, we also observed some confusion around coordinated/integrated care terminology, particularly for non-administrative and non-psychologist healthcare professionals. When provided with a copy of each definition, participants had minimal to no difficulties identifying or describing key components of integrated care, and participants provided specific examples of these care levels in their or others' practices. Similarly, no major difficulties were observed for the other side of the continuum with understanding the lowest levels of coordinated care. Healthcare professionals who perceived their clinic or specialty area to be integrated in nature also correctly identified their teams as such when compared to team IPAT scores.

Conversely, non-administrative healthcare professionals in fields other than psychology had more difficulty distinguishing between the higher level of coordinated care and the lower level of co-located care. Comments suggest this discrepancy may be related to the shared emphasis on basic collaboration and differing conceptualizations in specialty medicine (vs. primary care) of “separate facilities” and “onsite” given the context of a hospital system divided by departments and divisions, and variation in office spaces, locations, and clinic layouts. This confusion often resulted in participant perceptions that MBH services were less coordinated/integrated (at a lower level) than objectively calculated IPAT scores.

These findings are consistent with other previous work which has demonstrated that healthcare professionals and patients alike are confused by terminology and the experience of integrated care models, but are clear on the overarching concept and its active components or impacts on the team as a whole (47, 48). Findings are also consistent with work on co-located and integrated care models, suggesting the main predictor of positive perceptions of successful interprofessional collaboration in healthcare contexts is co-location of MBH care (49). This is often why formal trainings, especially in primary care contexts, have been developed to teach healthcare professionals about coordinated/integrated MBH models (47). Similar trainings may prove beneficial in pediatric specialty environments prior to adoption/implementation or changes to current MBH coordination/integration.

The observed differences (lower levels) of Management Support in the study as compared to other RCQ domains is also an important finding. Healthcare professionals often reported beliefs that their primary manager/administrator was supportive of MBH collaboration and organizational barriers were perceived to be greater at higher levels of hospital administration. Consistently, Administrators often expressed great enthusiasm and interest in increasing MBH supports within and outside of their specialty areas, though readily detailed organizational barriers at the hospital and higher policy/public health system levels (e.g., structure of the hospital system itself, U.S. healthcare structure/payment systems). Future work could benefit from including more senior managers and healthcare leaders when investigating barriers and facilitators to MBH integration. Research using focus groups or other shared learning methodologies, in addition to continued mixed methods designs may also improve our understanding of readiness to change regarding MBH integration across and within levels of a healthcare system/organization.

Limitations

The present study is not without its limitations. First, as the design was exploratory and predominately emphasized the qualitative components of the mixed methods design (QUAL-quant), the sample size limited the statistical plan and quantitative hypothesis testing and thus the level of comparisons included in our mixed methods analysis. Relatedly, given the number of MBH clinicians eligible to participate at the time of this work, our team was unable to parse out unique perspectives of MBH clinicians by type (i.e., psychologists vs. other healthcare professionals) in this manuscript without impacting participant confidentiality. Further, though several attempts were made to decrease conflicts of interest and social desirability (i.e., assigning interviewers with no professional working relationship with the participant), some healthcare professionals had some interactions with other authors/study investigators outside of the study. Further, as described for transparency in the author note, qualitative analyses may have been impacted by investigator worldviews, backgrounds, and experiences. Lastly, this study focused on a single healthcare system predominately serving rural Appalachia. These sociodemographic considerations are important, as they potentially impact applicability of findings to other settings or healthcare systems. However, applicability and generalizability are not the primary objective of qualitative or mixed methods research.

Strengths and contributions to mixed methods research, clinical practice, and policy

This study also has several strengths and contributions. First, an evidence-based approach was taken to study design, data collection, and data analysis. Validated measures were used to assess most constructs and non-validated tools (e.g., semi-structured interview guide) were piloted and modified based on feedback from individuals with relevant expertise. Purposive and theoretical sampling techniques were used for codebook stability and diverse representation of participant backgrounds. Real-time data also informed sampling decisions. These features are innovative for the integrated MBH care literature, where emphasis has been placed on primary care/a single pediatric subspecialty area; observational methods, qualitative, or survey data alone; and non-validated, study-specific tools (40, 50, 51).

The rigorous, empirically based convergent mixed methods design of this study, however, resulted in rich, comprehensive data and the ability to identify consistencies and contradictions across qualitative and quantitative reports. Taking a QUAL-quant approach also ensured findings were grounded in the experiences and voices of participants. Finally, the focus on a healthcare system in a historically marginalized and underserved community (i.e., rural Appalachia) allowed for unique examination of the construct (MBH coordination/integration) and its variation when a healthcare system has limited access to resources. Other studies often emphasize larger, well-resourced healthcare systems with more widespread adoption or structures in place to support higher MBH coordination/integration.

Across psychology, medicine, and mixed methods literatures, there is growing recognition of the strengths, applicability, and practical utility of mixed methods research to dissemination/implementation and health equity science, especially as it relates to reducing MBH science-to-practice gaps (52, 53). This manuscript applied mixed methods to a quality improvement initiative aimed to understand multilevel factors influencing MBH coordination/integration in pediatric subspecialty care. Consequently, our project resulted in data aligned with several of Dahler-Larsen's (54) proposed practical utilities of mixed methods research. Specifically, the mixed methods approach applied in the present study: (1) provided nuanced understanding of constructs and new insights; (2) produced the type of practical knowledge that best serves organizational partners, such as healthcare professionals and leadership; and (3) identified and catered to the diverse perspectives and needs across employee/manager (i.e., Healthcare Professional/administrator) types, including expression of joint commitment and conflicting views (54).

As previously stated, the present study was also designed with quality improvement in mind, and the potential to have impact on the healthcare system from which we sampled. Prior to this publication, results were shared with study participants, administrative leaders, and the larger pediatric faculty to spur ongoing conversation around MBH support within the system and inform recruitment efforts for additional MBH clinicians. Moreover, the present study serves as an exemplar for how to conduct rigorous, yet low-cost MBH-related research within a hospital system.

Overall, data suggests partnerships between healthcare professionals, leadership, and policymakers is warranted to address barriers and improve MBH access and equity in these settings, including improving policies related to fiscal resources (i.e., funding, reimbursement), productivity metrics/time (e.g., billable hours and relative value units); education and training related to MB; and protecting time to identify preferred models of MBH care, learn to collaborate with MBH clinicians, assess implementation outcomes, and refine procedures over time to support sustainability. Moreover, data facilitated understanding of the multilevel factors influencing MBH implementation and provided guidance for future research, clinical, and policy directions aimed to reduce MBH disparities.

Conclusion

Chronic illness and medical complexity places unique demands on patients, families, and the medical system; often coincides with increased risk for psychosocial concerns overall; and contributes to healthcare disparities. Future research should explore barriers and facilitators to MBH integration longitudinally and with various partners beyond healthcare professionals to include higher-level administrators/hospital leaders, policymakers, and patients/families. Specialty medicine conducted in rural and other marginalized contexts presents unique challenges and strengths which should also be explored further as should how lack of access to MBH exacerbates MBH concerns and places an undue burden on healthcare professionals without MBH training to address these needs as part of their care despite scope of practice, experience, and comfort with doing so. Future research could also involve comprehensive needs assessments or cost-benefit analyses to quantify gaps between current and desired MBH landscape within healthcare organizations; obtain additional context around systemic/logistical barriers and facilitators (e.g., cost, resources, time, space); and understand the feasibility and sustainability of different models of MBH coordination/integration.

In the present study and beyond, de-identification and limiting participant demographics are common strategies used to protect participant confidentiality. However, future research should consider novel methodological approaches to explore within- and between-group differences in perceptions while maintaining participant confidentiality, particularly in rural or smaller healthcare contexts. Practical guides and handbooks such as those on Rural Healthcare Ethics (55) can support researchers in thinking through these concerns while balancing benefits and risks. Researchers may also benefit from conducing multi-site studies and incorporating community-engaged approaches (56) by embedding healthcare professionals, administrators, and other key stakeholders into the research team and working together to develop new strategies to overcome and address barriers to confidentiality and support larger, engaged samples in this type of research (57).

Author's note

In the spirit of self-reflexivity, authors recognize their positionality as highly-educated individuals with employment and/or training in academic (university and medical) environments. Authors also self-identify as a multi-racial, Afro-Latina and Indigenous, cisgender female (DNW); and white, cisgender females (JK, DMD, CLD). We acknowledge the potential influences of these identities on the study design, conduct, and interpretation of findings and implications described in the present manuscript.

Data availability statement

The datasets for this article are not publicly available due to concerns regarding participant/patient anonymity. Requests to access the datasets should be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the West Virginia University Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

DNW: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. JK: Data curation, Formal analysis, Writing – original draft, Writing – review & editing. DMD: Conceptualization, Methodology, Writing – review & editing. CLD: Conceptualization, Methodology, Supervision, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. All phases of this study were supported with funding from an institutional dissertation research support program. At separate points during the preparation of this manuscript, Dr. Williford and Dr. Kelleher's time was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD T32HD068223).

Acknowledgements

We would like to thank the study participants for their interest, time, participation, and valuable perspectives shared during the study. We would also like to thank Dr. Barbara Cubic and Dr. Williford’s dissertation committee for their expertise and consultation during study design. Finally, we wish to thank Ms. Erin Flynn and trainees in the Psychology Fellowship Program at Cincinnati Children’s Hospital Medical Center for reviewing prior versions of this manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Barican JL, Yung D, Schwartz C, Zheng Y, Georgiades K, Waddell C. Prevalence of childhood mental disorders in high-income countries: a systematic review and meta-analysis to inform policymaking. BMJ Mental Health. (2022) 25(1):36–44. doi: 10.1136/ebmental-2021-300277

PubMed Abstract | Crossref Full Text | Google Scholar

2. Arias D, Saxena S, Verguet S. Quantifying the global burden of mental disorders and their economic value. eClinicalMed. (2022) 54(101675):1–10. doi: 10.1016/j.eclinm.2022.101675

Crossref Full Text | Google Scholar

3. Copeland WE, Wolke D, Shanahan L, Costello EJ. Adult functional outcomes of common childhood psychiatric problems: a prospective, longitudinal study. JAMA Psychiatry. (2015) 72(9):892–9. doi: 10.1001/jamapsychiatry.2015.0730

PubMed Abstract | Crossref Full Text | Google Scholar

4. Deng J, Zhou F, Hou W, Heybati K, Lohit S, Abbas U, et al. Prevalence of mental health symptoms in children and adolescents during the COVID-19 pandemic: a meta-analysis. Ann N Y Acad Sci. (2023) 1520(1):53–73. doi: 10.1111/nyas.14947

PubMed Abstract | Crossref Full Text | Google Scholar

5. Panchal U, de Pablo G S, Franco M, Moreno C, Parellada M, Arango C, et al. The impact of COVID-19 lockdown on child and adolescent mental health: systematic review. Eur Child Adolesc Psychiatry. (2023) 32:1151–77. doi: 10.1007/s00787-021-01856-w

PubMed Abstract | Crossref Full Text | Google Scholar

6. Delamater AM, Guzman A, Aparicio K. Mental health issues in children and adolescents with chronic illness. Int J Hum Rights Healthcare. (2017) 10(3):163–73. doi: 10.1108/IJHRH-05-2017-0020

Crossref Full Text | Google Scholar

7. Guzman Holst C, Bowes L, Waite P, Skripkauskaite S, Shum A, Pearcey S, et al. Examining children and adolescent mental health trajectories during the COVID-19 pandemic: findings from a year of the Co-SPACE study. JCPP Advances. (2023) 3(2):e12153. doi: 10.1002/jcv2.12153

PubMed Abstract | Crossref Full Text | Google Scholar

8. Staudacher HM, Black CJ, Teasdale SB, Mikocka-Walus A, Keefer L. Irritable bowel syndrome and mental health comorbidity—approach to multidisciplinary management. Nat Rev Gastroenterol Hepatol. (2023) 20:582–96. doi: 10.1038/s41575-023-00794-z

PubMed Abstract | Crossref Full Text | Google Scholar

9. ElSayed NA, Aleppo G, Aroda VR, Bannuru RR, Brown FM, Bruemmer D, et al. 5. Facilitating positive health behaviors and well-being to improve health outcomes: standards of care in diabetes—2023. Diabetes Care. (2022) 46(Supplement_1):S68–96. doi: 10.2337/dc23-S005

Crossref Full Text | Google Scholar

10. Quittner AL, Abbott J, Hussain S, Ong T, Uluer A, Hempstead S, et al. Integration of mental health screening and treatment into cystic fibrosis clinics: evaluation of initial implementation in 84 programs across the United States. Pediatr Pulmonol. (2020) 55(11):2995–3004. doi: 10.1002/ppul.24949

PubMed Abstract | Crossref Full Text | Google Scholar

11. Foy JM, Green CM, Earls MF, Committee on Psychosocial Aspects of Child and Family Health, Mental Health Leadership Work Group, Lavin A, et al. Mental health competencies for pediatric practice. Pediatrics. (2019) 144(5):e20192757. doi: 10.1542/peds.2019-2757

PubMed Abstract | Crossref Full Text | Google Scholar

12. American Academy of Pediatrics. Mental health initiatives. (2018). Available online at: https://www.aap.org/en-us/advocacy-and-policy/aap-health-initiatives/Mental-Health/Pages/About-Us.aspx (Accessed May 07, 2025).

13. Heath B, Wise Romero P, Reynolds K. A Review and Proposed Standard Framework for Levels of Integrated Healthcare. Washington, DC: SAMHSA-HRSA Center for Integrated Health Solutions (2013).

Google Scholar

14. Chandrashekar P, Jain SH. Understanding and fixing the growing divide between physicians and healthcare administrators. J Med Pract Manag. (2019) 34(5):264–8.

Google Scholar

15. Bahna SL, Assa'ad AH, Bellanti JA, Bernstein DI, Davis CM, Greenberger PA, et al. Where is academic medicine going?: part 2. Academic career and patient care. Ann Allergy Asthma Immunol. (2024) 133(4):364–5. doi: 10.1016/j.anai.2024.06.015

PubMed Abstract | Crossref Full Text | Google Scholar

16. Barrera M, Alexander S, Shama W, Mills D, Desjardins L, Hancock K. Perceived benefits of and barriers to psychosocial risk screening in pediatric oncology by health care providers. Pediatr Blood Cancer. (2018) 65(12):e27429. doi: 10.1002/pbc.27429

PubMed Abstract | Crossref Full Text | Google Scholar

17. Levitt HM, Bamberg M, Creswell JW, Frost DM, Josselson R, Suarez-Orozco C. Journal article reporting standards for qualitative research in psychology: the APA publications and communications board task force report. American Psychologist. (2018) 73(1):26–46. doi: 10.1037/amp0000151

PubMed Abstract | Crossref Full Text | Google Scholar

18. Goodyear-Smith F, York D, Petousis-Harris H, Turner N, Copp J, Kerse N, et al. Recruitment of practices in primary care research: the long and the short of it. Fam Pract. (2009) 26(2):128–36. doi: 10.1093/fampra/cmp015

PubMed Abstract | Crossref Full Text | Google Scholar

19. Johnston S, Liddy C, Hogg W, Donskov M, Russell G, Gyorfi-Dyke E. Barriers and facilitators to recruitment of physicians and practices for primary care health services research at one centre. BMC Med Res Methodol. (2010) 10:1–8. doi: 10.1186/1471-2288-10-109

PubMed Abstract | Crossref Full Text | Google Scholar

20. Creswell JW, Clark VLP. Analyzing and Interpreting Data in Mixed Methods Research. Designing and Conducting Mixed Methods Research. 3 ed. Thousand Oaks, CA: Sage (2018). p. 209–58.

Google Scholar

21. Coyne IT. Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? J Adv Nurs. (1997) 26(3):623–30. doi: 10.1046/j.1365-2648.1997.t01-25-00999.x

PubMed Abstract | Crossref Full Text | Google Scholar

22. Thorogood N, Green J. Qualitative Methods for Health Research. 4th ed. London: SAGE Publications (2018). p. 1–440. Available online at: https://alraziuni.edu.ye/book1/nursing/dMGvRe1D.pdf

Google Scholar

23. Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One. (2020) 15(5):e0232076. doi: 10.1371/journal.pone.0232076

PubMed Abstract | Crossref Full Text | Google Scholar

24. Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. (2017) 27(4):591–608. doi: 10.1177/1049732316665344

PubMed Abstract | Crossref Full Text | Google Scholar

25. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42(2):377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | Crossref Full Text | Google Scholar

26. Holt DT, Armenakis AA, Feild HS, Harris SG. Readiness for organizational change: the systematic development of a scale. J Appl Behav Sci. (2007) 43(2):232–55. doi: 10.1177/0021886306295295

Crossref Full Text | Google Scholar

27. Rafferty AE, Jimmieson NL, Armenakis AA. Change readiness: a multilevel review. J Manage. (2013) 39(1):110–35. doi: 10.1177/0149206312457417

Crossref Full Text | Google Scholar

28. Waxmonsky J, Auxier A, Wise-Romero P, Heath B. Integrated Practice Assessment Tool (IPAT). 2013.

29. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. (2006) 18(1):59–82. doi: 10.1177/1525822X05279903

Crossref Full Text | Google Scholar

30. Braun V, Clarke V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual Res Psychol. (2020) 3:1–25. doi: 10.1080/14780887.2020.1769238

Crossref Full Text | Google Scholar

31. Braun V, Clarke V. Successful Qualitative Research. A Practical Guide for Beginners: SAGE (2013).

Google Scholar

32. Braun V, Clarke V. Can I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern-based qualitative analytic approaches. Counselling and Psychotherapy Research. (2021) 21:37–47. doi: 10.1002/capr.12360

Crossref Full Text | Google Scholar

33. Terry G, Hayfield N, Clarke V, Braun V. Thematic analysis. In: Willig C, Rogers WS, editors. The SAGE Handbook of Qualitative Research in Psychology. London: SAGE Publications Ltd. (2017). p. 17–36. doi: 10.4135/9781526405555.n2

Crossref Full Text | Google Scholar

34. Creswell JW. A Concise introduction to Mixed Methods Research. 1st ed. Thousand Oaks, CA: SAGE Publications (2014).

Google Scholar

35. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs: principles and practices. Health Serv Res. (2013) 48(6pt2):2134–56. doi: 10.1111/1475-6773.12117

PubMed Abstract | Crossref Full Text | Google Scholar

36. O’Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. Br Med J. (2010) 341:c4587. doi: 10.1136/bmj.c4587

Crossref Full Text | Google Scholar

37. Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry. (2012) 169(8):790–804. doi: 10.1176/appi.ajp.2012.11111616

PubMed Abstract | Crossref Full Text | Google Scholar

38. Supper I, Catala O, Lustman M, Chemla C, Bourgueil Y, Letrilliart L. Interprofessional collaboration in primary health care: a review of facilitators and barriers perceived by involved actors. J Public Health. (2015) 37(4):716–27. doi: 10.1093/pubmed/fdu102

PubMed Abstract | Crossref Full Text | Google Scholar

39. Hodgkinson S, Godoy L, Beers LS, Lewin A. Improving mental health access for lowincome children and families in the primary care setting. Pediatrics. (2017) 139(1):e20151175. doi: 10.1542/peds.2015-1175

PubMed Abstract | Crossref Full Text | Google Scholar

40. Burkhart K, Asogwa K, Muzaffar N, Gabriel M. Pediatric integrated care models: a systematic review. Clin Pediatr (Phila). (2020) 59(2):148–53. doi: 10.1177/0009922819890004

PubMed Abstract | Crossref Full Text | Google Scholar

41. Gurfinkel D, Owen V, Kreisel C, Hosokawa P, Kluger S, Legge C, et al. Patient perspectives of integrated behavioral health in primary care: a mixed methods analysis. Journal of Patient Experience. (2024) 11: 23743735241293877. doi: 10.1177/23743735241293877

PubMed Abstract | Crossref Full Text | Google Scholar

42. Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated medical-behavioral care compared with usual primary care for child and adolescent behavioral health: a metaanalysis. JAMA Pediatr. (2015) 169(10):929–37. doi: 10.1001/jamapediatrics.2015.1141

PubMed Abstract | Crossref Full Text | Google Scholar

43. Roberts MC, Steele R. Handbook of Pediatric Psychology. 5 ed. New York: Guilford (2017).

Google Scholar

44. McGrady ME. Commentary: demonstrating cost-effectiveness in pediatric psychology. J Pediatr Psychol. (2014) 39(6):602–11. doi: 10.1093/jpepsy/jsu019

PubMed Abstract | Crossref Full Text | Google Scholar

45. Harris KR, Brown RM. Challenges and opportunities of pediatric mental health practice in rural America. Pediatric Clinics. (2025) 72(1):19–36. doi: 10.1016/j.pcl.2024.07.025

PubMed Abstract | Crossref Full Text | Google Scholar

46. Peterson M, Turgesen J, Fisk L, McCarthy S. Integrated care in rural health: seeking sustainability. Fam Syst Health. (2017) 35(2):167. doi: 10.1037/fsh0000267

PubMed Abstract | Crossref Full Text | Google Scholar

47. O'Donohue WT, Byrd MR, Cummings NA, Henderson DA. Behavioral Integrative Care: Treatments That Work in the Primary Care Setting. New York, NY: Routledge (2005).

Google Scholar

48. Walker KO, Labat A, Choi J, Schmittdiel J, Stewart AL, Grumbach K. Patient perceptions of integrated care: confused by the term, clear on the concept. Int J Integr Care. (2013) 13:e004. doi: 10.5334/ijic.985

PubMed Abstract | Crossref Full Text | Google Scholar

49. Rousseau C, Pontbriand A, Nadeau L, Johnson-Lafleur J. Perception of interprofessional collaboration and co-location of specialists and primary care teams in youth mental health. J Can Acad Child Adolesc Psychiatry. (2017) 26(3):198–204.29056982

PubMed Abstract | Google Scholar

50. Dalberg T, McNinch NL, Friebert S. Perceptions of barriers and facilitators to early integration of pediatric palliative care: a national survey of pediatric oncology providers. Pediatr Blood Cancer. (2018) 65(6):e26996. doi: 10.1002/pbc.26996

PubMed Abstract | Crossref Full Text | Google Scholar

51. Platt RE, Spencer AE, Burkey MD, Vidal C, Polk S, Bettencourt AF, et al. What’s known about implementing co-located paediatric integrated care: a scoping review. Int Rev Psychiatry. (2018) 30(6):242–71. doi: 10.1080/09540261.2018.1563530

PubMed Abstract | Crossref Full Text | Google Scholar

52. Price J, Beidas RS, Wolk CB, Genuario K, Kazak AE. Implementation science in pediatric psychology: the example of type 1 diabetes. J Pediatr Psychol. (2019) 44(9):1068–73. doi: 10.1093/jpepsy/jsz030

PubMed Abstract | Crossref Full Text | Google Scholar

53. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. (2011) 104(12):510–20. doi: 10.1258/jrsm.2011.110180

PubMed Abstract | Crossref Full Text | Google Scholar

54. Dahler-Larsen P. The practical utility of mixed methods: an empirical study. J Mix Methods Res. (2023) 17(2):187–208. doi: 10.1177/15586898211057359

Crossref Full Text | Google Scholar

55. Townsend T. Ethics conflicts in rural communities: privacy and confidentiality. Chapter 7. In: Nelson WA, editor. Handbook for Rural Health Care Ethics: A Practical Guide for Professionals. Lebanon, NH: Dartmouth College Press (2009). p. 126–41. Available online at: https://geiselmed.dartmouth.edu/cfm/wp-content/uploads/sites/97/2022/04/chapter-00.pdf

Google Scholar

56. Key KD, Furr-Holden D, Lewis EY, Cunningham R, Zimmerman MA, JohnsonLawrence V, et al. The continuum of community engagement in research: a roadmap for understanding and assessing progress. Prog Community Health Partnersh. (2019) 13(4):427–34. doi: 10.1353/cpr.2019.0064

PubMed Abstract | Crossref Full Text | Google Scholar

57. Wu J, Wang Y, Tao L, Peng J. Stakeholders in the healthcare service ecosystem. Procedia CIRP. (2019) 83:375–9. doi: 10.1016/j.procir.2019.04.085

Crossref Full Text | Google Scholar