Use of organizational change strategies and personalized agency feedback improves addressing tobacco in behavioral health outpatient treatment settings

Introduction: Implementing evidence-based interventions for tobacco use disorder (TUD) in community mental health agencies is critical, given the low adoption rates of these interventions and the high rates of TUD among patients, contributing to the high morbidity and shortened lifespan of this population. Implementation efforts require enhancing organizational preparedness to integrate these evidence-based interventions.

Purpose: When the Addressing Tobacco Through Organizational Change (ATTOC) model was evaluated in a cluster-randomized trial (with 13 clinics, 610 clients, and 222 staff) and compared with an education-only intervention, ATTOC proved to be better at having more TUD treatment, policies, and staff skills. This paper presents a secondary analysis focusing only on the ATTOC sites, examining whether clinic-level preparedness is associated with increased implementation activities and estimating the combined direct and indirect impact on patient referrals to evidence-based TUD interventions.

Methods: Seven sites applied the ATTOC model over 9 months, with the ATTOC Environmental Scan (ES) conducted at baseline and 3, 6, and 9 months to assess the following: (1) the environment inside and outside the building, (2) staff training and personal tobacco use, (3) clinical TUD services and documentation, and (4) tobacco policies. Summary statistics are provided, and generalized linear mixed model analyses for repeated measures were used to assess time trends and relationships among composite preparedness, activities, and number of referrals.

Results: Over the 9-month period, significant improvements were observed in ES composite preparedness (p < 0.001) and individual ES areas (p < 0.001 for each). Eight out of 11 ATTOC Dashboard items showed significant changes, including increased number of patients treated (p = 0.002); tobacco discussions (p = 0.022); provision of educational brochures (p = 0.034); referrals to a Nicotine Anonymous group (p < 0.001), an in-house wellness or tobacco group (p < 0.001), and state quitline (p = 0.012); and documentation in treatment plans (p = 0.008). Both composite preparedness (p = 0.006) and composite activities (p < 0.001) were significantly associated with the number of composite referrals.

Conclusion: Significant TUD intervention uptake was found over time through the ATTOC model organizational change intervention and tracking tools.

Introduction

Tobacco use rates among individuals with serious mental illness (SMI) in the USA are approximately double in clinical settings, with this population accounting for 44% of all cigarettes consumed nationwide (1, 2). This has resulted in significant health disparities in this population, including increased risks of tobacco-associated illnesses such as cancer and respiratory and cardiovascular diseases, along with a 25-year shorter lifespan (2). Attitudes toward smoking have shifted over time, with recent efforts in banning smoking in public and private domains, as smoking contributes to the economic burden exceeding $500 billion annually (3, 4). As a result, this population faces isolating and alienating social consequences, including stigma related to mental health and smoking, which further contributes to poor health outcomes and social disadvantages (3, 5).

Unfortunately, smokers with behavioral health disorders have been historically excluded from smoking cessation clinical trials. However, the EAGLES study (6) has included this population and confirmed that existing evidence-based treatments for tobacco use disorder (TUD) are effective for this population. Despite this evidence, <25% of smokers with SMI receive evidence-based TUD treatment, and mental health clinicians, compared with other specialists, are significantly less likely to address TUD in their practice (7). This may be due to long-held myths among providers, such as the belief that tobacco is used for self-medication, that individuals with SMI are not motivated or able to quit smoking, that psychiatric symptoms will exacerbate if they do quit, and that smoking is a low health priority for this population (8). Transforming the mental healthcare system to integrate and adhere to evidence-based guidelines for TUD treatment is a priority of the National Institute of Mental Health and is a critical component of the national effort to meet Healthy People 2030 target goals for tobacco use (9, 10).

There is growing recognition and emphasis in the field of implementation science to study the mechanics of implementation strategies and measures to advance the field and its impact on health and healthcare (11, 12). Such mechanistic studies are rare in the existing literature and often lack empirical data. This study provides a unique opportunity to delve into the mechanics of a set of implementation strategies and how they interact and act to effect organizational change, in this case, within clinical settings involved in treating TUD among individuals with SMI. For this paper, we use the term tobacco cessation to include smoking cessation and TUD treatment, which includes all tobacco products, e-cigarettes, and other nicotine related products.

The Addressing Tobacco Through Organizational Change (ATTOC) model

The ATTOC model is an effective approach in promoting organizational change to ensure the provision of TUD treatment in clinical settings (13). This model comprises multiple strategies (Table 1) that support the adoption of evidence-based TUD screening, assessment, and treatment, including psychosocial, pharmacological, and community-based interventions such as the quitline (14). The system changes in ATTOC align with the Agency for Healthcare Research and Quality's (AHRQ) “Systems Change: Treating Tobacco Use and Dependence” based on the Public Health Service (PHS) Clinical Practice Guideline—2008 Update (15). System changes refer to specific strategies that healthcare administrators, managed care organizations, and purchasers of health plans can implement to treat tobacco dependence. These strategies include implementing a tobacco-user identification system; providing training, resources, and feedback; assigning staff to provide tobacco dependence treatment and assessing treatment delivery through performance evaluations; and promoting hospital policies that support and provide tobacco dependence services.

Table 1

Table 1. ATTOC model—implementation strategies.

The ATTOC model provides a structured approach that links organizational change to improved implementation and outcomes in tobacco cessation by ensuring comprehensive support for quitting tobacco. It facilitates this link through an initial comprehensive assessment of current tobacco use, policies and procedures, tobacco cessation practices and strategies currently being utilized, and areas needing improvement. This model also allows for strategic planning to address identifiable gaps, define specific goals and timelines, and assign responsibilities for implementing tobacco cessation initiatives. Because ATTOC is an evidence-based and validated approach, it can help secure organizational leadership to gain commitment for implementation and sustainability. Leaders are influential for championing the areas discussed in the initial assessment, promoting tobacco-free policies and procedures, allocating resources, and creating a culture and behavioral change that prioritizes tobacco cessation.

Most importantly, the ATTOC model allows for more integrated care, which includes tobacco cessation interventions as routine and standard care, ensuring that all patients are screened for tobacco use and offered cessation support. It also allows for the support of resources that are readily available and facilitates improved patient access to educational materials, counseling services, patient monitoring, pharmacotherapy, and follow-up support.

Possible outcomes that may benefit from the ATTOC model include increased quit rates, a healthier environment, enhanced and more integrated patient care, patient cost savings, and an organizational culture and behavioral shift that makes tobacco cessation both a norm and a priority, with sustained efforts and long-term success in reducing tobacco use. By systematically addressing tobacco use through organizational change, the ATTOC model ensures that tobacco cessation efforts are comprehensive, integrated, and effective, leading to significant improvements in implementation and outcomes.

This approach utilizes core strategies, steps, and phases to help create organizational change. Tobacco cessation interventions are commonly influenced by attitudes and theories of what behavior change should look like, as well as other barriers to tobacco cessation efforts, including staffing limitations, lack of training, and insufficient reimbursement for TUD treatment services (16). Although it has been noted that resistance to change can be observed in both patients and staff, it allows observers to note the systemic nature of these barriers and suggests that organizational change is warranted for success (13, 17). Several studies have documented the impact of the ATTOC intervention and organizational changes on both healthcare teams and patients (13, 18), including building on related approaches and studies (19–21).

Our cluster-randomized trial (18), involving 13 clinics, 610 clients, and 222 staff, compared the ATTOC model to an education-only intervention. Clients at the ATTOC sites reported more TUD treatment from providers at Weeks 12 and 24 (ps < 0.05) and more TUD treatment and policies from clinics at Weeks 12, 24, 36, and 52 (ps < 0.05). The ATTOC staff reported higher skill levels in TUD treatment at Week 36 (p = 0.05). Given the promising findings for the ATTOC intervention in this study, this secondary analysis focuses on the implementation process and intervention tools used across the seven ATTOC sites.

We conceptualize organizational culture as the usual set of expectations and norms that influence behavior within an organization, which includes the historical values and collective experiences of the organization, and the usual ways that things are done (22, 23). The champions within organizations can be described as changing the organizational culture by addressing tobacco use, which counters norms such as not addressing tobacco use, giving cigarettes as a reward for good behavior, allowing smoking on the campus, and believing that tobacco use is an unimportant matter to address.

Implementation science focuses on identifying approaches to integrate evidence-based treatments and interventions into clinical settings in an effective manner. Evidence-based interventions, however, take, on average, approximately 17 years to become integrated into routine practice (24). Implementation science provides an overarching approach that considers the problem to be addressed in which population, what evidence-based treatments and interventions should be integrated, how organizations can be supported to develop the needed support, what organizational change methods might be used to support the leadership and staff, how to ensure sustainability of the change, and how to assess the effectiveness of the change. There is growing recognition in implementation research to allow for a distinct focus on organizational change (25–27). Despite the growing literature around the determinants of organizational change, few models actually provide a deeper understanding of the process of organizational change.

The ATTOC model is a process model specifically focused on healthcare organizations and is guided by the organization development theory (28–31). It provides guiding principles for the implementation of system-wide interventions using behavioral science strategies. This model provides a generalizable, validated set of strategies for use in healthcare settings. Although it is a unique process model, it aligns with the implementation outcome framework, proposed by Proctor et al. (32), in which they set the implementation outcomes in the pathway to achieving service and client health outcomes. We propose that the ATTOC model provides a set of strategies that impact implementation outcomes, especially in the context of organizational change. Indeed, in a recent review, Proctor et al. called for an increased focus on organizational processes and testing mechanistic pathways for implementation outcomes (33). Following this model, we specified our implementation outcomes as the preparedness and the activities undertaken at the organizational level, culminating in the referrals provided to TUD patients.

In this report, we examine the changes at the organizational level following the implementation of the ATTOC model within community mental health agencies to address TUD patients. We hypothesized that implementation indicators collected via the ATTOC Environmental Scan (ES) and the ATTOC Dashboard would increase over time. We next hypothesized that combined environmental, staff, and patient preparedness for the intervention and TUD patient activities are potential mechanistic pathways that would be positively associated with increases in patient referrals. We also investigate whether preparedness is associated with improvement in the number of activities and estimate the combined direct and indirect impact on referrals.

Materials and methods

Data were collected from clinical staff and leaders who participated in the parent trial (ClinicalTrials.gov ID: NCT02849652). The primary findings from this trial were published elsewhere (18). This secondary analysis examines the ATTOC sites’ organizational changes in depth to better understand the intervention and key effective components. Community mental health clinics (CMHCs) within Philadelphia's Community Behavioral Health system were randomized to one of the two approaches (training alone vs. training and ATTOC). Sites were eligible for this trial if they had an electronic health record, provided access to prescription data (tobacco use treatments), and could enroll at least 12 staff members in the clinical trial. A complete description of the study procedures is reported elsewhere (34).

This report includes data only from seven ATTOC sites. These agencies varied in size, with 28–140 clinicians, 110–1,500 patients, and 60%–80% minorities (patients). The measures of the ATTOC Environmental Scan (baseline and 3, 6, and 9 months) and the ATTOC Dashboard (monthly) were used at these ATTOC intervention sites, which also provided feedback to the clinicians and leadership.

The Institutional Review Boards of the University of Pennsylvania and the City of Philadelphia provided approval for the trial. Following randomization at the level of the CMHC, research personnel attended clinics to enroll staff prior to the intervention. Interested and eligible staff provided informed consent and completed a baseline self-report assessment.

ATTOC model intervention

The ATTOC model proposes a series of 10 steps, in three distinct phases of (1) preparing and organizing for change; (2) changing, integrating, and adapting; and (3) documenting, monitoring, and sustaining. Within the three phases is a series of 10 steps to guide healthcare delivery sites through organizational change. These phases and steps are flexible to accommodate the unique needs, barriers, and resources of the site implementing change, and organizations are encouraged to consider starting at the point that best matches their context.

During the 9-month ATTOC intervention, seven core strategies (35) were used by the ATTOC intervention consultants with the agencies for implementation, as shown in Table 1. Staff training of evidence-based interventions includes the provision of the AHRQ guideline document, Clinical Guidelines and Recommendations, and an Internet link for training materials on TUD assessment and treatment (14). In addition, the goal of reducing tobacco use on the campus (tobacco-free or restricted use) was an evidence-based public health strategy to support clinical evidence-based practices. The primary ATTOC activities occurred in 10 sessions (2 of which were in-person and on-site visits, and 8 were through videoconference/telephone). Each site set up unique and specific goals in three critical areas: (1) related to their patient care and treatment, (2) staff training and addressing staff smoking, and (3) the agency's campus becoming tobacco-free in total or in a restricted manner. Ongoing consultations were provided through videoconferencing to address any concerns, obstacles, or problems that may emerge during implementation. In addition, site staff were provided with access to an interactive website that provides information, tools, and materials for training and treatment. Site champions were encouraged to use the dashboard assessment of clinician and agency performance to provide formal performance feedback.

In Phase 2 (Weeks 4, 8, and 12), activities supported the implementation of the change plans. These activities included clinical training, TUD treatment service development, and environmental changes.

Environmental Scan

The Environmental Scan is an assessment tool used at baseline and at 3, 6, and 9 months post-baseline to assess four areas of preparedness of a health organization in addressing TUD: (1) environmental preparedness; (2) patient care preparedness; (3) staff competency, smoking patterns, and attitude; and (4) tobacco policies and procedures (see Table 2). All preparedness measures were scored from 1 to 5: not prepared (1), minimally prepared (2), moderately prepared (3), highly prepared (4), very highly prepared (5). This assessment was performed by the ATTOC intervention consulting team with multiple assessors doing independent scoring (with excellent reliability across assessors through consensus and only a few times an initial deviation greater than one point). Information was gathered through meetings with agency leaders, staff, and patients; review of policies; review of website and social media; walkthroughs of the campus (outside and inside the building); and patient chart reviews.

Table 2

Table 2. ATTOC Environmental Scan components.

Environmental preparedness was evaluated through observations of outdoor and indoor signage, smoking areas, the presence of cigarette butts, and individuals smoking. Signage may include “no smoking” posters and signs that designate smoking and non-smoking areas. An agency that has all non-smoking areas designated with visible and clear signage containing comprehensive, agency-specific messaging (e.g., written in large print, bilingual, and with the agency's logo) and has no evidence of smoking anywhere on the grounds would be considered optimally prepared. Website, social media, use of a carbon monoxide (CO) meter, and billing for TUD services were also considered in this assessment.

Patient care preparedness was evaluated by assessing patient charts, with an average, in this study, of five charts per assessment scan. A well-prepared agency would have documentation of a patient's screening and assessment of tobacco use, a diagnosis of TUD when necessary, motivational level of change, and a treatment plan. An agency would also have either a medication prescriber in place within the agency or in partnership with an external prescriber. Finally, individual or group psychosocial treatments offered on-site and matched to a client's motivational level should be indicated, including referrals to community resources such as quitlines, phone apps, Nicotine Anonymous (Nic A) meetings, or websites.

Staff competency, smoking patterns, and attitude toward tobacco use in general and the staff's own smoking patterns were assessed using a 15-item questionnaire. This scale assesses the staff's own tobacco use, attitude about addressing tobacco in treatment, and prior training. This staff-level assessment focuses on readiness to change of the staff and the common barriers to change: staff smoking and use of other tobacco products, level of training on TUD treatment, and their confidence in treating tobacco. Information for this scale was collected from conversations with staff and leadership and by reviewing agency documents and policies.

Finally, tobacco policies and procedures were reviewed to assess policies regarding tobacco use (e.g., restricted use, tobacco-free campus), services provided to patients, and support for staff TUD recovery. This assessment examines if there is dedication to a tobacco-free campus not only in discourse, but in clearly delineated policies and whether these are enforced. A comprehensive TUD policy and standard operating procedures would include descriptions of and how to integrate assessments, use of a CO meter, patient education, diagnosis, treatment planning, treatment access, and detailed documentation of such operations is expected for a well-prepared agency.

The Environmental Scan information and scoring were conducted independently by multiple ATTOC intervention consultants at each assessment time point, and individual findings were reviewed together to compare findings and determine a final version of the ES assessment and the scoring for each of the four areas. The four preparedness areas were scored on a range of 1–5, with a score of 5 marking excellence in preparedness. A table of anchoring points had been developed for this scoring purpose. The four individual scores were summed to provide an overall preparedness score. The findings and scoring were reviewed with the local agency champion and leaders after each ES was completed. In addition, an overall composite preparedness score was calculated by averaging the environmental preparedness score, the patient preparedness score, and the staff competency preparedness score. These three ES scores were significantly correlated (Pearson correlation >0.65 for all) with a standardized Cronbach's α = 0.89. We hypothesized that this composite measure can be used to model the association between preparedness and patient referrals during the study.

The Environmental Scan was developed empirically in the process of helping over 100 organizations to adopt evidence-based TUD treatments and restrict tobacco use on their campus. The scan was a comprehensive assessment of the organization that was given to the leadership as feedback on their baseline and then ongoing progress. Each scan had two or three raters who engaged in the same activity, and then the ratings were compared for concordance in evaluation. This process helped refine the instrument for the study. For this paper, we used composite variables and assessed correlations and Cronbach's α values to assess the reliability of the proposed composite measures for preparedness, referrals, and activities, described above.

ATTOC Dashboard

The ATTOC Dashboard is a one-page summary of the agency clinician behaviors related to the implementation of 11 TUD patient activities, documentation, and staff training. This monthly assessment is based on staff self-reports and chart reviews. The dashboard aggregated individual clinician data and summarized information such as patient caseload number, average amount of time spent with each patient and time spent on tobacco discussion, and the number of patients who received an educational brochure, undwerwent a CO test, and/or were referred to a Nic A group, an in-house wellness or tobacco group, state's quitline, or for TUD medications. Documentation of TUD and attendance at staff training were also assessed. Table 3 summarizes this information. A composite variable to represent patient care activities was created by averaging three items: the number of consumers spoken to about tobacco cessation, the number of brochures handed out, and the number of consumers who received a CO test. Variables in the composite activity measure were significantly correlated p < 0.03 (Pearson correlations between 0.89 and 0.31) and had a standardized Cronbach's α = 0.76. A composite referrals variable was computed by averaging four dashboard items: number of referrals to a Nic A group, number of referrals to an in-house wellness or tobacco group, number of referrals to the state's quitline, and number of consumers assisted in obtaining smoking cessation medications (Table 3). The composite referral variable had a standardized Cronbach's α = 0.89 with Pearson correlations among variables ≥0.47 and p < 0.001.

Table 3

Table 3. ATTOC Dashboard—patient care related items.

Data analysis

All Environmental Scan and ATTOC Dashboard data used in analyses are clinic-level data collected over time. Summary statistics were calculated for the four key Environmental Scan areas, for the composite preparedness score (range from 1, not prepared, to 5, highly prepared; Table 2), for continuous dashboard items, and for the composite activities and referrals scores (Table 3). Frequencies and percentages for categorical dashboard items (staff attendance at training events, tobacco interventions logged) were computed by week.

Generalized linear mixed model analyses for repeated measures were used to assess time trends for Environmental Scan and for ATTOC Dashboard items. The models assumed normally distributed error distributions except for a binary dashboard item for attendance at staff training (logistic model) and a three-level ordinal score for logging tobacco interventions into treatment or service plans (0 none, 1 some, or 2 all; Poisson model). We fitted the models with linear time and time as a categorical variable. The models were constructed with the first measurement week as the reference conditions so that positive coefficient estimates represent improved scores relative to baseline. For a one-arm pre-post design like this study, one must make the strong assumption that sites without an intervention would not change over time for the coefficients to be unbiased estimates of impact on implementation outcomes. Predicted means and 95% CI for each week are reported graphically. Distributions of residuals and plots of residuals vs. predicted values were examined to assess model assumptions. Non-parametric Spearman’s correlations with week and Kruskal–Wallis tests were used to assess the sensitivity of analysis conclusions to model assumptions. We also assessed the effects of missing dashboard data on time trend analyses using fully conditional multiple imputation (N = 25 imputations) (36). In addition, we conducted analyses to investigate how composite preparedness was associated with the number of activities and how these influenced the number of referrals via direct and indirect pathways (37, 38). Referrals, composite cessation activities, and composite preparedness measures are described above. The three variables were standardized to z-scores before analyses.

We used linear mixed model analyses with random site effects to assess relationships among preparedness, activities, and referrals. Four models were fitted (site random effect and residual error terms not shown):

Model A: Activities = i + a × Preparedness

Model B: Referrals = i + b × Activities

Model C: Referrals = i + c × Preparedness

Model D: Referrals = i + b′ × Activities + c′ × Preparedness

Estimated regression coefficients represent the change in the dependent variable in standard deviation units that are associated with a one standard deviation increase in the independent variable. Using coefficients from these models, we estimated unadjusted non-causal effects for Models A, B, and C. By including Model D, we were able to estimate the indirect effect of composite preparedness on composite referrals as mediated by composite activities as the product of the Model A preparedness coefficient, a, times the Model D activities coefficient, b′ (37, 38). To conduct the analysis, Week 1 preparedness was linked to Week 4 activities and referrals and by week for subsequent Weeks 12 and 36. Sensitivity to missing values was assessed by imputing missing preparedness and referrals with available nearest-neighbor values that were closest in time to the missing data for Site E (Week 12 for missing Week 4) and Sites F and G (Week 32 for missing Week 36). Statistical significance was set at α = 0.05, and analyses were conducted using SAS v9.4 (mixed, glimmix, causalmed, mi, and mianalyze). For this study of seven intervention-only sites, we did not adjust for multiple comparisons.

Results

To test our hypotheses around the mechanism of the ATTOC intervention, we evaluated the impact on preparedness and activities in the intervention sites.

Environmental Scan

ES reports were collected for all sites and weeks. Scores were relatively low at Week 1 and increased significantly to Week 36 (Figure 1, p < 0.001 for all). At Week 36, three out of seven clinics had a composite preparedness score of at least 4.0. Staff competency had the highest score at Week 36 (4.3 ± 0.5), and the tobacco policy and procedures component had the most between-clinic variation in scores (SD = 1.1). Over the course of the 9-month intervention, improvement in scores had a linear trend (p < 0.001 for all) with no apparent leveling off after Week 12.

Figure 1

Figure 1. Scores for individual Environmental Scan components and for the composite score. Black symbols and whiskers are means and 95% CI. Gray lines are individual sites, which may overlap and be obscured. All Environmental Scan measures increased significantly after Week 1, p < 0.001 for all.

ATTOC Dashboard

Eight out of 11 ATTOC Dashboard items showed significant changes over time (Figure 2), including increased number of patients treated (p = 0.002), number of patients spoken to about tobacco cessation (p = 0.022), number of patients who received educational brochures (p = 0.034), and number of patients referred to a Nicotine Anonymous group (p < 0.001), an in-house wellness or tobacco group (p < 0.001), and state quitline (p = 0.012). In addition, there was an increase in tobacco use and TUD documentation in treatment plans (p = 0.008). The amount of missing data was variable for clinic-collected dashboard items. In contrast to the ES measures, nearly half of the dashboard items were missing for Clinic B (49%), Clinic E had 22% missing, Clinic G had 15% missing, and the remaining clinics had <5% missing data. The results from fully conditional multiple imputation confirmed the conclusions based on available case analyses. Examination of residuals revealed potential outliers and variances possibly increasing with predicted values. Spearman's correlation analyses between dashboard items and week remained significant for all except the number of patients treated (p = 0.053); however, Spearman's correlations did not account for repeated measures.

Figure 2

Figure 2. Dashboard items by week for seven participating intervention sites. Black symbols and whiskers are means and 95% CI. Gray lines are individual sites, which may overlap and be obscured or may be missing. Site B did not report data for Weeks 16–28, Site E is missing Weeks 4 and 8, and Sites F and G are missing Week 36.

Impact of preparedness and tobacco cessation activities on referrals

We hypothesized that increasing preparedness and cessation activities would be predictive of referrals (Figure 3A). In the separate mixed model analyses, we noted significant unadjusted associations with the site-preparedness and the number of activities undertaken by the sites, including providing brochures and discussing tobacco cessation services. Both composite preparedness (Model C = 0.53 p = 0.006) and composite activities (Model B = 0.79; p < 0.001) were significantly associated with the number of composite referrals. We also found that the average number of activities increased significantly more when average preparedness was higher (Model A = 0.39; p = 0.018). Coefficients from our multivariable Model D are shown in Figure 3B. The adjusted association between referrals and preparedness was c′ = 0.20 (95% CI: −0.09 to 0.49, p = 0.158), and the activities coefficient was b′ = 0.73 (95% CI: 0.44–1.01, p < 0.001). Model D explained 64% of the variation in the number of referrals. The coefficient in Figure 3B is the same as in Model A in Figure 3A. The indirect effect pathway from preparedness through composite activities to referrals was a × b′ = 0.29 (SE = 0.12, p = 0.021), which accounted for 54% of the total effect of preparedness on the number of referrals (Model C = 0.52; 95% CI: 0.18–0.87, p = 0.005). The results from sensitivity analyses were not substantially different from the available data analyses.

Figure 3

Figure 3. Proposed conceptual model for the impact of preparedness and tobacco cessation activities on referrals. (A) Unadjusted standardized coefficients from three separate generalized linear mixed model analyses: Model A, Model B, and Model C. (B) Coefficients from Model A (a) and Model D (b′ and c′). The indirect effect of preparedness mediated by activities is a × b′ = 0.29 (SE = 0.12, p = 0.021).

Discussion

Tobacco use among individuals with SMI is common. Changing the organizational culture to integrate evidence-based treatments for TUD will be vital to address this health disparity. This paper confirms that support for preparedness and activities leads to increased number of patient interventions on-site (assessments, discussions with patients about tobacco use, brochures created and given, and CO monitoring) and referrals within the site and to community interventions (referrals to a Nic A group, an in-house wellness or tobacco group, and state quitline and number of patients obtaining TUD smoking cessation medications) during the 10 session, 9-month ATTOC model intervention.

This study also demonstrated the process by which organizational change (i.e., increased preparedness, increased activities, and increased number of referrals) can lead to implementation of evidence-based interventions for TUD in behavioral health settings. The same organizational ATTOC approach was used at each site; however, each site determined its change goals in three critical areas: (1) patient care and treatment, (2) staff training and staff smoking, and (3) their campus becoming tobacco-free in total or in a restricted manner. The proposed work builds on the substantial work supporting the feasibility and impact of ATTOC and is part of a larger study attempting to validate the ATTOC model for organizational change in a randomized control trial, building on previous work for a single-arm evaluation (18, 34). Informed by implementation science and organizational change theory, this paper focused on elucidating the mechanistic pathways to changing organizational culture and can be generalized to culture changes needed to enhance clinical outcomes through the integration of evidence-based treatments for the intervention sites.

This study found significant improvements through the ATTOC Environmental Scan assessment in critical areas of the clinic, including staff training, restrictions on tobacco use or fully tobacco-free campuses, new policies, integration of new evidence-based treatments (medications and psychosocial and community-based approaches), communication tools, and patients engaged in assessment and treatment. As described in the results, there was a significant change across the sites for the Environmental Scan composite preparedness score and for each of the four core individual areas (environmental, patient care, staff competency and recovery, and policy/procedures) of the ES.

The findings reported from the 11 ATTOC Dashboard items showed significant improvements in the number of patients treated, talked with about tobacco cessation, provided educational brochures, and referred to Nic A and in-house wellness or tobacco groups and state quitline, as well as documentation in treatment plans. Dashboard items were not available for all weeks from some sites (Figure 2), which increased variability when evaluating trends. Changes in preparedness and cessation activities were predictive of the number of referrals (Figure 3), with 54% of the preparedness impact due to its positive association with increasing activities. Future reports will focus on the clinical outcomes of the patients, whereas this report focuses on the ATTOC intervention being able to support the needed organizational changes.

Having frontline staff with skills to discuss TUD and provide evidence-based treatments for patients with varying levels of quit motivation appears critical to enhancing the integration of evidence-based treatments and improving TUD treatment engagement and outcomes. As such, staff were prepared to confidently provide general information and connect patients to resources through phone, online, internal tobacco cessation specialists, or external community resources. The ongoing nature of the training appears to support the culture change and the impact of more patients receiving the interventions. A potential barrier to addressing TUD with patients and building efficacy, confidence, and competence is the staff member's own challenges with quitting smoking. Addressing this within the ATTOC intervention appears to help increase the number of patients receiving care, as noted in the larger study (18). Stigma can impact staff members (and patients) who may be dealing with a sense of shame, embarrassment, or guilt, resulting in a lower likelihood of helping their patients address tobacco use. ATTOC addressed this concern directly in the training and also by supporting tobacco cessation among staff members by encouraging the organization to either cover costs or reduce co-pays for treatment and providing them time off for such treatment. Changes by the organization's CEO, clinical leaders, and frontline clinicians were noticed and assessed in the Environmental Scan, and these changes appear to be associated with providing more TUD patient care. The changes in agency readiness were also reflected by staff having been able to reflect on other times they were successful on large organizational changes, their values of the importance of the recovery and wellness in the organization, and their personal mission, and through the support of the local champion. A strong champion demonstrates leadership, management, and effective communication, knows the organizational change process, and has the ability to collaborate and support all aspects of the ATTOC processes, phases, and steps.

The study has potential implications for impacting clinical practice, healthcare policies, and future research directions. First, the findings demonstrate how the ATTOC intervention helped agencies follow both the PHS guidelines for clinical practice changes and organizational changes (i.e., clinical system policies, screenings and assessments, and trainings). By doing so, the ATTOC model led to sites making the organizational changes recommended and by having more clinical services and better trained staff than the control study [see the main study findings of Schnoll et al. (18)], which builds capacity in the system through organizational change. These expanded clinical services were advanced because of the staff trainings, mentoring of champions who were members of the agency and led the organizational change effort with the support of the ATTOC consultants, the development of new patient educational materials, provision of CO meters for clinical programs, trained staff on screenings and assessment, starting a quitting group or group for lower motivated patients to increase their motivation (i.e., learning about health living), and medication management training. From a policy perspective, this study builds on the PHS guidelines and has implications for changes in the Joint Commission standards for behavioral health care organizations to ensure safe, high-quality treatment. The Joint Commission standards are stronger for general medical facilities, and this study suggests that if there were higher standards in behavioral health settings that were similar to general medical settings, it would help facilitate needed organizational change as well as enhanced clinical services. Finally, future research should assess organizational change in both the intervention and control groups and further increase the sample size. Additional patient care outcomes might also be added in both the intervention group and the control group, including assessing CO levels. Future research might incorporate a focus on implementation costs and cost–benefit that can further strengthen the evidence for implementing the ATTOC model and also align with prevention research, which demonstrates the return on investment for these initially higher costs. Furthermore, using the Project Extension of Community Health Outcomes approach (39) could be a method to increase the sample size of sites dramatically and be a potential opportunity to deliver the training with low cost and with maximum reach, allowing for a group effect of the “all teach all learn” principle, rather that one program being supported in isolation.

We acknowledge that organizational change requires time; however, in the current study, by focusing on specific implementation indicators such as preparedness, activities, and referral, we were able to show positive organizational change. As one might expect, changing tobacco policy and procedures takes longer in some organizations due to the amount of discussion required and formal processes for getting policies added or changed. For example, one site needed 12 weeks to get policies modified. Addressing tobacco is a culture change, and in one site, this was more difficult, and there was limited change in staff competency, smoking, and attitudes that addressing tobacco was important. The policies and procedures at the site are influenced by both staff attitudes about addressing tobacco as well as staff recovery rates. In previous ATTOC studies, these policies and procedures can be more difficult at sites that have a high rate of smokers, limited resources for staff to quit, or when there are other competing organizational demands.

The study results should be interpreted in the light of limitations. First, since the control sites did not receive implementation support, the organizational change data (Environmental Scan and ATTOC Dashboard) were not collected in the control sites. For a one-arm pre-post design like this study, there is limited generalizability, and one must assume that sites without an intervention would not change over time for the coefficients to be unbiased estimates of impact on implementation outcomes. Second, there were some differences in client and staff characteristics between ATTOC sites and our control sites (18), suggesting caution in generalizing these results to all community mental health sites. Finally, we acknowledge the missing data and performed additional model diagnostics to ensure that there was no evidence of non-normality or heteroscedasticity due to the missing data. Future implementation efforts can focus on reducing missing data when collecting implementation outcomes and be supported through qualitative methods of inquiry.

Conclusion

The sites that participated in ATTOC showed improvement in the provision of evidence-based treatments for tobacco use in outpatient mental health treatment settings and enhanced campus tobacco use restrictions, staff training, staff TUD recovery, and TUD clinical documentation, policies, and new signage/educational materials for patients. It is important to include patients, staff, and environmental assessment to promote cultural change in tobacco cessation programs. Studying the indicators and mechanisms of implementation in the intervention sites provided much-needed information on the organizational change process and its impact.

A collaborative effort must be made to fight stigma that impacts vulnerable populations, such as individuals with SMI. Addressing environmental factors, including challenging long-held myths and beliefs about tobacco use, becomes critical in tobacco cessation treatment programming, procedures, and policies. And while the efficacy of tobacco cessation interventions has improved the lives of many populations across the globe, broader, innovative approaches that target underserved or vulnerable populations are needed to address various socioecological factors that contribute to tobacco use. Furthermore, interventions must move beyond only targeting TUD and move toward the promotion of wellness and recovery paradigms that address comprehensive healthy ways of living, including a system of social support, among individuals with SMI. Treatments that target mental health systems may show greater promise for tobacco cessation than individual-level interventions alone.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Human Research Protections Office—Institutional Review Board of the University of New Mexico’s Health Sciences Center. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

DZ: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing. RS: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing. OM: Conceptualization, Data curation, Formal analysis, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing. TA: Writing – original draft, Writing – review & editing. AB: Writing – original draft, Writing – review & editing. CO: Project administration, Writing – review & editing. FL: Funding acquisition, Resources, Supervision, Writing – review & editing. MH: Project administration, Writing – review & editing. PA: Conceptualization, Methodology, Visualization, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This research was funded by the National Cancer Institute, National Institutes of Health, grant number R01CA202699. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Acknowledgments

We acknowledge the contributions of the participating sites and individuals for their support in the research study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence, and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Lipari RN, Van Horn S. Smoking and mental illness among adults in the United States. The CBHSQ Report. Rockville (MD): Substance Abuse and Mental Health Services Administration (US) (2017). p. 1. Available online at: http://www.ncbi.nlm.nih.gov/books/NBK430654/ (Accessed August 03, 2024).

Google Scholar

2. Prochaska JJ. Smoking and mental illness — breaking the link. N Engl J Med. (2011) 365(3):196–8. doi: 10.1056/NEJMp1105248

PubMed Abstract | Crossref Full Text | Google Scholar

3. Castaldelli-Maia JM, Ventriglio A, Bhugra D. Tobacco smoking: from ‘glamour’ to ’stigma’. A comprehensive review. Psychiatry Clin Neurosci. (2016) 70(1):24–33. doi: 10.1111/pcn.12365

PubMed Abstract | Crossref Full Text | Google Scholar

4. Ekpu VU, Brown AK. The economic impact of smoking and of reducing smoking prevalence: review of evidence. Tob Use Insights. (2015) 8:TUI.S15628. doi: 10.4137/TUI.S15628

Crossref Full Text | Google Scholar

5. Graham H. Smoking, stigma and social class. J Soc Policy. (2012) 41(1):83–99. doi: 10.1017/S004727941100033X

Crossref Full Text | Google Scholar

6. Anthenelli RM, Benowitz NL, West R, St Aubin L, McRae T, Lawrence D, et al. Neuropsychiatric safety and efficacy of varenicline, bupropion, and nicotine patch in smokers with and without psychiatric disorders (EAGLES): a double-blind, randomised, placebo-controlled clinical trial. Lancet. (2016) 387(10037):2507–20. doi: 10.1016/S0140-6736(16)30272-0

PubMed Abstract | Crossref Full Text | Google Scholar

7. Rojewski AM, Baldassarri S, Cooperman NA, Gritz ER, Leone FT, Piper ME, et al. Exploring issues of comorbid conditions in people who smoke: table 1. Nicotine Tob Res. (2016) 18(8):1684–96. doi: 10.1093/ntr/ntw016

PubMed Abstract | Crossref Full Text | Google Scholar

8. Schroeder SA. Smoking cessation should be an integral part of serious mental illness treatment. World Psychiatry. (2016) 15(2):175–6. doi: 10.1002/wps.20332

PubMed Abstract | Crossref Full Text | Google Scholar

9. Ziedonis D, Hitsman B, Beckham JC, Zvolensky M, Adler LE, Audrain-McGovern J, et al. Tobacco use and cessation in psychiatric disorders: National Institute of Mental Health report. Nicotine Tob Res. (2008) 10(12):1691–715. doi: 10.1080/14622200802443569

PubMed Abstract | Crossref Full Text | Google Scholar

10. HP 2030. Healthy People 2030: Increase the Proportion of Females Who Get Screened for Cervical Cancer—C-09 (2022).

Google Scholar

11. Lewis CC, Klasnja P, Lyon AR, Powell BJ, Lengnick-Hall R, Buchanan G, et al. The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms. Implement Sci Commun. (2022) 3(1):114. doi: 10.1186/s43058-022-00358-3

PubMed Abstract | Crossref Full Text | Google Scholar

12. Weiner BJ, Lewis MA, Clauser SB, Stitzenberg KB. In search of synergy: strategies for combining interventions at multiple levels. JNCI Monogr. (2012) 2012(44):34–41. doi: 10.1093/jncimonographs/lgs001

PubMed Abstract | Crossref Full Text | Google Scholar

13. Guydish J, Ziedonis D, Tajima B, Seward G, Passalacqua E, Chan M, et al. Addressing Tobacco Through Organizational Change (ATTOC) in residential addiction treatment settings. Drug Alcohol Depend. (2012) 121(1–2):30–7. doi: 10.1016/j.drugalcdep.2011.08.003

PubMed Abstract | Crossref Full Text | Google Scholar

14. Clinical Guidelines and Recommendations. Rockville, MD: Agency for Healthcare Research and Quality (2021). Available online at: https://www.ahrq.gov/prevention/guidelines/index.html (Accessed August 03, 2024).

Google Scholar

15. Treating Tobacco Use and Dependence: 2008 Update. Rockville, MD: Agency for Healthcare Research and Quality (2020). Available online at: https://www.ahrq.gov/prevention/guidelines/tobacco/index.html (Accessed August 03, 2024).

Google Scholar

16. Guydish J, Passalacqua E, Tajima B, Manser ST. Staff smoking and other barriers to nicotine dependence intervention in addiction treatment settings: a review. J Psychoactive Drugs. (2007) 39(4):423–33. doi: 10.1080/02791072.2007.10399881

PubMed Abstract | Crossref Full Text | Google Scholar

17. Sussman L, Adams AJ, Kuzmits FE, Raho LE. No title found. J Bus Ethics. (2002) 40(4):313–29. doi: 10.1023/A:1020807700478

Crossref Full Text | Google Scholar

18. Schnoll RA, Leone FT, Quinn MH, Stevens N, Flitter A, Wileyto P, et al. A randomized clinical trial testing two implementation strategies to promote the treatment of tobacco dependence in community mental healthcare. Drug Alcohol Depend. (2023) 247:109873. doi: 10.1016/j.drugalcdep.2023.109873

PubMed Abstract | Crossref Full Text | Google Scholar

19. Capesius T, Betzner A. CommuniCare SMI / SUD Tobacco Cessation Programs Cumulative Summary Report. Professional Data Analysts. Inc. (2013). Connecticut Department of Health. Available online at: https://portal.ct.gov/-/media/OPM/FinalReportCCIFeb10toDec1233113pdf.pdf (Accessed September 08, 2025).

Google Scholar

20. Williams JM, Foulds J, Dwyer M, Order-Connors B, Springer M, Gadde P, et al.. The integration of tobacco dependence treatment and tobacco-free standards into residential addictions treatment in New Jersey. J Subst Abuse Treat. (2005) 28(4):331–40. doi: 10.1016/j.jsat.2005.02.010

PubMed Abstract | Crossref Full Text | Google Scholar

21. Hoffman AL, Kantor B, Leech T, Lindberg D, Order-Connors B, Schreiber J, et al. Drug-free is nicotine-free: a manual for chemical dependency treatment programs. New Brunswick, NJ: Tobacco Dependence Program (1997).

Google Scholar

22. Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood KE, Mayberg S, et al. Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Adm Policy Ment Health Ment Health Serv Res. (2008) 35(1–2):98–113. doi: 10.1007/s10488-007-0148-5

PubMed Abstract | Crossref Full Text | Google Scholar

23. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. (2011) 38(1):4–23. doi: 10.1007/s10488-010-0327-7

PubMed Abstract | Crossref Full Text | Google Scholar

24. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. (2011) 104(12):510–20. doi: 10.1258/jrsm.2011.110180

PubMed Abstract | Crossref Full Text | Google Scholar

25. Leeman J, Calancie L, Kegler MC, Escoffery CT, Herrmann AK, Thatcher E, et al. Developing theory to guide building practitioners’ capacity to implement evidence-based interventions. Health Educ Behav Off Publ Soc Public Health Educ. (2017) 44(1):59–69. doi: 10.1177/1090198115610572

Crossref Full Text | Google Scholar

26. Birken SA, Ko LK, Wangen M, Wagi CR, Bender M, Nilsen P, et al. Increasing access to organization theories for implementation science. Front Health Serv. (2022) 2:891507. doi: 10.3389/frhs.2022.891507

PubMed Abstract | Crossref Full Text | Google Scholar

27. Birken SA, Wagi CR, Peluso AG, Kegler MC, Baloh J, Adsul P, et al. Toward a more comprehensive understanding of organizational influences on implementation: the organization theory for implementation science framework. Front Health Serv. (2023) 3:1142598. doi: 10.3389/frhs.2023.1142598

PubMed Abstract | Crossref Full Text | Google Scholar

28. Cummings T. Organization development and change. In: Boonstra JJ, editor. Dynamics of Organizational Change and Learning. West Sussex: John Wiley & Sons Ltd (2004). p. 25–42. doi: 10.1002/9780470753408.ch2 (Accessed August 03, 2024).

Crossref Full Text | Google Scholar

29. Boonstra JJ. Dynamics of Organizational Change and Learning. West Essex, England; Hoboken, NJ, United States: J. Wiley & Sons Inc (2004). p. 492. Wiley Handbooks in the Psychology of Management in Organizations.

Google Scholar

30. Gregory BT, Armenakis AA, Moates KN, Albritton MD, Harris SG. Achieving scientific rigor in organizational diagnosis: an application of the diagnostic funnel. Consult Psychol J Pract Res. (2007) 59(2):79–90. doi: 10.1037/1065-9293.59.2.79

Crossref Full Text | Google Scholar

31. Levin M. Organizing change processes. In: Boonstra JJ, editor. Dynamics of Organizational Change and Learning. West Sussex: John Wiley & Sons Ltd (2004). p. 71–84. doi: 10.1002/9780470753408.ch4 (Accessed August 03, 2024).

Crossref Full Text | Google Scholar

32. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38(2):65–76. doi: 10.1007/s10488-010-0319-7

PubMed Abstract | Crossref Full Text | Google Scholar

33. Proctor EK, Bunger AC, Lengnick-Hall R, Gerke DR, Martin JK, Phillips RJ, et al. Ten years of implementation outcomes research: a scoping review. Implement Sci. (2023) 18(1):31. doi: 10.1186/s13012-023-01286-z

PubMed Abstract | Crossref Full Text | Google Scholar

34. Flitter AS, Lubitz SF, Ziedonis D, Stevens N, Leone FT, Mandell D, et al. A cluster-randomized clinical trial testing the effectiveness of the addressing tobacco through organizational change model for improving the treatment of tobacco use in community mental health care: preliminary study feasibility and baseline findings. Nicotine Tob Res. (2019) 21(5):559–67. doi: 10.1093/ntr/nty239

PubMed Abstract | Crossref Full Text | Google Scholar

35. Ziedonis DM, Zammarelli L, Seward G, Oliver K, Guydish J, Hobart M, et al. Addressing tobacco use through organizational change: a case study of an addiction treatment organization. J Psychoactive Drugs. (2007) 39(4):451–9. doi: 10.1080/02791072.2007.10399884

PubMed Abstract | Crossref Full Text | Google Scholar

36. Rubin DB. Frontmatter. In: Rubin DB, editor. Multiple Imputation for Nonresponse in Surveys. Canada: Wiley & Sons (1987). (Wiley Series in Probability and Statistics). Available online at: https://doi.org/10.1002/9780470316696.fmatter (Accessed July 29, 2025).

Google Scholar

37. Mackinnon DP, Dwyer JH. Estimating mediated effects in prevention studies. Eval Rev. (1993) 17(2):144–58. doi: 10.1177/0193841X9301700202

Crossref Full Text | Google Scholar

38. Vanderweele TJ, Vansteelandt S. Conceptual issues concerning mediation, interventions and composition. Stat Interface. (2009) 2(4):457–68. doi: 10.4310/SII.2009.v2.n4.a7

Crossref Full Text | Google Scholar

39. Arora S, Thornton K, Murata G, Deming P, Kalishman S, Dion D, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. (2011) 364(23):2199–207. doi: 10.1056/NEJMoa1009370

PubMed Abstract | Crossref Full Text | Google Scholar