Variation in research designs used to test the effectiveness of dissemination and implementation strategies: A review
- 1Prevention Research Center, Washington University in St. Louis, United States
- 2Department of Psychiatry, Washington University School of Medicine, United States
- 3George Warren Brown School of Social Work, Washington University in St. Louis, United States
- 4School of Social Work, University of Maryland, United States
- 5Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, United States
- 6Alvin J. Siteman Cancer Center, Washington University School of Medicine, United States
Background: The need for optimal study designs in dissemination and implementation (D&I) research is increasingly recognized. Despite the wide range of study designs available for D&I research, we lack understanding of the types of designs and methodologies that are routinely used in the field. This review assesses the designs and methodologies in recently proposed D&I studies and provides resources to guide design decisions.
Methods: We reviewed 404 study protocols published in the journal Implementation Science from 2/2006 to 9/2017. Eligible studies tested the efficacy or effectiveness of D&I strategies (e.g., not effectiveness of the underlying clinical or public health intervention); had a comparison by group and/or time; and used ≥1 quantitative measure. Several design elements were extracted: design category (e.g., randomized); design type (e.g., cluster randomized controlled trial); data type (e.g., quantitative); D&I theoretical framework; levels of treatment assignment, intervention, and measurement; and country in which the research was conducted. Each protocol was double-coded, and discrepancies were resolved through discussion.
Results: Of the 404 protocols reviewed, 212 (52%) studies tested one or more implementation strategy across 208 manuscripts, therefore meeting inclusion criteria. Of those studies included, 77% utilized randomized designs, primarily cluster RCTs. The use of alternative designs (e.g., stepped wedge) increased over time. Fewer studies were quasi-experimental (17%) or observational (6%). Many study design categories (e.g., controlled pre-post, matched pair cluster design) were represented by only one or two studies. Most articles proposed quantitative and qualitative methods (61%), with the remaining 39% proposing only quantitative. Half of protocols (52%) reported using a theoretical framework to guide the study. The four most frequently reported frameworks were CFIR and RE-AIM (n=16 each), followed by PARIHS and Theoretical Domains Framework (n=12 each).
Conclusions: While several novel designs for D&I research have been proposed (e.g., stepped-wedge, adaptive designs), the majority of the studies in our sample employed RCT designs. Alternative study designs are increasing in use but may be underutilized for a variety of reasons, including preference of funders or lack of awareness of these designs. Promisingly, the prevalent use of quantitative and qualitative methods together reflects methodological innovation in newer D&I research.
Keywords: Research study design, Research Methods, review, dissemination research, implementation research
Received: 17 Nov 2017;
Accepted: 30 Jan 2018.
Edited by:Connie J. Evashwick, George Washington University, United States
Reviewed by:Jo Ann Shoup, Kaiser Permanente Colorado, United States
Miruna Petrescu-Prahova, University of Washington, United States
Copyright: © 2018 Mazzucca, Tabak, Pilar, Ramsey, Baumann, Kryzer, Lewis, Padek, Powell and Brownson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: PhD. Stephanie Mazzucca, Prevention Research Center, Washington University in St. Louis, St. Louis, Missouri, United States, firstname.lastname@example.org