Improving child health service interventions through a Theory of Change: A scoping review

Abstract

Background:

The objective of this scoping review was to map how child health service interventions develop, utilise, and refine theories of change. A Theory of Change (ToC) is a tool for designing, implementing, and evaluating interventions that is being increasingly used by child health practitioners who are aiming to enact change in health services.

Methods:

A published protocol guided this scoping review. Relevant publications were identified through selected electronic databases and grey literature via a search strategy. The main inclusion criteria were any child health service intervention globally that described their ToC or ToC development process. These were applied by two independent reviewers. Data relevant to the research sub-questions were extracted, charted and discussed.

Findings:

38 studies were included in the analysis. This scoping review highlights the disparate and inconsistent use, and reporting of ToCs in the child health service intervention literature.

Conclusion:

A ToC may be a helpful tool to enact change in a child health service but careful consideration must be undertaken by the child health service regarding how to maximise the benefits of doing a ToC, and how to accurately report it.

1. Introduction

Child health service interventions (CHSI) target the access to, and the use, costs, quality, delivery, organisation, financing, and outcomes of child healthcare services (1–3). These interventions can be any organised activity, program, project, or initiative, that is supported by resources and established with the purpose of enacting change for children aged 0–19 years old. They can be based in the community, primary-care or hospital, with a child health focus, and may encompass multiple components including, healthcare access, human resourcing, training and education, health facility processes and policies, quality improvement, e-health, task shifting, and healthcare continuity.

CHSI often involve multiple stakeholders (e.g., parents and schools), diverse objectives (e.g., health and education), and are dependent on context, making them inherently complex. As this complexity is increasingly being recognised by stakeholders in child health including practitioners, researchers, and funders, there is more of a need to articulate the ways in which this complexity has been considered in the design, implementation, and evaluation of CHSI. A Theory of Change (ToC) is a tool that child health practitioners leading such interventions can use to navigate and report on this complexity. For example, child health practitioners developed a ToC for their 2020 BetterBirth Program which included leadership engagement, an educational and motivational program launch, and ongoing coaching visits all aiming to promote the use of the World Health Organization's Safe Childbirth Checklist (4). Similarly, the child health practitioners for The Future Health Systems Program, published their ToC in 2017 to demonstrate how they navigated the complexity of improving information systems and coordination of child health service providers in India (5).

A ToC is a an articulation of the hypothesised pathways of how and why an intervention is intended to bring about change (6–9). It presents an intervention's assumptions, activities, mechanisms, measurement indicators, outcomes, and context, as well as the linkages between these various components (7). For child health practitioners looking to develop or evaluate an intervention, a ToC can be used as a guiding tool for reflecting and making explicit these important components of an intervention. Doing so allows teams to better understand if an intervention is working in the way that they had hypothesised, and assess implementation successes and failures. It also allows for better communication of the intervention to community, colleagues and others looking to carry out, or scale up, similar interventions, and to stakeholders in order to obtain more resources. These benefits have been recognised by the Medical Research Council, UK, who highlight the value of developing program theory in their 2021 framework for developing and evaluating complex interventions (10).

Whilst ToC is one of many tools used to better understand an intervention, it differs from other theories, and frameworks and models in several ways. Firstly, in its focus, a ToC provides an explanation of how and why a specific intervention leads to change (6, 7, 9). This differentiates it from frameworks (e.g., RE-AIM) and models (e.g., PDSA) as frameworks and models do not explore explanations but focus on describing intervention implementation aspects or endeavours (often without exploring the relationship between these aspects) and by structuring them into categories (11). Secondly, ToC has a particular focus on causality and outlines the steps in the causal chain or logic that connects an intervention's activities to its outcomes (6, 7, 9). As per De Silva, Breuer, Lee et al. (9), theories that exist at a more abstract level (e.g., Normalisation Process Theory) can strengthen ToCs by contributing a theoretical basis of why particular casual links happen. Finally, a ToC differs in that it can be utilised at any stage of an intervention from planning to evaluating (6, 7, 9).

The origins of ToC, and the overarching field, theory-based evaluation were popularised in the 1990s (7, 8, 12–14). They have been used in many contexts, including health, education, business, social welfare and international development (6, 9, 13, 14). Benefits of the ToC development process include, engaging stakeholders, clarifying focus, and enhancing connection to the intervention. Benefits once completed include, providing a roadmap for the intervention, a clear anchor for evaluation, a base in which to refine thinking and monitor the intervention as context changes overtime, scale-up utility, and identifying intentional and unintentional consequences (7, 9, 13–17). Criticisms of ToC include, confusion around definition, the so-called reductionist modelling which may give a false sense of control to the intervention implementers, a lack of enforced academic rigour, and the provision of an excuse not to adapt when context changes (7–9, 13).

2. Objectives

To map how CHSI ToCs have been developed, utilised, and refined.

3. Methods

A scoping review was selected for this study as it provides an opportunity to map an overview of the available research evidence. A protocol for this scoping review was registered (OSF, DOI 10.17605/OSF.IO/5TPGM), published and peer-reviewed in January, 2022 (18). This protocol was used as the methodological basis for this scoping review except where explicitly stated. This scoping review was conducted in accordance with the widely utilised JBI methodology (19, 20). A PRISMA-ScR checklist was also used to guide the study (Supplementary File S1).

4. Eligibility criteria

Table 1 outlines the review inclusion and exclusion criteria.

5. Search strategy

MEDLINE, EMBASE, Global Health, WHO Global Index Medicus, CINAHL and SCOPUS were searched from 1946—present. A search strategy was developed for each database (Supplementary File S2). The first 10 pages of the google search “Child* health* “theory of change” filetype:pdf” were searched for grey literature. The term “health*” was added and differs from the protocol. The database searches were conducted in January and February 2022 by BJ.

6. Evidence selection

Citations were uploaded to Endnote V20 and duplicates removed. Rayyan software (21) was used by two independent reviewers (BJ and AP) to screen titles and abstracts using the eligibility criteria. A screening guide was developed (Supplementary File S3) to aid in title and abstract screening. All conflicts between the reviewers were resolved without the need for a third reviewer. The guide was refined based on these conflicts for use in full text screening. All eligible full texts were reviewed by BJ. A random 20% of these were reviewed by an additional reviewer (AP). Any disagreements at each stage of the selection process were resolved through discussion or review by a third reviewer (SN). The results of the search and the study inclusion process are reported in the adapted Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram (see Figure 1).

Figure 1

7. Data charting

Data were extracted from studies included in the scoping review by BJ using a data extraction tool developed and reviewed by all authors, to aid with consistency. The data extracted included: author, link to paper, year of publication, study country, name of intervention, ToC level, type of health service intervention, health service level, definition of ToC, process of development of ToC, stakeholders involved in ToC development, stage in which ToC was developed, ToC presentation in the literature, ToC purpose (articulation of why it is useful), ToC value (what it helped do), ToC refinement over time, and other. The components of the ToC were also extracted. In the absence of a gold standard ToC components checklist, a classification was been developed based on the work of Dhillon and Vaca (22) and Vogel (23). This classification of the components of a ToC, which uses the acronym COMMA, consists of the following elements; Context, Outcomes, Mechanisms, Measurement indicators, and Assumptions.

Raw data is presented in table form (Supplementary File S4). A narrative synthesis of the data was conducted using both deductive (pre-defined research questions) and inductive elements to look at consistent themes arising from the included studies. The literature was also quantitatively analysed for study location, type of health service intervention, ToC presentation, and ToC definition using a checklist of ToC components such as assumptions, activities, mechanisms, measurement indicators, outcomes, context, and linkages (see discussion for more detail of ToC components).

8. Results

8.1. Search results

5,354 abstracts and 309 full texts were screened using the eligibility criteria. A total of 38 full-texts were included for data extraction and analysis. This process is summarised in an adapted PRISMA-ScR model shown in Figure 1.

8.2. Included studies

The publication date of the included studies ranged from 2005 to 2022 with an increase in studies in the past five years (Figure 2). Of the 38 ToCs presented, 16 (42%) represented a single-component CHSI and 22 (58%) represented multi-component CHSIs and 15 (47%) included ToCs that encompassed multi-country sites. The majority of studies, [27/38 (71%)] were conducted in low- or middle-income countries, eight (21%) were in high-income countries, one across both and two did not specify where they were conducted. The most common countries were India (8 studies), Ethiopia (5 studies) and the UK (4 studies).

Figure 2

A number of different types of health service interventions were reported (Supplementary FileS4). These include sexual and reproductive health, mental health, infectious disease, maternal and childhood nutrition, supply chain, home visitation, safe antenatal care and childbirth, violence prevention, teamwork related, weight management, pain treatment, and vaccination services. Of the 38 studies, 20 involved (53%) community-based interventions, seven (18%) only included hospital-based interventions, six (16%) only featured primary-care interventions, five (13%) involved interventions across multiple settings and one was a policy -based intervention.

8.3. ToC definition and presentation

Twenty four studies (63%) provided a definition for ToC or program theory. The most common elements in the definition were; “how” and/or “why” (24–31), “assumptions” (4, 24, 29, 32), “hypothesis” (4, 25, 26, 33, 34), and “tool” (4, 5, 35–37). 26 studies (68%) described the diagrammatic representation of their model as a “ToC diagram” with the remainder (32%) describing their diagrams as “logic models”, “program theory”, or “causal web diagrams”.

8.4. ToC development

Only 13 studies (34%) provided commentary on which stakeholders were involved in the ToC development process. Of these, two mentioned the community/beneficiaries of the intervention as stakeholders involved in the process (4, 38). Further, only 10 studies (26%) spoke to the refinement of ToCs over time—one of the most important aspects of a ToC (5, 29–31, 35, 37, 39–42).

Exactly half of the ToCs (19/38) appeared to be developed pre-intervention. Of the remainder, 9 (24%) were developed post-intervention, 8 (21%) during the intervention, and the timing of the development of two was not clear. Under half of the studies (16/38, 42%) commented on the process of ToC development, with many describing a collaborative approach through workshops, meetings, or discussions (5, 25, 30, 31, 33–35, 37, 38, 43, 44). Seven studies (18%) provided a stepwise and more comprehensive explanation for their ToC development (5, 25, 28, 33, 38, 39, 42).

8.5. ToC purpose and value

A justification for the ToC approach being used was identified in 14 studies (37%). Of these the most common reasons were monitoring and evaluation (4, 5, 28, 30, 31, 45), testing causal links (38, 40, 45, 46), and collaborating with stakeholders (4, 28, 35, 39). 16 studies (42%) included some form of reflection on the value of a ToC. This included comments on using data more effectively (4, 5, 47, 48), engaging stakeholders (5, 28, 33, 38, 48), and learning about the intervention (5, 29). However, there were some studies that reported more critical reflections, for example, that ToCs can be too linear and lack detail (26).

8.6. ToC components

ToC components and the frequency of use are reported in Table 2.

9. Discussion

This scoping review provides a map of the literature for how ToC are used in CHSI. Many of the findings in this scoping review echo those found in Breuer, Lee, De Silva et al. (49) systematic review of public health interventions. Namely, that there is a wide variation in how ToCs are developed, utilised and refined, and the literature presents limited detail on the rationalisation and development processes. These variations may relate to the historical emergence of ToCs as a tool for theory-based evaluation within academia, but also used within business and management practice. When evaluators utilise a ToC they have to decide whether to align themselves more closely with the academic underpinnings of the tool, reading and referencing the works of Weiss, Rossi, Connell, Kubisch and Chen (7, 13, 50). The alternative is to freely adopt the elements of the tool that work for them, and use it more liberally for the purpose of their evaluation in a business consultant-like fashion, even if this sacrifices some of the original intended value of the ToC. This tension is potentially why we now see such disparate findings in how ToCs are used and why.

9.1. ToC components

One aspect of ToC often discussed is what components are necessary to constitute a ToC. Currently, there is no clear and consistent classification of essential elements for a ToC, likely leading to the variation observed in the literature. As mentioned in the data extraction section of the methods in this paper, our classification of the components of a ToC, uses the acronym COMMA, and consists of the following elements; Context, Outcomes, Mechanisms, Measurement indicators, and Assumptions. Interestingly, in this scoping review, two of the five elements of COMMA, context and outcomes, were evident in every paper whereas the other three components were all present in less than a third. These other elements, namely mechanisms, measurement indicators, and assumptions, are arguably the elements that differentiate ToCs from logic models and add the most depth of thought and understanding to a ToC and hence their common absence could be considered a missed opportunity. In order to encourage integration of all three of these components into standard ToCs it may be worth adjusting Dhillon and Vaca (22)'s diagram in the simplified way seen in Figure 3.

Figure 3

9.2. ToC use over time

The findings of this review suggest that the number of child health organisations reporting a ToC is increasing (see Figure 2 above). In fact, all studies included in this review except one were published after Breuer, Lee, De Silva et al. (49) systematic review of public health interventions. This finding is consistent with the broader ToC literature which indicates that the use of a ToC is increasing (5, 51). Another, theory-based evaluation approach being used increasingly are realist evaluations. A realist evaluation seeks to answer the practical questions of what works for whom, under what circumstances, and how (52–54)? Realist evaluations are distinct from ToCs in several ways. For example, realist approaches have a specific explanation of causation that is informed by the paradigm of realism. The generative causation model explains that mechanisms are triggered only under specific contexts to produce outcomes (52–54). Whilst ToCs explore mechanisms they do not detail that these can only be triggered in certain contexts. The realist conceptualisation of causation leads to the analytical unit for realist evaluations: context-mechanism-outcome configurations which differs to the analystical unit in the ToC approach—the ToC diagram. Whilst out of scope of this particular review, exploring the synergies between ToC and realist evaluation approaches, building on the work done by Blamey and Mackenzie (16) will be important to add to our depth of understanding for both approaches and how they may complement one another. More broadly, with the increasing use of ToC and theory-based evaluations as a whole, there is an increasing need for a better understanding of this tool by child health practitioners, researchers, program implementers and policy makers.

9.3. ToC mechanisms or “program theory”

One of the areas that requires further understanding is the actual program theory. Weiss (13) defines program theory as “the mechanisms that intervene between the delivery of the program service and the occurrence of the outcomes of interest.” That is, what is the true cognitive shift or causal mechanisms that occur within the program participant. It is an element of theory-based evaluations that requires considerable reflection by program implementers. Weiss (13) and more recently, Rogers and Weiss (55) and Blamey and Mackenzie (16) found that whilst many organisations claim to consider program theory, in practice it is often not done, or, poorly done. This scoping review mirrored these findings with less than a third of ToCs including reporting on mechanisms. Thus, the authors of this paper propose that further consideration of how mechanisms are developed and by whom is needed.

9.4. Child health practitioners and ToC

A ToC can be a useful tool in the development and/or evaluation of CHSI. A ToC may aid in promoting an understanding of an intervention and enabling evaluative learning. In its true form, a ToC is not merely a diagrammatic summary of what was done or what is planned. More research needs to be done into which child health organisations benefit from a ToC and in what way. Another consideration that needs further exploration is how a ToC may contribute to organisational learning and, more broadly, learning health systems. Finally, the authors recommend the use of a checklist such as that presented in Breuer, Lee, De Silva et al. (49) to aid in the ToC process and provide some standardisation in this potentially disparate field.

9.5. Strengths and limitations

A strength of this review was the screening process. The screening process was broad at the title and abstract stage because the inconsistency in the use of the term ToC meant that only selecting studies based on if they used the term “Theroy of Change” would have resulted in missed studies. The findings and discussion of this review effectively highlight the perceived strengths as well as several broader concerns with ToCs. This review also builds on the work of Breuer, Lee, De Silva et al. (49) by: demonstrating that the inconsistent use of ToC terminology has continued since Breuer, Lee, De Silva et al. (49) publication in 2016 and extends from the public health interventions field into the child health service interventions field, demonstrating that a broader search strategy can be used and leads to the inclusion of relevant studies that do not specifically mention Theory of Change, and includes a more detailed discussion on causal mechanisms of change. Regarding limitations, since this review only included studies published in English, the scope is limited. During screening conducting and reporting an inter-rater reliability assessment prior to resolving the conflicts would have enhanced the validity of the study. The findings presented in this review are related to the child health context which was the scope of this review because of the area of interest and expertise of the authors. These findings may be true in other contexts, however this would require further research.

10. Conclusion

This scoping review offers a review of how ToCs have been used in CHSI and in doing so, reveals both the potential strengths and current weaknesses of the tool when used for this purpose. This has several potential implications. Importantly, for practitioners and organisations interested in CHSI, it maps how ToCs have been used to enact system changes in these settings, and provides suggestions as to how the use of ToCs might be improved upon. Additionally, for implementation researchers, it further elucidates how ToCs are used and described in the literature. This includes the perceived meaning and value of ToCs and what important components are typically missing in use to date. This, in turn, encourages consideration of how the definitions, processes and classifications of ToCs could be clarified. Ultimately, this scoping review demonstrates the ways in which practioners involved in CHSI can maximise the benefits of ToCs.

References

• 1.

  Institute of Medicine Committee on Health Services Research. A working defintion of health services research. Washington, DC: National Academies Press (1994). Available at:https://www.ncbi.nlm.nih.gov/books/NBK231502/.

  • Google Scholar

• 2.

  SteinwachsDMHughesRG. Advances in patient safety health services research: scope and significance. In: HughesRG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US). (2008).

  • Google Scholar

• 3.

  LohrKSteinwachsD. Health services research: an evolving definition of the field. Health Serv Res. (2002) 37(1):7–9. 10.1111/1475-6773.01020

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  BarnhartDASemrauKEAZiglerCMMolinaRLDelaneyMMHirschhornLRet alOptimizing the development and evaluation of complex interventions: lessons learned from the BetterBirth program and associated trial. Implement Sci Commun. (2020) 1:29. 10.1186/s43058-020-00014-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  PainaLWilkinsonATetuiMEkirapa-KirachoEBarmanDAhmedTet alUsing theories of change to inform implementation of health systems research and innovation: experiences of future health systems consortium partners in Bangladesh, India and Uganda. Health Res Policy Syst. (2017) 15(Suppl 2):109. 10.1186/s12961-017-0272-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6.

  Center for Theory of Change. What is Theory of Change? (2021). Available at:https://www.theoryofchange.org/what-is-theory-of-change/

• 7.

  ConnellJKubischA. Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress. Prospects, and problems. New approaches to evaluat- ing community initiatives, vol. 2, theory, measurement, and analysis. Washington DC: Aspen Institute (1998).

  • Google Scholar

• 8.

  WeissCH. Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community-based initiatives for children and families. New approaches to evaluating community initiatives:, vol. 1, concepts, methods and contexts. Washington, DC: Aspen Institute (1995).

  • Google Scholar

• 9.

  De SilvaMJBreuerELeeLAsherLChowdharyNLundCet alTheory of change: a theory-driven approach to enhance the medical research Council's framework for complex interventions. Trials. (2014) 15(1):267. 10.1186/1745-6215-15-267

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  SkivingtonKMatthewsLSimpsonSACraigPBairdJBlazebyJMet alA new framework for developing and evaluating complex interventions: update of medical research council guidance. BMJ Br Med J. (2021) 374:n2061. 10.1136/bmj.n2061

  • CrossRef
  • Google Scholar

• 11.

  NilsenP. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. 10.1186/s13012-015-0242-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 12.

  ConnellJKubischASchorrLWeissCH. New approaches to evaluating community initiatives: concepts, methods and contexts. New York: Aspen Institute (1995).

  • Google Scholar

• 13.

  WeissCH. Theory-based evaluation: past, present, and future. New Dir Eval. (1997) 76:41–55. 10.1002/ev.1086

  • CrossRef
  • Google Scholar

• 14.

  WeissCH. How can theory-based evaluation make greater headway?Evaluation Rev. (1997) 21(4):501–24. 10.1177/0193841X9702100405

  • CrossRef
  • Google Scholar

• 15.

  AndersonAA. The community Builder's approach to theory of change: a practical guide to theory development. New York: Aspen Institute Roundtable on Community Change. (2006).

  • Google Scholar

• 16.

  BlameyAMackenzieM. Theories of change and realistic evaluation: peas in a pod or apples and oranges?Evaluation. (2007) 13(4):439–55. 10.1177/1356389007082129

  • CrossRef
  • Google Scholar

• 17.

  RogersPPetrosinoAHuebnerTHacsiT. Program theory evaluation: practice, promise, and problems. New Dir Eval. (2004) 2000:5–13.s 10.1002/ev.1177

  • CrossRef
  • Google Scholar

• 18.

  JonesBNagrajSEnglishM. Using theory of change in child health service interventions: a scoping review protocol. Wellcome Open Res. (2022) 7:30. 10.12688/wellcomeopenres.17553.1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 19.

  PetersMGodfreyCMcInerneyPSoaresCKhalilHParkerD. The Joanna Briggs Institute Reviewers' Manual 2015: Methodology for JBI Scoping Reviews. Adelaide, SA Australia: The Joanna Briggs Institute. (2015).

  • Google Scholar

• 20.

  PetersMMarnieCTriccoCPollockDMunnZAlexanderLet alUpdated methodological guidance for the conduct of scoping reviews. Jbi Evid Synth. (2020) 18(10):2119–26. 10.11124/JBIES-20-00167

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  OuzzaniMHammadyHFedorowiczZElmagarmidA. Rayyan-a web and mobile app for systematic reviews. Syst Rev. (2016) 5. 10.1186/s13643-016-0384-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  DhillonLVacaS. Refining theories of change. J MultiDiscip Eval. (2018) 14(30):64–87. https://assets.ctfassets.net/dsmojq7rshge/42gHh3lNgseO4kcEEamGqk/fdf20dd9db6f93b53875c0ce26a44751/Refining_Theories_of_Change.pdf

  • Google Scholar

• 23.

  VogelI. Review of the use of ‘theory of change’ in international development. London, UK: Department for International Development (DFID) (2012).

  • Google Scholar

• 24.

  Cordova-PozoKHoopesAJCordovaFVegaBSeguraZHagensA. Applying the results based management framework to the CERCA multi-component project in adolescent sexual and reproductive health: a retrospective analysis. Reprod Health. (2018) 15:1. 10.1186/s12978-018-0461-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  DaruwallaNJaswalSFernandesPPintoPHateKAmbavkarGet alA theory of change for community interventions to prevent domestic violence against women and girls in Mumbai, India. Wellcome Open Res. (2019) 4:54. 10.12688/wellcomeopenres.15128.1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  DiLibertoDDStaedkeSGNankyaFMaiteki-SebuguziCTaakaLNayigaSet alBehind the scenes of the PRIME intervention: designing a complex intervention to improve malaria care at public health centres in Uganda. Glob Health Action. (2015) 8:29067. 10.3402/gha.v8.29067

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  GuptaMKaurMChakrapaniVRanaMBosmaHVan SchayckOCPet alImpact of a multi-strategy community intervention to reduce maternal and child health inequalities in India: a qualitative study in Haryana. PLoS One. (2017) 12(1):e0170175. 10.1371/journal.pone.0170175

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  HanleyTSefiAGraubergJPrescottJEtchebarneA. A theory of change for web-based therapy and support services for children and young people: collaborative qualitative exploration. JMIR Pediatr Parent. (2021) 4(1):e23193. 10.2196/23193

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  KabongoEMMukumbangFCDelobellePNicolE. Combining the theory of change and realist evaluation approaches to elicit an initial program theory of the MomConnect program in South Africa. BMC Med Res Methodol. (2020) 20(1):282. 10.1186/s12874-020-01164-y

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30.

  LalliMRuysenHBlencoweHYeeKCluneKDeSilvaMet alSaving lives at birth; development of a retrospective theory of change, impact framework and prioritised metrics. Globalization Health. (2018) 14:1-N.PAG. 10.1186/s12992-018-0327-z

  • CrossRef
  • Google Scholar

• 31.

  MacKenzieMShiteJBerzinsKJudgeK. The Independent Evaluation of 'Starting Well': Final Report. In: Office CSs, editor. Edinburgh: The Scottish Executive, Chief Scientist's Office. (2005).

  • Google Scholar

• 32.

  GadsbyEWHothamSEidaTLawrenceCMerrittR. Impact of a community-based pilot intervention to tackle childhood obesity: a 'whole-system approach' case study. BMC Public Health. (2020) 20(1):1–12. 10.1186/s12889-020-09694-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  WilburJBrightTHameedSKuperHPolackSMahonTet alDeveloping behaviour change interventions for improving access to health and hygiene for people with disabilities: two case studies from Nepal and Malawi. Int J Environ Res Public Health. (2018) 15(12):2746. 10.3390/ijerph15122746

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  ZamboniKSinghSTyagiMHillZHansonCSchellenbergJ. Effect of collaborative quality improvement on stillbirths, neonatal mortality and newborn care practices in hospitals of Telangana and Andhra Pradesh, India: evidence from a quasi-experimental mixed-methods study. Implement Sci. (2021) 16(1):1–17. 10.1186/s13012-020-01058-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 35.

  FuhrDCAcarturkCSijbrandijMBrownFLJordansMJDWoodwardAet alPlanning the scale up of brief psychological interventions using theory of change. BMC Health Serv Res. (2020) 20(1):1–12. 10.1186/s12913-019-4778-6

  • CrossRef
  • Google Scholar

• 36.

  MakowieckaKMarchantTWuletaBAnuraagCLaboniJLimanAet alCharacterising innovations in maternal and newborn health based on a common theory of change: lessons from developing and applying a characterisation framework in Nigeria, Ethiopia and India. BMJ Global Health. (2019) 4(4):e001405. 10.1136/bmjgh-2019-001405

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  MbuthiaFReidMFichardtA. Development and validation of a mobile health communication framework for postnatal care in rural Kenya. Int J Africa Nurs Sci. (2021) 14:100304. 10.1016/j.ijans.2021.100304

  • CrossRef
  • Google Scholar

• 38.

  HurtubiseKBrousselleACamdenC. Using collaborative logic analysis evaluation to test the program theory of an intensive interdisciplinary pain treatment for youth with pain-related disability. Paediatr Neonatal Pain. (2020) 2(4):113–30. 10.1002/pne2.12018

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  HernandezMHodgesS. Applying a theory of change approach to interagency planning in child mental health. Am J Community Psychol. (2006) 38(3–4):165–73. 10.1007/s10464-006-9079-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  MulhernEApplefordG. Adolescents 360 evaluation process evaluation methodology. Adolescents. (2008) 360:1–22. https://www.itad.com/wp-content/uploads/2020/02/A360-Process-Evaluation-Protocol-Paper_Revised_August-2019_final-CLEAN-191028-1.pdf

  • Google Scholar

• 41.

  NimpagaritseMKorachaisCMeessenB. Effects in spite of tough constraints—a theory of change based investigation of contextual and implementation factors affecting the results of a performance based financing scheme extended to malnutrition in Burundi. PLoS One. (2020) 15(1):e0226376. 10.1371/journal.pone.0226376

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  van BelleSBMarchalBDubourgDKegelsG. How to develop a theory-driven evaluation design—lessons learned from an adolescent sexual and reproductive health programme in West Africa. BMC Public Health. (2010) 10:741. 10.1186/1471-2458-10-741

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  FaizTSaeedBAliSAbbasQMalikM. OR To ICU handoff: theory of change model for sustainable change in behavior. Asian Cardiovasc Thorac Ann. (2019) 27(6):452–8. 10.1177/0218492319850730

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  SewardNHanlonCAbdellaAAbrahamsZAlemAArayaoRet alHealth system StrEngThening in four sub-sSaharan African countries (ASSET) to achieve high-quality, evidence-informed surgical, maternal and newborn, and primary care: protocol for pre-implementation phase studies. Glob Health Action. (2022) 15(1):1987044. 10.1080/16549716.2021.1987044

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  ChandaniYAnderssonSHeatonANoelMShieshiaMMwirotsiAet alMaking products available among community health workers: evidence for improving community health supply chains from Ethiopia, Malawi, and Rwanda. J Glob Health. (2014) 4(2):020405. 10.7189/jogh.04.020405

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 46.

  Michaud-LetourneauIGayardMPelletierDL. Contribution of the alive & thrive-UNICEF advocacy efforts to improve infant and young child feeding policies in Southeast Asia. Maternal and Child Nutrition. (2019) 15:e12683. 10.1111/mcn.12683

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  GreenlandKChipunguJChilekwaJChilengiRCurtisV. Disentangling the effects of a multiple behaviour change intervention for diarrhoea control in Zambia: a theory-based process evaluation. Globalization Health. (2017) 13:1–18. 10.1186/s12992-017-0302-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48.

  MilnerKMBernal SalazarRBhopalSBrentaniABrittoPRDuaTet alContextual design choices and partnerships for scaling early child development programmes. Arch Dis Child. (2019) 104(Suppl 1):S3–S12. 10.1136/archdischild-2018-315433

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49.

  BreuerELeeLDe SilvaMLundC. Using theory of change to design and evaluate public health interventions: a systematic review. Implement Sci. (2016) 11:63. 10.1186/s13012-016-0422-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50.

  ChenHTRossiPH. The theory-driven approach to validity. Eval Program Plann. (1987) 10(1):95–103. 10.1016/0149-7189(87)90025-5

  • CrossRef
  • Google Scholar

• 51.

  MackenzieMBlameyA. The practice and the theory:lessons from the application of a theories of change approach. Evaluation. (2005) 11(2):151–68. 10.1177/1356389005055538

  • CrossRef
  • Google Scholar

• 52.

  PawsonRTilleyN. Realistic evaluation. London, UK: SAGE Publications (1997).

  • Google Scholar

• 53.

  PawsonR. The science of evaluation: a realist manifesto. London, UK: SAGE Publications (2013).

  • Google Scholar

• 54.

  WongGWesthorpGManzanoAGreenhalghJJagoshJGreenhalghT. RAMESES II reporting standards for realist evaluations. BMC Med. (2016) 14(1):96. 10.1186/s12916-016-0643-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55.

  RogersPJWeissCH. Theory-based evaluation: reflections ten years on: theory-based evaluation: past, present, and future. New Directions for Eval. (2007) 2007(114):63–81. 10.1002/ev.225

  • CrossRef
  • Google Scholar