Fostering continuous quality improvement in a European rare disease network

Background: The European Reference Network for rare Inherited Congenital Anomalies (ERNICA) is a clinical network dedicated to improving the quality of care for patients with rare and complex digestive and gastrointestinal diseases, many of whom require surgery in early life. The network brings together clinicians, researchers and patient representatives from 22 countries in Europe. By pooling expertise, ERNICA is able to facilitate improvement initiatives that may not otherwise be possible. However, describing the desired quality of care and transferring it to local practice remains a challenge, complicated by our low-prevalence patient population, multidisciplinary clinical involvement and heterogeneous European context. In an attempt to mitigate these challenges, and foster a system of continuous quality improvement, we present the “ERNICA quality cycle”.

Main body: The ERNICA quality cycle is comprised of five steps: (1) Describing the desired quality of care (2) Promoting guideline implementation (3) Measuring quality of care (4) Evaluating clinical practice (5) Conducting research. It offers a structured, continuous and collaborative approach to the improvement of care for patients with rare and complex digestive and gastrointestinal diseases. Evaluating the approach, through qualitative process evaluation, will be critical to capturing learning points.

Conclusions: The ERNICA quality cycle holds tremendous potential for improving the quality of care for patients with rare and complex conditions, both within ERNICA and for other European Reference Networks.

Background

High-quality healthcare can be defined as “doing the right thing, at the right time, in the right way, for the right person and having the best results possible” (1). In the field of rare diseases, it can be challenging to determine what “high-quality” care looks like and how it can be achieved. In Europe, rare diseases have been defined as conditions affecting less than one in 2,000 people, with seventy-five percent affecting children (2, 3). Although committed to their patients, healthcare professionals caring for those with rare diseases may have limited disease-specific knowledge. They may encounter a lack of resources, an absence of high-quality evidence and a very small patient population which is geographically dispersed. To improve care for individuals affected by rare diseases, collaborative networks and cross-border efforts are imperative.

The European Reference Network for rare Inherited Congenital Anomalies (ERNICA) is one of 24 European Reference Networks (ERNs) launched by the European Commission in 2017 (4). See Supplementary File 1 for a full list of ERNs. Bringing together multidisciplinary expert teams and patient representatives from across Europe, these networks aim to pool knowledge on rare and/or complex diseases requiring specialized care. Each ERN focuses on a thematic group of diseases. For ERNICA, this includes diseases and/or malformations of the digestive system, diaphragm and abdominal wall. Multidisciplinary care is essential for these patients, across the lifespan from before birth through to adulthood. In 2017, 20 hospitals across 10 European countries were accepted as ERNICA members. As of 2024, ERNICA is comprised of 39 member and 13 affiliated partner hospitals, from 21 European countries. Clinical representatives work in partnership with patient representatives, whose involvement is endorsed by recognized local, national and/or international patient organizations (5). See Figure 1 for an overview of countries with ERNICA member/affiliated partner involvement. Additional countries with patient groups represented are also displayed.

Figure 1

Figure 1. Map identifying European countries with ERNICA member/affiliated partner involvement. Additional countries with patient groups represented are also displayed.

While cross-border collaboration is of utmost importance, ERNICA's European reach comes hand-in-hand with context heterogeneity. Hospitals operate within heterogeneous healthcare systems and may be governed by national laws and regulations. Clinical practices may also vary between hospitals in a single country. This runs the risk of unwarranted practice variation; differences in healthcare processes or outcomes that cannot be explained by patient characteristics or patient preference (6, 7). In the absence of relevant and widely accepted metrics, high-quality evidence, clinical knowledge and experience, it can be challenging to define how “the best possible results” can be achieved for our patients. To overcome these challenges (8), and foster a system of continuous quality improvement, we present an iterative framework referred to as the “ERNICA quality cycle”.

The ERNICA quality cycle

The ERNICA quality cycle is comprised of five steps: (1) Describing the desired quality of care; (2) Promoting guideline implementation; (3) Measuring quality of care; (4) Evaluating clinical practice and (5) Conducting research. See Figure 2 for a visual outline of the ERNICA quality cycle. In this paper, we describe each step of the ERNICA quality cycle in turn. The structure, and underlying principles of the ERNICA quality cycle are inspired by the “Plan-Do-Study-Act” (PDSA) cycle (9), a widely used framework for continuous improvement popularized by W. Edwards Demming in the 1950s (10). This four-step cycle involves planning, testing and evaluating (context-specific) change initiatives (9) and is grounded in principles of iterative learning, collaboration and evidence-based decision making.

Figure 2

Figure 2. The ERNICA quality cycle. Created with support from Jana Steerneman.

1 Describing the desired quality of care

To describe high-quality care, ERNICA facilitates the development of clinical practice guidelines. This process involves multidisciplinary clinical experts, researchers and patient representatives and is supported by a methodologist (6). Participants are selected from expert centers connected to ERNICA for the disease of interest. Attention is given to ensure geographical spread and a balance of relevant disciplines and expertise. A prioritization exercise takes place to decide on the topics of focus and an evidence-based development method is followed. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system guides the assessment of evidence quality and the development of care recommendations (11, 12). This structured approach has been used most recently to develop ERNICA guidelines on necrotizing enterocolitis (13) and gastroschisis (14).

However, this process is often complicated by a lack of high-quality evidence and high heterogeneity in reported outcomes. Uncertain outcome estimates can make it difficult to reach agreement on specific recommendations (6). For diseases with very limited data, such as total colonic aganglionosis (TCA) and pediatric esophageal achalasia, the development of formal clinical consensus statements using a (modified) Delphi approach has proven a viable alternative (15). The development processes for both evidence-based guidelines and consensus statements are explained in Supplementary File 2.

ERNICA is also prioritizing the testing of adapted, innovative methodological strategies to support guideline development in our context (8). This involves the supplementary use of “real world” clinical audit data and the use of indirect evidence (e.g., data from adults with the same rare disease, data on ventilation outcomes in a general neonatal population), n = 1 trials (where multiple treatment strategies are tested on a single patient using a cross-over design), as well as the incorporation of expert opinion solicited through structured observation forms.

Recommendations reported in ERNICA clinical guidelines/consensus statements are also used to inform the development of disease-specific quality indicators (16). These include structural, process and outcome indicators. See Supplementary File 3 for descriptions of each. Baseline characteristics facilitate interpretation of results, altogether forming a “core indicator set”. A methodologically sound approach is followed for development, involving a literature review and (modified) Delphi study with international stakeholders, including multidisciplinary clinicians, researchers and patient representatives. This process has been completed for esophageal atresia (17, 18) and Hirschsprung's disease (19) and is underway for gastroschisis, omphalocele, intestinal atresia, necrotizing enterocolitis and intestinal malrotation. Once developed, these disease-specific indicator sets are integrated into the European Pediatric Surgical Audit (EPSA), a clinical audit tool used by both ERNICA and non-ERNICA centers (16, 20).

Variability in how terms are used and interpreted can pose challenges during the development of both guidelines and clinical indicator sets. Attempts to standardize definitions through group consensus are underway. In the meantime, terms are accompanied with clear and consistent explanations to promote shared understanding.

2 Promoting guideline implementation

Efforts are required to ensure the desired quality of care is implemented in clinical practice. When implemented successfully, ERNICA's clinical guidelines are expected to narrow unwarranted practice variation and ultimately, improve quality of care, clinical outcomes and patient experiences. However, there are long-standing challenges surrounding the uptake of clinical guideline recommendations in practice (21). How recommendations are developed and presented to clinicians may influence a guideline's trustworthiness, clinical credibility and implementability (6, 22). During the ERNICA guideline development process, resources designed to increase implementability, such as the Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence (AGREE-REX) reporting checklist (23, 24) are drawn upon.

A range of factors, pertaining to the individual health professional(s) [e.g., knowledge and skills, cognitions (including attitudes), professional behavior] and their professional interactions, the patients themselves (e.g., needs, beliefs, knowledge, preferences, motivation and behavior), the availability of incentives and resources, the capacity for organizational change and the wider social, political and legal environment may also hinder and/or facilitate successful guideline implementation (6, 22). A Central Implementation Support Team (the ERNICA CIST) has been established as a cross-disease ERNICA working group. Made up of key stakeholders with relevant position(s), expertise and/or interest, the CIST is dedicated to supporting the successful uptake of clinical guidelines in European practice.

To guide the CIST's ongoing efforts, an implementation agenda has been developed, initially focused on three diseases for which ERNICA guidelines/consensus statements exist or are planned: omphalocele, esophageal atresia and rectosigmoid Hirschsprung's disease. Surveys have been circulated to healthcare providers involved in ERNICA and/or the EPSA, to obtain disease-specific “baseline” overviews of care practices in Europe. These overviews will provide insight into areas of potential (unwarranted) practice variation, providing a view into differences between hospitals as well as deviations from established guideline recommendations. This understanding will lay the foundation for further investigation into the factors foreseen to hinder and/or facilitate successful implementation. Results will be used to inform the (collaborative) selection of implementation strategies for execution (21, 25). Example strategies include: professional (e.g., distribution of guideline material, presentation at meetings), financial (e.g., incentives, grants/allowance, penalties), organizational (e.g., human resources, consumer involvement), structural (e.g., organizational structure, setting/site), regulatory (e.g., legislation, accreditation), and patient/consumer strategies (e.g., printed material, patient education) (26, 27).

Throughout, the CIST will have a focus on providing local teams with practical, stepped support. To facilitate good communication, “implementation leads” are locally appointed at each center. Implementation leads will serve as the local point of contact, playing a key role in identifying and/or involving relevant team members and local stakeholders.

3 Measuring quality of care

To measure performance, and monitor implementation success, a standardized approach to data collection is warranted (28). The EPSA clinical audit tool facilitates the collection of prospective patient data aligned to the aforementioned (disease-specific) core indicator sets. The EPSA was originally developed by the Dutch Association of Pediatric Surgeons in 2014 and is hosted by the Dutch Institute for Clinical Auditing (DICA) (16, 20). Over recent years, ERNICA has facilitated international expansion of the EPSA, with 34 hospitals across 18 countries currently connected (see Supplementary File 4). Twenty-six of these hospitals are part of the ERNICA network.

For hospitals registering their patients, we will attempt to employ the EPSA as a centrally-led continuous feedback mechanism to monitor guideline implementation and its impact (6). Where not already aligned, this requires incorporating guideline recommendation-specific variables into the disease-specific core indicator sets. This will make it possible to compare center performance with guideline recommendation(s) as the “care standard”. For example, for a recommendation on post-surgical feeding practices, the core indicator set should capture differences between pre-and post-surgical feeding as well as feeding types.

Such a feedback mechanism relies heavily on the EPSA being used by healthcare professionals in practice. We will therefore continue to focus on areas considered to be influential in implementing multi-center comparative audits, namely encouraging audit participation, generating “trustworthy” data and ensuring audit sustainability (including financial sustainability) (16, 29, 30). As both a multicenter, and European audit, we will continue to facilitate the provision of legal/regulatory support to those interested in connecting. Reducing the administrative burden also remains fundamental (16).

While EPSA is named a “pediatric surgical audit”, its core indicator sets include variables relevant to multiple disciplines (e.g., prenatal specialists). Initiatives are therefore underway to promote locally coordinated patient registration. Long-term follow-up data and patient-reported outcomes are not yet registered but are under consideration for the future (16).

The aim of the EPSA is to facilitate the exchange of knowledge and best practice and drive (local) quality improvement. This is done by feeding back data to participating hospitals on their performance and by creating opportunities for collaborative learning. EPSA's Codman Dashboard provides users with “real time” feedback on their center's performance, benchmarked against the European mean. Annual feedback sessions take place, where aggregated data is presented and used to stimulate group discussion (16).

Providing feedback to healthcare professionals on their performance can be a highly effective way to stimulate behavior change towards desired practices (31, 32). EPSA “audit and feedback” (A&F) therefore holds great potential for improving center performance and increasing levels of guideline adherence. However, the effectiveness of A&F has proven to be highly variable (31, 32), with unintended, adverse effects having also been reported (33). The International A&F Metalab (34), a research group in the field, provides theory- and evidence-informed guidance to maximize the effectiveness of A&F (31, 32, 35, 36). Together with users, we will review such guidance, reflecting on the EPSA's current status, as well as future possibilities for the provision of feedback within our context.

One recurring recommendation is to provide “actionable” feedback, alongside explicit “action plans” (29, 31, 32, 36, 37). Driving local improvement through clinical auditing relies on planning and testing change initiatives (38). This is crucial to ensuring completion of the clinical audit cycle (See Supplementary File 5); a chain of events that is “only as strong as its weakest link” (38). To facilitate its completion, local teams will be supported to engage in action planning on the basis of EPSA feedback. Such plans may incorporate the tailored guideline implementation strategies identified in Step 2. In this way, the EPSA is employed as a centrally-led guideline implementation strategy with local site tailoring.

Considering our rare disease context, where numbers are low and guideline recommendations often conditional, supplementary measures may be needed to validate EPSA feedback. To identify practice variation that is truly unwarranted, patient characteristics (case-mix) and patient preference (e.g., as part of shared decision-making) should also be accounted for (16, 39). Such considerations will be further explored and addressed. Moreover, not all ERNICA centers are connected to the EPSA. For non-EPSA centers, alternative ways of monitoring implementation success will be explored.

4 Evaluating clinical practice

Evaluation is essential to assess the success of our efforts and plan for the next iteration of the cycle. We will attempt to use the EPSA to re-audit center practices and associated outcomes. Aggregated EPSA data (alongside other potential measures) can provide insight into levels of (unwarranted) practice, and outcome variation across Europe. Process evaluations with local teams, using both qualitative and quantitative approaches, will also be key to exploring the when, where, why, and how behind successful, or failed guideline implementation (40). Implementation strategies will be iteratively, and collaboratively refined. In this way, EPSA as a quality improvement initiative benefits from the incorporation of implementation science principles (41). Opportunities will also be created to capture and build on community learning experiences.

Further, such insights into practice/outcome variation can play a role in identifying the need for new guidelines, revisions and/or additional research. ERNICA guidelines/consensus statements are routinely assessed for revision every five years. Knowledge gaps for future research are also identified and prioritized (42). Patient journeys can help to identify and prioritize areas for patient-centered research (43, 44). Within ERNICA, patient journeys set out to describe the key stages in a patient's life from possible prenatal diagnosis to adulthood. Needs and ideal support scenarios from the patient perspective are mapped out at each stage (43).

5 Conducting research

Closing knowledge gaps demands ongoing, rigorous research. Such research helps advance our understanding of what constitutes the “desired quality of care”. Prioritized topics may be identified both in Step 4 and in Step 1 of the cycle. With “high-quality” care patient-centered, the appropriate use of psychometrically-robust Patient-Reported Outcome Measures (PROMs) in research is also key (45). This is particularly critical in the rare disease context, where medical knowledge is often limited.

Conducting high-quality research on topics relevant to low-prevalence conditions can, however, be challenging (6). Small sample sizes, observational study designs and heterogeneous (6) and/or low-quality (46) outcome reporting often prevent us from drawing firm conclusions. Furthermore, enrolling pediatric patients in clinical trials involving surgical interventions can pose legal and ethical challenges (6). International collaboration, facilitated by ERNICA, is fundamental to increasing opportunities for trials and other research activities involving larger samples. Disease-specific core outcome sets (COS) are critical to ensure standardized outcome reporting (6). ERNICA has and continues to play a role in initiating and facilitating the development of COS (47, 48).

Discussion

Applying and evaluating the ERNICA quality cycle

Engaging our network and instilling a non-judgemental culture of improvement will be fundamental to the success of this initiative (49, 50). Relevant stakeholders may include multidisciplinary clinicians, researchers, policy makers, hospital management and patient representatives. With support of the CIST, centers’ implementation leads will play a key role in engaging and liaising with local stakeholders.

Application of this structured approach to quality improvement is novel within our context. Capturing learning points through qualitative process evaluation is key to evaluating its success, exploring, for example, how activities progress and the experiences and opinions of those involved (40). The approach will be piloted for three ERNICA diseases; omphalocele, esophageal atresia and rectosigmoid Hirschsprung's disease. Rigorous evaluation will shed light on its potential value for other ERNICA diseases. The sustainability of care improvements also warrants specific thought and attention, considering the potential for changing guideline recommendations over time.

A European Commission report was recently published outlining the results of the comprehensive 5-year evaluation of the ERNs and their members (51). Implementing (and measuring the implementation) of clinical practice guidelines was specifically highlighted as an area for improvement. ERNICA's cyclical approach to improving care quality may therefore also be of value to other ERNs.

To promote inclusivity and increase equity, this initiative is not limited to centers considered to have disease-specific “expertise”. Amongst other criteria, ERNICA expert centers (“members”) are required to meet a disease-specific threshold of five new patients per year. Non-member centers may connect to the EPSA, if they treat a minimum of five new patients per year for at least one registered disease (16). However, it may be that not all centers treating patients with rare and complex digestive and gastrointestinal diseases are connected to ERNICA and/or the EPSA.

Although efforts have been made to promote broad geographic coverage within the ERN system, there are recognized opportunities for improvement (51). Current funding allows for a restricted number of EPSA connections. To promote equitable participation, ongoing and additional funding is warranted. Moreover, to facilitate centers’ involvement, we call for explicit national/international legislation on data collection for quality assurance purposes (30).

Individual countries are responsible for the designation of centers to ERNs. However, in some countries, care is centralized and in others, it is not. Centers situated in countries with de-centralized healthcare systems may never meet the aforementioned patient threshold. Engaging with national health ministries is therefore encouraged to optimize patient coverage (52). With caseload considered a “driving force for quality” (52), evaluating “high-quality” care should also be looked at in context and with this considered.

Conclusion

As an ERN, ERNICA strives to improve the quality of care for patients with rare and complex digestive and gastrointestinal diseases. To foster a system of continuous quality improvement, we present the “ERNICA quality cycle”. This approach will be applied and will be subject to rigorous evaluation. It holds tremendous potential for use in other rare disease networks (ERNs), in support of our collective mission to improve rare disease care in Europe.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

OS: Writing – original draft, Writing – review & editing. WI: Writing – original draft, Writing – review & editing. SH: Writing – review & editing. ER: Writing – review & editing. TW: Writing – review & editing. RW: Writing – review & editing.

Group authorship: ERNICA representatives

ERNICA disease-specific working group leads, members of the ERNICA Central Implementation Support Team (CIST) and chairs of the ERNICA Scientific Committee.

Mikko Pakarinen: ERNICA Intestinal Diseases working group lead (Department of Pediatric Surgery, Pediatric Liver and Gut Research Group, New Children's Hospital, University of Helsinki and Helsinki University Hospital, Helsinki, Finland); Annette Lemli: ERNICA Intestinal Diseases working group lead (SoMA, The German Patient Support Organization for Anorectal Malformations and Hirschsprung Disease, Munich, Germany); Stefaan Tytgat: ERNICA Esophageal Diseases working group lead (Department of Pediatric Surgery, Wilhelmina Children's Hospital, University Medical Center Utrecht, Netherlands); Frederic Gottrand: ERNICA Esophageal Diseases working group lead (Univ. Lille, CHU Lille, Department of pediatric gastroenterology hepatology and nutrition, reference center for rare esophageal diseases, U1286-INFINITE, F-59000 Lille, France); JoAnne Fruithof: ERNICA Esophageal Diseases working group lead (Esophageal Atresia Global Support Groups, Stuttgart, Germany); Annika Mutanen: ERNICA Intestinal Failure working group lead (Department of Pediatric Surgery, The New Children's Hospital, Helsinki University Hospital and University of Helsinki, Helsinki, Finland); Merit Tabbers: ERNICA Intestinal Failure working group lead (1. Pediatric Gastroenterology, Hepatology and Nutrition, Emma Children's Hospital, Amsterdam University Medical Centers University of Amsterdam, Amsterdam, Netherlands. 2. Amsterdam Reproduction and Development, Child development, Amsterdam, Netherlands. 3. Amsterdam Gastroenterology, Endocrinology, Metabolism, Amsterdam, Netherlands); Mark Bremholm Ellebæk: ERNICA Gastroenterological Diseases working group lead (1. Research Unit for Surgery, Odense University Hospital, Odense, Denmark; University of Southern Denmark, Odense Denmark. 2. Centre of Excellence in Gastrointestinal Diseases and Malformations in Infancy and Childhood (GAIN) Odense University Hospital, Odense, Denmark & University of Southern Denmark, Odense, Denmark. 3. Department of Clinical Research, University of Southern Denmark, Odense, Denmark); Jan Deprest: ERNICA Malformations of the Diaphragm working group lead (1. Department of Obstetrics and Gynaecology, University Hospitals Leuven, Belgium. 2. My FetUZ Lab, Department of Development and Regeneration, Biomedical Sciences, KU Leuven, Leuven, Belgium. 3. Institute for Women's Health, University College London, London, UK); Laurent Storme: ERNICA Malformations of the Diaphragm working group lead (1. Department of Neonatology, Jeanne de Flandre Hospital, University Hospital of Lille, F-59000 France. 2. METRICS, University of Lille, Lille F-59000 France); Paolo de Coppi: ERNICA Malformations of the Diaphragm working group lead (1. Great Ormond Street Hospital & Great Ormond Street Institute of Child Health, University College London, London, UK. 2. My FetUZ Lab, Department of Development and Regeneration, Biomedical Sciences, KU Leuven, Leuven, Belgium. 3. Ospedale Pediatrico Bambino Gesù, Rome, Italy); Beverley Power: ERNICA Malformations of the Diaphragm working group lead [CDH UK - The Congenital Diaphragmatic Hernia Charity reg no. 1106065 (England & Wales) SC042410 (Scotland)]; Alexandra Benachi: ERNICA Abdominal Wall Defects working group lead (1. Department of Obstetrics and Gynaecology. Antoine Béclère Hospital. AP-PH. Clamart France. 2. Paris Saclay University. Clamart. France); Carmen Mesas Burgos: ERNICA Abdominal Wall Defects working group lead and ERNICA CIST member (Department of Pediatric Surgery, Karolinska University Hospital and Karolinska Institutet, Stockholm, Sweden); Polona Studen Pauletić: ERNICA CIST member (Department of Pediatric Surgery, University Medical Center Ljubljana, Slovenia); Rony Sfeir: ERNICA CIST member (Department of Pediatric Surgery, CRACMO, CHRU Lille, France); Erwin Ista: ERNICA CIST member (1. Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC Sophia Children's Hospital, Rotterdam, Netherlands. 2. Department of Internal Medicine, Division of Nursing Science, Erasmus Medical Center, Rotterdam, Netherlands); Michaela Dellenmark-Blom: ERNICA CIST member (1. Department of Women's and Children's Health, Karolinska Institute, Stockholm, Sweden. 2. Department of Pediatrics, University of Gothenburg Institute of Clinical Sciences, Gothenburg, Sweden. 3. Department of Pediatric Surgery, Sahlgrenska University Hospital Queen Silvia Children's Hospital, Gothenburg, Sweden); Alessio Pini Prato: ERNICA CIST member (Department of Pediatric Surgery, Umberto Bosio Center for Digestive Diseases, The Children Hospital AO SS Antonio e Biagio e Cesare Arrigo, Alessandria, Italy); Anke Widenmann: ERNICA CIST member (Esophageal Atresia Global Support Groups, Stuttgart, Germany); Romee van Steekelenburg, Marlinde van der Kamp, Daniel Rossi: ERNICA CIST members (1. Department of Pediatric Surgery, Erasmus MC Sophia Children's Hospital, Rotterdam, Netherlands. 2. Dutch Institute for Clinical Auditing: Leiden, NL. Additional for Daniel Rossi: 3. Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden); Iris den Uijl: ERNICA CIST member (Department of Pediatric Surgery, Erasmus MC Sophia Children's Hospital, Rotterdam, Netherlands); Roel Bakx: ERNICA Scientific Committee Chair (Department of Pediatric Surgery, Emma Children's Hospital, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands); Francesca Russo: ERNICA Scientific Committee Chair (1. Department of Obstetrics and Gynaecology, University Hospitals Leuven, Belgium. 2. My FetUZ Lab, Department of Development and Regeneration, Biomedical Sciences, KU Leuven, Leuven, Belgium).

First author Olivia K.C. Spivack coordinates the ERNICA CIST. First author Willemijn F.E. Irvine and last author René M.H. Wijnen are also members of the ERNICA CIST.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This paper was generated within the European Reference Network for rare Inherited Congenital Anomalies (ERNICA). ERNICA is currently funded by the EU4Health Programme 2023–2027 (Project 101155831 - EU4H-2023-ERN-IBA). The manuscript's open-access publication costs were financed by the Department of Pediatric Surgery, Erasmus MC Sophia Children's Hospital, Rotterdam, Netherlands.

Acknowledgments

The authors would like to acknowledge Jana Steerneman for her support with designing the ERNICA quality cycle visual. This paper was generated within the European Reference Network for rare Inherited Congenital Anomalies (ERNICA). ERNICA is currently funded by the EU4Health Programme 2023–2027. The content of this publication represents the views of the authors only and it is their sole responsibility; it cannot be considered to reflect the views of the European Commission and/or the Health and Digital Executive Agency (HaDEA) or any other body of the European Union. The European Commission and the agency do not accept any responsibility for use that may be made of the information it contains.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2025.1609018/full#supplementary-material

References

1. Agency for Healthcare Research and Quality UDoH, and Human Services. National Healthcare Quality & Disparities Report. (2019).

Google Scholar

2. EURORDIS Rare Disease Europe. What is a rare disease? Available at: https://www.eurordis.org/information-support/what-is-a-rare-disease/ (accessed August 8, 2024)

Google Scholar

3. Nguengang Wakap S, Lambert DM, Olry A, Rodwell C, Gueydan C, Lanneau V, et al. Estimating cumulative point prevalence of rare diseases: analysis of the orphanet database. Eur J Hum Genet. (2020) 28(2):165–73. doi: 10.1038/s41431-019-0508-0

PubMed Abstract | Crossref Full Text | Google Scholar

4. European Commission. Rare diseases and European Reference Networks. Available at: https://health.ec.europa.eu/rare-diseases-and-european-reference-networks_en (accessed August 8, 2024)

Google Scholar

5. Spivack OKC, Klein Haneveld MJ, Louisse S, Slater G, Hernando I. Supporting the continuous development and use of a patient partnership framework in European rare disease networks (ERNs): a scoping review of frameworks in the scientific literature. J Community Genet. (2024) 16(1):15–28. doi: 10.1007/s12687-024-00763-2

PubMed Abstract | Crossref Full Text | Google Scholar

6. Irvine WFE, Spivack OKC, Ista E. Moving toward the development and effective implementation of high-quality guidelines in pediatric surgery: a review of the literature. Eur J Pediatr Surg. (2024) 34(2):115–27. doi: 10.1055/s-0043-1778020

PubMed Abstract | Crossref Full Text | Google Scholar

7. Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. Br Med J. (2002) 325(7370):961–4. doi: 10.1136/bmj.325.7370.961

PubMed Abstract | Crossref Full Text | Google Scholar

8. Walshe K, Freeman T. Effectiveness of quality improvement: learning from evaluations. Qual Saf Health Care. (2002) 11(1):85–7. doi: 10.1136/qhc.11.1.85

PubMed Abstract | Crossref Full Text | Google Scholar

9. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. (2014) 23(4):290–8. doi: 10.1136/bmjqs-2013-001862

PubMed Abstract | Crossref Full Text | Google Scholar

10. Moen R. Foundation and History of the PDSA Cycle. (2020).

Google Scholar

11. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J. (2008) 336(7650):924–6. doi: 10.1136/bmj.39489.470347.AD

PubMed Abstract | Crossref Full Text | Google Scholar

12. Moberg J, Oxman AD, Rosenbaum S, Schünemann HJ, Guyatt G, Flottorp S, et al. The GRADE evidence to decision (EtD) framework for health system and public health decisions. Health Res Policy Syst. (2018) 16(1):45. doi: 10.1186/s12961-018-0320-2

PubMed Abstract | Crossref Full Text | Google Scholar

13. Hulscher J, Irvine W, Conforti A, Di Cesare A, Ichino M, Sfeir R, et al. European reference network for inherited and congenital anomalies evidence-based guideline on surgical aspects of necrotizing enterocolitis in premature neonates. Neonatology. (2025) 122:376–84. doi: 10.1159/000542540

Crossref Full Text | Google Scholar

14. Burgos CM, Irvine W, Vivanti A, Conner P, Machtejeviene E, Peters N, et al. European reference network for rare inherited congenital anomalies (ERNICA) evidence based guideline on the management of gastroschisis. Orphanet J Rare Dis. (2024) 19(1):60. doi: 10.1186/s13023-024-03062-8

PubMed Abstract | Crossref Full Text | Google Scholar

15. Granström AL, Irvine W, Hoel AT, Tabbers M, Kyrklund K, Leon FF, et al. Ernica clinical consensus statements on total colonic and intestinal aganglionosis. J Pediatr Surg. (2024) 59(10):161565. doi: 10.1016/j.jpedsurg.2024.04.019

PubMed Abstract | Crossref Full Text | Google Scholar

16. Teunissen NM, Rossi D, Wouters MW, Eaton S, van Heurn LWE, Wijnen R. The European pediatric surgical audit: improving quality of care in rare congenital malformations. Eur J Pediatr Surg. (2025). doi: 10.1055/a-2551-2056

PubMed Abstract | Crossref Full Text | Google Scholar

17. Teunissen N, Brendel J, Eaton S, Hall N, Thursfield R, van Heurn ELW, et al. Variability in the reporting of baseline characteristics, treatment, and outcomes in esophageal atresia publications: a systematic review. Eur J Pediatr Surg. (2023) 33(2):129–37. doi: 10.1055/s-0042-1758828

PubMed Abstract | Crossref Full Text | Google Scholar

18. Teunissen NM, Brendel J, Heurn L, Ure B, Wijnen R, Eaton S. Selection of quality indicators to evaluate quality of care for patients with esophageal atresia using a Delphi method. Eur J Pediatr Surg. (2023) 34(5):398–409. doi: 10.1055/s-0043-1777100

PubMed Abstract | Crossref Full Text | Google Scholar

19. Rossi D, Löf Granström A, Teunissen NM, Wijnen RMH, Wester T, Sloots CEJ. Bridging the gap: a systematic review on reporting baseline characteristics, process, and outcome parameters in rectosigmoid Hirschsprung’s disease. Eur J Pediatr Surg. (2024) 34(2):189–98. doi: 10.1055/a-2198-9050

PubMed Abstract | Crossref Full Text | Google Scholar

20. Dutch Institute for Clinical Auditing (DICA). Kinderchirurgie—EPSA—European Pediatric Surgical Audit. Available at: https://dica.nl/registratie/kinderchirurgie-epsa/ (Accessed April 08, 2025).

Google Scholar

21. Wilson P, Kislov R. Implementation Science. (2022). Available at: https://books.google.nl/books?hl=nl&lr=&id=q62aEAAAQBAJ&oi=fnd&pg=PT11&dq=16.%09Wilson#v=onepage&q=16.%09Wilson&f=false (accessed August 8, 2024)

Google Scholar

22. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, et al. A checklist for identifying determinants of practice: a systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. (2013) 8:35. doi: 10.1186/1748-5908-8-35

PubMed Abstract | Crossref Full Text | Google Scholar

23. Brouwers MC, Spithoff K, Kerkvliet K, Alonso-Coello P, Burgers J, Cluzeau F, et al. Development and validation of a tool to assess the quality of clinical practice guideline recommendations. JAMA Netw Open. (2020) 3(5):e205535. doi: 10.1001/jamanetworkopen.2020.5535

PubMed Abstract | Crossref Full Text | Google Scholar

24. AGREE Trust. AGREE-REX: Recommendation EXcellence. Available at: https://www.agreetrust.org/resource-centre/agree-rex-recommendation-excellence/ (accessed August 8, 2024)

Google Scholar

25. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. doi: 10.1186/s13012-015-0242-0

PubMed Abstract | Crossref Full Text | Google Scholar

26. Mazza D, Bairstow P, Buchan H, Chakraborty SP, Van Hecke O, Grech C, et al. Refining a taxonomy for guideline implementation: results of an exercise in abstract classification. Implement Sci. (2013) 8:32. doi: 10.1186/1748-5908-8-32

PubMed Abstract | Crossref Full Text | Google Scholar

27. Gagliardi AR, Alhabib S. Trends in guideline implementation: a scoping systematic review. Implement Sci. (2015) 10:54. doi: 10.1186/s13012-015-0247-8

PubMed Abstract | Crossref Full Text | Google Scholar

28. Teunissen NM, Wijnen RMH. Unlocking the power of data: standardized data collection in pediatric surgery. Eur J Pediatr Surg. (2024) 34(2):113–4. doi: 10.1055/a-2206-6790

PubMed Abstract | Crossref Full Text | Google Scholar

29. McGlacken-Byrne SM, Murphy NP, Barry S. A realist synthesis of multicentre comparative audit implementation: exploring what works and in which healthcare contexts. BMJ Open Qual. (2024) 13(1):e002629. doi: 10.1136/bmjoq-2023-002629

PubMed Abstract | Crossref Full Text | Google Scholar

30. Teunissen N. Doctoral thesis: improving quality of care for rare congenital malformations: achieving standardized quality measurement and improvement through international clinical auditing. Erasmus University Rotterdam (2023).

Google Scholar

31. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. (2012) 2012(6):Cd000259. doi: 10.1002/14651858.CD000259.pub3

PubMed Abstract | Crossref Full Text | Google Scholar

32. Ivers N, Yogasingam S, Lacroix M, Brown KA, Antony J, Soobiah C, et al. Audit and feedback: effects on professional practice. Cochrane Database Syst Rev. (2025) 3(3):Cd000259. doi: 10.1002/14651858.CD000259.pub4

PubMed Abstract | Crossref Full Text | Google Scholar

33. Duncan EM, Ivers NM, Grimshaw JM. Channelling the force of audit and feedback: averting the dark side. BMJ Qual Saf. (2022) 31(10):695–7. doi: 10.1136/bmjqs-2021-014520

PubMed Abstract | Crossref Full Text | Google Scholar

34. Grimshaw JM, Ivers N, Linklater S, Foy R, Francis JJ, Gude WT, et al. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Qual Saf. (2019) 28(5):416–23. doi: 10.1136/bmjqs-2018-008355

PubMed Abstract | Crossref Full Text | Google Scholar

35. Brown B, Gude WT, Blakeman T, van der Veer SN, Ivers N, Francis JJ, et al. Clinical performance feedback intervention theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. (2019) 14(1):40. doi: 10.1186/s13012-019-0883-5

PubMed Abstract | Crossref Full Text | Google Scholar

36. Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. (2016) 164(6):435–41. doi: 10.7326/M15-2248

PubMed Abstract | Crossref Full Text | Google Scholar

37. Gude WT, Roos-Blom MJ, van der Veer SN, Dongelmans DA, de Jonge E, Peek N, et al. Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care. Implement Sci. (2019) 14(1):90. doi: 10.1186/s13012-019-0937-8

PubMed Abstract | Crossref Full Text | Google Scholar

38. Foy R, Skrypak M, Alderson S, Ivers NM, McInerney B, Stoddart J, et al. Revitalising audit and feedback to improve patient care. Br Med J. (2020) 368:m213. doi: 10.1136/bmj.m213

PubMed Abstract | Crossref Full Text | Google Scholar

39. Vink MDH, de Bekker P, Koolman X, van Tulder MW, de Vries R, Mol BWJ, et al. Design characteristics of studies on medical practice variation of caesarean section rates: a scoping review. BMC Pregnancy Childbirth. (2020) 20(1):478. doi: 10.1186/s12884-020-03169-3

PubMed Abstract | Crossref Full Text | Google Scholar

40. Hulscher M, Wensing M. Process evaluation of implementation strategies. In: Improving Patient Care: The Implementation of Change in Healthcare. 3rd ed. Bohn Stafleu van Loghum. Hoboken, NJ: Wiley-Blackwell (2020). p. 369–89.

Google Scholar

41. Sullivan GA, Schäfer WLA, Raval MV, Johnson JK. Implementation science for quality improvement in pediatric surgery. Semin Pediatr Surg. (2023) 32(2):151282. doi: 10.1016/j.sempedsurg.2023.151282

PubMed Abstract | Crossref Full Text | Google Scholar

42. Nagelkerke S, Lambe C, Wijnen R, Tabbers M. P1.19: top-ten research topics in paediatric chronic intestinal failure, on behalf of the European reference network for rare inherited and congenital anomalies (ERNICA)—intestinal failure working group*. Transplantation. (2019) 103(7S2):S68. doi: 10.1097/01.tp.0000575796.68593.e4

Crossref Full Text | Google Scholar

43. ERNICA. Patient Journeys. Available at: https://www.ern-ernica.eu/patient-journeys (accessed December 17, 2024)

Google Scholar

44. Power B. The CDH patient perspective journey. Front Pediatr. (2023) 11:1052422. doi: 10.3389/fped.2023.1052422

PubMed Abstract | Crossref Full Text | Google Scholar

45. Spivack OKC, Dellenmark-Blom M, Dingemann J, Ten Kate CA, Wallace V, Bramer WM, et al. A narrative review of patient-reported outcome measures and their application in recent pediatric surgical research: advancing knowledge and offering new perspectives to the field. Eur J Pediatr Surg. (2024) 34(2):143–61. doi: 10.1055/s-0043-1778108

PubMed Abstract | Crossref Full Text | Google Scholar

46. Abbassi F, Pfister M, Domenghino A, Puhan MA, Clavien PA. Surgical outcome reporting. Moving from a comic to a tragic opera? Ann Surg. (2024) 280(2):248–52. doi: 10.1097/SLA.0000000000006226

PubMed Abstract | Crossref Full Text | Google Scholar

47. Vergote S, De Bie FR, Duffy JMN, Bosteels J, Benachi A, Power B, et al. Core outcome set for perinatal interventions for congenital diaphragmatic hernia. Ultrasound Obstet Gynecol. (2023) 62(3):374–82. doi: 10.1002/uog.26235

PubMed Abstract | Crossref Full Text | Google Scholar

48. Demirok A, Nagelkerke SCJ, Benninga MA, Köglmeier J, Mutanen A, Arnell H, et al. Development of a core outcome set for pediatric chronic intestinal failure. JPEN J Parenter Enteral Nutr. (2023) 47(3):364–71. doi: 10.1002/jpen.2468

PubMed Abstract | Crossref Full Text | Google Scholar

49. Lukas CV, Holmes SK, Cohen AB, Restuccia J, Cramer IE, Shwartz M, et al. Transformational change in health care systems: an organizational model. Health Care Manage Rev. (2007) 32(4):309–20. doi: 10.1097/01.HMR.0000296785.29718.5d

PubMed Abstract | Crossref Full Text | Google Scholar

50. Wensing M, Grol R. Knowledge translation in health: how implementation science could contribute more. BMC Med. (2019) 17(1):88. doi: 10.1186/s12916-019-1322-9

PubMed Abstract | Crossref Full Text | Google Scholar

51. European Commission. Independent evaluation of European Reference Networks (ERNs) and of Healthcare Providers (HCPs): Final report. (2024).

Google Scholar

52. Spivack O, Gaillard L. ERN CRANIO patient coverage of craniosynostosis in Europe. Orphanet J Rare Dis. (2022) 17(1):333. doi: 10.1186/s13023-022-02475-7

PubMed Abstract | Crossref Full Text | Google Scholar