Targeted Training for Subspecialist Care in Children With Medical Complexity

Introduction

Preventable harm in healthcare occurs in about 6% of patients across medical care settings (primary care, hospitals, intensive care units), with 12% leading to permanent disability or death (1). Medication and procedure-related errors, as well as nosocomial infections, are the most common adverse events in hospitals (2). Peaks of healthcare-related adverse events have been observed to reoccur annually during July and August in the United States and the United Kingdom, decreasing progressively after 1 month (3, 4). This tendency coincides with the enrolment of many new trainees and fellows at teaching hospitals, annually over 32,000 and 100,000 in Europe and the United States (5, 6). Such data suggest that current medical education does not sufficiently prepare graduates for their future work environment (3, 7–9).

Children with medical complexity (CMC) account for an increasing proportion of hospitalized pediatric patients resulting in about one-third of total pediatric healthcare costs (10–13). These patients are at particular risk for medical errors and longer hospital stays (14–18). Several studies suggest that residents often feel overwhelmed, anxious, and in doubt of their skills when caring for CMC (11, 19, 20). There is a high need for novel measures to equip pediatric residents with the knowledge and skill set to improve their performance in early clinical training, thus increasing patient safety (12).

Simulation-based boot camps at the beginning of clinical training might be an effective way to prepare young physicians for their future clinical environment (9, 21). A web-based multimedia curriculum for pediatric residents increased satisfaction, knowledge, behavior change related to verbal handoffs, and comfort with clinical care of CMC (18). This curriculum primarily focuses on the care of CMC with neurological impairment and technology dependence (e.g., spasticity, tracheostomy tubes).

In the care for CMC, important differences exist between Europe and the United States. In Europe, tertiary care center subspecialists care for all patients, including CMC. In contrast, in the United States, hospitalists mainly provide the care for these patients. Hospitalists are defined as “a physician whose primary professional focus is the general medical care of hospitalized patients and whose activities include patient care, teaching, research, and leadership related to hospital medicine” (22). Consequently, different needs and challenges for new residents becoming future pediatric subspecialists might prevail.

In the literature, several national educational curricula, syllabi, and content outlines for general pediatrics and pediatric subspecialties exist (23–28), in addition to individual efforts of needs assessments in particular areas of medicine for future general pediatricians, e.g., pediatric palliative care, pediatric gastroenterology curricula, or essential hypertension for primary care pediatricians (29–31). However, as Abbott and First concluded, a “one-size-fits-all” approach to pediatric residency training cannot be justified as training needs for subspecialty pediatricians and hospitalists primarily working in hospital-based medicine are sufficiently distinct from those of subspecialty pediatricians and general pediatricians working mainly in ambulatory medicine (32).

Currently, no national or international educational guidance for the training of future pediatric tertiary care specialists outside their field of expertise involving the multidisciplinary care of CMC exists. Some examples of similar schools of thought have been published in internal medicine (33–36), but to the best of our knowledge, no needs assessment for the training of residents becoming future subspecialists at a pediatric tertiary care center caring for CMC has been published yet. The primary objective of this mixed-methods study was to evaluate the pediatric resident's past educational needs and challenges during their clinical rotation in specific subspecialty areas of pediatrics (e.g., pediatric nephrology, pediatric cardiology) and to define key learning content for future resident and subspecialty training involving care for CMC.

Methods

Setting

The Comprehensive Center for Pediatrics at the Medical University of Vienna is the largest Austrian academic tertiary care center that focuses almost exclusively on caring for children with severe or orphan acute and chronic diseases in both inpatient and outpatient medicine. The Austrian pediatric residency curriculum is set for 6 years. At our center, all residents need to accomplish their training at the same, a total of six, different specialized divisions (“rotations;” pulmonology, allergology, and endocrinology; neurooncology and epileptology; cardiology and hemostaseology; nephrology, gastroenterology, and rheumatology; pediatric intensive care medicine and neonatology; pediatric ambulatory care) to learn the whole spectrum of pediatrics, with approximately 80% inpatient and 20% outpatient care.

A prospective mixed-methods study was conducted at the Department of Pediatrics and Adolescent Medicine. This study consisted of two phases, namely, (1) qualitative explorative interviews with a convenience sample of pediatric residents at different stages of clinical training to explore and evaluate educational needs and challenges yielding a defined set of competencies in the form of knowledge, practical skills, and soft skills in specific highly specialized work environments; and (2) quantitative validation (online survey) with all pediatric residents and specialists to rank these individual competencies through a larger-scale survey to create a more profound content outline. Approval was granted by our university's data protection agency beforehand.

Qualitative Explorative Interviews

Qualitative explorative interviews were conducted from November 2019 until February 2020 by two researchers (FE and AH), supervised by an expert in qualitative research (VR). At the time of screening (10/2019), 58 residents were potentially eligible for inclusion in phase I of this study. In detail, we conducted open-ended, semi-structured individual interviews to allow in-depth discussion in a private setting to talk about their perceived resources and demands (37). Our developed and piloted semi-structured interview guide is given in the Supplementary Material. The interviews were audio-recorded and transcribed verbatim. Analysis of the collected data was conducted utilizing a qualitative thematic content analysis to identify codes (37, 38). First, an initial code list was created by reading through the collected data. Codes are defined as words that act as labels for specific concepts and describe the meaning of a piece of text (38). For each coding unit, two different labels were assigned, namely, theme and group. Theme refers to the content of the coded segment, and group to the type of learning by context (“knowledge,” “practical skill,” “soft skill”) and category refers to whether the theme of the coded unit was or was not learned by the resident during their rotation (“learned” vs. “missing,” respectively), as seen in the context of the coded unit. Examples are given in Table 1.

TABLE 1

Table 1. Coding examples of the qualitative thematic content analysis.

To introduce better reproducibility and international standardization of our results, the original coded themes were linked to corresponding domains and subdomains as outlined in the American Board of Pediatrics (ABP) content outline created for the Pediatrics Board Examination in the United States. These content outlines drafted by content experts are regularly updated and available for different subspecialties, e.g., general pediatrics and pediatric nephrology (25–27). The content outlines are structured in a major content domain and several subdomains, e.g., content domain—glomerular disorders; subdomain—nephropathies; lower-order subdomain—minimal change disease and variants (27). We linked our previously identified themes to these content domains and subdomains. In a few cases with added descriptors, the lowest appropriate content domain or subdomain was chosen to preserve the meaning of the themes in context with the interview's original quotes. This is a modified approach of the previously published rules for International Classification of Functioning, Disability and Health linking in qualitative research (39). Themes that fit into more than one subspecialty and themes where apparent linking was not possible were discussed between two researchers and a pediatrician and then consistently categorized. For example, the theme hemolytic uremic syndrome (HUS) (as given in Table 1) was linked to the lowest appropriate subdomain thrombotic microangiopathies within the ABP content outline, categorized within pediatric nephrology > glomerular disorders > nephropathies with systemic disease (27). To ensure validity of our qualitative data collection and analysis, the following strategies were embodied from the beginning until the completion of our study, namely, (1) audit trail to ensure optimal replicability (40), (2) peer review/debriefing with specialists from our department's education committee (41, 42), and (3) reflexivity of executive researchers (40, 43).

After initial analysis, we displayed the emerged themes according to Park's et al. model of the importance performance analysis (IPA, = needs assessment) (44), which aids to identify key themes of high importance with low performance, where focused improvement is needed and deemed to be fruitful (see Figure 1).

FIGURE 1

Figure 1. Importance performance analysis (IPA = needs assessment) adapted from Park et al. (44). The importance of each theme is derived from the frequency (number of occurrences = in how many interviews it was coded), the performance by the categorization (learned vs. missing).

Quantitative Validation

To test the validity of this qualitative IPA, we set out to identify key themes within the IPA's results for focused interventions by a quantitative assessment. As pediatric nephrology was one of the three topics with the most identified themes of our IPA, we concentrated further investigations on this subspecialty: We submitted the 20 most frequently mentioned themes in pediatric nephrology to an online ranking among all physicians (= residents and specialists), utilizing SoSci Survey (SoSci Survey GmbH, Munich, Germany) (45). Each participant was asked to select the five top relevant (“TOP 5”) and the five least relevant (“LEAST 5”) themes (with the remaining 10 themes categorized as “Intermediate”). Participants were explicitly asked to assign the relative importance of each theme for training to become future tertiary care hospitalists outside pediatric nephrology. Furthermore, participants were asked to state their level of training (resident vs. specialist), and specialists were asked to declare their field of expertise. Survey invitations were emitted on March 1, 2021, and were online for 4 weeks. Participants received two reminders (after 1 and 3 weeks) to increase attendance.

Statistical Analysis

Descriptive data were analyzed using absolute and relative frequencies, median and interquartile range (IQR), or mean and standard deviation (SD) depending on data distribution. For each theme submitted to the survey, an importance score (IS) was calculated as the mean reciprocal rank, defined as the mean of 1 over the ranking assigned by each participant, in accordance with similar qualitative research in pediatric nephrology (46). For example, if a theme was ranked into the “TOP 5” category by one participant and in “LEAST 5” by another participant, the IS for this theme would be calculated as the mean of 1/1 and 1/3, resulting in an IS of 0.67, respectively. Differences of IS between groups were calculated by utilizing Student's t-test as IS represent mean reciprocal ranks. Kendall's W was calculated to assess interrater agreement among participants. Kendall's rank correlation was used to calculate correlation coefficients. All themes were submitted to cluster analysis by calculating Euclidean distances of IS across different groups of subspecialties and residents and submitting them to hierarchical Ward clustering. Subspecialties were analyzed when at least four specialists of the respective subspecialty participated in our survey or categorized in “other.” Numbers of final clusters were evaluated based on contextual meaningfulness and discussed with experts in pediatric nephrology. Prioritization of themes and interrater agreement was further analyzed concerning experience, pooled by different training levels [Trainees (= residents), “trained” (= specialists outside pediatric nephrology), trainers [=specialists in pediatric nephrology)]. Statistical analysis was performed using IBM SPSS version 24.0.0.0 2016 (SPSS, Inc., Chicago, IL, USA) and R software (R Core Team 2020) (47).

Results

Qualitative Explorative Interviews

Over 4 months, 16 interviews were conducted with 16 residents [10 women, median age 30 years (IQR4)]. Baseline characteristics of the total and the studied population were similar, with 38 and 31% of residents based on neonatology and pediatric intensive care units, and 62 and 69% based on general wards and intermediate care units, respectively. Furthermore, sex distribution among the targeted and studied population was comparable, with 63 and 64% female residents, respectively. With a median number of in-house ward rotations of 3 (IQR 3), our study population accurately represents an equally distributed spectrum of residents with a wide and heterogeneous clinical exposure on different stages of their education. This is being indicated by the range of interviewed residents from early training up to almost completion of residency with a median at “half-time” (= 3 rotations).

The median duration of individual interviews was 69 min (IQR 35 min). Participants mentioned a mean of 28 (SD ± 13) themes in their interviews. In total, 165 unique themes were identified, resulting in a mean of 10 (SD ± 7) unique themes per participant. In addition, 86% (141/165) of themes were assigned to (theoretical) knowledge, 12% (20/165) to practical skills, and 2% (4/165) to soft skills. Of these themes, 62% (122/197) were assigned as sufficiently trained (category: learned) and 38% (75/197) to need additional training (category: missing), with double mentions by different participants. Assignment to these groups did not differ between ages, numbers of in-house rotations, and sex. Categorization of emerged themes into various pediatric subspecialties is displayed in Supplementary Table 1.

Practical skills mentioned by at least four interviewees (25%) are displayed in Table 2 and congregate into three distinct categories, namely, resuscitation, sonography, and vascular/cerebrospinal fluid access. Practical skills mentioned by <4 interviewees were bladder catheterization (19%, 3/16), blood products transfusions (19%, 3/16), clinical presentation in neurology (19%, 3/16), incision and drainage (19%, 3/16), and transport management in critical care medicine (13%, 2/16). Soft skills mentioned by at least two interviewees were patient-parent-pediatrician relationship (56%, 9/16) and end-of-life care (13%, 2/16).

TABLE 2

Table 2. Practical skills (mentioned by at least four interviewees) drafted from the qualitative explorative interviews (n = 16).

The IPA of all emerged themes of our qualitative explorative interviews is displayed in Supplementary Figure 2. Table 3 shows the extracted 23 (16%) key knowledge themes from 10 different pediatric subspecialties in the high-importance and low-performance panel, i.e., focused improvement is needed and deemed fruitful. In total, 20 themes of pediatric nephrology emerged during the qualitative explorative interviews (see Supplementary Table 2). In addition, 85% (17/20) of these themes were correctly identified by the IPA. Three themes (urinary tract infection, bacteriuria, and pyuria; acid-base disorders; hematuria and proteinuria) were correctly identified within the high-importance and low-performance panel, as occurrence of missing was higher than learned. Two themes (core diagnostics in nephrology; chronic kidney disease and end-stage kidney disease) that were 100% missing were not highlighted by the IPA as they were mentioned once only. The IPA also highlighted the theme of sodium and water balance although the majority declared it as learned (missing:learned = 3:6).

TABLE 3

Table 3. Extracted key knowledge themes from the importance performance analysis (IPA) high-importance and low-performance panel (n = 23) ranked by their importance (= number of occurrence) with nephrology themes in bold.

Quantitative Validation

Survey invitations were sent to all residents and specialists (n = 154). The response rate was 55% (84/154), with 14 being excluded due to doubled or incomplete entries, or due to ranking themes into “TOP 5” and “LEAST 5” categories simultaneously. A total of 70 responses (46%) could be included in our final analysis.

Of these, 46% (32/70) were residents and 54% (38/70) specialists, reflecting a mild overrepresentation of residents of the targeted medical staff (38% residents, 62% specialists). The remaining subspecialties were pooled and analyzed as a single category, namely, “other.” The distribution of specialists among subspecialties within the targeted (n = 84) and the surveyed (n = 38) population were similar, with 32 and 34% from neonatology and pediatric critical care, 18 and 11% from pediatric cardiology, 10 and 11% from pediatric nephrology, 7 and 11% from pediatric pulmonology, and 33 and 34% from “other.”

Figure 2 represents a significant high correlation between the occurrence of the theme in the qualitative interviews and the calculated IS with themes being assorted based on importance in the quantitative survey (tau = 0.57, p = 0.001). Themes of high importance in the qualitative interviews also display high importance within the quantitative survey. In contrast, themes that may be underrepresented in the interviews due to the lower number of participants show greater diversity along with the calculated ISs. Most importantly, core diagnostics nephrology and chronic kidney disease and end-stage kidney disease that the IPA, although classified as 100% “missing,” failed to identify as important were assigned with higher IS within the quantitative validation.

FIGURE 2

Figure 2. Validation of the importance of key nephrology themes within the qualitative interviews with the calculated importance scores (IS) of the quantitative survey. Color coding represents classification as the ratio of missing:learned, with red representing 100% missing and blue representing 100% learned. Kendall's rank correlation tau = 0.57, p = 0.001.

Overall interrater agreement on IS was poor (W = 0.3, p < 0.001), as well as between residents (W = 0.32, p < 0.001) and “trained” (W = 0.3, p < 0.001). The highest interrater agreement was achieved between trainers (W = 0.66, p < 0.001).

Hierarchical clustering of the most frequently mentioned themes in pediatric nephrology by IS of trainees, “trained,” and trainers resulted in three major contextual clusters, as displayed in Figure 3. Cluster number 1 representing the most important themes for future tertiary care hospitalists is normal and abnormal blood pressure, electrolyte disorders, urinary tract infection, bacteriuria, and pyuria, core diagnostics nephrology, and acute kidney injury. Cluster number 2 represents a total of five further themes of high importance, resulting in a contextual cluster of frequent encounters in both inpatient and outpatient care (urine assessments, nephrotic syndrome, hematuria and proteinuria, acid-base disorders, and sodium and water balance). Together, clusters 1 and 2 cover both the themes identified by the IPA and one of the two themes not identified by the IPA, although classified as 100% “missing” (core diagnostics nephrology).

FIGURE 3

Figure 3. Hierarchical clustering of the most frequently mentioned themes in pediatric nephrology by importance scores (IS) of trainees, “trained,” and trainers, as well as different specialists.

Although classified as 100% “missing” within the qualitative interviews, chronic kidney disease and end-stage kidney disease was not rated with high importance within the quantitative survey, neither by trainees, “trained,” nor by trainers. Cluster number 3 represents themes where in-depth knowledge and specialization is needed and consists of themes related to end-stage kidney disease, renal replacement therapy, other extracorporeal treatments, and complex kidney diseases. In subanalysis, the theme CAKUT (congenital anomalies of the kidney and urinary tract) is rated with high importance by pediatric cardiologists and neonatologists/intensivists.

Representative quotes of the themes identified in both most important clusters (1 and 2) describing clinical situations where the importance of these themes was stated are given in Supplementary Table 3.

Supplementary Figure 3 represents the calculated IS for different training levels [Trainees (= residents), “trained” (= specialists outside pediatric nephrology), trainers (= specialists in pediatric nephrology)] themes being assorted on the basis of importance, with decreasing importance from left to right. Statistical comparison of IS is displayed in Table 3.

Discussion

The aim of this explorative mixed-methods study was to evaluate educational needs and challenges faced by pediatric residents yielding competencies of knowledge, and practical and soft skills at an academic pediatric tertiary care center that focuses almost exclusively on caring for children with severe or orphan diseases. Several national curricula, outlines, and frameworks exist, defining the knowledge and skill set pediatricians should have acquired by the end of their residency or specialization training. However, the current literature does not offer consensus on specific training recommendations for interdisciplinary inpatient care for CMC with multiple-organ involvement (7, 14, 17, 18). Most importantly, a “one-size-fits-all” approach to pediatric residency training may not be appropriate as hospital-based pediatrics and pediatric primary care and ambulatory medicine are sufficiently different (32).

Our standardized qualitative analysis revealed the need for systematic incorporation of subject-specific clinical training in adjacent pediatric specialties for future highly specialized pediatric hospitalists caring for CMC with multiorgan disease. We reduced the 280 listed themes (= lowest domains) in the ABP content outline to 165 unique themes and finally extracted 23 key themes where improvement is necessary and deemed fruitful. A subset of key themes in pediatric nephrology was validated by importance for training of future hospitalists in pediatric tertiary care. Hierarchical clustering of nephrology themes resulted in two clusters of high importance dominated by normal and abnormal blood pressure, electrolyte disorders, urinary tract infection, bacteriuria, and pyuria, core diagnostics nephrology, and acute kidney injury, together with a contextual cluster of frequent encounters in both inpatient and outpatient care (urine assessments, nephrotic syndrome, hematuria and proteinuria, acid-base disorders, and sodium and water balance). The third cluster, including e.g., renal replacement therapy, was rated as of low importance, most probably because these patients are usually consulted and very closely monitored by pediatric nephrologists.

In this study, we present the results and a methodological framework based on the educational needs in pediatric nephrology for tertiary care hospitalists in training as key learning content for the development of new curricula and/or further studies on this matter.

Heterogeneity of Medical Training Programs in Pediatrics

National educational content outlines are highly diverse and generally broad or specifically tailored to specialists needs. The Austrian training content outline for general pediatrics (48) consists of an obligatory basic training module and seven different additional modules (e.g., nephrology and urology; neuropediatrics, sleep medicine, psychosomatics), of which three must be chosen for in-depth clinical training. In contrast, the Royal College of Pediatrics and Child Health (RCPCH) general pediatrics syllabus for general pediatrics training in the United Kingdom (24) describes five general learning outcomes (e.g., resuscitation, team leading, and management) and a sixth learning outcome consisting of pediatric conditions from 14 areas of medicine. The US-focused ABP general pediatrics content outline (26) provides a more detailed description with 280 learning themes structured within 25 different content domains relating to several subspecialties of pediatric medicine.

Practical and Soft Skills

The identified practical and soft skill set (Table 2) generally matches the Austrian training content outline well, whereas the ABP and RCPCH only mention few practical and soft skills. Details are given in Supplementary Table 4. Despite being required, quality improvement/assurance did not occur in our analysis, which is in line with recent literature where only one-third of pediatric residents reported positive educational experiences in quality improvement (49). The importance of mastering skills such as resuscitation (63%) and stabilization and transition of newborn infants (31%) is reflected by a pediatric tertiary care survey where 72–96% of third-year residents attended events of cardiopulmonary resuscitation during their clinical practice (50). Another important skill set being mentioned by 19% of participants is blood products transfusions. As a recent international evaluation among pediatric residents concluded, there is an urgent need for improved education in transfusion medicine as this large international surveyed group of pediatric residents performed poorly on a transfusion medicine knowledge exam (51).

In our analysis, patient-parent-pediatrician relationship was considered a relevant soft skill to be mastered by medical trainees, which is also included in the Accreditation Council for Graduate Medical Education (ACGME) milestones to trace pediatric residents' development but does not specifically appear in the RCPCH or ABP content outline. Notably, end-of-life care, despite being mentioned only in 13% of our interviews, yielded strong emotional responses by the interviewed residents, particularly anxiety as no uniform training had been established: “[…] Our daily hand-off for this patient was, that he was terminal and could die anytime soon. The first thing that came to my mind was, I hope this does not happen during my shift. At 5 a.m., 21 h into my shift, my only thought was: 3 h left until hand-off, thank God. My own mindset, that fact that I thought this way, is awful to me. Ideally, I should be able to say: okay, I am well-trained with this kind of situation, and I am able to care and provide support to the patient and his/her family. […]”. With more than 50% of deaths in childhood in the United States occurring in the hospital and only 39% of pediatric residents receiving adequate palliative care training (52), and as careful breakdown of our explorative qualitative analysis suggests, training in end-of-life care for pediatricians should be greatly enhanced. This has also been recommended by the American Academy of Pediatrics, the Institute of Medicine and the ACGME (52, 53). In line with other needs assessments specifically targeting pediatric palliative care, there is a clear need for increased efforts in pediatric palliative care education during residency training as residents perceived their training in palliative care to be inadequate with no improvement over time (30). Training future (pediatric) hospitalists in all of these skills is of great importance as some of these skills, e.g., blood products transfusions, are generally needed often and rarely produce adverse events, whereas others, e.g., end-of-life care, are needed rarely but have a high potential to go wrong.

Importance Performance Analysis: Knowledge

The 23 knowledge themes identified by our IPA with high importance and potential for improvement are generally well-represented in all three content outlines and might thus be regarded as extracted key themes for targeted training. Details of consistency and differences between content outlines and qualitative interviews are given in Supplementary Table 5. Noteworthy, the Austrian content outline includes additional aspects important for the care of CMC, e.g., developmental and social pediatrics, psychosomatics, ergotherapy, analgesia, palliative care, follow-up care, and transition to adult care, that were not identified by our IPA. In contrast, antimicrobial stewardship principles and handoffs across the continuum of care were identified with high importance and low performance by our IPA but are not mentioned in the analyzed national educational content outlines, although the ACGME milestones include handoffs as an important skill for residents. Especially antimicrobial stewardship principles is of valuable interest to hospitalists caring of CMC as the prevalence of multidrug-resistant bacterial infections in children has significantly increased during the last two decades (54). CMC may receive many antibiotic treatments during their lifetime, and studies have shown that antimicrobial stewardship programs can be effectively applied in pediatric inpatient settings, e.g., neonatal care (55). Medical errors are estimated to be attributed in approximately 80% to communication breakdowns. In particular for CMC, standardized high-quality handoffs across the continuum of care are essential for patient safety. However, as the committee on hospital care of the AAP states, “Hand-off communication is a skill requiring training and practice. Attending physicians are likely to benefit from ongoing training and monitoring of a standard approach to hand-offs” (56). This verifies our findings and highlights the importance of placing great emphasis on teaching antimicrobial stewardship principles and handoffs across the continuum of care to future hospitalists in pediatric tertiary care.

Themes in pediatric nephrology labeled with high importance (clusters 1 and 2 of Figure 3) in our analysis are generally well-covered by the RCPCH and ABP content outlines. In the Austrian general pediatrics content outline, no consistent standard exists on specific nephrology training for general pediatricians, let alone hospitalists/specialists outside pediatric nephrology. Therefore, to assume that future non-nephrology specialists, especially at tertiary care centers caring for CMC, should get by with the nephrological knowledge of general pediatricians is utopian fallacy. Our results indicate high importance of training in diagnostic tools essential to nephrology as the themes core diagnostics in nephrology and urine assessments emerged in a high-importance cluster in addition to hematuria and proteinuria, whereas the RCPCH syllabus and ABP content outline cast no additional focus on diagnostics. However, the ABP content outline explicitly features genetic disorders, diseases, and conditions in pediatric nephrology, which did not emerge in our analysis. Arguably there has been tremendous progress in the knowledge on genetic backgrounds of kidney disease, also dramatically influencing diagnostic progress and therapeutic pathways, e.g., for patients suffering from steroid-resistant nephrotic syndrome with genetic origins (57). This is of special importance since, on the one hand, communicative skills in the setting of patient-parent-pediatrician relationships emerged as highly relevant soft skill for young residents, but, on the other hand, literature reviews conclude that especially to young pediatric residents communication of complex medical information, especially genetic information, can be very challenging, and education of pediatric residents in these skills should be reinforced (58).

Quantitative Validation

The quantitative survey-based validation of key themes in pediatric nephrology of a subset of 20 identified nephrology themes revealed significantly high correlation (tau = 0.57, p = 0.001, as displayed in Figure 2). Themes of low importance display greater diversity along the calculated IS and may be underrepresented in the interviews due to a lower number of participants and free exploration of themes. Both themes classified as 100% “missing” and failed to be identified by the IPA received higher IS within the quantitative validation. This might reflect the implicit knowledge on diagnostic tools summarized in core diagnostics nephrology as well as for the care of children with chronic kidney disease and end-stage kidney disease. Themes of undoubtedly high importance, also after validation, were sodium and water balance, electrolyte disorders, and acid-base disorders, being an integral part of inpatient care.

As displayed in Figure 3, Supplementary Figure 3, and Table 4, perception of importance of themes between trainees, “trained,” and trainers was similar but not identical, likely reflecting distinct training needs for different subspecialties. Only the theme normal and abnormal blood pressure was ranked higher by trainers than by trainees, whereas other significant themes (plasmapheresis and other plasma-based therapies, staging chronic kidney disease/progression) were ranked lower by trainers, which might stem from the trainers' point of view of consultant requests in these areas. This comes with no surprise as similar needs assessments between pediatric residents and attending staff for education in clinical pharmacology also resulted in similar but not identical learning needs (59). Accordingly, interrater agreement of relevancy was high between pediatric nephrologists (= trainers) but low among trainees and “trained” (= i.e., non-nephrology specialists). This corroborates different emphasis of nephrological themes in diverse subspecialties of pediatrics as, for example, the importance of core diagnostics in nephrology is perceived significantly different among specialists of neonatology and pediatric critical care vs. pediatric nephrologists. However, as displayed in Figure 4 homogeneous importance of major thematic clusters of nephrological themes for non-nephrologists could be identified despite minor differences between specialties.

TABLE 4

Table 4. Comparison of importance scores (IS) between trainees, “trained,” and trainers.

Although the RCPCH syllabus includes acute nephritis, chronic kidney disease, and enuresis in their content outline, both acute nephritis and chronic kidney disease did not receive high ISs by non-nephrologists. This might stem from major involvement of pediatric nephrologists with these patients in the training environment of our tertiary care center since they represent a preselection of the most severe and complex cases. Enuresis did never emerge during our analysis as it being a generally common typical outpatient problem (60). CAKUT, being by far the major cause of chronic kidney disease (CKD) in childhood (48% of CKD), emerged as a topic of high importance to future non-nephrologist tertiary care specialists, especially for pediatric cardiologists (IS 0.8). These findings are not surprising given the association of CAKUT with congenital heart disease, e.g., DiGeorge syndrome (61). Given the high importance of CAKUT in multidisciplinary care, it seems surprising that this disease spectrum did not gain additional attention in the RCPCH syllabus.

Strengths and Limitations

To the best of our knowledge, this is the first study evaluating this special niche of residency training with “needs assessment” based on qualitative interview exploration, refined by quantitative importance validation. While several national and international educational frameworks and guidelines for pediatrics and pediatric subspecialties exist, they do not focus on the multidisciplinary treatment of CMC with multiorgan diseases involving collaboration among subspecialties. Several examples underlining the importance of interdisciplinary exchange in highly specialized areas of medicine can be drawn from literature in internal medicine, e.g., dialysis for non-nephrologists (33, 34, 36), CKD for non-nephrologists (35), or heart- and lung transplantation with associated renal complications (62). In addition, renal involvement in mitochondrial cytopathies (63) and cross-over areas such as pediatric onconephrology (64) might warrant relevant benefits for nephrological knowledge in other highly specialized areas of pediatrics. As this study was initiated by members of the pediatric nephrology team, we decided to focus on this subspecialty for validation, although themes from cardiology and general pediatrics were more frequently mentioned. Qualitative research explores the unexpected in specific populations by holistically seeking to understand the participants' perspectives on the phenomena of interest and might be more feasible than standard methods of quantitative research when evaluating processes and outcomes of medical education (38, 65). Due to the COVID-19 pandemic, we were compelled to abort qualitative interviews after 4 months. Regarding this shortcoming, some residents were deprived to mention important aspects, highly relevant to them, and rare themes might not have been emerged and identified by our analytic approach. Despite a limited number of qualitative interviews, interviewed residents are generally representative for target population in terms of baseline characteristics. Small participant numbers in each subcategory (i.e., nephrologists, cardiologists) limit accuracy of comparison for drawn conclusions on the basis of our survey. However, our analysis is strengthened by a survey with high interrater agreement between specialists of the thematic focus of this work.

Conclusion

This applied a mixed-methods approach to build key learning content for training needs of future subspecialty hospitalists caring for CMC, highlighting aspects of nephrology validated by an orthogonal method. A total of 280 listed themes within the ABP content outline were condensed to 23 key themes of high importance in need for improvement by an IPA, as well as to key themes in pediatric nephrology validated in a larger cohort for importance for training of future pediatric tertiary care hospitalists. Most importantly, future training should emphasize important aspects of patient safety beyond subspecialty training, such as antimicrobial stewardship principles and handoffs across the continuum of care, practical skills, such as point-of-care sonography and blood products transfusions, as well as end-of-life care. This study also introduces hierarchical clustering analysis to further tailor educational contents for a given subspecialty to distinct needs of pediatricians in training for another subspecialty. The knowledge basis and used methodologies of this study may lay the groundwork for future detailed analysis and the development of digital boot camps and might be able to aid the improvement of patient safety by decreasing preventable harm by medical errors, especially for vulnerable patient groups, such as CMC in tertiary care pediatrics.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

FE, CA, and MB developed concept and design, had full access to all the data in the study, take responsibility for the integrity and accuracy of the data and subsequent analysis, drafted the initial manuscript, conducted analysis and interpretation of data, and reviewed and revised the manuscript. IV, RS, AH, VR, and SB conducted further data analysis and interpretation. FE, AH, VR, IV, and RS collected all data presented in this manuscript. LK, LD-F, TS, KO, and ES performed critical revision for the manuscript for important intellectual content. All authors approved the final manuscript as submitted.

Funding

This research was funded by the Anniversary Fund of the Oesterreichische Nationalbank (OeNB) Nr. 17194.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We sincerely thank all participants of this study for their valued contribution, as well as our department's educational task force, particularly Dr. Bernadette Goeschl, for promoting and supporting this project at ourdepartment.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2022.851033/full#supplementary-material

References

1. Panagioti M, Khan K, Keers RN, Abuzour A, Phipps D, Kontopantelis E, et al. Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ. (2019) 366:l4185. doi: 10.1136/bmj.l4185

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al. 'Global trigger tool' shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff. (2011) 30:581–9. doi: 10.1377/hlthaff.2011.0190

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Barach P, Philibert I. The July effect: fertile ground for systems improvement. Ann Intern Med. (2011) 155:331–2. doi: 10.7326/0003-4819-155-5-201109060-00352

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Haller G, Myles PS, Taffe P, Perneger TV, Wu CL. Rate of undesirable events at beginning of academic year: retrospective cohort study. BMJ. (2009) 339:b3974. doi: 10.1136/bmj.b3974

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Barzansky B, Etzel SI. Medical schools in the United States, 2007-2008. Jama. (2008) 300:1221–7. doi: 10.1001/jama.300.10.1221

PubMed Abstract | CrossRef Full Text | Google Scholar

6. OECD. OECD Health Data 2008. Available online at: http://titania.sourceoecd.org/rpsv/dotstat.htm

7. Zhao X, Koutroulis I, Cohen J, Berkowitz D. Pediatric urgent care education: a survey-based needs assessment. BMC Health Serv Res. (2019) 19:388. doi: 10.1186/s12913-019-4241-8

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Jarrett MP. Impact of the “July effect” on patient outcomes. Ann Intern Med. (2012) 156:168. doi: 10.7326/0003-4819-156-2-201201170-00022

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Cohen ER, Barsuk JH, Moazed F, Caprio T, Didwania A, McGaghie WC, et al. Making july safer: simulation-based mastery learning during intern boot camp. Acad Med. (2013) 88:233–9. doi: 10.1097/ACM.0b013e31827bfc0a

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Berry JG, Hall DE, Kuo DZ, Cohen E, Agrawal R, Feudtner C, et al. Hospital utilization and characteristics of patients experiencing recurrent readmissions within children's hospitals. JAMA. (2011) 305:682–90. doi: 10.1001/jama.2011.122

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Bogetz JF, Bogetz AL, Bergman D, Turner T, Blankenburg R, Ballantine A. Challenges and potential solutions to educating learners about pediatric complex care. Acad Pediatr. (2014) 14:603–9. doi: 10.1016/j.acap.2014.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Frush BW. Preparing residents for children with complex medical needs. Pediatrics. (2020) 145:e20191731. doi: 10.1542/peds.2019-1731

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Simon TD, Berry J, Feudtner C, Stone BL, Sheng X, Bratton SL, et al. Children with complex chronic conditions in inpatient hospital settings in the United States. Pediatrics. (2010) 126:647–55. doi: 10.1542/peds.2009-3266

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Berry JG, Agrawal R, Kuo DZ, Cohen E, Risko W, Hall M, et al. Characteristics of hospitalizations for patients who use a structured clinical care program for children with medical complexity. J Pediatr. (2011) 159:284–90. doi: 10.1016/j.jpeds.2011.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Berry JG, Hall M, Cohen E, O'Neill M, Feudtner C. Ways to identify children with medical complexity and the importance of why. J Pediatr. (2015) 167:229–37. doi: 10.1016/j.jpeds.2015.04.068

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Cohen E, Berry JG, Camacho X, Anderson G, Wodchis W, Guttmann A. Patterns and costs of health care use of children with medical complexity. Pediatrics. (2012) 130:e1463–70. doi: 10.1542/peds.2012-0175

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Gordon JB, Colby HH, Bartelt T, Jablonski D, Krauthoefer ML, Havens P. A tertiary care-primary care partnership model for medically complex and fragile children and youth with special health care needs. Arch Pediatr Adolesc Med. (2007) 161:937–44. doi: 10.1001/archpedi.161.10.937

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Shah NH, Bhansali P, Barber A, Toner K, Kahn M, MacLean M, et al. Children with medical complexity: a web-based multimedia curriculum assessing pediatric residents across North America. Acad Pediatr. (2018) 18:79–85. doi: 10.1016/j.acap.2017.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Bogetz JF, Bogetz AL, Rassbach CE, Gabhart JM, Blankenburg RL. Caring for children with medical complexity: challenges and educational opportunities identified by pediatric residents. Acad Pediatr. (2015) 15:621–5. doi: 10.1016/j.acap.2015.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Nazarian BL, Glader L, Choueiri R, Shipman DL, Sadof M. Identifying what pediatric residents are taught about children and youth with special health care needs and the medical home. Pediatrics. (2010) 126 (Suppl. 3):S183–9. doi: 10.1542/peds.2010-1466O

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Nishisaki A, Hales R, Biagas K, Cheifetz I, Corriveau C, Garber N, et al. A multi-institutional high-fidelity simulation “boot camp” orientation and training program for first year pediatric critical care fellows. Pediatr Crit Care Med. (2009) 10:157–62. doi: 10.1097/PCC.0b013e3181956d29

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Section on Hospital Medicine. Guiding principles for pediatric hospital medicine programs. Pediatrics. (2013) 132:782–6. doi: 10.1542/peds.2013-2269

PubMed Abstract | CrossRef Full Text | Google Scholar

23. RCPCH. Paediatric Nephrology, Level 3, Paediatrics Sub-speciality Syllabus. RCPCH (2020). Available online at: https://www.rcpch.ac.uk/sites/default/files/2018-03/paediatric_nephrology_syllabus_final.pdf (accessed October 12, 2020).

Google Scholar

24. RCPCH. Sub-Specialties. RCPCH (2021). Available online at: https://www.rcpch.ac.uk/education-careers/careers-paediatrics/sub-specialties (accessed January 17, 2021).

25. The American Board of Pediatrics. The American Board of Pediatrics Content Outline. The American Board of Pediatrics (2020). Available online at: https://www.abp.org/content/general-pediatrics-content-outline (accessed December 5, 2020).

26. The American Board of Pediatrics. General Pediatrics Content Outline. The American Board of Pediatrics (2020). Available online at: https://www.abp.org/sites/abp/files/gp_contentoutline_2017.pdf (accessed December 5, 2020).

27. The American Board of Pediatrics. Pediatric Nephrology Content Outline. The American Board of Pediatrics (2020). Available online at: https://www.abp.org/sites/abp/files/pdf/neph_outline_updated.pdf (accessed December 5, 2020).

Google Scholar

28. Ärztekammer AMCÖ. Training content for Pediatrics Adolescent Medicine. (2021). Available online at: https://www.aerztekammer.at/ausbildung-fachaerzte#anlage13 (accessed August 30, 2021).

29. Cha SD, Chisolm DJ, Mahan JD. Essential pediatric hypertension: defining the educational needs of primary care pediatricians. BMC Med Educ. (2014) 14:154. doi: 10.1186/1472-6920-14-154

PubMed Abstract | CrossRef Full Text

30. Kolarik RC, Walker G, Arnold RM. Pediatric resident education in palliative care: a needs assessment. Pediatrics. (2006) 117:1949–54. doi: 10.1542/peds.2005-1111

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Pentiuk S, Baker R. Development of a gastroenterology educational curriculum for pediatric residents using fellows as teachers. J Pediatr Gastroenterol Nutr. (2012) 54:281–4. doi: 10.1097/MPG.0b013e31822cd2b7

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Abbott MB, First LR. Report of colloquium III: challenges for pediatric graduate medical education and how to meet them–a quality improvement approach to innovation in pediatric graduate medical education. Pediatrics. (2009) 123 (Suppl. 1):S22–5. doi: 10.1542/peds.2008-1578G

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Windpessl M, Prischl FC, Prenner A, Vychytil A. Managing hospitalized peritoneal dialysis patients: ten practical points for non-nephrologists. Am J Med. (2021) 134:833–9. doi: 10.1016/j.amjmed.2021.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Saeed F, Wong LP. Managing hospitalized hemodialysis patients: a guide for the non-nephrologist. Hosp Pract. (2015) 43:245–8. doi: 10.1080/21548331.2015.1077094

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Perez-Gomez MV, Bartsch LA, Castillo-Rodriguez E, Fernandez-Prado R, Fernandez-Fernandez B, Martin-Cleary C, et al. Clarifying the concept of chronic kidney disease for non-nephrologists. Clin Kidney J. (2019) 12:258–61. doi: 10.1093/ckj/sfz007

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Foy M, Sperati CJ. What the non-nephrologist needs to know about dialysis. Semin Dial. (2018) 31:183–92. doi: 10.1111/sdi.12671

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Stamm TA, Mattsson M, Mihai C, Stocker J, Binder A, Bauernfeind B, et al. Concepts of functioning and health important to people with systemic sclerosis: a qualitative study in four European countries. Ann Rheum Dis. (2011) 70:1074–9. doi: 10.1136/ard.2010.148767

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Hanson JL, Balmer DF, Giardino AP. Qualitative research methods for medical educators. Acad Pediatr. (2011) 11:375–86. doi: 10.1016/j.acap.2011.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Cieza A, Fayed N, Bickenbach J, Prodinger B. Refinements of the ICF linking rules to strengthen their potential for establishing comparability of health information. Disabil Rehabil. (2019) 41:574–83. doi: 10.3109/09638288.2016.1145258

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Depoy E, Gitlin LN. Introduction to Research: Understanding and Applying Multiple Strategies. Mosby, MO: Elsevier Inc (2005).

Google Scholar

41. Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Thousand Oaks, CA: SAGE Publications (2013).

Google Scholar

42. Lincoln YS, Guba EG, Publishing S. Naturalistic Inquiry. Thousand Oaks, CA: SAGE Publications (1985). doi: 10.1016/0147-1767(85)90062-8

CrossRef Full Text | Google Scholar

43. Curtin M, Fossey E. Appraising the trustworthiness of qualitative studies: guidelines for occupational therapists. Austral Occup Ther J. (2007) 54:88–94. doi: 10.1111/j.1440-1630.2007.00661.x

CrossRef Full Text | Google Scholar

44. Park EA, Chae IS, Jo MN. Importance-performance analysis (IPA) of foodservice operation, dietary life education, and nutrition counseling tasks of nutrition teachers and dietitians in Jeju, Korea. Nutrients. (2017) 9:1157. doi: 10.3390/nu9101157

PubMed Abstract | CrossRef Full Text | Google Scholar

45. SoSci Survey. SoSci Survey GmbH. Munich (2021). Available online at: https://www.soscisurvey.de/ (accessed March 1, 2021).

46. Hanson CS, Gutman T, Craig JC, Bernays S, Raman G, Zhang Y, et al. Identifying important outcomes for young people with CKD and their caregivers: a nominal group technique study. Am J Kidney Dis. (2019) 74:82–94. doi: 10.1053/j.ajkd.2018.12.040

PubMed Abstract | CrossRef Full Text | Google Scholar

47. R Core Team. R: A Language Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing (2020). Available online at: https://wwwR-projectorg/.

Google Scholar

48. Österreichische Ärztekammer. Ausbildungsinhalte der KEF und RZ-V 2015. Österreichische Ärztekammer (2002). Available online at: https://www.aerztekammer.at/ausbildung-fachaerzte#anlage13(accessed September 4, 2021).

49. Craig MS, Garfunkel LC, Baldwin CD, Mann KJ, Moses JM, Co JP, et al. Pediatric resident education in quality improvement (QI): a national survey. Acad Pediatr. (2014) 14:54–61. doi: 10.1016/j.acap.2013.10.004

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Hunt EA, Patel S, Vera K, Shaffner DH, Pronovost PJ. Survey of pediatric resident experiences with resuscitation training and attendance at actual cardiopulmonary arrests. Pediatr Crit Care Med. (2009) 10:96–105. doi: 10.1097/PCC.0b013e3181937170

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Wheeler AP, Delaney M, Fung M, Gorlin J, Kutner JM, Lam JCM, et al. Pediatric resident knowledge of transfusion medicine: results from the BEST-TEST3 international education needs assessment. Transfusion. (2021) 61:2487–95. doi: 10.1111/trf.16439

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Barnett MD, Maurer SH, Wood GJ. Pediatric palliative care pilot curriculum: impact of “pain cards” on resident education. Am J Hosp Palliat Care. (2016) 33:829–33. doi: 10.1177/1049909115590965

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Field MJ, Behrman RE editors. When Children Die: Improving Palliative and End-of-Life Care for Children and Their Families. Washington, DC: National Academies Press (2003).

PubMed Abstract | Google Scholar

54. Hyun DY, Hersh AL, Namtu K, Palazzi DL, Maples HD, Newland JG, et al. Antimicrobial stewardship in pediatrics: how every pediatrician can be a steward. JAMA Pediatr. (2013) 167:859–66. doi: 10.1001/jamapediatrics.2013.2241

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Araujo da Silva AR, Marques A, Di Biase C, Faitanin M, Murni I, Dramowski A, et al. Effectiveness of antimicrobial stewardship programmes in neonatology: a systematic review. Arch Dis Child. (2020) 105:563–8. doi: 10.1136/archdischild-2019-318026

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Jewell JA, Committee Committee on Hospital Care. Standardization of inpatient handoff communication. Pediatrics. (2016) 138:e20162681. doi: 10.1542/peds.2016-2681

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Schmidts M, Liebau MC. Editorial: genetic kidney diseases of childhood. Front Pediatr. (2018) 6:409. doi: 10.3389/fped.2018.00409

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Rosas-Blum E, Shirsat P, Leiner M. Communicating genetic information: a difficult challenge for future pediatricians. BMC Med Educ. (2007) 7:17. doi: 10.1186/1472-6920-7-17

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Ratnapalan S, Ito S. Pediatric resident education and needs assessment in clinical pharmacology. Can J Clin Pharmacol. (2004) 11:e150–5. Available online at: https://jptcp.com/index.php/jptcp/article/view/64

PubMed Abstract | Google Scholar

60. Neveus T, Fonseca E, Franco I, Kawauchi A, Kovacevic L, Nieuwhof-Leppink A, et al. Management and treatment of nocturnal enuresis-an updated standardization document from the international children's continence society. J Pediatr Urol. (2020) 16:10–9. doi: 10.1016/j.jpurol.2019.12.020

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Gabriel GC, Pazour GJ, Lo CW. Congenital heart defects and ciliopathies associated with renal phenotypes. Front Pediatr. (2018) 6:175. doi: 10.3389/fped.2018.00175

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Robinson PD, Shroff RC, Spencer H. Renal complications following lung and heart-lung transplantation. Pediatr Nephrol. (2013) 28:375–86. doi: 10.1007/s00467-012-2200-2

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Emma F, Bertini E, Salviati L, Montini G. Renal involvement in mitochondrial cytopathies. Pediatr Nephrol. (2012) 27:539–50. doi: 10.1007/s00467-011-1926-6

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Nada A, Jetton JG. Pediatric onco-nephrology: time to spread the word : part I: early kidney involvement in children with malignancy. Pediatr Nephrol. (2021) 36:2227–55. doi: 10.1007/s00467-020-04800-3

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Ramani S, Mann K. Introducing medical educators to qualitative study design: twelve tips from inception to completion. Med Teach. (2016) 38:456–63. doi: 10.3109/0142159X.2015.1035244

PubMed Abstract | CrossRef Full Text | Google Scholar