Building Surgical Research Capacity Globally: Efficacy of a Clinical Research Course for Surgeons in Low-Resource Settings

Introduction

A fundamental understanding of the concepts behind research design, implementation, and analysis is increasingly necessary on the part of physicians and surgeons in order to incorporate rapidly developing biomedical knowledge into practice and deliver the highest quality of patient care. Evidence-based medicine [EBM; or practice, as opposed to opinion-based medicine (Straus et al., 2004)] is the judicious and intentional application of the highest quality available medical knowledge, including epidemiology and clinical research, in conjunction with clinical acumen and experience to the treatment of an individual patient (Evidence Based Medicine Working Group, 1992; Sackett et al., 1996; CEBM, 2013).

Clinical research is intimately connected to EBM. The former provides material for the latter, and the practice of seeking data to answer clinical queries reveals when gaps in that material exist, provoking additional relevant research. Certain underlying knowledge and skills are necessary to both appraise and conduct such work (Evidence Based Medicine Working Group, 1992). Working knowledge of study design types, levels of evidence, power calculation, statistical significance, and outcome selection are all relevant to the critical interpretation of literature as well as to study design. Formulation of a clinical question to answer via literature review or for the purpose of study genesis is the same skill applied to two related purposes. The course examined in this study took as its purpose the instruction of health professionals in the conduct of clinical research. Doing so required introducing and emphasizing the aforementioned underlying common principles of EBM. The instruction did not extend to the interpretation and application of evidence-based guidelines to clinical care.

Prior literature has described the lack of knowledge and instruction in EBM in surgical training programs in North America as well as a growing receptiveness to acquire these skills (Poolman et al., 2007; Sprague et al., 2012). This is no less true in resource-scarce regions of the world. International organizations have characterized the need for health research capacity strengthening in low- and moderate-income countries (LMICs) (Commission on Health Research for Development, 1990; Changing Mindsets, 2008). The Global Forum for Health Research described the 10/90 gap, an observation that a mere 10% of the world’s health research budget is devoted to the health problems that affect 90% of the earth’s population, and the Commission on Health Research for Development described the crucial relationship between local health research capacity and sustainable, cost-effective health and development (Commission on Health Research for Development, 1990; Global Forum for Health Research and Davey, 2004).

From an orthopedic perspective, it is clear that musculoskeletal injury is a major contributor to disability and mortality in the developing world. Rates of extremity injury from falls and road traffic crashes are two to five times higher in LMICs than in high-income countries (HICs) (Mock and Cherian, 2008). Disability data consistently demonstrate that LMICs suffer an enormous burden of long-term injury-related disability. One study in Ghana found a 0.83% population prevalence of injury-related disability; 80% of this disability was due to extremity injuries (Mock et al., 2003; Mock and Cherian, 2008). Serious injuries are twice as likely to result in death in the developing world (Mock and Cherian, 2008). These numbers represent avoidable death and disability. Resolving this disparity is a complex problem involving training, resources, infrastructure, and political will. Supporting research capacity is an important part of developing robust global health systems and empowering local health professionals to examine and solve the healthcare challenges that their countries face (Global Forum for Health Research et al, 2009). Given the massive impact of orthopedic injury on the global burden of disease, this is particularly important for the orthopedic community to recognize and act upon at this time.

An ongoing collaboration exists between the Orthopaedic Trauma Institute at the University of California, San Francisco (UCSF) and the Centro de Investigaciones Medico Quirúrgicas (CIMEQ) hospital in Havana, Cuba. Via this collaboration, surgeons determined that despite some degree of infrastructure in Cuba for clinical research and professional interest in carrying it out, there was an inadequate understanding in the Orthopedic Surgical community on how to engage in clinical studies. Although Cuba has medical institutions (i.e., hospitals and clinics) with the capacity to support clinical research and ancillary workers who can assist with study design, data collection, and statistics, their system is not ideal because Cuban orthopedic surgeons do not receive sufficient training to engage clinical research.

In response, faculty from UCSF, CIMEQ, and the Mexican Orthopaedics and Trauma Association [Asociación Mexicana de Ortopedia y Traumatología (AMOT)] collaborated on the design of a one-day clinical research course, to be taught in conjunction with a national course in orthopedic trauma presented at CIMEQ in Havana. The course was designed to impart fundamental skills and knowledge to surgeons on the design, implementation, and publication of original research, employing basic EBM principles. Teaching attendees how to use and interpret EBM in their practice was not a specific course goal, but knowledge of EBM principles will arguably influence clinical practice. The aim of the present study was to evaluate and quantify the efficacy of this course.

Materials and Methods

Course Presentation and Content

During the clinical research course, surgeon researchers from the United States, Canada, Mexico, and Cuba presented lectures on study protocol, surgical research methodology, and navigation of the publication process (Figure 1). Surgeons presented the one-day training immediately prior to a course on orthopedic trauma. All slides were in Spanish, and live interpreters translated for those presenters who lectured in English. In addition, each participant received a 60-page syllabus, also translated into Spanish.

FIGURE 1

Figure 1. Plenary session topics.

Course Evaluation

No existing and validated tool specifically addressed the content of our course, which included instruction on fundamental elements of EBM incorporated into instruction on practical aspects of carrying out clinical research and preparing work for publication. Therefore, established quality guidelines (Shaneyfelt et al., 2006) were used to design our own assessment tool.

Evaluation instruments for education in EBM are diverse. Multiple authors conclude that no one tool would be appropriate to evaluate any proposed EBM training intervention—rather a tool designed to evaluate knowledge acquisition in the domains of intended knowledge gain should be thoughtfully selected (Straus et al., 2004; Shaneyfelt et al., 2006). Straus et al. proposed a conceptual framework for the evaluation of EBM that advised considering a physician’s “mode of practice” of EBM and choosing an evaluation instrument that reflects the different learners, interventions, and outcomes expected (Straus et al., 2004).

A recent review of instruments available for the evaluation of evidence-based practice suggested that the highest quality instruments were characterized by “interrater reliability (if applicable), objective (non-self-reported) outcome measures, and multiple (> = 3) types of established validity evidence (including evidence for discriminative validity)” (Shaneyfelt et al., 2006). Ultimately, an instrument was designed specific to the course, using the quality guidelines established by Shaneyfelt et al. and focusing on fundamental knowledge acquisition, appropriate for a one-day educational event. This was adapted in part from the questionnaire designed by Sprague and colleagues, who described the development and evaluation of a short training course in EBM for a surgeon audience in North America. Using a pre- and post-course questionnaire scheme, they reported a statistically significant increase in knowledge acquired based on relative increase in score as well as participant satisfaction with the training (Sprague et al., 2012). The instrument they employed was fundamentally based on the highly reliable and validated Fresno Test (Ramos et al., 2003; Thomas and Kreptul, 2015). The Fresno Test is a performance-based measure that was designed to evaluate a broad range of EBM knowledge and skills. Sprague and colleagues adapted it to their group’s more narrow goals and requirements, and the instrument was altered to apply to our training event.

By defining the learners (surgeons, physicians, etc.), intervention (education on clinical research and EBM), and the outcome (objective questionnaire results), our tool was consistent with the Straus conceptual model (Straus et al., 2004). A demographic survey, which specifies gender, age, and profession was also included. Once participants were separated into subgroups, it could then be determined which subgroups entered the course more knowledgeable in clinical research and EBM principles than others, and which subgroups benefited the most from the one-day course (as indicated by an increase in test scores). Pre-course and post-course questionnaires were intended to highlight objective (and not self-reported) measures, and the self-reported levels of knowledge were only present on the pre-test. In total, there were 18 multiple choice questions, eight of which referred to a specific clinical scenario case study, and 10 relating to clinical research design and methodology.

Participants received a printed pre-test upon entering the conference area, and these were collected before commencement of proceedings. Post-tests were handed out at the conclusion of the course and collected as participants exited the conference area. The outcome measure was defined as change in number of questions answered correctly on the post-test compared with the number answered correctly on the pre-test; thus, it was objective. Responsive validity was ensured by comparing individual participants’ scores from before and after the intervention. Discriminative validity was maintained by our ability to stratify participants based on their reported level of expertise in EBM and clinical research.

Statistical Analysis

Descriptive statistics were performed using means and proportions. The distribution of scores pre- and post-course were assessed for normality. The paired Student’s t-test was used to assess the difference in means of pre- and post-course scores for statistical significance, assuming an assumption of normal distribution of scores. The data were further analyzed for further trends and hypothesis setting. The data were stratified based on professional level (surgeon, physician, and other health professional) as well as by experience with EBM and by experience in clinical research, rated by participants based on six-point scales. All analyses were conducted using STATA version 12.0 (College Station, TX, USA).

Ethics Statement

This study was submitted to the University of California, San Francisco’s Human Research Protection Program (IRB number: 11-06489) and was certified as exempt from Institutional Review Board consideration under Exempt Category 2: Use of educational tests, surveys, interviews, or observations of public behavior.

As required by this body, participants were provided a document with details regarding risks and benefits of the study. This document informed them that completion of the survey was considered consent to participate. The Human Research Protection Program did not require a signature on this document.

Results

Participant Characteristics

Ninety six of 102 registered attendees (94.1%) completed both the pre-course and post-course questionnaires. Eight participants completed only one of the two instruments and were, therefore, dropped from the analysis. Attendees were asked beforehand to complete the questionnaires only if they planned on staying throughout the entirety of the course. Surveys were collected from 104 total attendees, indicating that at least two other non-registered attendees elected to participate in the questionnaires.

Though the course was aimed at a surgeon audience, professionals from a number of other healthcare disciplines also attended and completed the surveys. Due to the high participation rate among attendees, it is improbable that certain demographic groups were more likely to complete the survey than others. Thus, in terms of who completed the survey, there was little to no selection bias. The mean age of participants was 44.6 years (SD: 11.8 years) and 52% were male. 46% identified themselves as surgeons, 17% as physicians, 23% as other professionals (nurses, physical therapists, etc.), and 15% did not respond to this question (Figure 1; Table 1). Self-reported exposure to EBM and clinical research on the part of this group was low to moderate; 46% of participants judged their EBM experience to be “poor” or “very poor,” and 38% described it as “good” or “average.” Similarly, 30% of participants judged their clinical research knowledge to be “poor” or “very poor,” while 53% described it as “good” or “average.”

TABLE 1

Table 1. Demographics of course participants.

The mean pre-test score was 43.9% (95% CI: 41.1–46.6%). The mean post-test score was 59.3% (95% CI: 56.5–62.1%). Despite the relatively low mean post-test score of 59.3%, there were no clear trends in what questions participants answered correctly or incorrectly. Overall, participants demonstrated a significant increase in post-course questionnaire scores compared with pre-course scores. 68% of participants improved by at least 20% relative their pre-test scores, and 35% of participants improved by at least 50% relative to their pre-test scores.

All professional groups demonstrated relative score increases (Table 2; Figure 2). At 57.9% (CI: 43.1–72.7%), non-surgeon, non-physician professionals showed the highest relative score increase. Physicians showed the lowest relative increase at 31.6% (CI: 10.2–53.0%). Scores increased at all EBM and clinical research experience levels. There were no clear patterns relative to changes in score based on pre-course self-assessment of experience with EBM (Figure 3; Table 3) or clinical research experience (Figure 4; Table 4).

TABLE 2

Table 2. Change in scores from pre-course to post-course questionnaire by profession.

FIGURE 2

Figure 2. Relative increase in scores from pre-course to post-course questionnaire by profession.

FIGURE 3

Figure 3. Relative increase in scores from pre-course to post-course questionnaire by level of self-reported evidence-based medicine (EBM) experience.

TABLE 3

Table 3. Change in scores from pre-course to post-course questionnaire by level of evidence-based medicine (EBM) experience, participant reported.

FIGURE 4

Figure 4. Relative increase in scores from pre-course to post-course questionnaire by level of self-reported clinical research knowledge.

TABLE 4

Table 4. Change in scores from pre-course to post-course questionnaire by level of clinical research knowledge, participant reported.

Discussion

This study of 96 participants in a short course on clinical research in an international setting demonstrated a significant improvement in test of knowledge acquisition. These participants reprsented a range of healthcare disciplines and endorsed a spectrum of levels of experience with EBM and clinical research. The magnitude of increase in score on our evaluation instrument was consistent with significant improvement (Sibley et al., 1982; Sprague et al., 2012). The results of this study suggest that a brief, intensive course presented by surgical faculty on the fundamentals of clinical research using EBM principles can be an effective way to support clinical research capacity in a LMIC setting.

Intended as a pilot project for future courses in Latin America, the aim of quantifying the value of this course was achieved in terms of knowledge acquisition on the subject of clinical research. This finding is consistent with previous work. Sprague et al. demonstrated that participants who took a short course in clinical research skills taught by surgical faculty significantly increased their scores on a test of knowledge acquisition (Sprague et al., 2012). They used a pre- and post-test format to evaluate the efficacy of the Principles and Practice of Family Research Course, a two and one-half day course that was offered to medical students, medical residents, physicians, surgeons, and research coordinators. Medical students and residents in particular appeared to benefit from the course. Argimon-Pallàs et al. (2011) found a significant increase in EBM knowledge and skills among family medicine residents who underwent a short training course on evidence-based practice. Nadler et al. found that a career and research development course presented to Nigerian surgeons had at least short-term efficacy and observed that potential cultural differences were not an impediment to knowledge acquisition (Nadler et al., 2010). Dodani and LaPorte (2008) showed short- and long-term benefits of a relatively more extensive course on research methodology presented in Pakistan.

Strengths and Limitations

The strengths of this study included a design that utilized a pre- and post-course questionnaire. As these were matched for each participant, a more robust statistical analysis was possible. Published guidelines were engaged to develop a high-quality instrument specific to the course content, participants, and structure. Potential contamination was limited by implementing both parts of our instrument on the same day as the course—immediately prior to its start and before participants departed from the final lecture.

Though our intended audience was orthopedic surgeons, many professionals from other health disciplines also registered for and benefited from the course. It would seem that non-physician, non-surgeon participants benefited the most, although their relatively higher change in score is the result of a lower pre-test mean. The intervention appears on the whole to have raised all participants to a common, higher level of understanding of the fundamentals of clinical research as presented in the course.

Limitations to our study include the possibility that an unknown proportion of participants were unable to be present for all portions of the course. In addition, the degree of effort exercised by participants in completing the questionnaires could not be assessed. Also, because of the nature of this course, with distance and logistics as significant barriers, an instrument was not designed to test the knowledge retention of the attendees over time. However, other similarly structured 1-day courses in pediatric life-support and partner abuse suggest that trainees retain increased knowledge for at least 3 to 4 months after the course (Durojaiye and O’Meara, 2002; Madden et al., 2015). Finally, it is unknown whether the gain in knowledge and skills that were measured will translate into an increase in clinical research or improvement in patient outcomes. Other studies have shown that teaching EBM knowledge and research skills can change health care professional behavior (Straus et al., 2005), and international health organizations, including the World Health Organization (WHO), the Council on Health Research for Development, and the Global Forum on Health Research, have consistently emphasized the critical importance of health research capacity development for the success of health systems in LMICs (Commission on Health Research for Development, 1990; Global Forum for Health Research and Davey, 2004; Lansang and Dennis, 2004; Changing Mindsets, 2008).

Application to Global Orthopedic Practice

The lack of quality data on the burden of global orthopedic trauma and the availability of care to manage trauma-related injuries impede the establishment of policies that prevent avoidable morbidity and mortality (Elliott et al., 2015). It is most appropriate that those who live and practice in the affected communities conduct the research, collect the data, and apply the results to their practice (Gosselin et al., 2009).

A 2004 Bulletin of the WHO states: “A combination of short-term and long-term strategies, directed at individual, institutional, and country levels are necessary to develop a sustainable system of health research.” (Lansang and Dennis, 2004). The authors believe that directed coursework is an effective short-term strategy to improve clinical investigation. Following this course, academic and clinical research collaborations developed between the course supporters, including UCSF, CIMEQ, AMOT, and the Osteosynthesis and Trauma Care Foundation (OTCF). It was felt that the course enabled this collaboration, most importantly by improving the understanding of EBM and the ability of the Cuban orthopedic surgeons to access more established and resourced international partners. Subsequent to this course, there are plans to include Cuban surgeons in large prospective studies through McMaster University and the University of California, San Francisco. While not a direct indication of the long-term efficacy of the course, Cuban surgeons who participated in this work have become active members of the recently established international research consortium, Associación de Cirujanos Traumatologicos en las Americas (ACTUAR) that supports clinical research across the Americas (Chomsky-Higgins et al., 2017).

Based on the results of this work and successful experience with this course, the authors plan to offer additional clinical research courses to medical professionals in other countries in Central and South America. Similar results are anticipated, and the attendees of these courses are expected to not only improve their ability to better interpret the existing literature but also engage with local, regional, and international clinical research efforts. Ultimately, the authors hope to empower surgeons to create their own an international Latin American network, where the most relevant questions for their patient populations can be addressed.

Conclusion

Despite the favorable short-term impact of this course, future work will be needed to establish the long-term content retention of this type of course, as well as the impact the information has on the development of clinical research capacity. It would be informative, for example, to know whether attendees of such courses publish more papers, pursue additional further education in clinical research, develop practice patterns that incorporate EBM, or have improved patient outcomes.

Ethics Statement

This study was submitted to the University of California, San Francisco’s Human Research Protection Program (IRB number: 11-06489) and was certified as exempt from Institutional Review Board consideration under Exempt Category 2: Use of educational tests, surveys, interviews, or observations of public behavior. As required by this body, participants were provided a document with details regarding risks and benefits of the study. This document informed them that completion of the survey was considered consent to participate. The Human Research Protection Program did not require a signature on this document.

Author Contributions

TAM and KC-H contributed equally to the development of this article. All authors participated in the discussion and intellectual contribution that was the foundation of the report. TAM, KC-H, and TM wrote the draft manuscript. Each of the authors critically reviewed the content, approved the final version to be published, and agreed to be accountable for all aspects of the work.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

The authors gratefully acknowledge Dr. Nancy Hills of the UCSF Department of Neurology for her assistance with statistical analysis, as well as funding from the Osteosynthesis and Trauma Care (OTC) and Atlantic Philanthropies Foundations to TM to support course content development, faculty travel, and course logistics.

References

Argimon-Pallàs, J. M., Flores-Mateo, G., Jiménez-Villa, J., and Pujol-Ribera, E. (2011). Effectiveness of a short-course in improving knowledge and skills on evidence-based practice. BMC Fam. Pract. 12:64. doi: 10.1186/1471-2296-12-64

PubMed Abstract | CrossRef Full Text | Google Scholar

CEBM. (2013). What Is EBM? Available from: http://www.cebm.net

Google Scholar

Changing Mindsets. (2008). Research Capacity Strengthening in Low and Middle-Income Countries. Geneva: COHRED, Global Forum for Health Research and UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR).

Google Scholar

Chomsky-Higgins, K., Miclau, T. A., Mackechnie, M. C., Aguilar, D., Avila, J. R., Dos Reis, F. B., et al. (2017). Barriers to clinical research in Latin America. Front. Public Health 5:57. doi:10.3389/fpubh.2017.00057

CrossRef Full Text | Google Scholar

Commission on Health Research for Development. (1990). Health Research: Essential Link to Equity in Development. New York, NY: Oxford University Press.

Google Scholar

Dodani, S., and LaPorte, R. E. (2008). Ways to strengthen research capacity in developing countries: effectiveness of a research training workshop in Pakistan. Public Health 122, 578–587. doi:10.1016/j.puhe.2007.09.003

CrossRef Full Text | Google Scholar

Durojaiye, L., and O’Meara, M. (2002). Improvement in resuscitation knowledge after a one-day paediatric life-support course. J. Paediatr. Child Health 38, 241–245. doi:10.1046/j.1440-1754.2002.00773.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Elliott, I. S., Sonshine, D. B., Akhavan, S., Slade Shantz, A., Caldwell, A., Slade Shantz, J., et al. (2015). What factors influence the production of orthopaedic research in East Africa? A qualitative analysis of interviews. Clin. Orthop. Relat. Res. 473, 2120–2130. doi:10.1007/s11999-015-4254-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Evidence Based Medicine Working Group. (1992). Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA 268, 2420–2425. doi:10.1001/jama.1992.03490170092032

CrossRef Full Text | Google Scholar

Global Forum for Health Research, Davey, S. (2004). The 10/90 Report on Health Research, 2003–2004. Geneva, Switzerland: Global Forum for Health Research.

Google Scholar

Global Forum for Health Research, Landriault, E., Blanco, A., and Cruz, N. J. C. (2009). Cuba: Financing of Research and Innovation for Health. Monitoring Financial Flows for Health Research, 2009: Behind the Global Numbers. Geneva, Switzerland: Global Forum for Health Research.

Google Scholar

Gosselin, R. A., Spiegel, D. A., Coughlin, R., and Zirkle, L. G. (2009). Injuries: the neglected burden in developing countries. Bull. World Health Organ. 87, 246–246a. doi:10.2471/BLT.08.052290

CrossRef Full Text | Google Scholar

Lansang, M. A., and Dennis, R. (2004). Building capacity in health research in the developing world. Bull. World Health Organ. 82, 764–770.

Google Scholar

Madden, K., Sprague, S., Petrisor, B. A., Farrokhyar, F., Ghert, M. A., Kirmani, M., et al. (2015). Orthopaedic trainees retain knowledge after a partner abuse course: an education study. Clin. Orthop. Relat. Res. 473, 2415–2422. doi:10.1007/s11999-015-4325-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Mock, C., and Cherian, M. N. (2008). The global burden of musculoskeletal injuries: challenges and solutions. Clin. Orthop. Relat. Res. 466, 2306–2316. doi:10.1007/s11999-008-0416-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Mock, C. N., Boland, E., Acheampong, F., and Adjei, S. (2003). Long-term injury related disability in Ghana. Disabil. Rehabil. 25, 732–741. doi:10.1080/0963828031000090524

CrossRef Full Text | Google Scholar

Nadler, E. P., Krishnaswami, S., Brundage, S. I., Kim, L. T., Kingham, T. P., Olutoye, O. O., et al. (2010). Assessing the efficacy of the fundamentals of research and career development course overseas. J. Surg. Res. 163, 197–200. doi:10.1016/j.jss.2010.03.069

PubMed Abstract | CrossRef Full Text | Google Scholar

Poolman, R. W., Sierevelt, I. N., Farrokhyar, F., Mazel, J. A., Blankevoort, L., and Bhandari, M. (2007). Perceptions and competence in evidence-based medicine: are surgeons getting better? A questionnaire survey of members of the Dutch Orthopaedic Association. J. Bone Joint Surg. Am. 89, 206–215. doi:10.2106/JBJS.F.00633

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramos, K. D., Schafer, S., and Tracz, S. M. (2003). Validation of the Fresno test of competence in evidence based medicine. BMJ 326, 319–321. doi:10.1136/bmj.326.7384.319

PubMed Abstract | CrossRef Full Text | Google Scholar

Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., and Richardson, W. S. (1996). Evidence based medicine: what it is and what it isn’t. BMJ 312, 71–72. doi:10.1136/bmj.312.7023.71

CrossRef Full Text | Google Scholar

Shaneyfelt, T., Baum, K. D., Bell, D., Feldstein, D., Houston, T. K., Kaatz, S., et al. (2006). Instruments for evaluating education in evidence-based practice: a systematic review. JAMA 296, 1116–1127. doi:10.1001/jama.296.9.1116

CrossRef Full Text | Google Scholar

Sibley, J. C., Sackett, D. L., Neufeld, V., Gerrard, B., Rudnick, K. V., and Fraser, W. (1982). A randomized trial of continuing medical education. N. Engl. J. Med. 306, 511–515. doi:10.1056/NEJM198203043060904

PubMed Abstract | CrossRef Full Text | Google Scholar

Sprague, S., Pozdniakova, P., Kaempffer, E., Saccone, M., Schemitsch, E. H., and Bhandari, M. (2012). Principles and practice of clinical research course for surgeons: an evaluation of knowledge transfer and perceptions. Can. J. Surg. 55, 46–52. doi:10.1503/cjs.018610

PubMed Abstract | CrossRef Full Text | Google Scholar

Straus, S. E., Ball, C., Balcombe, N., Sheldon, J., and McAlister, F. A. (2005). Teaching evidence-based medicine skills can change practice in a community hospital. J. Gen. Intern. Med. 20, 340–343. doi:10.1111/j.1525-1497.2005.04045.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Straus, S. E., Green, M. L., Bell, D. S., Badgett, R., Davis, D., Gerrity, M., et al. (2004). Evaluating the teaching of evidence based medicine: conceptual framework. BMJ 329, 1029–1032. doi:10.1136/bmj.329.7473.1029

CrossRef Full Text | Google Scholar

Thomas, R. E., and Kreptul, D. (2015). Systematic review of evidence-based medicine tests for family physician residents. Fam. Med. 47, 101–117.

PubMed Abstract | Google Scholar