A Review of the AAOS International Scholars Program after 14 Years: Professional Development and Education of the Recipients

Introduction

The AAOS International Scholars Program was launched in 2002 as an International Surgical Skills (ISS) scholarship with the objective of providing recipients with short-term educational experience to enhance their knowledge and skills. Ideally, the information gained would be passed onto students and colleagues through cascade training. A secondary goal is to foster sustainable and longstanding relationships between scholarship recipients and both the AAOS and its members. Scholarship recipients attend a surgical skills course of their choice at the Orthopaedic Learning Center, AAOS Headquarters, followed by an observership for 3–5 days at a leading US institution with AAOS volunteers as mentors. An International Emerging Leaders (IEL) Scholarship Program was subsequently launched in 2013 with the goal of further enhancing professional development through additional training/mentorship for a subset of previous international scholarship recipients. IEL scholarship recipients have an opportunity to design their visit to USA based on their professional needs: attending the AAOS Annual Meeting or a meeting of a Specialty Orthopaedic Society, participating in the AAOS Orthopaedic Educators Course, shadowing the Head of Orthopaedic Department in the clinical and administrative responsibilities, meeting with the institutional Orthopaedic Residency Program Directors, etc. AAOS staff helps coordinate the requested program.

The goals of this retrospective review of information collected through scholarship applications and follow-up questionnaires were to (1) characterize the profile of scholarship recipients, (2) evaluate subjective information from questionnaires to indirectly assess their perceived impact of a short-term educational experience on their practice, (3) if/how the knowledge and skills have been disseminated to students and/or colleagues, and (4) to assess career development in that subset of ISS recipients who subsequently applied for the IEL scholarship.

Materials and Methods

The AAOS international scholar database was accessed to review the scholarship applications and all available follow-up assessments. All scholars were asked to fill out a post-course questionnaire (Annex 1), as well as a follow-up questionnaire 6 months or 1 year later (Annex 2), to provide general feedback on the skills course and the observership, assess what knowledge and/or skills were learned and are being utilized in their current practice, evaluate whether the knowledge and skills have been disseminated, and understand the barriers experienced in their practice. We evaluated the responses to selected elements within these questionnaires for the entire group of scholarship recipients, and also divided the group into recipients of only the ISS scholarship and IEL for the purposes of additional analysis. Recognizing that the IEL scholars were all in the ISS pool initially, furthermore, we elected to evaluate the career path of 44 ISS scholarship recipients (2005–2015) who subsequently applied for the IEL program, by comparing their applications (that contained four essays on their practice, achievements, professional goals, and barriers) and resumes at the two points in time based on a professional development index (PDI) as shown in Annex 3. This index was developed by members of our committee and has not been validated.

Results

A total of 103 scholarships were awarded over 14 years. Eleven additional ad hoc scholarships were awarded over these years but are not included in our review, for example, the AAOS-SICOT International Annual Meeting, the AAOS-Institute of Global Orthopaedics and Trauma, or AAOS-SIGN Fracture Care International Awards.

Data concerning practice environment were reviewed from surveys completed immediately after the scholarship (n = 65), and at 6 months or 1 year following the scholarship (n = 75). We were unable to provide quantitative information because both the specific questions asked on the questionnaires and the timing of their distribution evolved over this 14-year period. There is also the challenge of an overlap in responses, since some individuals completed both evaluations, while others completed either one or none of the evaluations. Since the true denominator cannot be calculated in most cases, we will report the number of responses. Our respondents practiced in a public hospital (25), private facility (21), non-governmental (NGO) or charity facility (6), an academic institution (44), or other (5). A subset of individuals practiced at both public and private facilities simultaneously, and some public facilities are also academic institutions. There were 24 professors (Assistant, Associate, and Full), 12 chairmen of their service, division or department, 2 residency program directors, and 6 presidents of National or Specialty Orthopaedic Societies.

There were 182 responses when the scholars were asked whether they had acquired new skills, and the results by anatomic region are as follows: knee (27%), hip or pelvis (13%), spine (12%), shoulder (10%), elbow (5%), and miscellaneous (23%). The most common procedures at the knee related to total knee arthroplasty and ligamentous injury. Most of the spinal procedures involved the cervical spine, especially upper cervical instrumentation techniques. Selected responses that could not be grouped by anatomic region include learning new approaches for surgical procedures, organization of the office or operating room, patient education, patient safety, postoperative pain management, avoiding complications, electronic medical record, evidence-based medicine, and multimodal techniques to reduce blood loss.

A number of scholars shared their knowledge and skills with others through conferences/lectures (32), clinical training (28), hands on training (32), individual discussions (26), and teaching residents and students (31). When asked to estimate how many individuals received this knowledge, responses were as follows: 1–10 (3), 11–20 (13), 21–30 (11), 30–40 (14), 41–50 (8), and more than 50 (12). If we assume the upper numbers within each range, we would estimate that at the time of evaluation up to 2,300 health care professionals had been exposed to the knowledge brought back by scholars.

Participants were also asked whether they implemented any changes in their practice in response to the scholarship experience. A total of 79 comments were recorded and may be best categorized according to surgical skills (66%), practice management (16%), clinical skills (16%), and skills used in teaching (1%). The surgical skills described include surgical approaches, revision total joint arthroplasty, confidence with procedures, and minimally invasive procedures. One scholar commented that he “started anterior cervical discectomy and fusion at my hospital.” Clinical skills included decision-making, preoperative planning and recognizing a greater number of options for treatment. Comments related to practice management included “patient education and discussion time with patients,” “better workplace organization,” and development of protocols for patient care.

Data concerning the impact of scholarship on their career development were available for 58 scholars, and 52 (90%) felt that the scholarship had a substantial impact on their practice and career through clinical decision-making (49), diagnostic skills (41), specific techniques and procedures (48), new protocols (47), and strategies for patient education (38).

In terms of professional development for the ISS recipients who applied for the IEL scholarship (N = 44), we found that 66% were working purely in the public sector, with 25% working in both the public and private sector, and 9% working exclusively in the private sector (N = 44). Almost half of applicants (48%) were promoted academically, and 55% were assigned to leadership positions within their institutions. The majority of the IEL applicants, 40 of 44 (91%), were active in professional societies, many were engaged in education and research, and have maintained some contact with their hosts and/or the AAOS. They are active in community service and often shared their knowledge, especially with residents (66%) (Table 1). Overall, our average PDI (Annex 3) was 6.5 (range 3–14), with the maximum being 18.

TABLE 1

Table 1. Professional development: academic and community service (44 respondents).

We also sought to assess any barriers experienced in their practice of orthopedic surgery, and the results are also illustrated in Table 1. Interestingly, a pushback from administration was cited by the greatest percentage of respondents (66%), followed by inadequate financing from the hospital or government (30%), lack of knowledge/skills (27%), and lack of equipment (23%). A much higher percentage of barriers were cited by those ISS scholars who subsequently applied for the IEL scholarship versus the ISS recipients from 2010 to 2014. In addition, we noted that for the ISS recipients, most of the barriers were reported by more senior scholars with >5 years in practice.

With regard to the career path of 44 ISS scholarship recipients (2005–2015) who subsequently applied to the IEL program, the time lapse between the initial scholarship and the second application ranged from 1 to 11 years, with the average being 4–5 years. We found that the recipients had an average PDI score (Annex 3) of 9.3 (n = 7) versus 5.9 (n = 37) for those who applied but did not receive the IEL scholarship. In a more focal analysis, we found that the PDI for ISS recipients was 6.5 versus 2.1 for non-recipients in 2011. The PDI for the ISS recipients and applicants was 4.5 in 2010 and 4.3 in 2011 versus 7.2 for the IEL recipients (2010–2011).

Discussion

The burden of musculoskeletal disease is enormous, and there are deficiencies in the provision of orthopedic services throughout the world, especially in low- and middle-income countries. One of the many challenges associated with improving access to safe and timely orthopedic surgical care has been the education and training of health providers. In recent years, there have been improvements in access to information, and in the ability of orthopedic surgeons around the globe to communicate effectively, and become “connected” through the internet and improvements in communication technologies. This has fostered the development of educational exchanges and/or partnerships between individuals, institutions, and professional societies, leading to better services for patients. Recognizing that orthopedic surgery in economically developed nations is often reliant on complex and expensive technologies and technical solutions to individual patient problems will depend on the local context, the general principles of management apply uniformly.

In an effort to enhance the knowledge and skills of orthopedic surgeons in other countries, especially those practicing in resource challenged areas, the AAOS embarked upon an educational program in 2002 in which international scholars attend a skills course at the AAOS headquarters, followed by an observership with an AAOS mentor. The AAOS International Scholars Program is generously funded by the contributions to the Scholarship Fund. The contributors include certain International Orthopaedic Associations, Orthopaedic Foundations, USA Specialty Orthopaedic Societies, hospitals, orthopedic practices, and orthopedic surgeons. The full list of contributors is available on the international section of the AAOS website. Over these 14 years, a total of 805 applications were received from 76 countries, and 103 (13%) scholarships were awarded to recipients from 46 countries. The number of applications increased steadily over the years from 49 (2002) to 139 (2016), with a maximum of 186 in 2015. All recipients have come from low- and middle-income countries, 16% from the lowest income bracket.

The purpose of this study was to characterize the individuals who had received the scholarships and to evaluate whether this short-term educational experience had an impact on their practice and career development. Our findings suggest that interest in the program has increased substantially over the years based on the number of applications and that our recipients have mainly practiced in an academic environment, have passed on their knowledge and skills to medical students, residents, and colleagues alike, and have often progressed significantly in their academic careers. While we cannot establish any causal relationship between attending a short-term educational course and/or an observership with an individual’s career development, it is extremely likely that the AAOS scholarship opportunity has attracted a group of highly motivated and talented individuals who are or will become leaders in their local orthopedic communities. As noted in the Section “Results,” several AAOS scholarship recipients have become chairman of their department, held offices in professional societies, and become recognized nationally or internationally for their research. These emerging leaders have generally kept in touch with colleagues from the AAOS and some have gone on to become members of AAOS. Their dedication to community service including care for the underprivileged has been inspiring. Finally, several scholars have made truly outstanding professional career achievements. For example, one scholar served as the Minister of Health of his country. Another scholar was elected to the Parliament and “will participate in leading the nation and the health sector of my country.” He stated that the AAOS Emerging Leaders Scholarship program was “an eye-opener for me. It helped me think broader and aim higher.”

We recognize that there are several limitations to our study. Once again, we cannot establish causality in assessing the impact of the short-term educational course on any individual’s career. Also, there are always concerns regarding the scientific value of non-validated questionnaires. The specific questions and the timing of distribution of these questionnaires evolved over this 14-year period. There may be recall bias, and individuals may not be able to provide reliable quantitative responses to some of our questions. Not all individuals responded to both questionnaires. A questionnaire with open-ended questions would certainly have provided more scientifically valid conclusion regarding barriers faced. There may also have been a greater likelihood that our scholars would avoid negative responses, considering they had been awarded a scholarship. We also recognize that we have no comparison group for career path, and it is possible that there would be no difference between recipients and non-recipients with regard to academic and professional activities.

From a broader perspective, short-course style learning experiences are becoming more common in medical and surgical education given demands on time, resources, and the complicated ethics of health-care education (Patel et al., 2008; Bansal et al., 2012). The effectiveness and maintenance of knowledge and skills garnered from a short-course format learning experience has been challenged in the literature (Mackay, 2006; Malling et al., 2009). On the contrary, there is substantial literature that supports the use of short-course format for medical and surgical primary education as well as continuing medical education (Morino et al., 1995; Keleher et al., 2005; Cheifetz and Phang, 2006; Patel et al., 2008; Scott et al., 2008; Devine et al., 2009; Bansal et al., 2012; Wilkins et al., 2015). There is objective and subjective knowledge retention and maintenance of skills at follow-up dates as long as 17 months (Keleher et al., 2005; Cheifetz and Phang, 2006; Scott et al., 2008; Devine et al., 2009; Bansal et al., 2012; Wilkins et al., 2015). There is evidence that maintenance of surgical skills requires continuing use of those skills garnered (Patel et al., 2008). In several studies, the short-course format was responsible for widely disseminating a higher quality of health care to a much larger population (Morino et al., 1995; Keleher et al., 2005; Devine et al., 2009; Wilkins et al., 2015). Our impression from reviewing information provided by our recipients is that a significant number of individuals were at least exposed, formally or informally, to the knowledge and/or skills presented in these short-term educational experiences, although we cannot make any conclusions as to whether the information changed their practice or stimulated them to seek additional training and/or knowledge which altered their practice. Most recipients came from practice environments where they can readily communicate new information with peers and trainees alike.

In summary, although difficult to prove objectively based on the data available, our impression is that the AAOS international scholars program and the IEL program have been successful in transferring knowledge and skills to recipients who have subsequently changed their practice and shared the information with many peers and trainees. Based on the available data, it is likely that those selected will play an important role in improving the development of the orthopedic knowledge and skills in their home countries. Professional connections have been established, which will likely lead to sustainable relationships between individual surgeons and between surgeons and AAOS, which will hopefully continue to solidify in the future. These relationships may be especially important for those practicing in resource challenged environments where access to educational materials and opportunities for a dialog with content experts may be limited. We hope that this scholarship program will attract the necessary resources to expand, leading to an even greater connectivity between the AAOS and its surgeons with our colleagues throughout the world. Further study will be required to gain a more detailed understanding of the transfer of knowledge and skills and the impact of the AAOS International Surgical Skills and Emerging Leaders scholarship programs.

Author Contributions

AG: came up with the concept/study design, collected the data, and wrote the first draft of the manuscript. DS: study design, data analysis, helped with writing first draft, and provided review of the manuscript. MC: study design, data analysis, wrote part of the Section “Discussion,” and critical review of the manuscript. SS: helped analyze the data, provided critical review of the manuscript. WS: came up with the concept, helped analyze the data, and provided critical review of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer, MB, and handling editor declared their shared affiliation, and the handling editor states that the process nevertheless met the standards of a fair and objective review.

Funding

The AAOS International Scholars Program is generously funded by the contributions to the Scholarship Fund. The contributors include International Orthopaedic Associations, Orthopaedic Foundations, USA Specialty Orthopaedic Societies, hospitals, orthopedic practices, and orthopedic surgeons. The full list of contributors is available on the international section of the AAOS website.

References

Bansal, V. K., Panwar, R., Misra, M., Bhattacharjee, H. K., Jindal, H., Loli, A., et al. (2012). Are short-term focused training courses on a phantom model using porcine gall bladder useful for trainees in acquiring basic laparoscopic skills? Surg. Laparaosc. Endosc. Percutan. Tech. 22, 154–160. doi: 10.1097/SLE.0b013e3182478e6c

CrossRef Full Text | Google Scholar

Cheifetz, R. E., and Phang, P. T. (2006). Evaluating leaning and knowledge retention after continuing medical education course on total mesorectal excision for surgeons. Am. J. Surg. 191, 687–690. doi:10.1016/j.amjsurg.2006.01.043

CrossRef Full Text | Google Scholar

Devine, S. G., Llewellyn-Jones, L., and Lloyd, J. (2009). Impact of a five-day short course on integration of health promotion into practice in north Queensland. Health Promot. J. Austr. 20, 69–71.

Google Scholar

Keleher, H., Round, R., Marshall, B., and Murphy, B. (2005). Impact evaluation of a five-day short-course in health promotion: workforce development in action. Health Promot. J. Austr. 16, 2.

Google Scholar

Mackay, S. J. (2006). The impact of a short course of study on the performance of radiographers when highlighting fractures on trauma radiographs: “The Red Dot System”. Br. J. Radiol. 79, 468–472. doi:10.1259/bjr/53513558

CrossRef Full Text | Google Scholar

Malling, B., Mortensen, L., Bonderup, T., Scherpbier, A., and Ringsted, C. (2009). Combining a leadership course and multi-source feedback has no effect on leadership skills of leaders in postgraduate medical education. An intervention study with a control group. BMC Med. Educ. 9:72. doi:10.1186/1472-6920-9-72

PubMed Abstract | CrossRef Full Text | Google Scholar

Morino, M., Festa, V., and Garrone, C. (1995). Survey on Torino courses: the impact of a two-day practical course on apprenticeship and diffusion of laparoscopic cholecystectomy in Italy. Surg. Endosc. 9, 46–48. doi:10.1007/BF00187884

PubMed Abstract | CrossRef Full Text | Google Scholar

Patel, M., Oosthuizen, G., Child, S., and Windsor, J. (2008). Training effect of skills courses on confidence of junior doctors performing clinical procedures. N. Z. Med. J. 121, 37–45.

PubMed Abstract | Google Scholar

Scott, T. M., Hameed, S. M., Evans, D. C., Simons, R. K., and Sidhu, R. S. (2008). Objective assessment of surgical decision making in trauma after a laboratory-based course: durability of cognitive skills. Am. J. Surg. 195, 599–603. doi:10.1016/j.amjsurg.2007.12.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilkins, A., Lobo, R., Griffin, D., and Woods, H. (2015). Evolution of health promotion training for the Western Australian Aboriginal maternal and child health sector. Health Promot. J. Austr. 26, 57–63. doi:10.1071/HE14032

CrossRef Full Text | Google Scholar

Appendix

Annex 1. AAOS International Surgical Skills Scholarship Post-Program Evaluation

1. Please use the space below to provide general feedback regarding the program.

2. How helpful were the logistics information in preparing you to participate in the scholarship program? On a scale of 1–5 where 1 = not useful, 2 = somewhat useful, 3 = useful, 4 = very useful, and 5 = extremely useful.

	1	2	3	4	5
Logistics information	□	□	□	□	□

3. Please rate the overall OLC course:
On a scale of 1–5 where 1 = poor, 2 = below average, 3 = average, 4 = above average,. and 5 = excellent.

	1	2	3	4	5
Quality of presentations	□	□	□	□	□
Value of information	□	□	□	□	□
Applicability to your practice	□	□	□	□	□

4. Please rate the overall observership:
On a scale of 1–5 where 1 = poor, 2 = below average, 3 = average, 4 = above average, and 5 = excellent.

	1	2	3	4	5
Knowledge of the doctors	□	□	□	□	□
Quality of the presentations	□	□	□	□	□
Value of information	□	□	□	□	□
Applicability to your practice	□	□	□	□	□
Time allocated	□	□	□	□	□

5. How well did the program (OLC course and observership) meet your learning objectives? On a scale of 1–5 where 1 = poor, 2 = below average, 3 = average, 4 = above average, and 5 = excellent.

1	2	3	4	5
□	□	□	□	□

6. From your participation in this program, what new information or procedures did you learn?

□ New protocols	□ Clinical decision-making
□ Diagnostic skills	□ Specific techniques and procedures
□ Patient education strategies	____________________________________

7. Based on your participation in this program, what is one thing you will change in your practice/hospital?

8. In your opinion, how could AAOS improve the scholarship program?

9. Please list any social media platforms you currently use on a regular basis.

□ LinkedIn	□ Twitter
□ Facebook	□ Other (please list) _____________________
□ Blogs

Annex 2. AAOS International Surgical Skills Scholarship 6-Month Evaluation

1. Please use the space below to provide general feedback regarding the program.

2. Impact of AAOS Scholarship program on your career:

□ Significant

□ Moderate

□ No impact

3. To what extent has your participation in this program impacted the following?
On a scale of 1–5 where 1 = not at all, 2 = a little, 3 = somewhat, 4 = moderately, and 5 = a lot.

Skills	1	2	3	4	5
Specific techniques and procedures	□	□	□	□	□
New protocols	□	□	□	□	□
Diagnostic skills	□	□	□	□	□
Patient education strategies	□	□	□	□	□
Clinical decision-making	□	□	□	□	□
Other	□	□	□	□	□

4. What specific techniques learned have proven most useful in your practice?

5. In the last 6 months, with whom have you shared the information you gained from the program?

□ Orthopedic surgeons	□ Staff
□ Health-care professionals	□ Other _____________________________
□ Residents	□ I have been unable to disseminate any information. Please explain
□ Medical students	_________________________

6. How many people did you share this information with?

□ 1–10	□ 30–40
□ 10–20	□ 40–50
□ 20–30	□ 50+

7. Were you able to accomplish the one change that you specified in your post-program evaluation?

8. Of the knowledge gained from the program, what have you been able to incorporate into your practice to accomplish change at your hospital?

□ Clinical skills	□ Diagnostic skills
□ Operative skills	□ I have been unable to incorporate any knowledge gained through the program
□ Skills used in teaching	Please explain _____________________________

9. Do you participate in the AAOS International Scholarship Facebook page? If not, please explain.

Annex 3. Professional Development Index (PDI) Scale

Academic rank: score = (1–3)

1: Promotion to a higher academic rank—associate professor

2: Promotion to a higher academic rank—full professor

3: Appointment to a leadership position in hospital (director, Head of Department, service)

Research presentations: score = (1–3)

1: Presentations/research at local, national or international meetings (1–10)

2: Presentations/research at local, national or international meetings (11–20)

3: Presentations/research at local, national or international meetings (>20)

Research publications: score = (1–3)

1: Publications in peer reviewed journals/venues (1–10)

2: Publications in peer reviewed journals/venues (11–20)

3: Publications in peer reviewed journals/venues (>20)

Book chapters: score = (1–2)

1: 1–4 chapters

2: 5 or more chapters

Membership in professional societies: score = (1–3)

1: One to three memberships

2: >3 memberships

3: Election to a leadership position (President, V-President, Board Member/Chair, etc.)

Community service (1–4)

1: Volunteer in disaster/war areas

2: Community service: treating patients in underserved communities/charity clinics

3: Community service: volunteering in professional/community organizations

4: Community service: other (please list in comments)

Total score (maximum = 18)