Evaluating Clinical Educators' Competence in an East Asian Context: Who Values What?

Background How to evaluate clinical educators is an important question in faculty development. The issue of who are best placed to evaluate their performance is also critical. However, the whos and the hows of clinical educator evaluation may differ culturally. This study aims to understand what comprises suitable evaluation criteria, alongside who is best placed to undertake the evaluation of clinical educators in medicine within an East Asian culture: specifically Taiwan. Methods An 84-item web-based questionnaire was created based on a literature review and medical educational experts' opinions focusing on potential raters (i.e., who) and domains (i.e., what) for evaluating clinical educators. Using purposive sampling, we sent 500 questionnaires to clinical educators, residents, Post-Graduate Year Trainees (PGYs), Year-4~6/Year-7 medical students (M4~6/M7) and nurses. Results We received 258 respondents with 52% response rate. All groups, except nurses, chose “teaching ability” as the most important domain. This contrasts with research from Western contexts that highlights role modeling, leadership and enthusiasm. The clinical educators and nurses have the same choices of the top five items in the “personal qualities” domain, but different choices in “assessment ability” and “curriculum planning” domains. The best fit rater groups for evaluating clinical educators were educators themselves and PGYs. Conclusions There may well be specific suitable domains and populations for evaluating clinical educators' competence in East Asian culture contexts. Further research in these contexts is required to examine the reach of these findings.


INTRODUCTION
Teaching effectiveness and faculty development are central to healthcare professional excellence (1,2). Indeed, clinical educators are asked to play numerous roles in relation to their students. For example, Harden and Crosby identify twelve roles of clinical teachers in medical education accross six areas: the information provider in the lecture, the role model on the clinical job, the facilitator as a mentor, the assessor of learners, the curriculum planner, and the resource developer (3). Thus, in medical education, clinical educators focus not only on educating students in terms of their medical knowledge, but also on the learners' learning state, living conditions and emotional state. Indeed, such psychosocial support includes counseling, friendship, acceptance, confirmation and rolemodeling (4,5). Due to the multiple functions of clinical educators, adequate evaluation of their performance is a challenging and complex task.

What Competencies Need to Be Evaluated in Our Clinical Educators?
In 2010, a systemic review of over 30 years' research of current questionnaires for evaluating clinical educators identified 54 manuscripts focusing on 32 different instruments from predominately Western (mainly USA) cultures (6). The authors identified the four most common foci for the evaluation of clinical educators that comprised (in order of frequency): teacher (i.e., mapping onto the information provider in Harden and Crosby's work) and supporter (i.e., Harden and Crosby's mentoring role); role model; and feedback provider (i.e., relating to aspects of Harden and Crosby's assessor role). Planner and assessor roles were less common across intruments/items (featuring in 18 and 5 instruments respectively). The majority of manuscripts cited literature around the concept of good clinical teaching, but no clear description of what makes a good clinical educator is offered. Furthermore, the authors highlight that doctors' competencies "as communicators, collaborators, health advocates, and managers" are missing elements across these evaluation tools (6). More recently, however, within the health professions education literature there has been an attempt to develop clinical educator evaluation tools that variously cover aspects such as: teaching skills (7,8), establishing learning climates (9), curriculum planning (10), communicating with learners (9), providing feedback (7,8), assessing learners (9,10), promoting self-directed learning (7,10), demonstrating educational goals (10), showing responsibility (10), integrating learning into the clinical setting (9,10), personal support (7). Many of the above abilities have been showed in the medical competencies described by the Canadian Medical Educational Directives (CanMEDS). It is reasonable that good clinical educators may be expected to act as role models of these competencies (6,8).

Who Are Best Placed to Evaluate Clinical Educators in Medical Education?
In their review of the literature, Fluit et al. (6) identified medical students as the most common raters of clinical educators (56% of the time). Indeed, some tools have been specifically developed for this purpose (11)(12)(13). It is easy to understand how medical students can be thought of as being the most suitable candidates for this task given that they are the recipients of education, and can observe the process directly. However, the power relationship between students and educators may interfere with the evaluation process. The crux of the issue is that educators have the power to fail students. As such, students may be reluctant to offend their educators as it might put them at risk of failure (14)(15)(16)(17)(18).
In addition to students, program coordinators also commonly evaluate clinical educators (19). While it is reasonable that program coordinators might score the clinical educators they assign, they may not have many opportunities to observe clinical educators' performances directly. In addition to students and program coordinators, clinical educators have been evaluated by their peers (2,20,21). Fellow educators may be able to observe one another during clinical teaching, or gather feedback about other educators from students. Finally, self-evaluation, a reflective process, has also been considered in several educator evaluation studies (2,(20)(21)(22).
Irrespective of whom is being evaluated and who undertakes the evaluation, the issue of the specific culture in which evaluation occurs is of importance. Here we focus on national culture (rather than local or organizational cultures). In East Asian cultures, such as Taiwan, China, Hong Kong, and Singapore, students are brought up to respect education and their educators (23)(24)(25)(26). This impacts not only students' evaluation of their clinical educators, but also the evaluation of educators by others who are not afforded such reverence (e.g., program coordinators). As such this reverence for educators in general is likely to have an impact on if and how students, and other groups, might evaluate them. Furthermore, in certain contexts and cultures, competition between clinical educators may create a conflict of interest situation (27). Indeed, the literature on clinical educators' evaluation in the medical domain so far appears to come predominately from a Western perspective, with educators' accountability to students having been high on the agenda for a number of years, leading to a variety of evaluation contexts and processes (28-30). Given the different foci between an East Asian perspective and a Western perspective, an understanding of whether Taiwanese students' evaluation of their clinical educators differs from that of Western students would be of great interest; particularly due to the sharp rise in the internationalization of education in general (31) and healthcare professionals more specifically (32).
To date, research has studied evaluation systems available to examine the teaching quality of clinical educators (1,7). However, there is no single, agreed source for this evaluation nor tool with which to evaluate all aspects of the educators' role: observing educators' effectiveness on different aspects of their role, and by a variety of raters, might be considered a more appropriate way to achieve an holistic view of performance (21,33). Further, by integrating various subjective observations, we can begin to form a relatively objective evaluation of them. The aim of this study is to find out the suitable evaluation domains (what) and the appropriate raters (who) for evaluating clinical educators' competence in the context of medical education an East Asian culture: specifically Taiwan.

Study Setting
The study was conducted in the largest teaching hospital in Taiwan: a 3,000-bed Medical Center in the north of the country. There were approximately 920 clinical educators, 600 residents, 130 post graduate year physicians (PGY), 200 year-7 medical students (M7), 150 year-4, year-5, and year-6 medical students (M4-6), and at least 3,000 nurses in this institution during the study period.

Ethical Approval
This study was approved by the Chang Gung Medical Foundation Institutional Review Board (REF: 103-1928B, anonymised to protect participants). Participants consented and were notified that they had the right to withdraw from the study at any time. They were also informed that they could omit any questions they did not want to respond to without consequences to their studies or work.

Participants
A purposive sampling technique was used with the numbers of invitations being stratified by staffing ratios at the hospital. We invited a total of 500 particiants by sending questionnaires to 184 clinical educators, 120 residents, 26 PGYs, 40 M7, 30 M4-6, and 100 nurses at the study institution: All of whom should have had experiences of providing, witbessing or receiving clincal education in a hospital setting. The participants were invited by e-mail and were requested to complete the questionnaire online.

Research Design
A quantitative cross-sectional survey design was used. The research questions (RQs) of this study were: (RQ1) Which evaluation domains for the role of clinical educator do different stakeholder groups prioritize? (RQ2) Which study population(s) do different stakeholder groups identify as being most suitable for the evaluation of clinical educators?

Questionnaire Development
Drawing on our review of the literature (including: 1, 4, 5, 14, 21, 34-36), and our understanding of the educational environment of the institution, we itemized potential raters (i.e., who) and potential domains (i.e., what) for evaluating clinical educators. Five healthcare professional experts, including three physician educators, one nursing educator, and one questionnaire development expert at the study institution participated in a Delphi procedure (37,38) to refine and validate the contents for the questionnaire. They exchanged their ideas under the condition of complete anonymity with the e-mail organized by the research assisstant. Ultimately, with the four rounds of written discussion, this process resulted in an 84item web-based questionnaire with which we used to gather participants' opinions.
The questionnaire was divided into two parts. The first part, comprising 70 items and five domains focused on what we need to evaluate: teaching ability (16 items) (34,(39)(40)(41)(42)(43), assessment ability (12 items) (44)(45)(46), personal qualities (16 items) (4,5,40,43,47,48), interpersonal relationships (14 items) (1,2,36,43), and curriculum planning (12 items) (1,22,43,49). The participants were asked to give a score to the question "this item is suitable to be evaluated" according to each domain. The second part, comprised 10 items, and focused on who is best placed to be the rater. The following possible rater groups, identified by evaluation experts in the study institution, were: clinical educator self-evaluation, clinical educators' peers, PGYs, residents, medical students, nurses, education associated administrative staff, outpatient services staff, and the clinical educators' supervisors. In this section, participants were asked to give a score for each of the potential rater group that is best placed to evaluate the clinical educators. To avoid a neutral option for respondents, both parts used the same 4-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = agree, 4 = strongly agree). The anonymous questionnaire collected participants' age, gender, occupation, learning and teaching experience in years.

Data Analysis
The quantitative responses were analyzed using SPSS 19.0 for Windows software package. For the descriptive statistics, we present the means ± standard deviations. Comparisons among the different groups were analyzed via a one-way analysis of variance (ANOVA) using the Scheffe test for post hoc analysis (p < 0.05).

Which Evaluation Domains Are Prioritized?
All respondent groups, except nurses, agreed that teaching ability was the most important domain. We applied ANOVA test to evaluate the scores of teaching ability from the different groups showed a significant difference [F (92) = 0.838, p = 0.004]. The post hoc Scheffe test showed a significant difference between the clinical educator and the nurse groups for teaching ability (p = 0.016). The clinical educator, resident, M7 and M4-6 groups chose assessment ability as the least important domain. The M7 and M4-6 groups prioritized the five domains in an identical pattern to each other, with the clinical educator and resident groups prioritizing three of the five domains identically. The PGY group placed curriculum planning as the least important domain, although this was placed as the most important for the nurse group ( Table 2).
We undertook an in-depth comparison between the clinical educator and nurse groups due to the stark differences in their initial ranking of domains. When comparing the top five ranked items across all domains for these two groups, we can see that both groups of respondents selected identical items as their number one priority across all domains (Table 3). Additionally, for the domain of personal qualities, both groups selected the same top five items, albeit in a slightly different order. For the domains of teaching ability and interpersonal relationships, both groups prioritized 4/5 identical items in their top five: interestingly, in terms of interpersonal relationships, respondents from the nursing group only prioritized demonstrates good interprofessionalism, whereas respondents from the clinical educator group only prioritized demonstrates mutual respect (between themselves and learners). For the domain of assessment ability, the nurse group prioritized uses assessment techniques well and good at formative assessment, whereas the clinical educator group focused on demonstrates good assessemt for clinical skills and clinical reasoning. For the domain of curriculum planning, the nurse group prioritized applies teaching resources effectively and demonstrates good curriculum management, whereas the clinical educator group focused more on demonstrates good time management and educates in accordance with the training program and schedule.

Which Group (s) Are Best Placed for Evaluating Clinical Educators?
We now turn our attentions to the who question: namely which stakeholder group respondents identify as being the most appropriate to undertake evaluation of clinical educators. Our data illustrates that clinical educators' supervisors, clinical educators themselves (so, self-evaluation), clinical educators' peers, residents, PGYs, M7, M4-6, and nurses featured in the top 5 suitable rater sources for each participant group. All six participant groups identified self-evaluation by clinical educators and PGYs in the top five of suitable rater sources ( Table 4).

DISCUSSION
This study investigated the suitable content and raters for doctors in their roles as clinical educators. In terms of content, Fluit et al. (6) identified role model, teacher, feedback provider, and supporter being the most common aspects of existing tools, with assessor and planner roles being less common. In our study, we identified the roles of assessor and planner as key factors. With the current international movement toward competencybased training programmes, and East Asia being no exception, the ability to assess the development of students' and trainees' knowledge, skills and practice is becoming a crucial component of the clinical educators' role. Clinical educators' abilities to plan learning activities in busy clinical environments, creating and protecting occasions for the execution of relevant clinical activities, provides a necessary structure for educators and learners. As such the planning role is important for clinical educators, and one for which they can be evaluated.
We found that clinical educators and medical learners prioritize educators' skills over other aspects of the clinical teaching role, including their personal qualities (e.g., enthusiasm, leadership, empathy), which were often ranked below interpersonal relationships by the learners. Indeed, both the clinical educators and medical learners in this study paid attention to the "learning process" and the "learning outcome" when thinking about eudcator evaluation. This contrasts with studies from Western cultural contexts which has found that clinical educators as role models, in which personal aspects such as leadership and enthusiasm are central components (50)(51)(52), and is more compatible with the results of studies that focus on the effectiveness of teaching (20,35). Our findings show that both clinical educators and learners ranked the five domains in a similar order: teaching ability rated as highest priority, assessment ability as the lowest (or very low). However, we believe that the rationale for each groups' ranking might be different. Thus, for the educators, their low ranking of assessment ability might be due to the relatively low confidence that clinical educators have in their assessment role (53). Indeed, clinical educators sometimes pass underperformance in their students (known as failing to fail): a difficult process tied up with educators' beliefs around self-efficacy (i.e., whether they have the ability to fail the student), their own knowledge and skills (e.g., whether they taught the student correct information, and in an appropriate way) as well as organizational constraints (the consequences of failing students on the organization, the program or fellow educators (53)(54)(55)(56). Furthermore, the process of assessment and feedback can sometimes have a negative emotional impact on the educator (15). As such, these aspects of assessment behaviors might lead clinical educators to rate this domain as the lowest priority. As for students, their rationale for rating assessment ability as the lowest priority may be linked to assessment anxiety. This is supported in a recent systematic review examining assessment and psychological distress among medical students, which found that irrespective of the type of assessment involved or the level of learner, overwhelmingly assessment invokes stress or anxiety (57). As such, we might expect that students would display a reluctance to prioritize this aspect of their teachers' role.
In Stein et al.'s study (20) examining teaching effectiveness, different health professions prioritized different aspects of effective teaching according to their own professional culture. We also see this patterning whereby respondents in the nursing group ranked the clinical educator domains differently to respondents in the medical educator or learner groups, most notably for the domains of curriculum planning and assessment ability and teaching ability: which they ranked #1, #2, and #5 respectively, and which others ranked #3-5, #4-5, and #1 respectively. This stark difference in ranking might reflect differences in professional culture and focus alongside professional distance. Thus, in the Taiwanese culture in which this study was conducted, nurse education is highly prioritized and nurse education research comprises a high degree of teaching and curricula innovation evaluations (58)(59)(60)(61). This may potentially explain why nurses prioritized the curriculum planning as their numberone domain. That nurses prioritize assessment more than do clinical educators and learners is likely to be due to the fact that they are far removed from the direct assessor-assessee relationship (62)(63)(64), and when assessing the learners they do so as part of a group and so the sole responsibility of passing or failing them does not primarily fall on their shoulders.
We undertook an additional comparison between the clinical educator and nurse groups based on their stark differences in opinions around the importance domains for clinical educator evaluations. We found that both groups selected enthusiasm as their #1 item across all the personal qualities domain, in addition to selecting the same top-5 for the domain of personal qualities. This suggests that there are more similarities than there are differences between the two groups. Our finding is compatible with several previous studies for both physicians (2,22) and nurses (43) which showed that enthusiasm was identified as an important parameter of an effective teacher.
Where differences occurred between clinical educator and nurse groups, this was mainly in the domains of assessment abilities and curriculum planning; furthermore, only the nurse group prioritized the item demonstrates good interprofessionalism within the domain of interpersonal relationships. This latter finding around the respective prioritizing of interprofessional cooperation echoes research undertaken in the undergraduate domain that has found medical students hold more negative attitudes toward interprofessional communication and collaboration, and this holds over time despite interprofessional learning experiences (65)(66)(67). Furthermore, in everyday clinical practice and workplace learning, issues of interprofessional hierarchies, roles and conflicts come into play, leading to healthcare practitioners and learners experiencing interprofessional dilemmas (68): notably, due to the power asymmetry between nurses and doctors, it is the nurses who tend to experience greater distress as a result. It is therefore unsurprising that nurses might prioritize interprofessionalism.
Finally, our data showed that the self-evaluation and PGYs were the two most common sources identified for the evaluation of clinical educators. The importance of self-evaluation for the clinical educator evaluation has also been identified in previous studies (2,20,22). The Johari window (69), a technique that helps people understand relationships between themselves and others, highlights the importance of self-evaluation. The triangulation of evaluation including learners, peers, and selfevaluation (10,65) should be considered in the design of future clinical educator evaluation.
As with any study, there are some limitations. Our study was conducted at a single site, with a relatively low response rate in comparison to the entire population targetted and inconsistentcy regarding to the different rater sources. Indeed, we had relatively high response rates from medical learnersincluding residents, PGYs and medical students-with the lowest response rates from clinical educators and nurses. Given that learners are on the receiving end of clinical educators' teaching, and are presently the main stakeholder group undertaking clinical educators' evaluation, it might be that they are more motivated to respond to invitations to complete the questionnaire about their clinical educators' evaluation. On the other hand, in East Asian cultures, educators comprise an occupation with a relatively high social status. In contrast to the learner-centered education, the educator-centered education may still exist in the East Asian cultures (14). It might be, therefore, that the relatively low response rate from the educators themselves is due to their unwillingness to be evaluated. Furthermore, due to the male-dominated composition of the clinical educators and medical students in Taiwan, our study participants comprised around 70% male. Thus, taken together, caution should be used when considering how the findings might be extrapolated to other settings.

CONCLUSION
The evaluation of clinical educators is a critical, complicated, and difficult task. The purposes of educator evaluations are to provide learning opportunities for educators, to evaluate how educators able assist their students' learning, to help educators to enhance teaching abilities, and to reward excellent educators. The final goal of educator evaluation is to improve education effectiveness. In this study, we applied a questionnaire survey to find out the suitable evaluation domains/aspects and rater sources for the evaluation of clinical educators. We found that most participants listed teaching ability as their first priority, which was different to the research findings from Western contexts. In terms of the evaluation aspects in each domain, the study showed that the rater sources have both similar and different choices. This reflects the rater sources' opinions of the evaluation of clinical educators according to their experiences. Regarding the issue of who is best placed to assess clinical educators, the study showed the best fit rater groups were educators themselves and PGYs. This study gathered the information about the similar and different opinions from different sources in a single East Asian context. Further studies may be drawn upon for the creation of a comprehensive evaluation system for clinical educators in other East Asian contexts according to the results of this study.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by Chang Gung Medical Foundation Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.