Development of individual competencies and team performance in interprofessional ward rounds: results of a study with multimodal observations at the Heidelberg Interprofessional Training Ward

1. Introduction

Improving interprofessional collaborative practice (IPCP) has been formulated as a policy goal in healthcare worldwide (1), acknowledging the associations that have been found between IPCP, quality of care, and patient safety (2–5). Accordingly, in recent years, the topic of interprofessional care has also gained relevance in the educational policy of health professions (medicine, nursing, and other allied health professions) and is demanded as a curricular concept for these vocational training and study programs (1, 6–13). In Germany, interprofessional education (IPE) and interprofessional learning (IPL) have been implemented in the curricula at many sites (14, 15). In medicine, interprofessional competencies should be taught longitudinally, according to the new National Competency-Based Learning Objective Catalog of Undergraduate Medical Education (16). Interprofessional competencies are also explicitly described for nursing in the new vocational training regulations (17). Competencies should be acquired at the level of independent and situation-appropriate performance by the end of training or study. Interprofessional training wards (IPTW) are of particular importance for this level of competence, as they exhibit a high degree of complexity in direct patient care that enables learners to interact self-determinedly and self-responsibly to the greatest possible extent (18–20). IPTWs have been implemented at many hospitals worldwide (18, 21–36). IPTW addresses both profession-specific and interprofessional learning objectives, namely by having undergraduates from different healthcare professions (2–12 undergraduates, depending on the concept) take over the care of a certain number of patients as independently as possible under supervision by learning facilitators. Competency frameworks (37–40) are often used to formulate interprofessional learning objectives. The didactic concept builds on adult learning theories (41, 42), such as cognitive constructivism (43) and socio-constructivism (44). Interprofessional learning is also promoted through real-life placement (45, 46). Positive short-term effects of IPTW are described, especially with regard to a better understanding of professional roles, as well as the long-term effects of interprofessional competencies. Most studies on IPTW document learning outcomes based on students' self-reported evaluations (20). IPTW is also increasingly being implemented in Germany (47–49).

Together with the implementation of IPE/IPL in the curricula of health professions, there is an increasing need to evaluate it, especially with regard to its impact on the competencies for IPCP (50, 51). Questionnaires are often used for this purpose, which collect a structured self-assessment of the participant in IPE/IPL (52–54) and are mostly oriented toward competency frameworks (1, 39, 55). In Kirkpatrick's classification (56) of learning-related outcomes, as modified by Barr et al. (57), these studies primarily map knowledge and attitude-related changes (levels 1 and 2a/b). Studies that assess behavioral change (level 3) or impact on quality of care (level 4a/b) through third-party assessment are rare.

1.1. The Heidelberg interprofessional training ward

In 2017, an interprofessional training ward (Heidelberger Interprofessionelle Ausbildungsstation, HIPSTA) was implemented in an abdominal surgery ward at the university hospital in Heidelberg, Germany (46). Together with the IPTW in Mannheim and Freiburg, which started shortly thereafter, it was the first IPTW in Germany. At that time, the “ward within a ward” included two three-bed patient rooms and a dedicated ward office. Utilities were shared with the surrounding ward. Four students of human medicine (medical undergraduates, MU) in their practical year (the last year of a total of 6 years of study) and four nursing trainees (nursing undergraduates, NU) in their third year of training (the last year of a total of 3 years of vocational training) spent a 3- to 5-week placement on the HIPSTA, during which they were responsible for the patient as far as possible independently and on their own responsibility. The undergraduates work in two shifts on weekdays, with a 2-h overlap at noon. On weekends and at night, patients are cared for by the ward's regular nursing staff. The cohorts (4 NU + 4 MU) were assigned to early and late shifts in the weekly rotation. The respective teams of one shift (2 NU + 2 MU) were divided into two interprofessional tandems, which took over the care of the patients in one room each. One team of four participants was planned to start with the early shift for the whole first week; the other four were to start with the late shift. In the second week, the groups switch, and the participants who worked early shift in the first week work late shift in the second week, and vice versa. In weeks 3 and 4, they changed again, enabling each group to work in one shift for 5 days in a row and a weekly alternation of early and late shifts, resulting in 2 weeks of early and 2 weeks of late shifts for each participant in total.

As shown by a retrospective analysis of patient data (58), the patients to be cared for did not significantly differ from patients on the surrounding ward with regard to age, comorbidities, reason for admission, or data concerning surgery. The undergraduates were supervised by nursing and physician learning facilitators. A nursing facilitator was present throughout the morning shift. The physician facilitator was present for the morning ward rounds, midday handovers, and afternoon short rounds and was on call by phone the rest of the time. During the afternoon shift, an experienced nurse from the surrounding ward who had been well introduced to the HIPSTA concept was the contact person for the undergraduates. The learning facilitators interfered with patient care only when there was a concern that patient safety would otherwise be compromised. Otherwise, they remained in the background and only became active when requested by the undergraduates, answering questions or, in case participants asked for it, guided certain actions on the patient or in administration. They also provided feedback and fostered reflection and problem-solving processes. The daily routine at HIPSTA was structured by different practical learning phases in which the undergraduates learned both professionally and interprofessionally. On the morning shift, the interprofessional ward round took place. It started at approximately 8 a.m. A tandem of one NU and one MU conducted the round in the room they were caring for. The facilitators took part in the rounds but remained in the background. In addition, the nursing shift leader of the surrounding ward, a pharmacist, and other medical staff may also have been involved. The other tandem of the team also passively participated in the ward round. The round in the patient's room was usually preceded by a brief exchange outside the room on the current situation or on aspects that could not be discussed in front of the patient for certain reasons. After all patients in both rooms had been visited, a joint comprehensive debriefing of the information gained took place in the HIPSTA ward office. The further treatment, therapy, and care plan were developed jointly in tandem and coordinated with the nursing and physician facilitators.

For the overall evaluation of HIPSTA, a mixed-methods approach was chosen (59), which included quantitative and qualitative analyses. The results of the quantitative analyses of self-assessment questionnaires (60), the reconstructive analyses based on group interviews (61), and qualitative content analysis of personal interviews (62) show an acquisition of competence experienced by the learners with regard to collaboration, roles, responsibilities, and communication, more positive attitudes toward IPL and teamwork, and partial development of an (inter-)professional identity and socialization.

In addition to the self-reported assessment of the HIPSTA evaluation, behavioral change was captured via third-party observation. The interprofessional ward rounds were chosen as the observational setting because it was anticipated that observable interaction between the undergraduates and with the patients would show up particularly often. For this purpose, an instrument was developed (63), which is multimodal in design and assesses both individual competencies and team performance. By observing the ward rounds, it was intended to record whether and how individual competencies and team performance change over time by means of external assessment.

1.2. Aims and research questions

The aim of this article is to present and discuss the results of the structured ward round observation. Research questions were: How do nursing and medical undergraduates develop individual competences and team performance during their 4-week HIPSTA placement measured by external assessment in ward rounding? To what extent does the development of nursing and medical undergraduates differ? To what extent are individual competencies and team performance interdependent?

2. Methods

2.1. Design and data collection

Data were collected from January to May 2018 in four cohorts of HIPSTA in a pre- (beginning) post- (end) design. For this purpose, participants' rounds were observed in their first (t0) and last week (t1) of their morning shift, when ward rounds took place. Observation was conducted by two researchers each using the IP-VITA^pre [Individual competencies and team performance assessment tool (63)]. The instrument was developed empirically by first testing three instruments (64–66) available for the evaluation of interprofessional learning interventions with patient contact in a pre-study. The “Individual Teamwork Assessment Scale” (iTOFT) (64), the “Teamwork Assessment Scale” (TAS) (66), and the “McMaster-Ottawa Scale” (McMOS) (67) were used in at least one cohort (n = 4 observations). Afterwards, their use in the HIPSTA evaluation was discussed. It was decided that data should be collected at both the individual and team levels and that a separate instrument would be needed for this purpose. Therefore, an instrument, the IP-VITA, was developed from the experience made with the former instruments. As shown in Figure 1, the data presented were collected using the preliminary version of the instrument (IP-VITA^pre). In this version, observable individual behavior was assessed by 16 items. Nine items were further developed from the instrument testing and adapted to the specific context of ward rounding. These items were evaluated on a 6-point Likert scale. Seven items compressed the CanMEDS model (68). Observable interaction within the tandems was assessed by 11 items on a 4-point Likert scale. Definitions (strongly agree/don't agree at all, to a very high degree/to a very low degree) were given only for the maximum expressions. Gradations were scored at intervals in relation to these two poles. Figure 1 gives an overview of the study design and data collection.

FIGURE 1

Figure 1. Study design.

Each researcher observed one person in the tandem regarding individual competencies. Both took notes on the interaction and completed the team performance scale jointly after the observation in terms of intersubjective interpretation for each tandem.

2.2. Statistical analysis

All data analysis was performed in the IBM SPSS Statistics 22 software, except for the mixed model calculated for the team performance scale, which was performed in R. The dataset was cleaned by identifying outliers and extreme values and checking the dataset for plausibility. Missing values occurred when a skill or behavior was not observable. They were excluded on an item-by-item basis, as no systematic correlation between the missing values could be identified.

For statistical analysis, interval scaling was assumed for the Likert-scale data. Therefore, the mean and standard deviation are presented to describe the dataset.

Descriptive statistics for the sample were compiled. Welch t-test was calculated for the age difference in groups by profession (NU, MU).

The IP-VITA^pre was checked with regard to its internal consistency. Cronbach's alpha was calculated. Based on an exploratory factor analysis and theoretical considerations, three scores were formed for the individual competency scale and the team performance scale. For this purpose and for better comparability of graphical representation, six-point scaled items were converted to a four-point scale. Subscales were checked for internal consistency using Cronbach's alpha.

The following hypotheses were formulated for testing:

1. H1: values of individual competencies (item and score) differ in t0 and t1.

2. H1: values of team performance (item and score) differ in t0 and t1.

3. H1: there are differences in the mean values of the NU and MU groups at t0 and t1, and there are differences in the mean change over time.

4. H1: the values of individual competencies and team performance are correlated.

To describe the change in individual competencies and team performance over time and within the professional groups, a linear mixed model with restricted maximum likelihood (REML) and Satterthwaite's method with an F-test were calculated. For individual differences, the model included group (Hypothesis 3), time (Hypothesis 1), and their interaction as fixed, and participants as a random factor (Hypothesis 3). For team performance, the model included time as fixed and the NU/MU group as random factors (Hypothesis 2). Effects with p < 0.05 were considered significant. Trends that appeared to be particularly interesting were plotted graphically or described, even if they did not show a significant value.

Pearson correlations were performed to determine relationships between individual competencies and team performance (Hypothesis 4).

3. Results

3.1. Participant characteristics

Observations with the IP-VITA^pre took place in HIPSTA cohorts 8–11. A total of 16 nursing undergraduates (10 women and 6 men) and 16 medical undergraduates (3 women and 13 men) were included. No observation could be conducted with one medical undergraduate in cohort 10, and only one observation took place with one nursing undergraduate due to illness/shift change. In these cohorts, therefore, two other medical and one nursing undergraduate were observed at three instead of two data collection points. For individual competencies, the third observation was not included in the data analysis. At the team level, the observation was considered to be regular t0 or t1, as most of the other tandems also changed partners. Only four tandems remained constant across measurement time points t0 and t1, and the other 12 tandems worked with different partners at t1 than at t0.

Table 1 and Figure 2 provide an overview of the included participants. The mean age was 21.8 ± 1 in the nursing group and 27.7 ± 3.4 in the medical group overall. This difference is significant (p < 0.001). There were significantly more women among nursing than among medical undergraduates (Fisher's test, p = 0.029). Based on this, the mean age difference is significantly different between male and female participants (p = 0.002).

TABLE 1

Table 1. Study sample: profession, gender, and age.

FIGURE 2

Figure 2. Dot plot of the study sample.

The number of patients in the two rooms varied according to surgery or overlapping times of discharge/admission. During 15 ward rounds at t0 and t1, each of the three patients had to be discussed. In 14 rounds at t1 and 13 rounds at t1, there had been two patients. In two rounds (t0 and t1), the rooms were occupied only by one patient. The patients' clinical appearance was heterogeneous and covered the whole spectrum of a general abdominal surgery ward. The nursing and physician learning facilitators were present in all except for two rounds, where once the physician and once the nurse were not present. The nursing ward manager was present in some rounds, a pharmacist in two, and an intern in two ward rounds.

3.2. Analysis of the IP-VITA^pre

Exploratory factor analysis yielded a three-component solution for the individual scale and a four-component solution for the team performance scale. This was checked for plausibility in terms of content. Cronbach's alpha was calculated for all subscales. For the individual competencies, 14 of the 16 items could be combined in the three subscales “Roles and Responsibilities” (6 items, α = 0.891), “Patient-Centeredness” (3 items, α = 0.850), and “Leadership” (5 items, α = 0.845). The items “Active participation” and “CanMEDS Collaborator” were not included. The three subscales explain almost 70% of the variance in the data.

For the team performance scale, the subscales “Roles and Responsibilities” (2 items, α = 0.808), “Patient-Centeredness” (4 items, α = 0.844), and “Decision-Making/Collaborative Clinical Reasoning (CCR)” (3 items, α = 0.739) excelled. The fourth component was discarded due to a lack of content plausibility and low inner consistency. The items “Exchange between NU and MU present” and “Swift effective round” were not included. The team scores explained 70% of the total variance of the team performance scale.

3.3. Development of individual competencies

We hypothesized (Hypothesis 1) that there is a mean value change from t0 to t1. In the linear mixed model calculated for the influence of time (t0, t1), significant differences between moments in time showed up in all subscale scores, as can be seen in Table 2. The mean value of all except one item of the subscale “Roles and Responsibilities” increased from t0 to t1. The increase was highly significant in the item “defines clear goals for further treatment” (mean change 0.80, p = 0.001). This means, according to the descriptors of the item, that in the ward rounds observed at the end of the assignment on HIPSTA, the undergraduates more often explicitly addressed further treatment, prioritized actions, and named feasible goals, including their realization in time. The difference in time was also highly significant for the item “CanMEDS Manager” (mean change 0.74, p = 0.001). This means that the undergraduates were better organized and more effective with available resources in the second observed ward round. Overall, the increase in the subscale “Roles and Responsibilities” was significant (mean change 0.44, p = 0.016). The mean value of all items in the subscale “Patient-Centeredness” increased from t0 to t1. The increase was significant for the item “Discusses current patient information with patient involvement” (mean change 0.55, p = 0.019). This means that in the second observation, the undergraduates shared more information with each other and actively and clearly approached the patient to obtain or verify information. The increase in the item “CanMEDS Health Advocate” remained slightly below the significance threshold. No significant change could be observed regarding the handling of the patient's questions. The change in the subscale “Patient-Centeredness” over time is at the significance threshold (mean change 0.40, p = 0.049). The differences in means over time are significant in the subscale “Leadership” (mean change 0.39, p = 0.015). At the item level, the mean change of “Self-confident/sovereign appearance” (mean change 0.052, p = 0.032) and “CanMEDS Professional” (mean change 0.32, p = 0.030) was significant. This means that the undergraduates were more confident, including in terms of verbal expression, and gave the impression of being confident about the process of ward rounding. The mean chance over time in the item “CanMEDS Expertise” was highly significant (mean change 0.52, p = 0.001). This means that a higher level of diagnostic and therapeutic skills could be observed in t1. For the items that were not listed in a subscale, a significant difference in the mean value for the item “Active participation” could be shown (mean change 0.55, p = 0.002). This means that the undergraduates showed up more proactive and less reactive. The change over time in the item “CanMEDS Collaborator” was not significant.

TABLE 2

Table 2. Development of individual competencies.

3.4. Development of team performance

We hypothesized (Hypothesis 2) that the mean values of the team performance scale (item and score) differ in t0 and t1. A linear mixed model was calculated that considered that most of the tandems were not constant from t0 to t1 of the observation of team performance, which means that different nursing and medical undergraduates performed the ward round, respectively. As can be seen in Table 3, there was a non-significant negative trend in the difference of mean scores from t0 to t1 in the items “Patient questions are answered” and “Swift effective rounds”. For all other items, there was a positive trend, but it was also mostly non-significant. The significant change in the mean sum scores of “Roles and Responsibilities” (mean change 0.67, p = 0.008) is accounted for by the highly significant difference t0 to t1 in the item “Roles are clearly assigned” (mean change 0.75, p = 0.002). This means that the undergraduates showed a higher level of role awareness in the second observed ward round, behaved more according to their own professional roles, and seemed to acknowledge the role of the other profession.

TABLE 3

Table 3. Development of team performance.

It is important to emphasize that most of the tandems (n = 12) at t1 were not composed identically to the observation at t0. The analysis of the development in the overall sample describes that two random individuals interacted at the beginning and end of their assignment in a ward round. Thus, the tandem is more of a theoretical construct than an empirical one, since individuals (NU and MU) in the tandems did not remain constant as would have been intended in the study design (Figure 1). In the following, therefore, the mean change over time is presented only for the small group of tandems (n = 4) that were identical at t0 and t1. No p-values are given for the change in means over time for the group of random tandems. Rather, these data can be viewed as a cross-sectional investigation with random tandems at t0 and t1, respectively, with which the group of constant tandems (seen as cross-sectional) is compared in Figure 3 and Table 4. Within the group of constant tandems, all items and scores except for the item “Swift effective round”, which remained the same from t0 to t1 (mean change 0.00, p = 1), showed a clear positive tendency. However, this trend is significant only for the item “Roles are clearly assigned” (mean change 1.00, p = 0.017) and the score “Decision-making/CCR” (mean change 1.00, p = 0.48). Compared with all other tandems in t1, the team performance of the constant tandems (CT) was better across all items and scores than in the group of random tandems (RT). An exception is the item “swift effective round” (CT mean 3.00 ± 0.916, RT mean 3.00 ± 0.603, P = 1), which was identical. However, the differences between constant and random tandems are significant only for the item “Further procedure is panned by the team” (CT mean 3.67 ± 0.577, RT mean 2.82 ± 0.405, p = 0.012).

FIGURE 3

Figure 3. Team performance-spider chart of mean values.

TABLE 4

Table 4. Development of team performance in constant tandems.

3.5. Differences between groups by profession

We hypothesized (Hypothesis 3) that there are group differences in the mean values of NU and MU in t0 and t1 and in the mean change over time. The linear mixed model was calculated to check for the subscales in each case: whether the groups of nursing undergraduates (NU group) and medical undergraduates (MU group) differed at time points t0 and t1, whether there was a change from t0 to t1 in each group itself, and whether these changes differed between groups. With regard to the two items not listed in the subscales, the item “Active participation” showed a highly significant group difference at t0 (NU mean 2.5 ± 0.903, MU mean 3.6 ± 0.529, p < 0.001), which leveled off somewhat at t1 but remained significant (NU mean 3.3 ± 0.844, MU mean 3.84 ± 0.356 p = 0.036). The difference over time t0 to t1 was highly significant for the NU group (mean change 0.8, p = 0.008) but not for the MU group (mean change 0.24, p = 0.162). However, the difference in trend was not significant (0.078) which means, that both groups developed to a similar extent, albeit at a different level. There was no significant group difference for the item “CanMEDS Collaborator” in either t0 (NU mean 2.56 ± 0.814, MU mean 2.80 ± 0.561, p = 0.355) or t1 (NU mean 2,73 ± 0.961, MU mean 2.80 ± 0.862, p = 0.843). None of the groups had a significant change over time (NU mean change 0.17, p = 0.597, MU mean change 0.00, p = 1). There was also no difference in the range of development over time (p = 0.676).

3.5.1. Roles and responsibilities

As shown in Table 5 and Figure 4, at t0, the MU group has higher mean values on all items than the NU group. This is highly significant for the items “Distributes tasks” (NU mean 1.24 ± 0.419, MU mean 2.15 ± 0.827, p = 0.005) and “CanMEDS Manager” (NU mean 1.69 ± 0.793, MU mean 2.47 ± 0.649, p = 0.006). This means that medical undergraduates asked nursing undergraduates more often to complete specific tasks later in the day and also exchanged about the timing of completion than the other way around. And for the “CanMEDS Manager”, which was also significant in the total sample, they showed more behaviors that served the effective organization of the ward routine. Accordingly, the mean value of the sum score “Roles and Responsibilities” is significantly higher than in the NU group. Interestingly, none of these differences are still significant in t1. Instead, the difference in means between the groups NU and MU in the item “Defines clear goals” is significant in t1 (NU mean 2.71 ± 1.125, MU mean 3.40 ± 0.600, p = 0.46). This means that the medical undergraduates more often showed behaviors that served to prioritize the further course of treatment and more often named explicit feasible goals, including their implementation in terms of time. Looking at the development from t0 to t1 in the respective groups, we find that the difference in terms of time is highly significant in the MU group (mean change 0.004, p = 0.004) but not in the NU group. But as seen in the data, the NU group also developed to a relatively high degree with respect to goal setting, although not at the 0.05 significance level. However, the differences in group development overall are not significant. Both groups developed similarly in all items and scores, albeit with different initial mean values. While the MU group evolved primarily in terms of treatment goal setting, the change over time in the NU group in the “CanMEDS Manager” item was highly significant (mean change 0.91, p = 0.006). We observed that the nursing undergraduates in t1 took more responsibility for the effective organization of ward procedures. Another finding is that while in t0 the differences in the mean values of the MU group compared to those of the NU group in the item “Distributes tasks” were highly significant, they are no longer so in t1. Instead, there is a significant change over time in the MU group in the item “Takes over tasks”. This means that the medical undergraduates observably expressed more frequently which tasks they would complete in the further course of the day.

TABLE 5

Table 5. Differences by profession – score “Roles and responsibility.”

FIGURE 4

Figure 4. Differences by profession: roles and responsibilities-spider chart of mean values.

3.5.2. Patient-centeredness

As shown in Table 6 and Figure 5, there were also differences in the mean values of the NU group compared to the MU group in the scale “Patient-Centeredness” at t0. Interestingly, all the mean values of the MU group are higher than those of the NU group. The difference is significant for the items “Discusses current patient information with patient involvement” (NU mean 2.27 ± 0.951, MU mean 2.92 ± 0.758, p = 0.47), “Ensures that the patients' questions are asked and answered” (NU mean 2.00 ± 0.740, MU mean 2.72 ± 0.844, p = 0.019), and the sum score (NU mean 2.08 ± 0.683, MU mean 2.65 ± 0.672, p = 0.026). This means that for the medical undergraduates, it was observed more frequently that they actively approached the patient to obtain and verify information, responded to the patient's questions, and included the patient in the goal-setting process for further treatment. In t1, the difference in mean values between the groups is still significant with regard to the item “Patients question” (NU mean 2.03 ± 0.1007, MU mean 3.08 ± 0.035, p = 0.017). In both groups, there is a positive trend in the mean values of t0 compared with t1. However, this was not significant for any of the groups. The two groups of NU and MU develop similarly in all items and the score, with different initial mean values.

TABLE 6

Table 6. Differences by profession – score “Patient-centeredness.”

FIGURE 5

Figure 5. Differences by Profession: patient-centeredness-spider chart of mean values.

3.5.3. Leadership

For the subscale “Leadership”, as shown in Table 7 and Figure 6, no significant difference was found in the sum score between the two groups at either measurement time point. In t0, there was a significant difference in the mean values of both groups in the item “Ensures that all team members receive all information (team info)” (NU mean 2.35, MU mean 3.04, p = 0.027). At both t0 and t1, the mean values of the MU group showed a slightly higher value than the NU group in almost all items. However, all these differences are not significant. An exception is the item “CanMEDS Expertise”, where in t0 the mean value of the NU group was very slightly higher (NU mean 2.25, MU mean 2.20) than that of the MU group. This is exactly the other way around at t1 (NU mean 2.62, MU mean 2.86). Looking at both groups separately over time, the positive trend in the mean differences from t0 to t1 is highly significant for the MU group for this item (mean change 0.66, p = 0.003). This means that the medical undergraduates were more likely to be observed accessing and applying information to clinical practice and demonstrating diagnostic and therapeutic skills in t1 than in t0. In all other items, there was a positive trend in the MU group except for the item “CanMEDS Professional”, in which the mean value remained the same. On the other hand, the difference in the mean value for this item in the NU group is significant over time (mean change 0.62, p = 0.025). This means that the nursing undergraduates in t1 showed more often than in t0 behavior, which aimed to deliver high-quality care, and it was observed that they were involved in the ward round with professionalism and integrity. For the NU group, the differences of mean values t0 to t1 for the sum score (mean chance 0.53, p = 0.039) and the item “Team info” (mean change 0.45, p = 0.038) are also significant. The latter means that at t1, it was observed more frequently how the nursing undergraduates made sure, e.g., through eye contact or active inquiry, that the tandem partner took note of and understood the information that he or she had provided.

TABLE 7

Table 7. Differences by profession – score “Leadership.”

FIGURE 6

Figure 6. Differences by profession: leadership-spider chart of mean values.

As can be seen in Figure 7, the maximum positive value (4.00) was not reached in any of the two groups at any time. The highest value in t1 (MU, Patient-Centeredness, mean 3.03) corresponds to 75.75% of the maximum value. The lowest value in t1 (NU, Roles and Responsibilities, mean 61.25) corresponds to 61.25% of the maximum value.

FIGURE 7

Figure 7. Bar chart of sum scores by group and time.

3.6. Relationship between individual competencies and team performance

We hypothesized (Hypothesis 4) that individual competencies and team performance are related. Pearson correlations were calculated to investigate whether there is a relation between individual competencies and team performance for all subscale sum scores.

As shown in Table 8, overall, within these four cohorts, at t0, there was only one significant moderate correlation for the individual and team score on “Roles and Responsibilities” (r = 0.416, p < 0.05). At t1, these scores showed a strong and highly significant correlation (r = 0.501, p < 0.005). The correlation between the individual and team scores on “Patient-Centeredness” was moderate and significant at t1 (r = 0.422, p < 0.05). Also at t1, there was a strong and highly significant correlation between both the individual scores on “Roles and Responsibilities” and “Patient-Centeredness” and the team score on “Decision-Making/CCR” (RR: p = 0.597, p < 0.001, PC: r = 0.516, p < 0.01). The non-significant negative correlation of the individual score on “Roles and Responsibilities” with the team score on “Patient-Centeredness” became a positive, although not significant, correlation at t1.

TABLE 8

Table 8. Correlations of individual and team scores t0/t1.

Looking at group differences at t0 and t1 (Table 9), two significant (p < 0.05) strong correlations were found between the individual competencies of the MU group and team performance in t0, respectively, in the same categories “Roles and Responsibilities (RR)” (r = 0.567) and “Patient-Centeredness (PC)” (r = 0.562). Both correlations become even stronger at t1 (RR r = 0.715, PC r = 0.617) and highly significant (p < 0.01) in the former. For the NU group, the relation between individual and team scores for “Roles and Responsibilities” is also apparent but not significant. Also, not significant but noteworthy is the negative correlation in the NU group between all individual sum scores and the team score “Patient-Centeredness”. In t1, however, the correlations are positive, although not significant. Looking at the team scores for “Decision-Making/Clinical Reasoning” (CCR), it is noticeable that there are no significant correlations with the individual scores at t0, but that there are moderately to strong positive correlations with all individual scores in both groups at t1, which are significant or highly significant for the NU group (RR r = 0.767, p < 0.01, PC r = 0.611, p < 0.05, LS r = 0.633, p < 0.05).

TABLE 9

Table 9. Correlations of individual and team scores t0/t1.

4. Discussion

4.1. Discussion of content

4.1.1. Summary of key findings

For the evaluation of HIPSTA, structured ward round observation was conducted during the first and last weeks of the undergraduates' placement. In this subtask, conducting an interprofessional ward round, the NU and MU demonstrated significant competence acquisition in all three competence domains: “Roles and Responsibilities”, “Patient-Centeredness”, and “Leadership”. However, the two groups developed differently; while the NU mainly acquired competence in leadership and management, the MU developed professional expertise and was better able to define treatment goals and take over tasks at the end of their placement. Team performance also showed that roles and responsibilities were much more observable. It was striking that the mean values of the group of constant tandems at t1 were higher than those of the random tandems in all items except the “Swift effective rounds”. These differences were not significant. The development of individual competencies and team performance are related. It has been shown that this correlation increases over time.

4.1.2. Integration into the body of research with external assessment on IPTW

Studies with a similar methodological approach are scarce, making it difficult to contextualize these results within the body of research. Brätz et al. (48) examined whether IPTW placement at the University Medical Center Hamburg-Eppendorf, Germany, had an impact on medical students' entrustable professional activities (EPA). After a 4-week placement in an IPTW (intervention group) or regular training (control group), 12 EPAs were recorded using a competency-based telemedicine assessment in a simulation of the first day of residency. The overall mean entrustment level was significantly higher (p < 0.001) in the IPTW group compared to the control group. Reeves et al. (32) and Freeth et al. (23) also conducted observations in the pilot of the Royal London Hospital's rheumatological and orthopaedical IPTW, UK. However, these were analyzed qualitatively and triangulated with data from individual and group interviews, so a systematic comparison of the results is not possible. Lidskog et al. (69) conducted unstructured observation when evaluating an IPTW within care for older people in Örebro, Sweden. In these studies, results were also triangulated with other qualitative data sources, so no comparison of the results of observation is possible. Same with the ethnographic observation conducted on an orthopedic ward at Karolinska Institutet, Stockholm, Sweden, by Ivarson et al. (70) that focused on a special learning intervention. The first result of the evaluation of the Mannheimer IPTW, Germany (37), identified ward round skills as a self-reported topic in which students gained competence.

4.1.3. Discussion of findings concerning individual competencies and team performance

About the overall scale of individual competencies, it is interesting to note that the group-specific differences that existed at the beginning of the IPTW placement were only significant in two items at the end of the placement, which will be discussed in more detail. The groups did not develop differently, which may indicate that the educational concept of HIPSTA adequately supports both professional groups.

There was a statistically significant mean change in all subcategories of the individual competency scale, showing that participants seem to have improved their competencies, especially in terms of defining treatment goals, involving patients, and acting self-confidently. This confirms the results of the quantitative and qualitative (60) analysis of learners self-assessment of competency development and interprofessional socialization (61, 62) from an external, observational perspective. With regard to the development of the individual competencies subscale “Roles and Responsibilities”, the results described from questionnaire and interview studies (33, 49, 60, 62, 71–75) are substantiated and supplemented. The gain in understanding roles was also evident in the ward round observation. The participants behaved according to their professional roles and acknowledged the others' roles to a greater extent at the second team performance observation than at the first. Within the individual competency observation, an increase in self-confident/sovereign demeanor could also indicate a risen understanding of and identification with the professional identity. Higher confidence was also described in the interviews with participants 1–1.5 years after their placement on the HIPSTA (62).

4.1.3.1. Patient-centeredness

With regard to the development of subscale “Patient-Centeredness”, no IPTW study so far has explicitly reported any effects. However, the Assessment of Interprofessional Collaboration Scale (AITCS) (76) and the Interprofessional Socialization and Valuing Scale (77, 78) were also used in the self-reported evaluation of HIPSTA (60). The AITCS, which includes aspects of patient-centeredness in the subscale “Coordination” showed a highly significant change in the mean sum score both in the pre-post as well as in the pre-follow-up comparison, and the ISVS, which covers patient-centeredness in terms of involving patients' interests and understanding and conducting collaborative decision-making together with patients, showed significant pre-post and pre-follow-up mean changes in the sum score and in the specific items (60). Within the analysis of learners retrospective evaluations of the HIPSTA, it has been shown that especially medical undergraduates had the impression of improved competencies in interprofessional communication in terms of listening to and understanding patient's needs (62). Still, these are all self-reported competencies, which do not guarantee performance. Hence, the results of this study give a better impression of how the undergraduates actually demonstrate their self-perceived competencies. The patient is central to the frameworks for interprofessional collaboration and the starting point for the call for more ICPC and IPE (1, 6, 38, 39). Analyses of the concept of “patient-centeredness” show that it is rich in perspectives and dimensions and requires further research to be operationalized for the health professions (79–82). Spaulding et al. (83) identify a lack of research on the patient-centeredness outcome of IPE. Orchard (84) sees the nursing leader's role as key between patients and other health providers. Interestingly, no significant differences in the patient-centeredness items from t0 to t1 were found in the NU group, and the sum score in the subscale “Patient-Centeredness” was significantly lower in the group of NU compared to the group of MU sum scores at both points in time. This could be related to the way in which patient-centeredness was recorded in the ward rounds, namely primarily with the extent to which the patient was involved in obtaining information and the extent to which questions were motivated and answered. Also, since ward rounds serve to clarify the patient's medical condition, it is not that surprising that there was a certain patient-centeredness present and observable. This explains the difference in “ensures that the patients' questions are asked and answered”, which is also significant at t1. Furthermore, there was a given structure for the ward round that was co-developed by the learners, saying that in the first step, the nursing undergraduate introduces the patient and reports on the process; second, the medical undergraduate takes the lead of the round; third, the patient is asked an open question (“how are you”); and fourth, a joint evaluation of the situation and background takes place. Having the lead of the round could have made the medical undergraduates feel more responsible for patient involvement than the nursing undergraduates in this specific situation. In addition to that, most nursing undergraduates have already visited the patients' rooms and talked to them in the morning before the ward round. For them, it might have been artificial to have the same conversation a second time. Since the undergraduates followed a structure, medical undergraduates' improvement in patient-centeredness does not necessarily mean that they actually change their attitude toward the patient; it could also indicate that they were better able to implement the instructions on the round. Within the team performance scale, no statistically significant improvement in the sum score “Patient-Centeredness” was shown. Still, the mean score of patient involvement in information collection was rather positive at t1.

4.1.3.2. Individual development of nursing and medical undergraduates

Regarding the subscale “Leadership”, the nursing undergraduates underwent a significant change, although the item “CanMEDS professional” was the only item where the mean change of both groups differed significantly. Also, at t1, the nursing undergraduates paid more attention to ensuring that all team members received all relevant information and backed this up. They also took on more of a managerial role (CanMEDS Manager) compared to that in t0. If this is viewed in parallel with the development of medical undergraduates, who developed especially in terms of goal definition and the assumption of tasks, professional socialization can be surmised. Wenger (85) described in the “communities of practice”, which was later described as a concept in health education (86), that professional identity forms in dependency on the relations and activities of other members of the community. This can also be seen in the statistically significant improvement of values regarding role assignment within the team performance scale. From this perspective, the interprofessional setting of the ward round could be conducive to the professional identity of medical and nursing undergraduates. This has also been shown by the longitudinal quantitative analysis of the HIPSTA within the ISVS that also measures role clarification (60) and within follow-up interviews with the participants, where they described an improvement in confidence and self-efficacy in their professional role due to the experience on the HIPSTA (62). In this study, it has been observed that medical students acquired and applied skills in collaborative clinical reasoning and decision-making, whereas nursing undergraduates acquired leadership skills. The importance of collaborative clinical reasoning skills in medical education has been widely acknowledged (87). Leadership skills for nursing are key for ward management and team performance in healthcare. It might have a central role in education and should be further investigated. In her review, Cummings et al. (88) analyzed factors and educational interventions that influence nursing leadership. However, they were unable to include any studies in an interprofessional setting. Orchard et al. (89) advocate nursing leadership as a dual role in interprofessional teams, namely, managerial and disciplinary. They suggest that in areas nurse leaders are responsible for, “their ability to support health providers use of knowledge, skills, and expertise to address the complex and uncertain needs of those persons seeking help can result in improved care”.

The concept of professional identity was further developed by Khalili et al. (90, 91) for the interprofessional context, forming the concept of dual identity and professional socialization (interprofessional socialization framework). Thistlethwaite (92), referring to Miller's competence pyramid (93), describes it similarly by asking if it needs a fifth competency level “is” above “does”. Mink et al. (61), referring to the concept of dual identities, examine in a reconstructive analysis of the focus groups of the first cohorts of HIPSTA the extent to which interprofessional socialization has occurred and conclude that it cannot be reliably anticipated. The data of cohorts 8–11 examined here show, in comparison to the constant with random tandems, that the former tended to plan the further procedure together significantly more often than the random tandems. The mean score for the constant tandem at t1 is above the middle, resembling a positive evaluation of their collaborative planning. However, no statement can be made about the participants' sense of belonging to the interprofessional community or about how sustainably team performance can be implemented in the subsequent everyday work.

The problem, also in evaluating interprofessional teaching by means of patient-relevant outcomes, is that little insight is gained into the black box between IPE and IPCP. IPE is important and has a positive impact on attitudes and competencies. Good IPCP increases patient safety and quality of care. But how does the former relate to the latter? One approach could be to break down the huge field of IPCP into small bites by asking what clinical problem is specifically to be solved by better interprofessional collaboration. This study is based on the premise that interprofessional competencies can be observed particularly well in interprofessional rounds. The ward round could therefore be seen as a clinical problem, as a unit of care structure in the clinical setting in need of optimization, which should be solved or optimized through IPCP. About half of all adverse events in the surgical setting occur outside the operating room (94) and are associated with poor organization of inpatient care. The ward round is a central element of quality assurance because it is used to exchange information, record the patient's condition, and plan further procedures within the team, if things are going well. If things are not going well, this can have a correspondingly negative effect on patient care. Klaas et al. (95) propose a taxonomy of non-technical skills for the surgical ward round and define good and bad behavior for the team and the team leader in four categories, namely, “Leadership”, “Situation awareness”, “Decision-making”, and “Communication and teamwork”, which was evaluated for nurses (96), and which are complemented by our study results very well. In this respect, a very small crack in the black box is opened in that the IPTW setting enhances a concrete clinical activity, ward rounds.

4.2. Discussion of methods and limitations

The study was single-centered and was conducted without a control group. The sample is small, which limits the statistical possibilities, and due to this, it should be considered that statistical tests have low power with small effect sizes. Therefore, both “almost” significant and non-significant changes have been reported and discussed.

For the interpretation of the results of this study, differences in the group comparisons between nursing and medical undergraduates might not (only) result from the professional background but from age or gender, or at least co-variances exist, which could not be examined in more detail due to the small sample size. In Germany, the medical study program takes twice as long as nursing school, and nursing is still predominantly female.

The data presented here were collected using the previous version of the IP-VITA (IP-VITA^pre), which has so far only been validated descriptively (63, 97) and not statistically. There were a few adaptations after the observations in the four cohorts, the results of which are presented here. The CanMEDS roles in IP-VITA^pre were adopted as items without critically appreciating previous publications on the recording of CanMEDS and its complexity (98–104). Although this framework was initially physician-specific, it has been successfully transferred to other health professions (105–108). However, the researchers did not operationalize further into the items but instead referred to the role model in interpretive intersubjective exchange to assess the undergraduate's behavior. This worked well for the observations in the HIPSTA setting. However, for a transfer to other sites and the use of the instrument, possibly with only one observer, more extensive descriptors and a further operationalization of the roles would have been necessary. As a consequence, the CanMEDS roles were removed. The two individual items “Discusses current patient information with patient involvement” and “Ensures that the patients' questions are asked and answered,” were split into five more distinct items (see IP-VITA in the Supplementary material). Adjustments were also made to the team performance scale, aimed at a clearer delineation of the items. In addition, it was also scaled to six-point instead of four-point Likert, and a distinction was made between observable interaction during the ward round and during debriefing.

The ward round observations were conducted by three researchers (AMit, CA, and JM) in different constellations of two. The observers had nursing (n = 2) and gerontological (n = 1) backgrounds, each with academic degrees. None had a medical background, and none of the three had specific training as observers; nonetheless, all had experience in quantitative and qualitative research. Because assessment practices are by no means trivial (109), assessment literacy (110) may be questioned, at least with respect to items related to medicine (e.g., physician decision-making and goal setting). The way the researchers handled this was to involve the physician learning facilitators when there was uncertainty.

For the interpretation of the data, especially the changes from t0 to t1, it is important to point out again that the group of tandems was naturally half as large as the group of individuals. The fact that there were more significant changes in the mean values of the individual scale from t0 to t1 could be due to the fact that the tandem sample is smaller than the individual one.

IPTWs are highly complex learning interventions. This complexity is highly conducive to the cause of IPL/IPE late in vocational training and study (111) – but not to its research. The setting of the observations that produced the data presented varied in terms of the number of patients, others (passively) involved, and the participants themselves to be observed. The setting was not meticulously recorded for each observation in this study. For exploration in the pre-post design, it would be necessary for the tandems to be composed of the same individuals at t1 as at t0. This was only the case to a very small extent in this study. The effects of the different individuals in the tandems were accounted for in the linear mixed model, but this still limits the interpretation.

For further studies, it would be advisable to reduce some of the complexity of the setting and to standardize the framework conditions as far as possible. Since patients cannot be standardized with regard to their illnesses and real-patient contact is the special attraction of observing interprofessional interaction, the other parameters should be adjusted. First and foremost, the tandems or teams from which data are collected should be identical at the time of collection. On the other hand, perhaps the very fact that they are not is the right approach. If the premise were that individual competencies in any healthcare team should have a positive impact on quality of care, then this might be an interesting idea to think about further, at least for formative feedback. What could be standardized for a follow-up survey would be that the individuals who participate in the ward round are defined, and clear guidelines also apply regarding their contribution. Furthermore, the course of the round could be standardized insofar as it was often not clear in this study when exactly the visit and thus the observation began and when it ended. Our study collected data on a small but important part of the IPTW, the ward round. However, multi-center approaches should also have the learning process of the IPTW itself in focus. Further studies could observe other team tasks on IPTW, like handovers and collaborative clinical reasoning. Also, longitudinal studies with repeated data collection and analysis several weeks and months after an IPTW placement should be conducted.

5. Conclusion

This study describes how interprofessional tandems at the end of an IPTW assignment interact more clearly in terms of their roles and tasks, are more patient-focused, and are better able to obtain and share information to set goals for treatment and plan next steps as a team. Our evidence suggests that tandems that stay consistently together perform better than tandems with changing partners. If this finding manifests itself, IPTW could be organized so that learning teams should be stable and not change. Alternately, IPTW research could focus on developing learning support approaches with prompts and intermediate learning goals that allow medical and nursing post-graduates to bring interprofessional competencies to performance independent of the tandem partner or team. We consider the latter to be the more promising way to foster the transferability of individual competencies to later team performance. In a work environment, healthcare teams change quite regularly. Therefore, the aim of IPTW should be to prepare healthcare team members for this change. Further studies will also focus on the translation of learned interprofessional competence into later professional practice. Perhaps IPTW, with its externally valid approach and high complexity, are one of the messiest research settings in healthcare education. Because of their high cost and organizational effort, it is our duty as healthcare education researchers to design IPTW for the best learning environment possible. Aside from team stability, there is much to be found out.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Ethics Committee of the Medical Faculty Heidelberg (S-072/2017). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AMit, JM, and CM conducted the study, acquired financial support, and developed the design of methodology. AMit, JM, and CA collected data, conducted the research, and investigation process. BT-H and AMih provided resources that enabled the study to be conducted. AMit conceptualized this work, led the analysis, interpretation, and wrote the initial draft of this manuscript. AMit and AMö performed the statistical analysis. CU supervised the development of the IP-VITA instrument and the qualitative aspects of the observations. JK supervised the work. All authors corrected and approved the revisions and final version of the manuscript.

Funding

The Robert Bosch Stiftung, a German charitable foundation, funded the development and implementation of the Heidelberg Interprofessional Training Ward from 08/2016 until 07/2018 (32.5.A381.0026.0). The funder was not involved in the study design, collection, analysis, and interpretation of data, the writing of this article, or the decision to submit it for publication.

Acknowledgments

The authors thank Dr. Katja Krug for giving statistical support and feedback during the development of the IP-VITA instrument and Dr. Larry Kamees for the native language proofreading.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1241557/full#supplementary-material

References

1. WHO. Framework for Action on Interprofessional Education and Collaborative Practice. Geneva: WHO (2010).

Google Scholar

2. Lillebo B, Faxvaag A. Continuous interprofessional coordination in perioperative work: an exploratory study. J Interprof Care. (2015) 29:125–30. doi: 10.3109/13561820.2014.950724

PubMed Abstract | CrossRef Full Text | Google Scholar

3. van Leijen-Zeelenberg JE, van Raak AJ, Duimel-Peeters IG, Kroese ME, Brink PR, Vrijhoef HJ. Interprofessional communication failures in acute care chains: how can we identify the causes? J Interprof Care. (2015) 29:320–30. doi: 10.3109/13561820.2014.1003802

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. Washington, DC: National Academies Press (US) (2000).

Google Scholar

5. Francis R. Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. London: The Stationery Office. (2013).

Google Scholar

6. Barr H, Reeves S, Hammick M, Freeth D. Effective Interprofessional Education: Argument, Assumption and Evidence. Oxford: Blackwell (2005).

PubMed Abstract | Google Scholar

7. Collaborative Practice Through Learning Together to Work Together: Statement of Purpose. Centre for the Advancement of Interprofessional Education. (2016). Available online at: https://www.caipe.org/resource/CAIPE-Statement-of-Purpose-2016.pdf (accessed November 18, 2019).

Google Scholar

8. Cuff P, Schmitt M, Zierler B, Cox M, De Maeseneer J, Maine LL, et al. Interprofessional education for collaborative practice: views from a global forum workshop. J Interprof Care. (2014) 28:2–4. doi: 10.3109/13561820.2013.828910

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. (2010) 376:1923–58. doi: 10.1016/S0140-6736(10)61854-5

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Walkenhorst U, Mahler C, Aistleithner R, Hahn EG, Kaap-Frohlich S, Karstens S, et al. Position statement GMA committee–“interprofessional education for the health care professions”. GMS Z Med Ausbild. (2015) 32:Doc22. doi: 10.3205/zma000964

PubMed Abstract | CrossRef Full Text | Google Scholar

11. You P, Malik N, Scott G, Fung K. Current state of interprofessional education in Canadian medical schools: Findings from a national survey. J Interprof Care. (2017) 31:670–2. doi: 10.1080/13561820.2017.1315060

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Fischer MR, Bauer D, Mohn K. Finally finished! National competence based catalogues of learning objectives for undergraduate medical education (NKLM) and dental education (NKLZ) ready for trial. GMS Zeitschrift Med Ausbildung. (2015) 32:Doc35. doi: 10.3205/zma000977

PubMed Abstract | CrossRef Full Text | Google Scholar

13. BMBF. Masterplan Medizinstudium 2020. Berlin. (2016). Available online at: https://www.bmbf.de/files/2017-03-31_Masterplan%20Beschlusstext.pdf (accessed March 31, 2017).

Google Scholar

14. Nock L. Handlungshilfe zur Entwicklung von Interprofessionellen Lehrveranstaltungen in den Gesundheitsberufen. Stuttgart: Robert Bosch Stiftung GmbH. (2016).

Google Scholar

15. Nock L. Gemeinsam Besser Werden für den Patienten. Interprofessionelle Lehrkonzepte aus der Förderung der Robert Bosch. Stiftung Stuttgart: Robert Bosch Stiftung. (2018).

Google Scholar

16. Medizinischer Fakultätentag. Kapitel VIII.3 Interprofessionelle Kompetenzen. Nationaler Kompetenzbasierter Lernzielkatalog Medizin Version 2.0. Berlin: Charité Universitätsmedizin Berlin (2018).

Google Scholar

17. Federal Ministry of Justice Consumer Protection FOoJ. Act on the Nursing Profession (Pflegeberufegesetzt - PflBG). (2020). Available online at: https://www.bundesgesundheitsministerium.de/fileadmin/Dateien/3_Downloads/Gesetze_und_Verordnungen/GuV/P/PflBG_gueltig_am_1.1.2020_EN.pdf2017 (accessed June 15, 2023).

Google Scholar

18. Anderson ES, Ford J, Kinnair DJ. Interprofessional education and practice guide no. 6: developing practice-based interprofessional learning using a short placement model. J Interprof Care. (2016) 30:433–40. doi: 10.3109/13561820.2016.1160040

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lie DA, Forest CP, Kysh L, Sinclair L. Interprofessional education and practice guide No. 5: interprofessional teaching for prequalification students in clinical settings. J Interprof Care. (2016) 30:324–30. doi: 10.3109/13561820.2016.1141752

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Oosterom N, Floren LC, Ten Cate O, Westerveld HE. A review of interprofessional training wards: enhancing student learning and patient outcomes. Med Teacher. (2018) 5:1–8. doi: 10.1080/0142159X.2018.1503410

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Brewer ML, Stewart-Wynne EG. An Australian hospital-based student training ward delivering safe, client-centred care while developing students' interprofessional practice capabilities. J Interprof Care. (2013) 27:482–8. doi: 10.3109/13561820.2013.811639

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Dando N. d'Avray L, Colman J, Hoy A, Todd J. Evaluation of an interprofessional practice placement in a UK in-patient palliative care unit. Palliative Med. (2012) 26:178–84. doi: 10.1177/0269216311400479

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Freeth D, Reeves S, Goreham C, Parker P, Haynes S, Pearson S. 'Real life' clinical learning on an interprofessional training ward. Nurse Educ Today. (2001) 21:366–72. doi: 10.1054/nedt.2001.0567

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hallin K, Kiessling A. A safe place with space for learning: experiences from an interprofessional training ward. J Interprof Care. (2016) 30:141–8. doi: 10.3109/13561820.2015.1113164

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hood K, Cant R, Leech M, Baulch J, Gilbee A. Trying on the professional self: nursing students' perceptions of learning about roles, identity and teamwork in an interprofessional clinical placement. Appl Nurs Res. (2014) 27:109–14. doi: 10.1016/j.apnr.2013.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Jacobsen F, Fink AM, Marcussen V, Larsen K, Hansen TB. Interprofessional undergraduate clinical learning: results from a three year project in a Danish Interprofessional training unit. J Interprof Care. (2009) 23:30–40. doi: 10.1080/13561820802490909

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Lie DA, Forest CP, Walsh A, Banzali Y, Lohenry K. What and how do students learn in an interprofessional student-run clinic? An educational framework for team-based care. Med Educ Online. (2016) 21:31900. doi: 10.3402/meo.v21.31900

PubMed Abstract | CrossRef Full Text | Google Scholar

28. McGettigan P, McKendree J. Interprofessional training for final year healthcare students: a mixed methods evaluation of the impact on ward staff and students of a two-week placement and of factors affecting sustainability. BMC Med Educ. (2015) 15:185. doi: 10.1186/s12909-015-0436-9

PubMed Abstract | CrossRef Full Text | Google Scholar

29. McVey C, Vessey JA, Kenner CA, Pressler JL. Interprofessional dedicated education unit: an academic practice partnership. Nurse Educ. (2014) 39:153–4. doi: 10.1097/NNE.0000000000000051

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Morphet J, Hood K, Cant R, Baulch J, Gilbee A, Sandry K. Teaching teamwork: an evaluation of an interprofessional training ward placement for health care students. Adv Med Educ Pract. (2014) 5:197–204. doi: 10.2147/AMEP.S61189

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Pelling S, Kalen A, Hammar M, Wahlstrom O. Preparation for becoming members of health care teams: findings from a 5-year evaluation of a student interprofessional training ward. J Interprof Care. (2011) 25:328–32. doi: 10.3109/13561820.2011.578222

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Reeves S, Freeth D. The London training ward: an innovative interprofessional learning initiative. J Interprof Care. (2002) 16:41–52. doi: 10.1080/13561820220104159

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Reeves S, Freeth D, McCrorie P, Perry D. 'It teaches you what to expect in future. Interprofessional learning on a training ward for medical, nursing, occupational therapy and physiotherapy students. Med Educ. (2002) 36:337–44. doi: 10.1046/j.1365-2923.2002.01169.x

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Vanderzalm J, Hall MD, McFarlane LA, Rutherford L, Patterson SK. Fostering interprofessional learning in a rehabilitation setting: development of an interprofessional clinical learning unit. Rehabil Nurs. (2013) 38:178–85. doi: 10.1002/rnj.78

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Visser CLF, Kusurkar RA, Croiset G, Ten Cate O, Westerveld HE. Students' motivation for interprofessional collaboration after their experience on an IPE ward: a qualitative analysis framed by self-determination theory. Med Teacher. (2018) 14:1–9. doi: 10.1080/0142159X.2018.1436759

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Wilhelmsson M, Pelling S, Ludvigsson J, Hammar M, Dahlgren LO, Faresjo T. Twenty years experiences of interprofessional education in linkoping–ground-breaking and sustainable. J Interprof Care. (2009) 23:121–33. doi: 10.1080/13561820902728984

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Curtin University. Interprofessional Capability Framework. Perth: Curtin University (2023).

Google Scholar

38. Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, DC: Interprofessional Education Collaborative (2011).

Google Scholar

39. Interprofessional Education Collaborative. Core Competencies for Interprofessional Collaborative Practice. Update. Washington, DC: Interprofessional Education Collaborative (2016).

Google Scholar

40. Canadian Interprofessional Health Collaborative. A National Interprofessional Competency Framework 07.2023. (2016). Available online at: https://phabc.org/wp-content/uploads/2015/07/CIHC-National-Interprofessional-Competency-Framework.pdf (accessed June 15, 2023).

Google Scholar

41. Knowles M. The Adult Learner: A Neglected Species. Houston, TX: Gulf Publishing. (1990).

Google Scholar

42. Hean S, Craddock D, O'Halloran C. Learning theories and interprofessional education: a user's guide. Learn Health Soc Care. (2009) 8:250–62. doi: 10.1111/j.1473-6861.2009.00227.x

CrossRef Full Text | Google Scholar

43. Kaufman DM. Applying educational theory in practice. BMJ. (2003) 326:7382. doi: 10.1136/bmj.326.7382.213

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Vygotsky L. Mind in Society. The Development of Higher Psychological Process. Harvard: Harvard Univerity Press. (1980).

Google Scholar

45. Cooper H, Braye S, Geyer R. Complexity in interprofessional education. Learn Health Soc Care. (2004) 3:179–89. doi: 10.1111/j.1473-6861.2004.00076.x

CrossRef Full Text | Google Scholar

46. Mihaljevic AL, Schmidt J, Mitzkat A, Probst P, Kenngott T, Mink J, et al. Heidelberger Interprofessionelle Ausbildungsstation (HIPSTA): a practice- and theory-guided approach to development and implementation of Germany's first interprofessional training ward. GMS J Med Educ. (2018) 35:Doc33. doi: 10.3205/zma001179

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Mette M, Baur C, Hinrichs J, Oestreicher-Krebs E, Narciß E. Implementing MIA - Mannheim's interprofessional training ward: first evaluation results. GMS J Med Educ. (2018) 36:Doc35. doi: 10.3205/zma001243

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Brätz J, Bußenius L, Brätz I, Grahn H, Prediger S, Harendza S. Assessment of final-year medical students' entrustable professional activities after education on an interprofessional training ward: a case-control study. Perspect Med Educ. (2022) 11:266–72. doi: 10.1007/S40037-022-00720-0

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Bode SFN, Friedrich S, Straub C. 'We just did it as a team': learning and working on a paediatric interprofessional training ward improves interprofessional competencies in the short- and in the long-term. Med Teacher. (2022) 5:1–8. doi: 10.1080/0142159X.2022.2128998

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Blue AV, Chesluk BJ, Conforti LN, Holmboe ES. Assessment and evaluation in interprofessional education: exploring the field. J Allied Health. (2015) 44:73–82.

PubMed Abstract | Google Scholar

51. Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, et al. Saving lives: a meta-analysis of team training in healthcare. J Appl Psychol. (2016) 101:1266–304. doi: 10.1037/apl0000120

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Baik D, Abu-Rish Blakeney E, Willgerodt M, Woodard N, Vogel M, Zierler B. Examining interprofessional team interventions designed to improve nursing and team outcomes in practice: a descriptive and methodological review. J Interprof Care. (2018) 5:1–9. doi: 10.1080/13561820.2018.1505714

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Thistlethwaite J, Kumar K, Moran M, Saunders R, Carr S. An exploratory review of pre-qualification interprofessional education evaluations. J Interprof Care. (2015) 29:292–7. doi: 10.3109/13561820.2014.985292

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Reeves S, Palaganas J, Zierler B. An updated synthesis of review evidence of interprofessional education. J Allied Health. (2017) 46:56–61.

PubMed Abstract | Google Scholar

55. Interprofessional Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice. Report of an Expert Panel. Washington, DC: Interprofessional Education Collaborative (2011).

Google Scholar

56. Kirkpatrick DL. Evaluationg Training Programs. San Fransisco: BErrett-Koehler Publisher Inc. (1994).

Google Scholar

57. Barr H, Freeth D, Hammick M, Koppel I, Reeves JW. Evaluations of interprofessional education - a united kingdom review for health and social care. Fareham: UK Centre for the Advancement of Interprofessional Education. (2000).

PubMed Abstract | Google Scholar

58. Kuner C, Doerr-Harim C, Feisst M, Klotz R, Heger P, Probst P, et al. Clinical outcomes of patients treated on the Heidelberg interprofessional training ward vs. care on a conventional surgical ward: A retrospective cohort study. J Interprof Care. (2022) 36:552–9. doi: 10.1080/13561820.2021.1975667

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Mink J, Mitzkat A, Mihaljevic AL, Trierweiler-Hauke B, Gotsch B, Schmidt J, et al. The impact of an interprofessional training ward on the development of interprofessional competencies: study protocol of a longitudinal mixed-methods study. BMC Med Educ. (2019) 19:48. doi: 10.1186/s12909-019-1478-1

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Mink J, Mitzkat A, Krug K, Mihaljevic A, Trierweiler-Hauke B, Götsch B, et al. Impact of an interprofessional training ward on interprofessional competencies - a quantitative longitudinal study. J Interprof Care. (2021) 35:751–9. doi: 10.1080/13561820.2020.1802240

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Mink J, Mitzkat A, Scharzbeck V, Mihaljevic A, Trierweiler-Hauke B, Gotsch B, et al. Interprofessional socialization and collaboration on an interprofessional training ward - a reconstructive analysis. Z Evid Fortbild Qual Gesundhwes. (2022) 169:94–102. doi: 10.1016/j.zefq.2022.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Mink J, Zurek B, Gotsch B, Mihaljevic AL, Mitzkat A, Trierweiler-Hauke B, et al. How do former medical and nursing undergraduates describe their learning on an interprofessional training Ward 12-18 months later? - A retrospective qualitative analysis. BMC Med Educ. (2023) 23:275. doi: 10.1186/s12909-023-04212-5

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Mitzkat A, Mink J, Arnold C, Krug K, Mahler C, Trierweiler-Hauke B, et al. Individuelle Kompetenzen und Team-Performance in klinischen Lernsettings der interprofessionellen Zusammenarbeit erfassen: Die empirische Entwicklung des Interprofessionelle Visiten Individual und Team Assessment Tools (IP-VITA) [Measuring individual competencies and team performance in clinical learning settings of interprofessional collaborative practice: Empirical development of the Interprofessional Ward Round Individual and Team Assessment Tool (IP-VITA)]. Z Evid Fortbild Qual Gesundhwes. (2023) 179:39–48. doi: 10.1016/j.zefq.2023.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Thistlethwaite J, Dallest K, Moran M, Dunston R, Roberts C, Eley D, et al. Introducing the individual teamwork observation and feedback tool (iTOFT): development and description of a new interprofessional teamwork measure. J Interprof Care. (2016) 30:526–8. doi: 10.3109/13561820.2016.1169262

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Lie DA, Richter-Lagha R, Forest CP, Walsh A, Lohenry K. When less is more: validating a brief scale to rate interprofessional team competencies. Med Educ Online. (2017) 22:1314751. doi: 10.1080/10872981.2017.1314751

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Kiesewetter J, Fischer MR. The teamwork assessment scale: a novel instrument to assess quality of undergraduate medical students' teamwork using the example of simulation-based ward-rounds. GMS Zeitschrift fur medizinische Ausbildung. (2015) 32:Doc19. doi: 10.3205/zma000961

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Lie D, May W, Richter-Lagha R, Forest C, Banzali Y, Lohenry K. Adapting the McMaster-Ottawa scale and developing behavioral anchors for assessing performance in an interprofessional team observed structured clinical encounter. Med Educ Online. (2015) 20:26691. doi: 10.3402/meo.v20.26691

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. (2007) 29:642–7. doi: 10.1080/01421590701746983

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Lidskog M, Löfmark A, Ahlström G. Learning through participating on an interprofessional training ward. J Interprof Care. (2009) 23:486–97. doi: 10.1080/13561820902921878

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Ivarson J, Zelic L, Sondén A, Samnegård E, Bolander Laksov K. Call the On-Call: a study of student learning on an interprofessional training ward. J Interprof Care. (2021) 35:275–83. doi: 10.1080/13561820.2020.1725452

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Hallin K, Kiessling A, Waldner A, Henriksson P. Active interprofessional education in a patient based setting increases perceived collaborative and professional competence. Med Teach. (2009) 31:151–7. doi: 10.1080/01421590802216258

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Hylin U, Lonka K, Ponzer S. Students' approaches to learning in clinical interprofessional context. Med Teach. (2011) 33:e204–10. doi: 10.3109/0142159X.2011.557410

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Ericson A, Masiello I, Bolinder G. Interprofessional clinical training for undergraduate students in an emergency department setting. J Interprof Care. (2012) 26:319–25. doi: 10.3109/13561820.2012.676109

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Lachmann H, Ponzer S, Johansson UB, Benson L, Karlgren K. Capturing students' learning experiences and academic emotions at an interprofessional training ward. J Interprof Care. (2013) 27:137–45. doi: 10.3109/13561820.2012.724124

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Mette M, Baur C, Hinrichs J, Narciß E. Gaining interprofessional knowledge and interprofessional competence on a training ward. Med Teach. (2021) 43:583–9. doi: 10.1080/0142159X.2021.1885638

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of interprofessional team collaboration scale (AITCS): development and testing of the instrument. J Contin Educ Health Prof. (2012) 32:58–67. doi: 10.1002/chp.21123

PubMed Abstract | CrossRef Full Text | Google Scholar

77. King G, Shaw L, Orchard CA, Miller S. The interprofessional socialization and valuing scale: a tool for evaluating the shift toward collaborative care approaches in health care settings. Work. (2010) 35:77–85. doi: 10.3233/WOR-2010-0959

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Mahler C, Orchard C, Berger S, Krisam J, Mink J, Krug K, et al. Translation and psychometric properties of the German version of the “interprofessional socialization and valuing scale” (ISVS-21-D). J Interprof Care. (2023) 37:655–61. doi: 10.1080/13561820.2022.2115024

PubMed Abstract | CrossRef Full Text | Google Scholar

79. Scholl I, Zill JM, Härter M, Dirmaier J. An integrative model of patient-centeredness - a systematic review and concept analysis. PLoS ONE. (2014) 9:e107828. doi: 10.1371/journal.pone.0107828

PubMed Abstract | CrossRef Full Text | Google Scholar

80. Mihaljevic AL, Michalski C, Kaisers U, Strunk G. Patient-centeredness. Chirurgie. (2022) 93:861–9. doi: 10.1007/s00104-022-01629-4

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Kim EJ, Nam IC, Koo YR. Reframing patient experience approaches and methods to achieve patient-centeredness in healthcare: scoping review. Int J Environ Res Pub Health. (2022) 19:15. doi: 10.3390/ijerph19159163

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Castro EM, Van Regenmortel T, Vanhaecht K, Sermeus W, Van Hecke A. Patient empowerment, patient participation and patient-centeredness in hospital care: a concept analysis based on a literature review. Patient Educ Couns. (2016) 99:1923–39. doi: 10.1016/j.pec.2016.07.026

PubMed Abstract | CrossRef Full Text | Google Scholar

83. Spaulding EM, Marvel FA, Jacob E, Rahman A, Hansen BR, Hanyok LA, et al. Interprofessional education and collaboration among healthcare students and professionals: a systematic review and call for action. J Interprof Care. (2021) 35:612–21. doi: 10.1080/13561820.2019.1697214

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Orchard CA, Sonibare O, Morse A, Collins J, Al-Hamad A. Collaborative leadership, part 2: the role of the nurse leader in interprofessional team-based practice - shifting from task- to collaborative patient-/family-focused care. Nurs Leadersh (Tor Ont). (2017) 30:26–38. doi: 10.12927/cjnl.2017.25257

PubMed Abstract | CrossRef Full Text | Google Scholar

85. Wenger E. Communities of Practice. Learning, Meaning and Idetity. New York, NY: Cambridge University Press (1998).

Google Scholar

86. Nicolini D, Scarbrough H, Gracheva J. Communities of Practice and Situated Learning in Health Care. Oxford: Oxford University Press. (2016).

PubMed Abstract | Google Scholar

87. Kiesewetter J, Fischer F, Fischer MR. Collaborative clinical reasoning-a systematic review of empirical studies. J Contin Educ Health Prof. (2017) 37:123–8. doi: 10.1097/CEH.0000000000000158

PubMed Abstract | CrossRef Full Text | Google Scholar

88. Cummings GG, Lee S, Tate K, Penconek T, Micaroni SPM, Paananen T, et al. The essentials of nursing leadership: a systematic review of factors and educational interventions influencing nursing leadership. Int J Nurs Stud. (2021) 115:103842. doi: 10.1016/j.ijnurstu.2020.103842

PubMed Abstract | CrossRef Full Text | Google Scholar

89. Orchard CA, Sonibare O, Morse A, Collins J, Al-Hamad A. Collaborative leadership, Part 1: the nurse leader's role within interprofessional teams. Nurs Leadersh. (2017) 30:14–25. doi: 10.12927/cjnl.2017.25258

PubMed Abstract | CrossRef Full Text | Google Scholar

90. Khalili H, Orchard C, Laschinger HK, Farah R. An interprofessional socialization framework for developing an interprofessional identity among health professions students. J Interprof Care. (2013) 27:448–53. doi: 10.3109/13561820.2013.804042

PubMed Abstract | CrossRef Full Text | Google Scholar

91. Khalili H, Price SL. From uniprofessionality to interprofessionality: dual vs dueling identities in healthcare. J Interprof Care. (2022) 36:473–8. doi: 10.1080/13561820.2021.1928029

PubMed Abstract | CrossRef Full Text | Google Scholar

92. Thistlethwaite J. Interprofessional education: 50 years and counting. Med Educ. (2016) 50:1082–6. doi: 10.1111/medu.12959

PubMed Abstract | CrossRef Full Text | Google Scholar

93. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. (1990) 65:S63–7. doi: 10.1097/00001888-199009000-00045

PubMed Abstract | CrossRef Full Text | Google Scholar

94. Anderson O, Davis R, Hanna GB, Vincent CA. Surgical adverse events: a systematic review. Am J Surg. (2013) 206:253–62. doi: 10.1016/j.amjsurg.2012.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

95. Klaas S, Kara M, Nikki M, Rhona F, Simon PB. A ward-round non-technical skills for surgery (WANTSS) taxonomy. J Surg Educ. (2020) 77:369–79. doi: 10.1016/j.jsurg.2019.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar

96. Murray K, McKenzie K, Kelleher M. The evaluation of a framework for measuring the non-technical ward round skills of final year nursing students: an observational study. Nurse Educ Today. (2016) 45:87–90. doi: 10.1016/j.nedt.2016.06.024

PubMed Abstract | CrossRef Full Text | Google Scholar

97. Mitzkat A, Mink J. Visitenbeobachtungen: Interprofessionelle Zusammenarbeit quantitativ und qualitativ erfassen - Ein Erfahrungsaustausch. Jahrestagung der Gesellschaft für Medizinische Ausbildung. (2018) 179:39–48.

Google Scholar

98. Tromp F, Vernooij-Dassen M, Grol R, Kramer A, Bottema B. Assessment of CanMEDS roles in postgraduate training: the validation of the Compass. Patient Educ Couns. (2012) 89:199–204. doi: 10.1016/j.pec.2012.06.028

PubMed Abstract | CrossRef Full Text | Google Scholar

99. Probyn L, Lang C, Tomlinson G, Bandiera G. Multisource feedback and self-assessment of the communicator, collaborator, and professional CanMEDS roles for diagnostic radiology residents. Can Assoc Radiol J. (2014) 65:379–84. doi: 10.1016/j.carj.2014.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

100. Michels NR, Denekens J, Driessen EW, Van Gaal LF, Bossaert LL, De Winter BY, et al. Delphi study to construct a CanMEDS competence based inventory applicable for workplace assessment. BMC Med Educ. (2012) 12:86. doi: 10.1186/1472-6920-12-86

PubMed Abstract | CrossRef Full Text | Google Scholar

101. Fluit C, Bolhuis S, Grol R, Ham M, Feskens R, Laan R, et al. Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles. Med Teach. (2012) 34:893–901. doi: 10.3109/0142159X.2012.699114

PubMed Abstract | CrossRef Full Text | Google Scholar

102. Esteves A, McConnell M, Ferretti E, Garber A, Fung-Kee-Fung K. “When in doubt, ask the patient”: a quantitative, patient-oriented approach to formative assessment of canmeds roles. MedEdPORTAL. (2021) 17:11169. doi: 10.15766/mep_2374-8265.11169

PubMed Abstract | CrossRef Full Text | Google Scholar

103. Doja A, Eady K, Warren A, Wiesenfeld L, Writer H. Utilization of evidence-based tools and medical education literature by Canadian postgraduate program directors in the teaching and assessment of the CanMEDS roles. Can Med Educ J. (2021) 12:e57–62. doi: 10.36834/cmej.70183

PubMed Abstract | CrossRef Full Text | Google Scholar

104. Andreou V, Peters S, Eggermont J, Embo M, Michels NR, Schoenmakers B. Fitness-for-purpose of the CanMEDS competencies for workplace-based assessment in General Practitioner's Training: a Delphi study. BMC Med Educ. (2023) 23:204. doi: 10.1186/s12909-023-04207-2

PubMed Abstract | CrossRef Full Text | Google Scholar

105. Herion C, Egger L, Greif R, Violato C. Validating international CanMEDS-based standards defining education and safe practice of nurse anesthetists. Int Nurs Rev. (2019) 66:404–15. doi: 10.1111/inr.12503

PubMed Abstract | CrossRef Full Text | Google Scholar

106. Züricher Hochschule für Angewandte Wissenschaft (ZHAW). Abschlusskompetenzen (2023).

Google Scholar

107. Janssens O, Embo M, Valcke M, Haerens L. An online Delphi study to investigate the completeness of the CanMEDS Roles and the relevance, formulation, and measurability of their key competencies within eight healthcare disciplines in Flanders. BMC Med Educ. (2022) 22:260. doi: 10.1186/s12909-022-03308-8

PubMed Abstract | CrossRef Full Text | Google Scholar

108. Fontys Hogeschool Mens en Geondheid. Competentiegebieden Verpleegkundige in CanMEDS (2020).

Google Scholar

109. Wong WYA, Moni K, Roberts C, Thistlethwaite J. Engaging clinical examiners with structured feedback to enhance assessment practices. Med Teach. (2022) 44:914–21. doi: 10.1080/0142159X.2022.2049734

PubMed Abstract | CrossRef Full Text | Google Scholar

110. JISC. Assessment Literacy (2023).

Google Scholar

111. Berger S, Whelan B, Mahler C, Szecsenyi J, Krug K. Encountering complexity in collaborative learning activities: an exploratory case study with undergraduate health professionals. J Interprof Care. (2019) 33:490–6. doi: 10.1080/13561820.2018.1562423

PubMed Abstract | CrossRef Full Text | Google Scholar