<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="brief-report" dtd-version="2.3" xml:lang="EN">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Educ.</journal-id>
<journal-title>Frontiers in Education</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Educ.</abbrev-journal-title>
<issn pub-type="epub">2504-284X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/feduc.2024.1345661</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Education</subject>
<subj-group>
<subject>Perspective</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Empowering examiners to develop doctoral assessment literacy: a situated learning perspective</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Tan</surname>
<given-names>Wee Chun</given-names>
</name>
<xref ref-type="corresp" rid="c001"><sup>&#x002A;</sup></xref>
<uri xlink:href="https://loop.frontiersin.org/people/2589586/overview"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-original-draft/"/>
<role content-type="https://credit.niso.org/contributor-roles/writing-review-editing/"/>
</contrib>
</contrib-group>
<aff><institution>Universiti Putra Malaysia</institution>, <addr-line>Serdang, Selangor Darul Ehsan</addr-line>, <country>Malaysia</country></aff>
<author-notes>
<fn fn-type="edited-by" id="fn0001"><p>Edited by: Paitoon Pimdee, King Mongkut's Institute of Technology Ladkrabang, Thailand</p></fn>
<fn fn-type="edited-by" id="fn0002"><p>Reviewed by: Maresi Nerad, University of Washington, United States</p></fn>
<corresp id="c001">&#x002A;Correspondence: Wee Chun Tan, <email>weechun@upm.edu.my</email></corresp>
</author-notes>
<pub-date pub-type="epub">
<day>29</day>
<month>01</month>
<year>2024</year>
</pub-date>
<pub-date pub-type="collection">
<year>2024</year>
</pub-date>
<volume>9</volume>
<elocation-id>1345661</elocation-id>
<history>
<date date-type="received">
<day>28</day>
<month>11</month>
<year>2023</year>
</date>
<date date-type="accepted">
<day>16</day>
<month>01</month>
<year>2024</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x00A9; 2024 Tan.</copyright-statement>
<copyright-year>2024</copyright-year>
<copyright-holder>Tan</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p>
</license>
</permissions>
<abstract>
<p>Doctoral examiners play a pivotal role in upholding academic standards; therefore, they must be supported in their work. Drawing on situated learning theory, this paper proposes a conceptual training framework to empower examiners, particularly those participating in the PhD viva. The framework comprises three key initiatives: a professional development programme, a peer review, and an accreditation programme. The paper further highlights the potential benefits of the training initiatives and discusses implementation challenges aimed at fostering examiners&#x2019; continuous professional development, promoting a deeper understanding of assessment literacy, and maintaining the overall quality of doctoral assessment.</p>
</abstract>
<kwd-group>
<kwd>PhD viva</kwd>
<kwd>professional development</kwd>
<kwd>assessment literacy</kwd>
<kwd>examiners</kwd>
<kwd>doctoral education</kwd>
</kwd-group>
<counts>
<fig-count count="2"/>
<table-count count="0"/>
<equation-count count="0"/>
<ref-count count="20"/>
<page-count count="5"/>
<word-count count="3208"/>
</counts>
<custom-meta-wrap>
<custom-meta>
<meta-name>section-at-acceptance</meta-name>
<meta-value>Higher Education</meta-value>
</custom-meta>
</custom-meta-wrap>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="sec1">
<title>Introduction</title>
<p>Universities play a pivotal role in global higher education, ensuring that the doctoral programmes they offer are not only rigorous but also internationally recognised. While considerable resources are often invested in research supervision, the professional development of examiners receives less attention. Although progress is being made in some universities, such as the University of Otago, New Zealand, offering training for internal examiners, there is a scarcity of training specifically designed to prepare examiners for the viva. This gap raises questions about the readiness of examiners to navigate the multifaceted complexities of doctoral assessment, as it directly impacts the credibility and standing of the awarded qualifications (<xref ref-type="bibr" rid="ref1">Bernstein et al., 2014</xref>; <xref ref-type="bibr" rid="ref4">Chetcuti et al., 2022</xref>).</p>
<p>In the context of Malaysian doctoral education, the British model of assessment is adopted, involving a written thesis examination and a mandatory closed-door viva (oral examination). Notably, examiners are not part of the supervisory committee; instead, they are independently appointed from within and outside the university. Examiners take part in the PhD viva, which consists of three parts: the pre-viva meeting, the viva talk, and the post-viva meeting (<xref ref-type="bibr" rid="ref17">Tan, 2023a</xref>). Despite variations in doctoral assessment systems across countries (<xref ref-type="bibr" rid="ref15">Powell and Green, 2007</xref>), the potential benefits derived from examiner training remain universally applicable.</p>
<p>This study addresses the need for examiner support and training in doctoral assessment. With an understanding that examiners play a crucial role in upholding academic standards, it becomes imperative to enhance their capacity for rigorous and purpose-driven assessment. Drawing on my experience as a higher education researcher investigating the PhD viva and examiner practises, this paper extends my earlier work (<xref ref-type="bibr" rid="ref18">Tan, 2023b</xref>). In the prior study, given the limited learning opportunities, I advocated for supporting examiners in their assessment endeavours and called for institutional support within the Malaysian doctoral education context. In this study, I asked the question: <italic>How should examiners and their practises in the PhD viva be supported by the university for effective assessment?</italic></p>
<p>The data for this study were sourced through a review of existing professional learning initiatives for academics at selected universities. This includes universities, such as the University of Otago, New Zealand, University of Auckland, New Zealand, and University of Bristol, United Kingdom. Moreover, the IELTS training guide for prospective examiners was referred to gain insights on examiner recruitment and training.</p>
</sec>
<sec id="sec2">
<title>Situated learning theory</title>
<p>Situated learning theory (SLT) is adopted as the theoretical framework in this study to understand doctoral examiners and their assessment practises. The theory emphasises the importance of learning within authentic contexts and social interactions. It is grounded in the work of <xref ref-type="bibr" rid="ref12">Lave and Wenger (1991)</xref>, who argue that learning is a process that occurs through legitimate peripheral participation (LPP) in communities of practise (CoP). SLT proposes that knowledge acquisition and skill development are most effective when learners actively engage in real-world tasks and experiences. According to this perspective, learning is not solely a cognitive process that occurs within the individual, but rather a social and collaborative endeavour that occurs through active participation in meaningful activities.</p>
<p>The integration of SLT also extends into the recognition of the importance of social participation and the establishment of communities of practise. This perspective, emphasising learning as membership in a community, aligns with the dynamic and interactive nature of examiner roles within the doctoral assessment process. The acknowledgement of collective knowledge acquisition and shared activities complements the intricate nature of the assessment. Furthermore, the theory&#x2019;s acknowledgement of cultural dimensions underscores the fact that assessment practises are situated within broader social contexts, thereby influencing the outcomes and interpretations. By embracing the principles of situated learning, examiners can have meaningful learning experiences that bridge the gap between theory and practise, enhance learner engagement, and foster the development of practical examining skills and knowledge.</p>
</sec>
<sec id="sec3">
<title>Professional learning initiatives</title>
<p>Three initiatives are proposed to support examiners: a PDP, a peer review of the viva, and an accreditation programme. These initiatives:</p>
<list list-type="bullet">
<list-item><p>provide a platform for conversation;</p></list-item>
<list-item><p>encourage examiners to learn, reflect on their practise and enhance it, if necessary;</p></list-item>
<list-item><p>encourage examiners to share their practise with others; and</p></list-item>
<list-item><p>provide ongoing, accessible institutional support systems to enable effective learning and assessment.</p></list-item>
</list>
<p>When the three initiatives are implemented together, they form a provisional training framework that universities could use to support examiners and their examining practises (<xref ref-type="fig" rid="fig1">Figure 1</xref>). The framework could be included in examiner education either as informal, non-structured, formal, or structured support, provided it achieves the aim of supporting examiners for the doctoral assessment. Each of the initiatives is explained below.</p>
<fig position="float" id="fig1">
<label>Figure 1</label>
<caption>
<p>A provisional training framework for supporting examiners.</p>
</caption>
<graphic xlink:href="feduc-09-1345661-g001.tif"/>
</fig>
<sec id="sec4">
<title>Initiative I: professional development programme</title>
<p>A professional development programme (PDP) focusing on the viva should be offered. PDPs are designed to target learners on particular topics, are facilitated in some way, and aim to foster learning (<xref ref-type="bibr" rid="ref2">Borko, 2004</xref>). Nonetheless, current PDPs for academics tend to focus on how to examine a written thesis rather than on how to examine in the viva. As the viva is an essential part of the doctoral assessment, training specific to the viva is indispensable.</p>
<p>Inspired by existing doctoral supervision PDPs, a similar viva training programme could be developed. For instance, the University of Bristol offers three-session seminars on supervision, focusing on institutional policies and experiences, facilitated by an academic developer. This adaptable model supports academics aiming to learn supervision amidst a hectic workload. Illustrated in <xref ref-type="fig" rid="fig2">Figure 2</xref>, the three-session PDP emphasises institutional policies, examiner practises, and experiences, proposing potential learning topics. It can be conducted face-to-face or online, facilitated by academic developers, fostering a research discourse community of examiner practise (<xref ref-type="bibr" rid="ref12">Lave and Wenger, 1991</xref>).</p>
<fig position="float" id="fig2">
<label>Figure 2</label>
<caption>
<p>A PDP for examiners who are involved in the doctoral assessment.</p>
</caption>
<graphic xlink:href="feduc-09-1345661-g002.tif"/>
</fig>
<p>Potential and newly appointed examiners would be anticipated to attend the three sessions of the PDP to familiarise themselves with the PhD viva at the university. They would be introduced to the university&#x2019;s doctoral assessment and oral examination policy and gain the knowledge and skills to examine. The knowledge basis of the PDP could be informed by the current literature, such as thesis examination practises (e.g., <xref ref-type="bibr" rid="ref4">Chetcuti et al., 2022</xref>; <xref ref-type="bibr" rid="ref11">Kobayashi and Emmeche, 2023</xref>) and viva practises (e.g., <xref ref-type="bibr" rid="ref16">Tan, 2022</xref>, <xref ref-type="bibr" rid="ref17">2023a</xref>; <xref ref-type="bibr" rid="ref22">Wisker et al., 2022</xref>), and complemented by examiners&#x2019; experience sharing. The three sessions of the PDP could also be offered as an intensive one-day training. Regardless of the delivery mode, the learning outcome would be to ensure that examiners have a clearer view of the institutional policy and practises of the doctoral assessment expected of them at the university.</p>
</sec>
<sec id="sec5">
<title>Initiative II: peer review</title>
<p>Peer review should be introduced to examiners of the PhD viva. Peer review is a purposeful, non-judgemental, collaborative process whereby a colleague or peer is invited to observe a selected aspect/s of another&#x2019;s teaching or supervision and provide constructive feedback on its effectiveness in promoting student learning (<xref ref-type="bibr" rid="ref21">University of Otago, n.d.</xref>). Although peer review intends to help academics review and enhance their teaching and supervision practise, it can also be adapted as a professional learning tool and help academics in doctoral assessment, particularly the oral examination.</p>
<p>There are various ways to conduct peer review. This study suggests learning from the peer review of the teaching/supervision model, initiated at the University of Otago, New Zealand. This is a voluntary, collaborative academic activity that takes place among colleagues for learning. There are five steps to conducting a peer review of doctoral assessment for examiners:</p>
<list list-type="roman-lower">
<list-item><p>Choosing an appropriate peer for the peer review.</p></list-item>
<list-item><p>Identifying the aims, focus and roles in the briefing session.</p></list-item>
<list-item><p>Conducting the review.</p></list-item>
<list-item><p>Having a debriefing session after the review.</p></list-item>
<list-item><p>Encouraging critical reflection for learning.</p></list-item>
</list>
<p>The five-step peer review is a learning activity that could be used as part of an academic mentoring programme, with experienced academics guiding less experienced academics. Academics who are experienced examiners in the department, faculty, division, or university could be selected as mentors to assist novice examiners, such as by providing feedback on their examining practises. The review might involve a discussion session or observation of the examiner during the viva followed by a discussion. Less experienced examiners would discuss their practise with experienced colleagues to learn how to examine better in the viva, drawing on their reflections from the peer review. Examiners could also exchange their viva experiences with colleagues to help them identify good practises. This sharing of practise among examiners would foster more effective examining practise with an aim to encourage student learning in the oral examination (<xref ref-type="bibr" rid="ref19">Tinkler and Jackson, 2004</xref>).</p>
</sec>
<sec id="sec6">
<title>Initiative III: accreditation programme</title>
<p>An accreditation programme for examiners could be implemented by the university. &#x2018;Accreditation&#x2019; refers to training, monitoring, and certifying an academic to conduct an assessment. Since a PhD has global recognition, and the viva is a high-stakes examination, examiners should be properly trained and certified. Holding a PhD does not mean an academic can examine well in the assessment. To be appointed as an examiner, one should be accredited. This initiative is inspired by the implemented guidelines on becoming and supporting supervisors at the <xref ref-type="bibr" rid="ref20">University of Auckland (2012)</xref>, where academics need to be accredited before supervising postgraduates. The goal is to ensure consistent and high-quality supervision and research across the university, and similar guidelines could be developed for examiners.</p>
<p>Given the crucial role of examiners in the viva, they are responsible for ensuring that the assessment decision is valid and reliable. One way to ensure that examiners are accountable is through training and certification; however, with rare exceptions, this is not being done. A comparison with the appointment of examiners for the high-stakes English language test, the International English Language Testing System (IELTS) (<xref ref-type="bibr" rid="ref6">IELTS, n.d.</xref>), reveals that examiners must be trained and certified every 2&#x2009;years to ensure practise quality. This quality control is lacking in the doctoral assessment, where examiner practises rely on experience rather than formal training. Therefore, it is recommended that examiners receive training and perhaps be certified to allow them to fulfil their responsibilities in the assessment, uphold thesis quality, and align with university expectations.</p>
<p>While the IELTS and PhD viva differ in their focus&#x2014;English language proficiency for IELTS and research competency and communicative ability for the viva&#x2014;they are alike because both have an oral component. This study adapts the IELTS examiner recruitment and training model, incorporating insights from the Guidelines on Accreditation of Supervisors of <xref ref-type="bibr" rid="ref20">University of Auckland (2012)</xref>, to propose an accreditation programme for examiners. The six-step examiner recruitment and training guide includes:</p>
<list list-type="order">
<list-item><p><italic>Recruitment</italic></p>
<list list-type="simple">
<list-item><p>Academics considered for the role of examiner should have a relevant doctoral degree, experience supervising doctoral students, and be research active.</p></list-item>
</list></list-item>
<list-item><p><italic>Induction</italic></p>
<list list-type="simple">
<list-item><p>Potential examiners attend a PDP on the university&#x2019;s doctoral examination policy.</p></list-item>
</list></list-item>
<list-item><p><italic>Training</italic></p>
<list list-type="simple">
<list-item><p>Potential examiners attend the PDP to obtain doctoral examination knowledge and skills.</p></list-item>
</list>
</list-item>
<list-item><p><italic>Certification</italic></p>
<list list-type="simple">
<list-item>
<p>Examiners undertake a mock examination, evaluate a thesis and participate in a recorded or mock viva to demonstrate their PhD assessment abilities. Successful completion results in accreditation.</p>
</list-item>
</list>
</list-item>
<list-item><p><italic>Monitoring</italic></p>
<list list-type="simple">
<list-item>
<p>Regular monitoring by senior colleagues or examiners occurs at least annually, with the potential for peer reviews as part of the monitoring process.</p>
</list-item>
</list>
</list-item>
<list-item><p><italic>Standardisation and re-certification</italic></p>
<list list-type="simple">
<list-item>
<p>To maintain standardisation, examiners may undergo re-certification every 5&#x2009;years, involving assessments or performance appraisals, as appropriate.</p>
</list-item>
</list>
</list-item>
</list>
</sec>
</sec>
<sec sec-type="discussion" id="sec7">
<title>Discussion</title>
<p>I have presented three initiatives to support examiners and their practises of the PhD viva: a PDP, a peer review of the viva, and an accreditation programme. These initiatives are professional development opportunities to assist novice examiners in enhancing their competence and effectiveness in conducting the doctoral assessment. The overarching aim is to support examiners&#x2019; community of practise (<xref ref-type="bibr" rid="ref12">Lave and Wenger, 1991</xref>) by establishing standardised examining practises across the university, ensuring that potential examiners are well-equipped to meet the professional and personality requirements of the PhD assessment (<xref ref-type="bibr" rid="ref8">Kiley, 2009</xref>).</p>
<p>While acknowledging the merits of these initiatives, it is crucial to recognise that their implementation may not be without challenges. The PDP is considered the most feasible initiative, with existing programmes related to supervision and thesis examination serving as favourable precedents. Various supervisory development workshops are also available (<xref ref-type="bibr" rid="ref3">Brew and Peseta, 2004</xref>; <xref ref-type="bibr" rid="ref9">Kiley, 2011</xref>) and are known to be effective (<xref ref-type="bibr" rid="ref13">McCulloch and Loeser, 2016</xref>). Despite the non-mandatory nature of a viva-focused PDP, its integration into professional development opportunities holds significant potential in enhancing examiner capacities.</p>
<p>The second initiative, peer review of the viva, may present notable implementation complexities due to its reliance on observation and experience sharing. Although many academics recognise the benefits of peer observation, such as in developing teaching practise (<xref ref-type="bibr" rid="ref5">Hendry and Oliver, 2012</xref>), some academics might be sceptical of using peer review in the viva. Potential concerns about the evaluative nature of the review and the resultant impact on examiner dialogue must be carefully addressed. Moreover, the intricate power dynamics within academia, particularly concerning esteemed senior professors undergoing review by junior colleagues, present additional considerations. The cultural milieu, especially in collectivist societies like Malaysia, introduces nuances in how reviewers might provide critiques, emphasising the need for thoughtful navigation of potential challenges.</p>
<p>The third initiative, establishing an accreditation programme, stands out as the most intricate undertaking due to its substantial resource investment. While essential for maintaining the quality of doctoral assessment, executing the programme necessitates the development of new policies, training, and allocation of financial and human resources. Appointing experienced trainers familiar with examining practises and university culture is pivotal for the programme&#x2019;s efficacy. The success of the accreditation initiative, ensuring reliability and standardisation of doctoral assessment across the university, demands a high level of commitment given the ongoing and time-consuming nature of the process.</p>
<p>The three initiatives are believed to be beneficial in better preparing examiners for the viva and doctoral assessment, impacting both examiner and supervisory practises (<xref ref-type="bibr" rid="ref13">McCulloch and Loeser, 2016</xref>). Nevertheless, their effectiveness and impact require careful evaluation, considering potential weaknesses within the Malaysian or international context. Applying <xref ref-type="bibr" rid="ref10">Kirkpatrick and Kirkpatrick&#x2019;s (2006)</xref> training evaluation model is recommended for this purpose, alongside a thorough examination of academics&#x2019; receptiveness and a comprehensive study of the risks and consequences before, during, and after implementation.</p>
<p>Despite the study&#x2019;s foundation in personal experience and existing institutional initiatives, the training initiatives suggest a potential for implementation and further enhancement and expansion. The paper emphasises the need for future research to transcend its boundaries, exploring and implementing these initiatives to augment examiner support and refine their practical application in doctoral education worldwide.</p>
</sec>
<sec sec-type="data-availability" id="sec8">
<title>Data availability statement</title>
<p>The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.</p>
</sec>
<sec sec-type="author-contributions" id="sec9">
<title>Author contributions</title>
<p>WT: Writing &#x2013; original draft, Writing &#x2013; review &#x0026; editing.</p>
</sec>
</body>
<back>
<sec sec-type="funding-information" id="sec10">
<title>Funding</title>
<p>The author declares that no financial support was received for the research, authorship, and/or publication of this article.</p>
</sec>
<sec sec-type="COI-statement" id="sec11">
<title>Conflict of interest</title>
<p>The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="sec100" sec-type="disclaimer">
<title>Publisher&#x2019;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list>
<title>References</title>
<ref id="ref1"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Bernstein</surname> <given-names>B. L.</given-names></name> <name><surname>Evans</surname> <given-names>B.</given-names></name> <name><surname>Fyffe</surname> <given-names>J.</given-names></name> <name><surname>Halai</surname> <given-names>N.</given-names></name> <name><surname>Hall</surname> <given-names>F. L.</given-names></name> <name><surname>Marsh</surname> <given-names>H.</given-names></name> <etal/></person-group>. (<year>2014</year>). &#x201C;<article-title>The continuing evolution of the research doctorate</article-title>&#x201D; in <source>Globalization and Its Impacts on the Quality of PhD Education: Forces and Forms in Doctoral Education Worldwide</source>. eds. <person-group person-group-type="editor"><name><surname>Nerad</surname> <given-names>M.</given-names></name> <name><surname>Evans</surname> <given-names>B.</given-names></name></person-group> (<publisher-loc>Rotterdam</publisher-loc>: <publisher-name>Sense Publishers</publisher-name>), <fpage>5</fpage>&#x2013;<lpage>30</lpage>.</citation></ref>
<ref id="ref2"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Borko</surname> <given-names>H.</given-names></name></person-group> (<year>2004</year>). <article-title>Professional development and teacher learning: mapping the terrain</article-title>. <source>Educ. Res.</source> <volume>33</volume>, <fpage>3</fpage>&#x2013;<lpage>15</lpage>. doi: <pub-id pub-id-type="doi">10.3102/0013189X033008003</pub-id></citation></ref>
<ref id="ref3"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Brew</surname> <given-names>A.</given-names></name> <name><surname>Peseta</surname> <given-names>T.</given-names></name></person-group> (<year>2004</year>). <article-title>Changing postgraduate supervision practice: a programme to encourage learning through reflection and feedback</article-title>. <source>Innov. Educ. Teach. Int.</source> <volume>41</volume>, <fpage>5</fpage>&#x2013;<lpage>22</lpage>. doi: <pub-id pub-id-type="doi">10.1080/1470329032000172685</pub-id></citation></ref>
<ref id="ref4"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chetcuti</surname> <given-names>D.</given-names></name> <name><surname>Cacciottolo</surname> <given-names>J.</given-names></name> <name><surname>Vella</surname> <given-names>N.</given-names></name></person-group> (<year>2022</year>). <article-title>What do examiners look for in a PhD thesis? Explicit and implicit criteria used by examiners across disciplines</article-title>. <source>Assess. Eval. High. Educ.</source> <volume>47</volume>, <fpage>1358</fpage>&#x2013;<lpage>1373</lpage>. doi: <pub-id pub-id-type="doi">10.1080/02602938.2022.2048293</pub-id></citation></ref>
<ref id="ref5"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hendry</surname> <given-names>G. D.</given-names></name> <name><surname>Oliver</surname> <given-names>G. R.</given-names></name></person-group> (<year>2012</year>). <article-title>Seeing is believing: the benefits of peer observation</article-title>. <source>J. Univ. Teach. Learn. Pract.</source> <volume>9</volume>, <fpage>87</fpage>&#x2013;<lpage>96</lpage>. doi: <pub-id pub-id-type="doi">10.53761/1.9.1.7</pub-id></citation></ref>
<ref id="ref6"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll1">IELTS</collab></person-group> (<year>n.d.</year>). Examiner recruitment and training. Available at: <ext-link xlink:href="https://www.ielts.org/for-teachers/examiner-recruitment-and-training" ext-link-type="uri">https://www.ielts.org/for-teachers/examiner-recruitment-and-training</ext-link></citation></ref>
<ref id="ref8"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kiley</surname> <given-names>M.</given-names></name></person-group> (<year>2009</year>). <article-title>You don't want a smart alec': selecting examiners to assess doctoral dissertations</article-title>. <source>Stud. High. Educ.</source> <volume>34</volume>, <fpage>889</fpage>&#x2013;<lpage>903</lpage>. doi: <pub-id pub-id-type="doi">10.1080/03075070802713112</pub-id></citation></ref>
<ref id="ref9"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kiley</surname> <given-names>M.</given-names></name></person-group> (<year>2011</year>). <article-title>Developments in research supervisor training: causes and responses</article-title>. <source>Stud. High. Educ.</source> <volume>36</volume>, <fpage>585</fpage>&#x2013;<lpage>599</lpage>. doi: <pub-id pub-id-type="doi">10.1080/03075079.2011.594595</pub-id></citation></ref>
<ref id="ref10"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Kirkpatrick</surname> <given-names>D.</given-names></name> <name><surname>Kirkpatrick</surname> <given-names>J.</given-names></name></person-group> (<year>2006</year>). <source>Evaluating Training Programmes: The Four Levels</source> (<edition>3rd Edn.</edition>). <publisher-loc>San Francisco</publisher-loc>: <publisher-name>Berrett-Koehler</publisher-name></citation></ref>
<ref id="ref11"><citation citation-type="other"><person-group person-group-type="author"><name><surname>Kobayashi</surname> <given-names>S.</given-names></name> <name><surname>Emmeche</surname> <given-names>C</given-names></name></person-group>. (<year>2023</year>). <article-title>What makes a good PhD thesis? Norms of science as reflected in written assessments of PhD theses</article-title>. <source>Assess. Eval. High. Educ.</source>. doi: <pub-id pub-id-type="doi">10.1080/02602938.2023.2200917</pub-id> (Epub ahead of print).</citation></ref>
<ref id="ref12"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Lave</surname> <given-names>J.</given-names></name> <name><surname>Wenger</surname> <given-names>E.</given-names></name></person-group> (<year>1991</year>). <source>Situated Learning: Legitimate Peripheral Participation</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>.</citation></ref>
<ref id="ref13"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>McCulloch</surname> <given-names>A.</given-names></name> <name><surname>Loeser</surname> <given-names>C.</given-names></name></person-group> (<year>2016</year>). <article-title>Does research degree supervisor training work? The impact of a professional development induction workshop on supervision practice</article-title>. <source>High. Educ. Res. Dev.</source> <volume>35</volume>, <fpage>968</fpage>&#x2013;<lpage>982</lpage>. doi: <pub-id pub-id-type="doi">10.1080/07294360.2016.1139547</pub-id></citation></ref>
<ref id="ref15"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Powell</surname> <given-names>S.</given-names></name> <name><surname>Green</surname> <given-names>H.</given-names></name></person-group> (<year>2007</year>). <source>The Doctorate Worldwide</source>. <publisher-loc>Maidenhead</publisher-loc>: <publisher-name>McGraw-Hill</publisher-name>.</citation></ref>
<ref id="ref16"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tan</surname> <given-names>W. C.</given-names></name></person-group> (<year>2022</year>). <article-title>Speaking the language of defence: narratives of doctoral examiners on the PhD viva</article-title>. <source>Qual. Res. J.</source> <volume>22</volume>, <fpage>478</fpage>&#x2013;<lpage>488</lpage>. doi: <pub-id pub-id-type="doi">10.1108/QRJ-01-2022-0009</pub-id></citation></ref>
<ref id="ref17"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tan</surname> <given-names>W. C.</given-names></name></person-group> (<year>2023a</year>). <article-title>Purpose-driven oral examination: insights from doctoral viva examiners</article-title>. <source>Discov. Educ.</source> <volume>2</volume>:<fpage>52</fpage>. doi: <pub-id pub-id-type="doi">10.1007/s44217-023-00083-6</pub-id></citation></ref>
<ref id="ref18"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tan</surname> <given-names>W. C.</given-names></name></person-group> (<year>2023b</year>). <article-title>Doctoral examiners&#x2019; narratives of learning to examine in the PhD viva: a call for support</article-title>. <source>High. Educ.</source> <volume>86</volume>, <fpage>527</fpage>&#x2013;<lpage>539</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10734-022-00913-w</pub-id></citation></ref>
<ref id="ref19"><citation citation-type="book"><person-group person-group-type="author"><name><surname>Tinkler</surname> <given-names>P.</given-names></name> <name><surname>Jackson</surname> <given-names>C.</given-names></name></person-group> (<year>2004</year>). <source>The Doctoral Examination Process: A Handbook for Students, Examiners and Supervisors</source>. <publisher-loc>Maidenhead</publisher-loc>: <publisher-name>Open University Press</publisher-name>.</citation></ref>
<ref id="ref20"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll2">University of Auckland</collab></person-group> (<year>2012</year>). Guidelines on accreditation of supervisors, University of Auckland, New Zealand. Available at: <ext-link xlink:href="https://cdn.auckland.ac.nz/assets/central/about/the-university/how-the-university-works/policy-and-administration/Supervision/accreditation-of-supervisors.pdf" ext-link-type="uri">https://cdn.auckland.ac.nz/assets/central/about/the-university/how-the-university-works/policy-and-administration/Supervision/accreditation-of-supervisors.pdf</ext-link></citation></ref>
<ref id="ref21"><citation citation-type="other"><person-group person-group-type="author"><collab id="coll3">University of Otago</collab></person-group> (<year>n.d.</year>). Peer review and feedback about teaching and supervision. Available at: <ext-link xlink:href="https://www.otago.ac.nz/hedc/otago703482.pdf" ext-link-type="uri">https://www.otago.ac.nz/hedc/otago703482.pdf</ext-link></citation></ref>
<ref id="ref22"><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wisker</surname> <given-names>G.</given-names></name> <name><surname>Highman</surname> <given-names>L.</given-names></name> <name><surname>Spronken-Smith</surname> <given-names>R.</given-names></name> <name><surname>Waghorne</surname> <given-names>J.</given-names></name></person-group> (<year>2022</year>). <article-title>Across time and space: examiner and candidate experiences of online doctoral vivas</article-title>. <source>Innov. Educ. Teach. Int.</source> <volume>59</volume>, <fpage>131</fpage>&#x2013;<lpage>141</lpage>. doi: <pub-id pub-id-type="doi">10.1080/14703297.2021.2022528</pub-id></citation></ref>
</ref-list>
</back>
</article>