Editorial: Multimodality in face-to-face teaching and learning: contemporary re-evaluations in theory, method, and pedagogy

Editorial on the Research Topic
Multimodality in face-to-face teaching and learning: contemporary re-evaluations in theory, method, and pedagogy

Sites of teaching and learning have long been a concern in the study of multimodality because of the impact education has on the production of future social subjects and the individual and collective potentials they embody. As such, it is incumbent upon teachers and researchers in education to take into account the impact that modes such as gesture, gaze, image, and the use of classroom space have in teaching, both independent of and in conjunction with spoken or written language. For instance, in one early multimodal classroom study, Kress et al. (2005) examined the multimodal structuring of knowledge in subject English at three state secondary schools in London, and found that image, gesture, and the use of classroom space communicated curricular elements which would be impossible in speech or writing alone. Other work on classroom multimodality of note includes de Silva Joyce and Feez's (2018) collection of studies, primarily from the pioneering social semiotic approach inspired by Kress and van Leeuwen (1996), containing chapters examining the classroom role of a range of modes, from gesture, to gaze, music, and design.

Building upon this prior work, this Research Topic highlights the diversity of theoretical and methodological approaches that characterizes the broad field of multimodality. It includes multimodal studies from around the world, including formal pedagogic contexts ranging from primary school to university, as well as a school for migrant and refugee children, and extends beyond traditional classrooms to include parent-child and professor-student office hour interactions. As such, this Research Topic provides a slice in time on the state of multimodal classroom research in the early 21st century.

Since multimodality transcends disciplinary boundaries and research traditions (e.g. Jewitt, 2009; Seizov and Wildfeuer, 2017), this Research Topic features contributions from multiple fields within multimodal studies, including gesture studies, multimodal Conversation Analysis, multimodal interaction analysis, and social semiotics. In their study of learner interactions in a school for migrant and refugee children in Greece, Tsikou and Papadopoulou explore how to enhance individuals' multimodal awareness, emphasizing its role in daily interactions and potential for intercultural misunderstandings. Their analysis of children's evolving “tactics” and “ways of doing” (de Certeau, 1990) highlights the need for reflection on communication strategies. Using video-stimulated recall interviews, they show how fostering this awareness can empower refugee and migrant children in utilizing their communicative resources. In his multimodal conversation analytic (e.g. Streek et al., 2011) study examining both audio-visual recordings as well as scanned assessment materials of K-12 reading assessments using commercially produced materials at an international school in Japan, Tomasine examines the practice of documenting feedback, or how teachers and students collaboratively create assessment interactions, both materially and discursively. Tomasine shows how, in formal, formative reading assessments in an elementary school, the degree to which participants focused on the formal assessment itself affected the record of the talk-in-interaction that was being assessed. Continuing to examine traditional educational contexts outside of formal teaching, Opazo et al.'s study investigates gesture alignment (Pickering and Garrod, 2006) in teacher–student office hour consultations conducted in English as a lingua franca. Analyzing 12 sessions with Spanish students and English-speaking lecturers, the researchers found that gestures were most often copied consecutively, mainly by teachers. Gesture alignment helped achieve mutual understanding, especially in L1–L2 contexts, and was driven by recurrent gesture forms. It played roles in negotiation, agreement, and clarifying meaning in academic interactions. Lopez-Ozieblo's study explores whether gesture-enriched grammatical explanations, grounded in a cognitive linguistics approach (e.g., Larsson and Stolpe, 2023), can benefit native speakers' understanding of language structures through multimodality. Addressing a research gap regarding use of gesture to teach abstract concepts, it shifts focus from vocabulary and phonetics to how young adult learners develop grammatical understanding. The study contributes to ongoing discussions through findings from an action-research project conducted in a naturalistic, multimodal learning environment. Finally, Tirosh and Chitrit explore mother–child interactions during joint computer gameplay, examining how maternal scaffolding occurs in informal, home-based digital settings. Using multimodal interaction analysis (e.g., Norris, 2004) and grounded theory heuristically, the researchers observed 20 dyads engaging with problem-solving games. They propose a three-stage model: initial child-dominance, role-based conflict, and eventual cooperation that allows for shared play and affective scaffolding. The study extends traditional concepts of scaffolding by highlighting how digital media and multimodal communication reshape parent–child dynamics. It offers valuable insight into learning processes outside formal education, where children often possess greater technological fluency than their parents.

In closing, these five contributions highlight the theoretically varied and methodologically rich field that is the study of multimodality in face-to-face learning environments. While this Research Topic does not feature any explicit comparison or critique of one approach against others, and is not a complete depiction of all studies of pedagogic interaction that have been called multimodal, it nevertheless provides an essential snapshot of the state of the art in this broad field. The editors would like to thank all contributors for their submissions. We would also like to acknowledge the community created through the Facebook Multimodal Researchers group, where the four Co-Editors met in response to a call for interest from Thomas Amundrud, and without which this multinational collaboration could not have occurred.

Author contributions

TA: Writing – original draft, Writing – review & editing. BA: Writing – original draft, Writing – review & editing. GC: Writing – review & editing, Writing – original draft. CS: Writing – original draft, Writing – review & editing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Gen AI was used in the creation of this manuscript.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

de Certeau, M. (1990). L'invention du quotidien. 1. Arts de faire. Paris: Gallimard.

Google Scholar

de Silva Joyce, H., and Feez, S. (2018). Multimodality across Classrooms: Learning About and Through Different Modalities. London: Routledge.

PubMed Abstract | Google Scholar

Jewitt, C. (2009). “Introduction to multimodal analysis,” in The Routledge Handbook of Multimodal Analysis, ed. C. Jewitt (London: Routledge), 14–27.

Google Scholar

Kress, G., Jewitt, C., Bourne, J., Franks, A., Hardcastle, J., Jones, K., and Reid, E. (2005). English in Urban Classrooms: A Multimodal Perspective on Teaching and Learning. London: Routledge Falmer.

Google Scholar

Kress, G., and van Leeuwen, T. (1996). Reading Images: The Grammar of Visual Design. London: Routledge.

Google Scholar

Larsson, A., and Stolpe, K. (2023). Hands on programming: teachers' use of metaphors in gesture and speech make abstract concepts tangible. Int. J. Technol. Des. Educ. 33, 901–919. doi: 10.1007/s10798-022-09755-0

Crossref Full Text | Google Scholar

Norris, S. (2004). Analyzing Multimodal Interaction. London: Routledge.

Google Scholar

Pickering, M. J., and Garrod, S. (2006). Alignment as the basis for successful communication. Res. Lang. Comput. 4, 203–228. doi: 10.1007/s11168-006-9004-0

Crossref Full Text | Google Scholar

Seizov, O., and Wildfeuer, J. (2017). “Introduction: rethinking multimodality in the twenty-first century,” in New Studies in Multimodality Conceptual and Methodological Elaborations, eds. O. Seizov and J. Wildfeuer (London: Bloomsbury), 1–13.

Google Scholar

Streek, J., Goodwin, C., and LeBaron, C. (2011). Embodied Interaction: Language and Body in the Material World. Cambridge, MA: Cambridge University Press.

PubMed Abstract | Google Scholar