A Metalanguage for Learning: Rebalancing the Cognitive with the Socio-Material

Multiliteracies for the Digital Age

After the COVID-19 crisis, where the role of digital media in education and society has been highlighted, it is timely to reflect on what this new normal should be as well as recognize the social fault lines that have surfaced and need attention (Adami et al., 2020). The saving grace during the pandemic was the advancement of technology made over the years that enabled much of life to continue despite physical lockdowns and social distancing imposed all over the world. Interactions and transactions for work have also been sustained through technological platforms as work-from-home became the main mode of operation for knowledge economy workers, including teachers. Social communication and play shifted mostly online, with social media activities and digital gaming participation and purchases skyrocketing (Nagata et al., 2020). Schooling took the form of online learning in many countries (König et al., 2020; Ng 2020). Some might rightly wonder how life would have been if the pandemic had happened just a decade ago when we did not have the quality of digital tools and resources we have today. Those with access to networked digital communications can be grateful for the state-of-the-art technologies we presently enjoy. Indeed, our collective experience during COVID has highlighted a changed perception of technology, from its role as ancillary support to that of an enabler—and during the pandemic, ensuring the continuity of work, play, and learning (Peters et al., 2020).

The incorporation of technology as an integral part of our lives was not a sudden change, but a shift that has been occurring incrementally for some time, only to be accelerated and accentuated by the pandemic. More than 20 years into the 21st century, and almost 40 years after the revolutionary invention of hypertext markup language, the core technology of the World Wide Web, by Tim Berners-Lee in 1989 at CERN in Switzerland, we are now truly in the digital age. The recognition of the fast-changing communication environment is fundamental for educators in considering how best to prepare the young for their future and their active and meaningful social lives. This recognition requires educators to understand and examine the nature of literacy practices, that is the ways of meaning-making that young people are engaged in to ensure the continual relevance of what we are offering in our literacy curriculum. The conviction and courage to reflect on the relevance of and, if necessary, challenge the established philosophies guiding our literacy curriculum, as well as associated in-grained educational practices, will prevent a disconnect between the students’ literacy learning in the classroom and what they are experiencing in their lifeworlds (Figure 1) (Husserl 1936/1970).

FIGURE 1

FIGURE 1. The lifeworld contexts of learning.

One response to the challenge of connecting the changing dimensions of students’ lifeworld experiences with the practices of schooling is a multiliteracies pedagogy proposed by the New London Group (1996). Multiliteracies recognizes the many forms of literacies needed in the digital age, reflecting an “increasing local diversity and global connectedness” (The New London Group 1996, 61). A multiliteracies pedagogy supplements the traditional literacy pedagogy and focuses on modes of meaning-making beyond language. It is focused on meanings in the world in all their inter-related forms, not just language, and their material manifestations, not just their conceptualization. Its analyses seek patterns in these meanings, including the subtle and complex differences that arise in the agency of meaning-makers and their unique contexts. Multiliteracies includes both the diversity of situations and the multiplicity of text forms. The latter we describe as multimodal literacy. Following Jewitt and Kress (2003), we focus on the multimodal meaning-making in multiliteracies learning and use their term “multimodal literacy” (O’Halloran and Lim 2011; van Leeuwen 2017; Mills and Unsworth 2018) to draw attention to the codified set of knowledge and skills as well as a sensitivity to semiotic choices in the interpretation and creation of multimodal artefacts (Lim 2021a).

The agenda for a multiliteracies pedagogy is to engage all our students, not just the privileged few, so they can become “active designers–makers–of social futures” (New London Group 1996, 7). The attention to education justice has been expressed in many of the works on multiliteracies over the last quarter of the century (Unsworth 2001; Macken-Horarik 2009; Zammit 2011; Cope and Kalantzis 2015; Luke 2018; Suwalska 2021). Indeed, the role of the education system as a social leveler and to ensure equitable educational access to social opportunities and resources can also be served when the curriculum requirements, pedagogical practices, and assessment processes value the relevant literacies that reflect the possibilities and practices of the contemporary communicative environment in the digital age. Schools thus must address the new social fault lines caused by differential access and participation using digital technologies and mitigate the inequities resultant from the diverse lifeworld origins of students. For instance, the disparate experiences of students in their access to digital tools, social participation, and the range of literacy practices in their digital learning and play, across different social classes, can be disempowering to marginalized groups. Such inequities can be rendered invisible and the system fails to recognize that the uneven access to multimodal meaning-making can have profound consequences on students’ ability to optimize and participate fully in the opportunities that the digital age affords them (Lim et al., 2021a). In this light, educational justice demands that educators recognize the changing landscape of literacy practices in the digital age and include these expanded ways of meaning-making in the curriculum, develop pedagogies for teaching this broader range of literacies by making explicit what is often left tacit, analyzing how they function, and valuing them in assessment practices.

Educational justice is the ultimate impetus for the work in multiliteracies (Cope and Kalantzis in review). In this paper, we present a case for designing for educational justice that begins with an introduction of a pedagogic metalanguage of transposition for teaching and learning multimodal literacy. The purpose of this metalanguage is to find terms to describe the processes of meaning-making that are not limited to language, but capable of describing shared meanings between and across text, image, space, object, body, sound and speech. In this way, the same meanings can be expressed in these different forms (or modes), though never in the same ways. This is why we use the terms “multimodal” or “multiform” to describe meaning-making in the digital age, thereby extending and elaborating on the social functions of such forms. We argue that in order to mediate the invisible inequities caused by privileging specific or single ways of meaning-making in the education system, we must not ignore the diversity in students’ lifeworlds and their immersion in multiform meanings, particularly in the digital age. As a consequence, shifts need to be made in the curriculum, pedagogy, and assessment of traditional language-based literacy learning. We argue that a pedagogic metalanguage of transposition for teaching and learning multimodal literacy can serve the foundational purpose of providing teachers and students a shared vocabulary to describe and discuss the meanings made in multimodal discourse (Lim and Tan 2017) and support students’ development of broader semiotic awareness (Towndrow et al., 2013).

A Pedagogic Metalanguage of Transposition

A metalanguage is a language to describe and represent language itself (Jaworski et al., 2004). Within multimodality, the term “metalanguage” has been used to describe and represent other meaning-making systems, such as images (Kress and van Leeuwen 2021). In the educational context, a pedagogic metalanguage is a type of conceptual language that is appropriated and used in the school context as a resource for teachers and students. In this case, the disciplinary knowledge of multimodality has been appropriated or translated into a pedagogic metalanguage that is teachable and relevant for students. While based on the theoretical foundations of various frameworks in the research domain of multimodality (for example, Kress 2010; Kress and van Leeuwen 2021), a pedagogic metalanguage is centrally concerned with issues of teaching and learning. It needs to be “capable of supporting sophisticated critical analysis” of multimodal meaning-making, yet “not make unrealistic demands on teacher and learner knowledge” as well as “not immediately conjure up teachers’ accumulated and often justified antipathies towards formalism” (New London Group 1996, 77). As such, a pedagogic metalanguage needs to be both robust and accessible while at the same time motivating its use in multimodal literacy teaching and learning. In other words, a pedagogic metalanguage needs to be theoretically grounded yet practically useful for the classroom. Even as academics continue to improve the conceptual rigor and seek compelling empirical evidence for each conception behind the theoretical frameworks of multimodality (Bateman 2021), teaching and learning cannot wait for “gold standard” and education research must work with “sufficing standards” (Fullan 1994; Hargreaves and Fink 2000; Hung et al., 2015) of what is presently available to inform the urgent pedagogical changes we need to make to address the literacy demands of the digital age.

Additional education-related considerations in the development of a pedagogic metalanguage for teaching and learning multimodal meaning-making include that it has to be rendered age-appropriate across the primary and secondary years of education. Its design and use should also allow for differentiation across ability profiles of students. The terms and definitions used need to be sufficiently stable to be codified in the curriculum documents and adequately reliable to be tested in assessment. Lim and Tan (2017, 181) discuss the importance of a “translational process” from theoretical frameworks to a pedagogic metalanguage and describe the iterative work with teachers to “judiciously identify the necessary descriptions and choice of descriptors that are aligned to what they are already using to teach similar concepts in English language learning”. The translational research (Woolf 2008) in developing a pedagogic metalanguage needs to take account of the range of teachers’ abilities, readiness and motivation to engage with new knowledge (Albright and Kramer-Dahl 2009; Teo 2014; Macken-Horarik and Horarik 2019; Lim et al., 2021b). It should also be explicitly aligned with other aspects of the literacy curriculum to show clear connection and coherence across the curriculum.

A pedagogic metalanguage needs to be both structured and flexible in its design. It needs to be sufficiently structured to offer a systematic set of tools for discourse analysis and interpretation. Yet, it needs to be flexible to allow for the meanings that continue to shift with new forms of media over time and account for the influence of co-text and contexts which can expand or constrain the meanings made. The New London Group (1996, 77) has maintained that “Flexibility is critical because the relationship between descriptive and analytical categories and actual events is, by its nature, shifting, provisional, unsure, and relative to the contexts and purposes of analysis”. Despite the advancements made in multimodal studies over the decades, this principle continues to hold true as empirical findings continue to challenge the prevailing theoretical conceptions we have for a multimodal grammar (Bateman et al., 2017).

The case for a metalanguage for multimodal meaning-making in the classroom has been thoroughly articulated. For example, Unsworth (2006) explains that a metalanguage allows teachers and students to “describe meaning in various realms” (Cope and Kalantzis 2000, 24). Unsworth (2014, 13) also argues that “teachers and students need this kind of metalanguage for talking about language, images, sound, and so forth, and for their meaning-making interactions”. A metalanguage organizes students’ viewing around the meaning potential in particular multimodal genres to achieve specific purposes—that is, how the textual features fulfill typical functions.

Serafini (2011), 344 argues that a metalanguage allows students to take a “more critical reading position” and interrogate the meanings which creators expressed through the semiotic choices in their artefacts. Researchers have also reported on the positive results of teaching a metalanguage, or a grammar of multimodality, to primary and secondary school students in Australian schools. They explained that the metalanguage could “structure their noticing, offering a fresh view of choices that may have been taken for granted in diegetic reading” (Macken-Horarik et al., 2017, 255). Indeed, the heartening observation was that “The metalanguage was not only “turning up” in their writing and talking but appeared to re-shape their ways of thinking about images and their contribution to higher orders of meaning in multimodal texts” (Macken-Horarik et al., 2017, 259). A central tenet has been to develop in students “a common terminology to integrate their analyses in an interpretive response in ways acceptable to English teachers” (Macken-Horarik et al., 2017, 19). In other words, the metalanguage offers students tools for semiotic awareness of how meanings are made multimodally.

While it is useful to have a pedagogic metalanguage coded in curriculum documents to encourage motivated and equitable implementation for all stakeholders, it must be emphasized that the metalanguage is neither a set of rules nor standards of correctness. Rather, it is a resource which offers “a common tongue” to support the discussion, analysis, and interpretation of multimodal discourse. In this light, a pedagogic metalanguage of transposition for multimodal literacy learning should ultimately serve teachers’ and students’ semiotic work. The key principle should be fit for purpose, where teachers and students exercise agency in choosing the most apt tool in the tool kit for the interpretation and creation of multimodal artefacts. As such, it is imperative that a metalanguage can support the identification of similarities and differences across the features in various discourses and relate them to the purpose, the audience as well as context of situation and culture in which the discourse is inextricably embedded.

To this end, there has been much work in the development of various sets of metalanguage, as resources to support the investigation of specific semiotic relations, systems of meanings, and discourses for multimodal meaning-making. Unsworth (2004), Unsworth (2006) proposed a framework to map the image-text interactions in picture books for multimodal literacy learning, which was later extended to examine animated movies (Unsworth, 2013; 2014). More recently, Unsworth and Mills (2020) and Mills and others (2020) applied the appraisal framework (Martin and White 2005) from Systemic Functional Theory to teach students the systems of attitudinal meanings and emotions in their digital multimodal composing. Also drawing from Systemic Functional Theory and the work on visual grammar (Kress and van Leeuwen (2021), we have proposed a pedagogic metalanguage for films (Liang and Lim 2020), video games (Toh and Lim 2021), and educational apps (Lim and Toh, 2022).

Building on this work and responding to the call for a metalanguage that is sufficiently sophisticated yet flexible enough to support the interpretation and creation of multimodal artefacts (Macken-Horarik 2009; Macken-Horarik and Horarik 2019), Cope and Kalantzis, 2020; Kalantzis and Cope, 2020; Cope and Kalantzis, 2021; 2022) have proposed the development of a “transpositional grammar”. Rather than focusing on specific aspects of multimodal meaning-making, or specific genres, a “transpositional grammar” offers theoretical tools that can be translated into the pedagogic context as ways of thinking with and talking about multimodal meaning-making.

A pedagogic metalanguage of transposition reflects the multimodal nature of meaning-making and accounts for the multiplicities in the forms of literacy practices we can engage with in today’s digital age (van Leeuwen 2017). This metalanguage is, in a sense, the grammar of multimodality. It is a grammar, not in the traditional formalist and structuralist sense of prescriptive rules insisting on the correct and standard ways of usage. Rather, it is a grammar in the sense of mapping the patterns in multimodal meaning-making and “a tool kit for working on semiotic activities” with “fuzzy-edged, overlapping concepts” (New London Group 1996, 77). A pedagogic metalanguage of transposition does not assign fixed and context-neutral meanings to forms. Indeed, meanings can never be interpreted independent of context. As such, transposition reflects the shifts in the patterns of meanings—foregrounding the dynamic and fluid nature of multimodal communication, where choices in the semiotic modes are used to instantiate discernible meanings across the time-space continuum in communication (Cope and Kalantzis in review).

The focus on multimodal or multiform meaning presented in a transpositional grammar does not undermine the central role of language in meaning-making. Instead, it recognizes that language is nestled within an ensemble of other semiotic modes (Jewitt 2008) and the orchestration of meanings often involves shifts and movements across these modes. The interaction and interplay of semiotic modes, often described as intersemiosis (Royce, 1998; O’Halloran 2008), result in a multiplication of meanings (Lemke 1998). The intermodal relations, such as image-text relations (Liu and O’Halloran 2009; Unsworth 2001; 2006; 2013), and language-gesture relations (Martinec 2004; Hood 2011; Lim 2021b), and transmediation (Mills 2011; Mills and Brown 2021) have been the topic of study for many scholars working in multimodal studies. Of particular interest is the concept of “transduction” (Kress 2010) and “transmodal transformation” (Newfield 2014) which have been used in multimodal studies to examine shifts or resemiotization (Iedema 2003) in meanings across different modes. Cope and Kalantzis (2020) and Kalantzis and Cope (2020) explain that the notion of transposition is similar in its interest in the reconstitution of meanings. They outline a metalanguage of transposition in its expression on two dimensions—across meaning forms and across meaning functions. Meaning forms, such as written text, speech, sound, body, object, space, and image, are shaped by the materiality of the media. Meaning functions include the purposes of meaning in relation to what it is about (reference), who is doing it (agency), how it hangs together (structure), when and where it is situated (context), and why it is produced or what it is for (interest).

Thinking and Talking About Multimodal Meanings

Forms of Meaning

Cope and Kalantzis (2020) and Kalantzis and Cope (2020) identify seven forms of meaning based on the materiality of their media and the mix of human senses used in their representation and communication. They are written text, speech, sound, body, object, space, and image (Figure 2). The first concern of this transpositional grammar lies in the ways in which the material forms allow and constrain the meanings that can be made (Bateman et al., 2017) as well as how they insist in their affordances that certain meaning forms are used (Kress 2010). The second concern is the moveability of meanings from one form to another, and the patterns in the movement. For example, the concept of “love” is expressed as such in written text in the English language and pronounced as /lʌv/in speech, transcribed here using the International Phonetic Alphabet (IPA). While the relations between these forms and meanings may be arbitrary but conventionalized within the discourse community of English language speakers, what may be of greater interest is how the concept of “love” looks like when it is expressed with the body, an image or an object. Can space express love? Perhaps intimacy can be expressed with close proximity. And what are the wonderfully varied ways in which love can be expressed through images with conceptual metaphors? Drawing attention to the transposition in multimodal meaning-making encourages students to develop a rigorous understanding of multimodal literacy as they grow in their semiotic awareness when reflecting on the affordances of each form.

FIGURE 2

FIGURE 2. Transpositions between forms of meaning (Cope and Kalantzi, in review).

The meanings that can potentially be transposed across material forms are limitless. What may be of greater interest is the question of gains and losses in the movement across forms. For example, Kress (2005) examines the changes in meaning as more scientific representations shift from writing mode to the use of images in secondary school science textbooks over the years. He reflects on how the changes in dominant ways of representation affect the ways in which scientific knowledge is taught and learnt in schools. Similarly, beyond specific concepts to the higher narrative level, a pedagogic metalanguage of multimodality, expressed in terms of transposition, support students in their study of the shifts in meanings when a story, such as The Fantastic Flying Books of Mr. Morris Lessmore, is transposed from a picture book to an animated film, and to an interactive educational app (Djonov et al., 2021). The popularity of transmedia narratives (Jenkins 2003) offers rich resources which engage with students’ lifeworlds as they reflect on the patterns of meanings that change in each transposition. A transpositional grammar can support the description and discussion of these meaning patterns.

Transpositional grammar is an integrative grammar of multimodal meanings, and not just of a specific genre or the combinational relationships between two or three forms. It offers teachers and students a way to think about how meaning is made and shifted as it is expressed across different forms or modes and discourses and provide them with a common vocabulary to talk about the changes in meaning that occur in each transposition. The focus on the seven forms in this transpositional grammar is a powerful starting point for students to understand and undertake semiotic work. Each of the forms can then be deconstructed into various semiotic modes, that is the “socially shaped and culturally given resources for meaning-making” (Kress 2010, 79). For example, in the written text form, fonts and layout are the semiotic modes where specific selections made are meaningful besides the meaning of the words. Likewise, with the body, gestures and facial expressions are embodied modes of meaning-making (Lim 2021c). Color is a semiotic mode within the image form that can be used to convey both precise denotative and broad connotative meanings (Kress and van Leeuwen 2002; van Leeuwen 2010). While the parsing of the semiotic modes and their meaning potential within each form requires a more sophisticated mapping, which could be gradually introduced, where necessary, in a more detailed curriculum, the pedagogic metalanguage of transposition introduced here focuses on the distinction in material forms and offers a fundamental and interactive approach to learning multimodal literacy.

Functions of Meaning

Complementing the material forms in the pedagogic metalanguage of transposition is the functional dimension. Here, Kalantzis and Cope (2022) propose five functions—reference, agency, structure, context, and interest (Figure 3). The development of the five functions and the connection of these terms with concepts from multimodal studies, including those from a systemic functional theoretical orientation, are discussed in Cope and Kalantzis (2020). Kalantzis and Cope (2020) and Kalantzis and Cope (2022), these five functions are always present in meaning-making regardless of its form or combination of forms. Reference addresses the question of what the discourse is about, or what is represented in the discourse. Agency relates to roles of the meaning-maker as a participant in meaning and the patterns of action in the world to which the meaning refers. Structure is about the coherence of the discourse and how it is organized. Context determines the frames of reference within which the meanings in the discourse can be understood and the scope for interpretation. Interest explains the motivation for meaning-making and alludes to the question of whose interests are served with such representations.

FIGURE 3

FIGURE 3. The functions of meaning of transpositional grammar (Cope and Kalantzis in review).

In the classroom setting, teachers can draw attention to the functions of meaning which the discourse serves and guide students to parse these meanings in their analysis and interpretation of the discourse systematically. For example, the pedagogic metalanguage can be used in a discussion of a poster created by SG United, part of the Singapore government effort to support and encourage its citizens during the COVID-19 pandemic. The poster is situated within the SG United website under the Happiness Initiative (https://happinessinitiative.sg/stay-strong-sg/). The form of communication is a digital poster with image and text (Figure 4). Of particular interest for discussion could be the semiotic choices made within these two forms, for example, fonts, color, and cartoon emojis.

FIGURE 4

FIGURE 4. Poster from the SG United Website (retrieved from www.sgunited.gov.sg).

In terms of Reference, students could discuss what is the message being conveyed in the poster through the use of text and image. In this case, the poster clearly sends an encouragement to the whole nation of Singapore in the form of an imperative text “Stay Strong Singapore”. This encouragement is at the same time expressed through the image of a mask-wearing, flag-waving emoji demonstrating its strength by stepping on a coronavirus. In terms of Structure, students may notice that the coherence of the poster is well achieved by the fact that the image and the imperative text help to convey similar meanings of “staying strong”. Moreover, the repetition of Singapore as a meaning is multimodally realized by words, by logo and by the national colors of red and white in the poster. In terms of Agency, students might reflect on the use of cartoons to appeal to young people and to bring across the message in a relatable and friendly manner. They might also discuss how the use of bright colors attracts their attention and also note the color contrast of red and white to emphasize the written slogan “Stay Strong Singapore”. In terms of Context, students may situate the exhortatory message expressed in the poster in relation to the current pandemic, in particular the stress and psychological toll that the health crisis has imposed on many Singaporeans. Students may also recognize that the message is part of the Government propaganda to foster resilience amongst its people. Crucially, in terms of Interest, students may probe the motivation for the production of such posters in this particular way. They might wish to understand more about the people behind the Happiness Initiative who created the poster and debate on the effectiveness of the poster in communicating the message and in representing the initiative.

In another classroom example from an English Language lesson in a primary school in Singapore, the teacher used the pedagogic metalanguage as a resource to help students explore different ways of representing meanings. After reading a picture book of a story about a conflict between mother and daughter, in the students, in small groups of three to four learn about perspective-taking by role-playing a conflict situation at home and later taking a photo of a freeze-frame from the role-play. Through the role-play, students learn about how the forms of meanings such as speech, sound, object, and space can be used to represent the tension amongst the family members. The transposition of the role-play actions to a freeze-frame image (Figure 5) allows the students to discuss and reflect on how a photograph can depict specific meanings by offering one perspective of the conflict situation as well as influence the interpretation of the viewers.

FIGURE 5

FIGURE 5. Students’ taking a Freeze Frame.

The five functions of meaning can serve as a resource for the students thinking and talking about the meanings they want to express in the role-play of the conflict situation. For example, in terms of Reference, the students decide on the specific issue of conflict to play out, who the participants are, and what they should be doing in relation to Agency. Given that the role-play depicts a certain duration of the conflict, the students would need to discuss the Structure and decide how the specific scene works coherently. The Context also offers important consideration for the students to situate the scene they are acting within the broader situation and culture. The students have to make decisions based on the Interest and reflect on the purpose of the role-play as a pedagogic activity, such as being mindful of how the activity can help them appreciate multimodal meaning-making and learn about perspective taking.

As illustrated through a example of artefact analysis and interpretation as well as a representing activity in the classroom, a pedagogic metalanguage of transposition can be useful as a resource to support the thinking and talking about multimodal meaning-making. Figure 6 below shows how the form and function dimensions of transposition can come together in a matrix to be used as a toolkit for teaching and learning.

FIGURE 6

FIGURE 6. Form meets function in the matrix of transpositional grammar (Cope and Kalantzis in review).

Pedagogical Designs for Multimodal Literacy Learning

Refining and elaborating upon the pedagogy of multiliteracies (New London Group 1996), Cope and Kalantzis (2015) outline a repertoire of knowledge processes to guide the learning of multimodal literacy in the classroom. The four knowledge processes are experiencing, conceptualizing, analyzing and applying. Figure 7 shows how each of these processes is grounded in established pedagogical practices. In drawing from all major traditions of pedagogical practice, the knowledge processes reflect a balanced repertoire of pedagogical moves (Cope and Kalantzis in review) in line with a “principled eclecticism” (Cazden 2006) in literacies pedagogies.

FIGURE 7

FIGURE 7. Multiliteracies pedagogy: recruiting and supplementing traditions of educational practice (Cope and Kalantzis in review).

The knowledge processes suggest that teachers design strategic moves between one knowledge process and another, though in no necessary order. For example, a typical learning experience design could have students engage with multimodal resources from their lifeworlds through the knowledge process of experiencing, acquire the pedagogic metalanguage to discuss the discourse though conceptualizing, explore the meanings of the discourse collaboratively through analyzing, and creating multimodal artefacts through applying.

Matthiessen (2019) describes the swings in educational paradigms between teacher-centered learning and student-centered learning. Responding to the dominance of the thesis of teacher-centeredness in the educational paradigm of didactic pedagogy is the swing to the antithesis of student-centeredness of progressive pedagogy. Matthiessen (2019) argues, however, for the value of a synthesis in relationship-centered learning. The repertoire of knowledge processes expresses that synthesis and balance. For example, in conceptualizing, teachers can be didactic in guiding students to notice how meanings are made in multimodal discourse and introducing to them a pedagogic metalanguage of transposition. In analyzing, teachers can also apply an inductive approach to learning, where students explore various multimodal artefacts amongst themselves and elicit the principles of multimodal meaning-making on their own, before the teacher consolidates their understanding. The use of an inductive approach engenders greater engagement and ownership in students as they embark on epistemological work for themselves, and, while more time-consuming, it has been regarded as being more effective for learning (Haight et al., 2008; Qi and Lai 2017). The blend of explicit teaching and inductive teaching is thus an expression of a “reflexive pedagogy” (Cope and Kalantzis 2015). Through a “reflexive pedagogy” of blending both inquiry-based learning and didactic teaching, teachers can design for students’ multimodal literacy learning.

Formal education has long been biased towards the cognitive domain of learning, frequently at the expense of what we call the “socio-material domain”. This latter domain is closely connected with the materiality of lifeworld experience, as opposed to the cognitive domain which is frequently abstracted away from the everyday in its formal educational practices. The socio-material is intrinsically sociable, contrasted with the cognitive orientation of conventional schooling where knowledge is mostly framed through learning activities and testing, narrowly focused on the capacities for memory and procedural operations of individual, separated brains. Cognition is oriented to objectivity and formalized reasoning, contrasted with experiential lifeworld knowledge which is motivated by emotion, feeling and embodied subjectivity. Of course, the cognitive and the socio-material can never be separated, so we are talking about orientations to learning, where we are advocating a re-balancing of the two perspectives, socio-material and cognitive. With the engagement of their lifeworlds and the valuing of their emotions and embodied subjectivity, we argue that teachers and students can learn to recognize the influence of the pedagogies of self-reflection and their choices in meaning-making.

Here is a historical aside, symptomatic of the longstanding bias towards the cognitive in education: Benjamin Bloom and colleagues began in 1949 the enormously ambitious task of developing a taxonomy of educational goals in order to assess comparable outcomes across subject domains and educational institutions. The taxonomy they created involved a wide range of leading educational leaders. It has since become the pre-eminent framework for setting high-level educational objectives.

The Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook I, Cognitive Domain, was presented as an “educational-logical-psychological classification system” (Bloom et al., 1956, 6). Although the complete taxonomy was intended to cover three major domains—cognitive, affective, and psychomotor—by the time of its publication in 1956 the group had only managed to address the first, the cognitive domain. For this, they developed a series of six hierarchically ordered classes (Bloom et al., 1956, 17–20). Running from the lower to the higher order, these were: 1) knowledge, such as facts, terminology and theories; then a series of increasingly complex skills and abilities: 2) comprehension; 3) application; 4) analysis; 5) synthesis and 6) evaluation (Bloom et al., 1956).

The original group never did complete these other parts of their project on the affective and psychomotor. Eventually, in 1964, a research assistant on the original project, David Krathwohl, led a new team to add the affective domain: Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook II, Affective Domain. Krathwohl’s affective domain was ordered in the form of a hierarchy of internalization, starting with 1) receiving or attention, followed by 2) responding or reacting in a meaningful way, 3) valuing or attributing worth, 4) organizing into a value system, and 5) characterization by a value system or a characteristic lifestyle (Krathwohl. et al., 1964).

None of the original group got around to working on the missing third of the taxonomy, the psychomotor domain, a domain that we would today call “embodiment”. However, with a grant from the US government in 1965, a colleague of Krathwohl, Elizabeth Simpson, did (Simpson, 1966). Her Classification of Educational Objectives, Psychomotor Domain, adopted the same methodology as the other parts of the taxonomy project: literature review, consultations with leaders in the field, a conference that attracted national experts, and checking the objectives to exemplify them against a range of curricula (Krathwohl et al., 1964).

Today, only the cognitive part of Bloom’s taxonomy is used, as if that could serve as a comprehensive set of educational objectives. The revised 2001 edition of the taxonomy is single-mindedly cognitivist as if other major learning domains did not exist (Anderson and Krathwohl 2001). In our terms, the taxonomy is individualistic and mentalistic, without balance of crucial socio-material, embodied, or emotional aspects of learning.

This, we have tried to rectify in the knowledge process pedagogy which moves backwards and forwards between a cognitive and a socio-material orientation. In this pedagogy, the knowledge processes of experiencing and applying are socio-material in their primary orientation, and those of conceptualizing and analysing are more cognitive in their orientation (Figure 8). We use the word “orientation” because there can never be socio-material activity without cognition, and never cognition that is disembodied or isolated away from the social. We are speaking here of the foci of pedagogical attention. These always beg refocusing; the cognitive begs refocusing on the socio-material and the socio-material on the cognitive. Pedagogy is thus a matter of moving backwards and forwards across these different kinds of focus. This is another example of the process of transposition, now focused specifically on the processes of learning and teaching.

FIGURE 8

FIGURE 8. Socio-material and cognitive orientations to learning.

We have mapped the knowledge process pedagogy onto the full taxonomy by Bloom, Krathwohl and Simpson in the Supplementary Appendix SA1.

The knowledge process framework for educational objectives covers the full range of the Bloom et al. taxonomy, recovering crucial socio-material, embodied and emotional aspects that have been long neglected, not only in the taxonomy but education generally. However, there is one important respect in which the knowledge process framework differs from the Bloom taxonomy—these capacities cannot and should not be ordered into a hierarchy. All are important; all are equally challenging for learners. There is no developmental hierarchy and there is no necessary order of delivery. The knowledge processes present a repertoire of pedagogical alternatives. Any good learning activity should have a mix of at least some and potentially all knowledge processes, or at least a justifiable explanation for a certain emphasis on some knowledge processes ahead of others. The art of teaching is the apt orchestration of moves across processes—another transposition in other words.

Table 1 shows a newly elaborated overview of the knowledge processes.

TABLE 1

TABLE 1. Elaboration of knowledge processes.

In this way, the knowledge processes perform the role of a pedagogical grammar, calling out different kinds of learning activity. This is not only for teachers—to map the range of learning activities they are planning for their curriculum, suggesting perhaps that they adopt a wider repertoire of activity types. This pedagogical metalanguage can also be translated into explicit, age-appropriate markers for learners. In classroom interventions documented in a number of research projects, teachers have applied learner-friendly versions of these knowledge processes (Neville 2010; Arvanitis and Vitsilaki 2015; Cope and Kalantzis 2015; van Haren 2015; Zapata 2018).

Following are several examples of the knowledge process pedagogy at work. Experiencing the known involves students sharing the artefacts from their lifeworlds in the classroom. A common way to do this is to invite students to bring artefacts or tell stories from their daily lives as rich resources for discussion. For instance, in a lesson on understanding how persuasion works in multimodal texts, the teacher could have students take a photo or save an image of an interesting advertisement which they have come across and have them discuss with their peers how semiotic choices were made in the advertisement to appeal to them.

Experiencing the new, the teacher creates for learners an encounter that engages the socio-material domain of learning. The focus here is to encourage students to respond to the multimodal artefacts. It could be an intuitive response, such as seeing a burger depicted in the print makes me feel hungry and want to buy the food or a banal response that the discourse bores me. It could also be more sophisticated such as the film makes me feel nostalgia with a sense of poignancy, loss, and disquiet at the urban changes happening in my once familiar neighborhood as a result of gentrification.

Moving between the knowledge processes of analyzing functionally and conceptualizing by classifying and conceptualizing with theory, the teacher draws on the knowledge of the transpositional grammar described earlier. The focus here is on having students identify and offer textual evidence in support of the thoughts and feelings they have towards the artefact. Students operate within the cognitive domain and perform a close reading of the discourse to identify the specific semiotic choices that lead to their preliminary thoughts. Their analysis and interpretation of the artefact enable them to surface textual evidence to explain their response. For example, younger students in primary schools could identify the villain’s facial features and expression to explain why they were positioned to feel antipathy towards him. Older students in secondary schools could identify the use of minor keys in the musical score of a video to generate suspense.

Analyzing critically entails activities often highly prized as critical thinking and reading in schools. Here, students are encouraged to evaluate critically the composition both in terms of how successfully it has been created and whether the embedded values have been appropriately and effectively conveyed in the work. In other words, students adopt a critical inquiry stance in their engagement with the discourse and question the ideas represented in the artefact. Even as they recognize the discourse offers a single perspective which it advocates, they are encouraged to consider other possible perspectives that are absent, suppressed, or ignored. For example, primary school students could explore alternative endings of a fairytale or experiment with what might have happened if the gender roles were reversed. Secondary school students could identify sexism present in a discourse where women’s bodies are being commodified.

Applying appropriately is important in reinforcing the inextricable connection between the viewing and representing of multimodal artefacts (Lim et al., 2021a). Beyond responding, understanding and interpreting multimodal discourse, students must be given opportunities to create multimodal artefacts. Studies have highlighted the value of multimodal composing (e.g., Nelson 2008; Anderson 2017; Thibault and Curwood 2018; Smith et al., 2021) in students’ learning of multimodal literacy as well as in other aspects such as empathy (Jiang and Gao 2020).

Pushing at times in the direction of innovative or hybrid expressions of meaning, applying creatively offers students wider opportunities to be active and agentive designers of meaning. Such creative work could also be done in groups where students engage in collaborative meaning-making and negotiation of meanings with one another. For example, primary school students could be guided in the creation of a deck of presentation slides for their show-and-tell. Secondary school students could be given a theme with choices in topics for them to explore bending genre conventions in their multimodal composing as an expression of their creativity. In this aspect of multimodal representation, digital technology offers powerful tools such as sophisticated images or video editing and production tools, which were once available to professionals only, but are now mediated by simple interfaces and freely accessible to students in many countries.

Mediating Invisible Inequities in the Digital Age

In relation to the outcomes of schools, Cope and Kalantzis (in review) describe “epistemic capital” as the capacity to meld the ideal of conscious meaning with the materials of its media. Following Bourdieu, (1986) theory of cultural capital, epistemic capital reflects the knowledge and skills from learning, enabling students to act effectively, share meanings with others and to interpret, value, and work with the differences in others. Importantly, epistemic capital is not just cognitive; it is also socio-material, including the practical, embodied activities of meaning-making, with their affective or socio-emotional underpinnings and effects.

In this paper, we have described a pedagogic metalanguage of transposition to account for the processes of multimodal or multiform meaning-making, as well as a grammar of learning itself in the theory and practice of multiliteracies pedagogy and its taxonomy of knowledge processes. The idea of a pedagogic metalanguage has been applied to inform the design of learning all over the world (Purcell-Gates 2007; Hilton et al., 2010; Pahl and Roswell 2012). In Singapore, for example, some of the concepts discussed in this paper has been used to guide the design of lesson packages on multiliteracies for primary and secondary school students in a design-based research project to develop pedagogies for multiliteracies (Lim and Tan-Chia, 2018).

We argue that incorporating multimodal literacy in the curriculum, offering teachers and students a metalanguage as a resource for all students to engage in multimodal meaning-making, and supporting teachers to design the learning experiences based on knowledge processes that rebalances both the socio-material and the cognitive, will contribute to educational justice. Multiliteracies is the meaning-making essence of this epistemic capital and its pedagogy of access. Education is the pathway to epistemic capital and educational justice holds us accountable towards a more equitable distribution of resources, not just financial, but also attentional, in our schools today. Educational justice begins by valuing the diversity in students’ lifeworlds and recognizing the need to provide opportunities in schools that both legitimize and harness the new ways of meaning-making that young people are engaged with in the digital age. More crucially, a program of educational justice sets out to provide every child, regardless of their background, the tools and resources for thinking and talking about meaning-making to support their interpretation and creation of multimodal artefacts. In this paper, we have proposed that a pedagogic metalanguage of transposition can serve as a step towards this endeavor. Having a codified set of knowledge and skills to develop multimodal literacy in students provides a common set of resources that can inform the development of curriculum requirements and explicate learning outcomes in concrete terms. More importantly, as discussed in this paper, a pedagogic metalanguage of transposition can offer a framework for teachers in the design for students’ multimodal literacy learning. For example, the repertoire of knowledge processes can guide teachers in their pedagogical practice. A pedagogic metalanguage of transposition can also be appropriated as a set of criteria in the design of assessment for multimodal literacy. Teachers’ concerns over what to assess and how to measure the multimodal literacy of students can now be addressed by what is the knowledge and skills that have been introduced to the students.

While language will remain fundamental in the literacy classroom, the incorporation of multimodal literacy contributes to educational justice to ensure that the literacies needed for young people to participate agentively and fully in the digital age are not ignored in formal learning. Schools are powerful in shaping the future society through privileging a particular configuration of knowledge and skills to be learnt in the curriculum, determining the emphases and ways these knowledge and skills are to be learnt, and shaping the agenda of what is ultimately to be valued in the assessment.

The multiliteracies agenda has highlighted the centrality of multimodal meaning-making both as a reflection of the students’ present lifeworlds and a necessity as part of their future work competencies (New London Group 1996; Cope and Kalantzis 2000; The). Neglecting the learning of multimodal literacy in schools is to do injustice to students. While some students, because of their socio-economic backgrounds, access to resources, and richer experiences, may be more ready to engage in multimodal meaning-making, there are students who will appreciate the opportunities that school can provide in lieu of the distance between their home experience and the characteristic epistemologies of schooling.

Given the diversity of students, some may require further support and guidance from their teachers in the development of multimodal literacy. Attention to multimodal literacy paves the way toward educational justice in making explicit to some students what may be intuitive and tacit for others. The pedagogic metalanguage of transposition levels the playing field by offering a common shared set of vocabulary, with concepts, as tools for students to think and talk about multimodal meaning-making explicitly with teachers and amongst themselves. Demystifying multimodal meaning-making challenges the flawed assumption that multimodal interpretation and creation belongs to those naturally gifted with the intelligence and sensitivity to perceive nuances in meaning. As such, we posit that the teaching of multimodal literacy in schools will mediate the invisible inequities that may incidentally accompany the digital age. Such recognition, supported by proactive intervention, will contribute to the project of educational justice advanced by the advocates of multiliteracies.

Author Contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This study was funded by Singapore Ministry of Education (MOE) under the Education Research Funding Programme (DEV 01/18 VL) and administered by National Institute of Education (NIE), Nanyang Technological University, Singapore. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the Singapore MOE and NIE. This research has received clearance from the NTU-Institutional Review Board (IRB-2019-2-038).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomm.2022.830613/full#supplementary-material

References

Adami, E., Al Zidjaly, N., Canale, G., Djonov, E., Ghiasian, M. S., Gualberto, C., et al. (2020). “PanMeMic Manifesto: Making Meaning in the COVID-19 Pandemic and the Future of Social Interaction,” in Working Papers in Urban Language & Literacies.

Google Scholar

Albright, J., and Kramer‐Dahl, A. (2009). The Legacy of Instrumentality in Policy and Pedagogy in the Teaching of English: The Case of Singapore. Res. Pap. Educ. 24 (2), 201–222. doi:10.1080/02671520902867200

CrossRef Full Text | Google Scholar

Anderson, K. T. (2017). Leveraging Researcher Reflexivity to Consider a Classroom Event over Time: Reflexive Discourse Analysis of 'What Counts'. Classroom Discourse 8 (1), 36–54. doi:10.1080/19463014.2016.1271742

CrossRef Full Text | Google Scholar

Anderson, L. W., and Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.

Google Scholar

Arvanitis, E., and Vitsilaki, C. (2015). “Collaborative Professional Learning and Differentiated Teacher Practice: Learning by Design in Greece,” in A Pedagogy of Multiliteracies: Learning by Design. Editors B. Cope, and M. Kalantzis (London: Palgrave), 49–69. doi:10.1057/9781137539724_3

CrossRef Full Text | Google Scholar

Bateman, J. A. (2021). What Are Digital media? Discourse, Context & Media 41, 100502. doi:10.1016/j.dcm.2021.100502

CrossRef Full Text | Google Scholar

Bateman, J., Wildfeuer, J., and Hiippala, T. (2017). Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction. Mouton: De Gruyter.

Google Scholar

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., and Krathwohl, D. R. (1956). Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook I, Cognitive Domain. London: Longmans.

Google Scholar

Bourdieu, P. (1986). “Forms of Capital,” in Sociolology of Education: A Critical Reader. Editors A. Sadovnik, and R. Coughlan (New York: Routledge), 83–95.

Google Scholar

Cazden, C. B. (2006). The Value of Principled Eclecticism in Education Reform: 1965-2005. Pedagogies: Int. J. 1 (2), 93–104. doi:10.1207/s15544818ped0102_2

CrossRef Full Text | Google Scholar

Cope, B., and Kalantzis, M. (2015). “An Introduction to the Pedagogy of Multiliteracies,” in A Pedagogy of Multiliteracies: Learning by Design. Editors B. Cope, and M. Kalantzis (Hampshire: Palgrave Macmillan), 1–36.

CrossRef Full Text | Google Scholar

Cope, B., and Kalantzis, M. (2000). Multiliteracies: Literacy Learning and the Design of Social Futures. London, New York: Routledge.

Google Scholar

Cope, B., and Kalantzis, M. (2020). “Making Sense: Reference, Agency, and Structure of Multimodal Meaning. Cambridge, UK: Cambridge University Press.

Google Scholar

Cope, B., and Kalantzis, M. (2022). Review. “Towards Education Justice: A Pedagogy of Multiliteracies, Revisited”.

Google Scholar

Djonov, E., Tseng, C.-I., and Lim, F. V. (2021). Children's Experiences with a Transmedia Narrative: Insights for Promoting Critical Multimodal Literacy in the Digital Age. Discourse, Context & Media 43, 100493. doi:10.1016/j.dcm.2021.100493

CrossRef Full Text | Google Scholar

Evan Nelson, M. (2008). Multimodal Synthesis and the Voice of the Multimedia Author in a Japanese EFL Context. Innovation Lang. Learn. Teach. 2 (1), 65–82. doi:10.1080/17501220802158826

CrossRef Full Text | Google Scholar

Fullan, M. (1994). “Coordinating Top-Down and Bottom-Up Strategies for Educational Reform,” in The Governance of Curriculum: Yearbook for the Association for Supervision and Curriculum Development. Editors R. F. Elmore, and S. H. Fuhraman (Alexandria, VA: Association for Supervision and Curriculum Development), 186–202.

Google Scholar

Haight, C. E., Herron, C., and Cole, S. P. (2008). The Effects of Deductive and Guided Inductive Instructional Approaches on the Learning of Grammar in the Elementary Foreign Language College Classroom. Foreign Lang. Ann. 40, 288–310. doi:10.1111/j.1944-9720.2007.tb03202.x

CrossRef Full Text | Google Scholar

Hargreaves, A., and Fink, D. (2000). The Three Dimensions of Reform. Educ. Leadersh. 57 (7), 30–34.

Google Scholar

Hilton, A., Nichols, K., and Gitsaki, C. (2010). “Multiliteracies in Secondary ChemistryMultiliteracies in Secondary Chemistry: A Model for Using Digital Technologies to Scaffold the Development of Students’ Chemical Literacy,” in Multiliteracies and Technology Enhanced Education: Social Practice and the Global Classroom. Editors D. L. Pullen, and D. Cole (Hershey PA: IGI Global), 186–208. doi:10.4018/978-1-60566-673-0.ch012

CrossRef Full Text | Google Scholar

Hood, S. E. (2011). “Body Language in Face-To-Face Teaching: A Focus on Textual and Interpersonal Meaning,” in Semiotic Margins: Meaning in Multimodalities. Editor E. A. Thompson, S. E. Hood, and M. Stenglin. Editors (London: Continuum), 31–52.

Google Scholar

Hung, D., Lee, S.-S., and Wu, L. (2015). Toward an Educational View of Scaling: Sufficing Standard and Not a Gold Standard. Educ. Res. Pol. Prac 14, 77–91. doi:10.1007/s10671-014-9164-x

CrossRef Full Text | Google Scholar

Husserl, E. (1936/1970). The Crisis of the European Sciences. Evanston: North Western University.

Google Scholar

Iedema, R. (2003). Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-Semiotic Practice. Vis. Commun. 2 (1), 29–57. doi:10.1177/1470357203002001751

CrossRef Full Text | Google Scholar

Jaworski, A., Coupland, N., and Galasinski, D. (2004). Metalanguage: Social and Ideological Perspectives (Language, Power and Social Process. Berlin: Walter de Gruyter.

Google Scholar

Jenkins, H. (2003). Transmedia Storytelling: Moving Characters from Books to Films to Video Games Can Make Them Stronger and More Compelling. MIT Technology Review.

Google Scholar

Jewitt, C., and Kress, G. (2003). Multimodal Literacy. New York: Peter Lang.

Google Scholar

Jewitt, C. (2008). “Multimodal Discourses across the Curriculum,” in Encyclopedia of Language and Education. Editor N. H. Hornberger (Boston, MA: Springer), 1098–1108. doi:10.1007/978-0-387-30424-3_84

CrossRef Full Text | Google Scholar

Jiang, L., and Gao, J. (2020). Fostering EFL Learners' Digital Empathy through Multimodal Composing. RELC J. 51 (1), 70–85. doi:10.1177/0033688219898565

CrossRef Full Text | Google Scholar

Kalantzis, M., and Cope, B. (2020). Adding Sense: Context and Interest in a Grammar of Multimodal Meaning. Cambridge UK: Cambridge University Press.

Google Scholar

Kalantzis, M., and Cope, B. (2022). “After Language: A Grammar of Multimodal Transposition,” in Foreign Language Learning in the Digital Age: Theory and Pedagogy for Developing Literacies. Editor Lütge. Christiane (”London: Routledge), 34–64.

Google Scholar

König, J., Jäger-Biela, D. J., and Glutsch, N. (2020). Adapting to Online Teaching during COVID-19 School Closure: Teacher Education and Teacher Competence Effects Among Early Career Teachers in Germany. Eur. J. Teacher Educ. 43 (4), 608–622. doi:10.1080/02619768.2020.1809650

CrossRef Full Text | Google Scholar

Krathwohl, D. R., Bloom, B. S., and Masia, B. B. (1964). Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook II, Affective Domain. New York: David McKay Company.

Google Scholar

Kress, G. (2005). Gains and Losses: New Forms of Texts, Knowledge, and Learning. Comput. Compost. 22 (1), 5–22. doi:10.1016/j.compcom.2004.12.004

CrossRef Full Text | Google Scholar

Kress, G. (2010). Multimodality: A Social Semiotic Approach to Contemporary Communication. Abingdon, New York: Routledge.

Google Scholar

Kress, G., and van Leeuwen, T. (2002). Colour as a Semiotic Mode: Notes for a Grammar of Colour. Vis. Commun. 1 (3), 343–368. doi:10.1177/147035720200100306

CrossRef Full Text | Google Scholar

Kress, G., and van Leeuwen, T. (20211996). Reading Images: The Grammar of Visual Design. New York: Routledge [first edition.

Google Scholar

Lemke, J. (1998). “Multiplying Meaning: Visual and Verbal Semiotics in Scientific Text,” in Reading Science. Editors J. R. Martin, and R. Veel (London: Routledge), 87–113.

Google Scholar

Liang, W. J., and Lim, F. V. (2020). A Pedagogical Framework for Digital Multimodal Authoring in the English Language Classroom. Innov. Lang. Learn. Teach. 15 (4), 306–320. doi:10.1080/17501229.2020.1800709

CrossRef Full Text | Google Scholar

Lim, F. V. (2021c). Designing Learning with Embodied Teaching: Perspectives from Multimodality. London: Routledge.

Google Scholar

Lim, F. V. (2021b). Investigating Intersemiosis: A Systemic Functional Multimodal Discourse Analysis of the Relationship between Language and Gesture in Classroom Discourse. Vis. Commun. 20 (1), 34–58. doi:10.1177/1470357218820695

CrossRef Full Text | Google Scholar

Lim, F. V., and Tan, K. Y. S. (2017). “Multimodal Translational Research: Teaching Visual Texts,” in New Studies in Multimodality: Conceptual and Methodological Elaborations. Editors O. Seizov, and J. Wildfeuer (London/New York: Bloomsbury), 175–200.

Google Scholar

Lim, F. V., and Tan-Chia, L. (2018). Designing Learning for Multimodal Literacy: Teaching Viewing and Representing. London: Routledge.

Google Scholar

Lim, F. V., and Toh, W. (2022). “Developing a Metafunctional Framework for Educational Apps,” in Multimodality in English Language Learning, Routledge Studies in Multimodality. Editors S. Diamantopoulou, and S. Ørevik (London, New York: Routledge).

Google Scholar

Lim, F. V., Towndrow, P. A., and Tan, J. M. (2021a). Unpacking the Teachers’ Multimodal Pedagogies in the Singapore Primary English Language Classroom. RELC J. Online first.

Google Scholar

Lim, F. V., Weninger, C., and Nguyen, T. T. H. (2021b). "I Expect Boredom": Students' Experiences and Expectations of Multiliteracies Learning. Literacy 55 (2), 102–112. doi:10.1111/lit.12243

CrossRef Full Text | Google Scholar

Lim, F. V. (2021a). “Towards Education 4.0: An Agenda for Teaching Multiliteracies in the English Language Classroom,” in Literacies, Culture, and Society towards Industrial Revolution 4.0: Reviewing Policies, Expanding Research, Enriching Practices in Asia. Editor F. A. Hamied (New York: Nova Science), 11–30.

Google Scholar

Liu, Y., and O'Halloran, K. L. (2009). Intersemiotic Texture: Analyzing Cohesive Devices between Language and Images. Social Semiotics 19 (4), 367–388. doi:10.1080/10350330903361059

CrossRef Full Text | Google Scholar

Luke, A. (2018). Critical Literacy, Schooling, and Social Justice: The Selected Works of Allan Luke. London: Routledge.

Google Scholar

Macken-Horarik, M., and Horarik, S. (2019). “‘Multiliteracies? I’m All for it … Just Don’t Ask Me How to Teach it’: Insights from a Survey of Australian Teachers,” in Paper presented at the 29th European Systemic Functional Linguistics Conference (Leiria PT.

Google Scholar

Macken-Horarik, M., Love, K., Sandiford, C., and Unsworth, L. (2017). Functional Grammatics: Re-conceptualising Knowledge about Language and Image for School English. London: Routledge.

Google Scholar

Macken-Horarik, M. (2009). Multiliteracies, Metalanguage and the Protean Mind: Navigating School English in a Sea of Change. English Aust. 44 (1), 33–43.

Google Scholar

Martin, J., and White, P. (2005). The Language of Evaluation: Appraisal in English. New York, NY: Macmillan.

Google Scholar

Martinec, R. (2004). Gestures that Co‐occur with Speech as a Systematic Resource: the Realization of Experiential Meanings in Indexes. Soc. Semiotics 14 (2), 193–213. doi:10.1080/1035033042000238259

CrossRef Full Text | Google Scholar

Matthiessen, C. M. I. M. (2019). Register in Systemic Functional Linguistics*. Rs 1 (1), 10–41. doi:10.1075/rs.18010.mat

CrossRef Full Text | Google Scholar

Mills, K. (2011). I’m Making it Different to the Book’: Transmediation in Young Children’s Multimodal and Digital TextsAustralas. J. Early Child. 36 (3), 56–65. doi:10.1177/183693911103600308

CrossRef Full Text | Google Scholar

Mills, K. A., and Brown, A. (2021). Immersive Virtual Reality (VR) for Digital Media Making: Transmediation Is Key. Online First. doi:10.1080/17439884.2021.1952428

CrossRef Full Text | Google Scholar

Mills, K. A., Stone, B. G., Unsworth, L., and Friend, L. (2020). Multimodal Language of Attitude in Digital Composition. Written Commun. 37 (2), 135–166. doi:10.1177/0741088319897978

CrossRef Full Text | Google Scholar

Mills, K. A., and Unsworth, L. (2018). iPad Animations: Powerful Multimodal Practices for Adolescent Literacy and Emotional Language. J. Adolesc. Adult Liter 61 (6), 609–620. doi:10.1002/jaal.717

CrossRef Full Text | Google Scholar

Nagata, J. M., Abdel Magid, H. S., and Pettee Gabriel, K. (2020). Screen Time for Children and Adolescents during the Coronavirus Disease 2019 Pandemic. Obesity 28 (9), 1582–1583. doi:10.1002/oby.22917

PubMed Abstract | CrossRef Full Text | Google Scholar

Neville, M. (2010). Meaning Making Using New Media: Learning by Design Case Studies. E-Learning and Digital Media 7 (3), 237–247. doi:10.2304/elea.2010.7.3.237

CrossRef Full Text | Google Scholar

New London Group (1996). A Pedagogy of Multiliteracies: Designing Social Futures. Harv. Educ. Rev. 66 (1), 60–92. doi:10.17763/haer.66.1.17370n67v22j160u

CrossRef Full Text | Google Scholar

Newfield, D. (2014). Transformation, Transduction and the Transmodal Moment. Routledge Handbook Multimodal Anal., 100–113.

Google Scholar

Ng, P. T. (2020). Timely Change and Timeless Constants: COVID-19 and Educational Change in Singapore. Educ. Res. Pol. Prac 20, 19–27. doi:10.1007/s10671-020-09285-3

CrossRef Full Text | Google Scholar

O'Halloran, K. L. (2008). Systemic Functional-Multimodal Discourse Analysis (SF-MDA): Constructing Ideational Meaning Using Language and Visual Imagery. Vis. Commun. 7 (4), 443–475. doi:10.1177/1470357208096210

CrossRef Full Text | Google Scholar

O’Halloran, K. L., and Lim, F. V. (2011). “Viden Om Læsning” (Knowledge about Reading). Dimensions Of Multimodal Literacy.

Google Scholar

Pahl, K., and Rowsell, J. (2012). Literacy and Education: Understanding the New Literacy Studies in the Classroom. Thousand Oaks CA: Sage.

Google Scholar

Peters, M. A., Rizvi, F., McCulloch, G., Gibbs, P., Gorur, R., Hong, M., et al. (2020). Reimagining the New Pedagogical Possibilities for Universities Post-Covid-19, Educ. Philos. Theor., 1–44. doi:10.1080/00131857.2020.1777655

CrossRef Full Text | Google Scholar

Purcell-Gates, V. (2007). Cultural Practices of Literacy: Case Studies of Language, Literacy, Social Practice, and Power. Hillsdale NJ: Lawrence Erlbaum Associates.

Google Scholar

Qi, X., and Lai, C. (2017). The Effects of Deductive Instruction and Inductive Instruction on Learners' Development of Pragmatic Competence in the Teaching of Chinese as a Second Language. System 70, 26–37. doi:10.1016/j.system.2017.08.011

CrossRef Full Text | Google Scholar

Royce, T. (1998). Synergy on the Page: Exploring Intersemiotic Complementarity in Page-based Multimodal Text. JASFL Occas. Pap. 1 (1), 25–50.

Google Scholar

Serafini, F. (2011). Expanding Perspectives for Comprehending Visual Images in Multimodal Texts. J. Adolesc. Adult Literacy 54 (5), 342–350. doi:10.1598/jaal.54.5.4

CrossRef Full Text | Google Scholar

Simpson, E. J. (1966). The Classification of Educational Objectives, Psychomotor Domain. Ill. J. Home Econ. 10 (4), 110–144. doi:10.1093/oxfordjournals.afraf.a095474

CrossRef Full Text | Google Scholar

Smith, B. E., Pacheco, M. B., and Khorosheva, M. (2021). Emergent Bilingual Students and Digital Multimodal Composition: A Systematic Review of Research in Secondary Classrooms. Read. Res. Q. 56 (1), 33–52. doi:10.1002/rrq.298

CrossRef Full Text | Google Scholar

Suwalska, A. (2021). “The Last Educational Change in Finland - its Dimensions and Multiliteracy Teaching.” 10.14746/se 60: 199–211. doi:10.14746/se.2021.60.11

CrossRef Full Text | Google Scholar

Teo, P. (2014). Making the Familiar Strange and the Strange Familiar: A Project for Teaching Critical Reading and Writing. Lang. Educ. 28 (6), 539–551. doi:10.1080/09500782.2014.921191

CrossRef Full Text | Google Scholar

Thibault, P., and Curwood, J. S. (2018). Multiliteracies in Practice: Integrating Multimodal Production across the Curriculum. Theor. into Pract. 57, 48–55.

Google Scholar

Toh, W., and Lim, F. V. (2021). Let's Play Together: Ways of Parent-Child Digital Co-play for Learning. Interactive Learn. Environments, 1–11. doi:10.1080/10494820.2021.1951768

CrossRef Full Text | Google Scholar

Towndrow, P. A., Nelson, M. E., and Yusuf, W. F. B. M. (2013). Squaring Literacy Assessment with Multimodal Design. J. Literacy Res. 45 (4), 327–355. doi:10.1177/1086296x13504155

CrossRef Full Text | Google Scholar

Unsworth, L. (2001). Teaching Multiliteracies across the Curriculum: Changing Contexts of Text and Image in Classroom Practice. Buckingham UK: Open University Press.

Google Scholar

Unsworth, L. (2004). Comparing School Science Explanations in Books and Computer-Based Formats: The Role of Images, Image/text Relations and Hyperlinks. Int. J. Instructional Media 31 (3), 283–301.

Google Scholar

Unsworth, L. (2006). Towards a Metalanguage for Multiliteracies Education: Describing the Meaning Making Resources of Language-Image Interaction. English Teach. Pract. Critique 5 (1), 55–76.

Google Scholar

Unsworth, L. (2013). “Re-configuring Image-Language Relations and Interpretive Possibilities in Picture Books as Animated Movies: A Site for Developing Multimodal Literacy Pedagogy.” Ilha do Desterro 0, no. 64: 15–48.doi:10.5007/2175-8026.2013n64p15

CrossRef Full Text | Google Scholar

Unsworth, L. (2014). “Point of View in Picture Books and Animated Film Adaptations: Informing Critical Multimodal Comprehension and Composition Pedagogy,” in ” in Critical Multimodal Studies of Popular Discourse. Editors E. Djonov, and S. Zhao (London, New York: Routledge), 202–216.

Google Scholar

Unsworth, L., and Mills, K. A. (2020). English Language Teaching of Attitude and Emotion in Digital Multimodal Composition. J. Second Lang. Writing 47, 100712. doi:10.1016/j.jslw.2020.100712

CrossRef Full Text | Google Scholar

van Haren, R. (2015). “Engaging Learner Diversity through Learning by Design,” in A Pedagogy of Multiliteracies: Learning by Design. Editors B. Cope, and M. Kalantzis (London: Palgrave), 258–271. doi:10.1057/9781137539724_15

CrossRef Full Text | Google Scholar

van Leeuwen, T. (2017). Multimodal Literacy. mETAphor 4, 17–23.

Google Scholar

van Leeuwen, T. (2010). The Language of Colour: An Introduction. London, New York: Routledge.

Google Scholar

Woolf, S. H. (2008). The Meaning of Translational Research and Why it Matters. JAMA 299 (2), 211–213. doi:10.1001/jama.2007.26

PubMed Abstract | CrossRef Full Text | Google Scholar

Zammit, K. P. (2011). “Connecting Multiliteracies and Engagement of Students from Low Socio-Economic Backgrounds: Using Bernstein's Pedagogic Discourse as a Bridge.” Lang. Educ. 25, no. 3: 203–220. doi:10.1080/09500782.2011.560945

CrossRef Full Text | Google Scholar

Zapata, G. C. (2018). “The Role of Digital, Learning by Design Instructional Materials in the Development of Spanish Heritage Learners' Literacy Skills,” in Multiliteracies Pedagogy and Language Learning: Teaching Spanish to Heritage Speakers. Editors G. C. Zapata, and M. Lacorte (Cham CH: Palgrave Macmillan), 67–106. doi:10.1007/978-3-319-63103-5_3

CrossRef Full Text | Google Scholar

Zollo, S. A., and Kalantzis, M. (2020). “Bill Cope and Mary Kalantzis. 2020. Making Sense. Reference, agency, and Structure in a Grammar of Multimodal Meaning.” Ld 10, no. 3: 443–446. doi:10.1075/ld.00078.zol

CrossRef Full Text | Google Scholar