Scholars working in both multimodal interaction analysis (MMIA; cf. Norris, 2004) and multi-modal discourse analysis (MMDA; cf. Kress, 2011) share the consensus that their objects of study are first of all text-like artefacts. This view holds despite a variety of labels in use, such as an event, ensemble, or piece of communication. Unity and connectedness of the various informational and structural units in a communicative whole can count as the hallmark of text, textuality or texture, a notion mostly captured by the concept of multimodal coherence. For realizing it, various expressive resources, i.e., semiotic modes must meaningfully link and cooperate to build a multi-modal text structure. The process of multimodal meaning-making then is inherently one of constructing discourse coherence within and between modes, an activity that is crucially shaped by the affordances of the medium and the demands of the genre.
This first Research Topic in the Multimodality of Communication specialty section of Frontiers in Communication focuses on the concept and practices of multimodal coherence in different genres and media. Bateman (2014a, pp. 161–174 and 186–221; 2014b) outlines major linguistic approaches to coherence and shows how they have been adapted to multimodal analysis. Three frameworks emerge as central in this: cohesive ties, propositional or conjunctive relations, and rhetorical structure. The notion of multimodal coherence has also been referred to as inter-semiotic complementarity (Royce, 1998) or inter-semiotic harmony (Norris/Maier, 2014, 390) and manifests in inter-semiotic relationships (Jewitt, 2014, pp. 26–27). Caple (2013, p. 122 and 175) has usefully sub-divided such cohesive relationships into inter-semiotic and intra-semiotic relations. It is probably fair to argue that coherence is “the core idea of multimodality”, which “posits the linking of semiotic modes and their formal, semantic, and functional integration” (Stöckl, 2019, p. 53). Patterns and styles of mode-linking and integration very likely vary with genre and medium.
For this Research Topic, we invite papers which:
• add to, refine, or extend notions and analytical frameworks for modelling coherence
• show how the discourse semantics of individual modes link to create a joint one in a given genre
• describe genre- or media-specific multimodal textures or compare them
• address issues of multimodal inferencing processes and the role of pragmatic relevance
• enquire into the multimodal coherence required/produced in a re-contextualization of content
• look at instances of incoherence or discontinuity and their effects on texture.
This list is an orientation only, other options are of course open. However, the central concern of the papers must be with multimodal coherence, which ought to preferably be addressed in an empirical, corpus-/case-studies-based fashion or in a theoretical approach with sufficient illustration.
Matters of Form
• Articles are to be of a maximum length of 8.000 words.
• As the articles are digital, open access only, Frontiers provides the opportunity to include extra illustrations, such as charts, diagrams, images, audio-visual data and other appended materials needed to enrich the presentation and discussion of the topic at issue.
Image credit: Max Williams
Scholars working in both multimodal interaction analysis (MMIA; cf. Norris, 2004) and multi-modal discourse analysis (MMDA; cf. Kress, 2011) share the consensus that their objects of study are first of all text-like artefacts. This view holds despite a variety of labels in use, such as an event, ensemble, or piece of communication. Unity and connectedness of the various informational and structural units in a communicative whole can count as the hallmark of text, textuality or texture, a notion mostly captured by the concept of multimodal coherence. For realizing it, various expressive resources, i.e., semiotic modes must meaningfully link and cooperate to build a multi-modal text structure. The process of multimodal meaning-making then is inherently one of constructing discourse coherence within and between modes, an activity that is crucially shaped by the affordances of the medium and the demands of the genre.
This first Research Topic in the Multimodality of Communication specialty section of Frontiers in Communication focuses on the concept and practices of multimodal coherence in different genres and media. Bateman (2014a, pp. 161–174 and 186–221; 2014b) outlines major linguistic approaches to coherence and shows how they have been adapted to multimodal analysis. Three frameworks emerge as central in this: cohesive ties, propositional or conjunctive relations, and rhetorical structure. The notion of multimodal coherence has also been referred to as inter-semiotic complementarity (Royce, 1998) or inter-semiotic harmony (Norris/Maier, 2014, 390) and manifests in inter-semiotic relationships (Jewitt, 2014, pp. 26–27). Caple (2013, p. 122 and 175) has usefully sub-divided such cohesive relationships into inter-semiotic and intra-semiotic relations. It is probably fair to argue that coherence is “the core idea of multimodality”, which “posits the linking of semiotic modes and their formal, semantic, and functional integration” (Stöckl, 2019, p. 53). Patterns and styles of mode-linking and integration very likely vary with genre and medium.
For this Research Topic, we invite papers which:
• add to, refine, or extend notions and analytical frameworks for modelling coherence
• show how the discourse semantics of individual modes link to create a joint one in a given genre
• describe genre- or media-specific multimodal textures or compare them
• address issues of multimodal inferencing processes and the role of pragmatic relevance
• enquire into the multimodal coherence required/produced in a re-contextualization of content
• look at instances of incoherence or discontinuity and their effects on texture.
This list is an orientation only, other options are of course open. However, the central concern of the papers must be with multimodal coherence, which ought to preferably be addressed in an empirical, corpus-/case-studies-based fashion or in a theoretical approach with sufficient illustration.
Matters of Form
• Articles are to be of a maximum length of 8.000 words.
• As the articles are digital, open access only, Frontiers provides the opportunity to include extra illustrations, such as charts, diagrams, images, audio-visual data and other appended materials needed to enrich the presentation and discussion of the topic at issue.
Image credit: Max Williams
