About this Research Topic
Scholars working in both multimodal interaction analysis (MMIA; cf. NORRIS, 2004) and multi-modal discourse analysis (MMDA; cf. KRESS, 2011) share the consensus that their objects of study are first of all text-like artefacts. This view holds despite a variety of labels in use, such as an event, ensemble, or piece of communication. Unity and connectedness of the various informational and structural units in a communicative whole can count as the hallmark of text, textuality or texture, a notion mostly captured by the concept of multimodal coherence. For realizing it, various expressive resources, i.e., semiotic modes must meaningfully link and cooperate to build a multi-modal text structure. The process of multimodal meaning-making then is inherently one of constructing discourse coherence within and between modes, an activity that is crucially shaped by the affordances of the medium and the demands of the genre.
This first call in the FRONTIERS-section ‘Multimodality of Communication’ focuses on the concept and practices of multimodal coherence in different genres and media. BATEMAN (2014a, pp. 161–174 and 186–221; 2014b) outlines major linguistic approaches to coherence and shows how they have been adapted to multimodal analysis. Three frameworks emerge as central in this: cohesive ties, propositional or conjunctive relations, and rhetorical structure. The notion of multimodal coherence has also been referred to as inter-semiotic complementarity (ROYCE, 1998) or inter-semiotic harmony (NORRIS/ MAIER, 2014, p. 390) and manifests in inter-semiotic relationships (JEWITT, 2014, p. 26–27). CAPLE (2013, p. 122 and 175) has usefully sub-divided such cohesive relationships into inter-semiotic and intra-semiotic relations. It is probably fair to argue that coherence is “the core idea of multimodality”, which “posits the linking of semiotic modes and their formal, semantic, and functional integration” (STÖCKL, 2019, p. 53). Patterns and styles of mode-linking and integration very likely vary with genre and medium.
For the research topic in the ‘Multimodality of Communication’ section, we invite papers which:
1. add to, refine or extend notions and analytical frameworks for modelling coherence,
2. show how the discourse semantics of individual modes link to create a joint one in a given genre,
3. describe genre- or media-specific multimodal textures or compare them,
4. address issues of multimodal inferencing processes and the role of pragmatic relevance,
5. enquire into the multimodal coherence required/produced in a re-contextualization of content,
6. look at instances of incoherence or discontinuity and their effects on texture.
This list is an orientation only, other options are of course open. However, the central concern of the papers must be with multimodal coherence, which ought to preferably be addressed in an empirical, corpus-/case-studies-based fashion or in a theoretical approach with sufficient illustration.
REFERENCES
BATEMAN, John A. (2014a) Text and Image. A Critical Introduction to the Visual/Verbal Divide. London/New York: Routledge.
BATEMAN, John (2014b) ‘Multimodal Coherence Research and its Applications’. In Gruber, Helmut and Redeker, Gisela (eds.) The Pragmatics of Discourse Coherence. Amsterdam: Benjamins, pp. 145–177.
CAPLE, Helen (2013) Photojournalism: A Social Semiotic Approach. Basingstoke and New York: Palgrave Macmillan.
JEWITT, Carey (2014) ‘An Introduction to Multimodality’. In Jewitt, Carey (ed.) The Routledge Handbook of Multi-modal Analysis. London/New York: Routledge, pp. 15–30.
KRESS, Gunther (2011): ‘Multimodal Discourse Analysis’. In Gee, James P. and Handford, Michael (eds.) The Routledge Handbook of Discourse Analysis. London/New York: Routledge, pp. 35–50.
NORRIS, Sigrid (2004) Analysing Multimodal Interaction: A Methodological Framework. London/New York: Routledge.
NORRIS, Sigrid and MAIER, Carmen D. (2004) Interactions, Images and Texts: A Reader in Multimodality. Boston/Berlin: de Gruyter.
ROYCE, Terry (1998) ‘Synergy on the Page: Exploring Intersemiotic Complementarity in Page-Based Multimodal Text’. Japan Association for Systemic Functional Linguistics (JASFL) Occasional Papers 1(1), 25–49.
STÖCKL, Hartmut (2019) ‘Linguistic Multimodality – Multimodal Linguistics: A State-of-the-Art Sketch’. In Wildfeuer, Janina et al. (eds.) Multimodality: Disciplinary Thoughts and the Challenge of Diversity. Berlin/Boston: de Gruyter, pp. 41–68.
MATTERS OF FORM
1. The papers are to be of a maximum length of 8.000 words.
2. As the papers are digital, open-access only, FRONTIERS provides the opportunity to include extra illustrations, such as charts, diagrams, images, audio-visual data and other appended materials needed to enrich the presentation and discussion of the topic at issue.
Image credit: Max Williams
This first call in the FRONTIERS-section ‘Multimodality of Communication’ focuses on the concept and practices of multimodal coherence in different genres and media. BATEMAN (2014a, pp. 161–174 and 186–221; 2014b) outlines major linguistic approaches to coherence and shows how they have been adapted to multimodal analysis. Three frameworks emerge as central in this: cohesive ties, propositional or conjunctive relations, and rhetorical structure. The notion of multimodal coherence has also been referred to as inter-semiotic complementarity (ROYCE, 1998) or inter-semiotic harmony (NORRIS/ MAIER, 2014, p. 390) and manifests in inter-semiotic relationships (JEWITT, 2014, p. 26–27). CAPLE (2013, p. 122 and 175) has usefully sub-divided such cohesive relationships into inter-semiotic and intra-semiotic relations. It is probably fair to argue that coherence is “the core idea of multimodality”, which “posits the linking of semiotic modes and their formal, semantic, and functional integration” (STÖCKL, 2019, p. 53). Patterns and styles of mode-linking and integration very likely vary with genre and medium.
For the research topic in the ‘Multimodality of Communication’ section, we invite papers which:
1. add to, refine or extend notions and analytical frameworks for modelling coherence,
2. show how the discourse semantics of individual modes link to create a joint one in a given genre,
3. describe genre- or media-specific multimodal textures or compare them,
4. address issues of multimodal inferencing processes and the role of pragmatic relevance,
5. enquire into the multimodal coherence required/produced in a re-contextualization of content,
6. look at instances of incoherence or discontinuity and their effects on texture.
This list is an orientation only, other options are of course open. However, the central concern of the papers must be with multimodal coherence, which ought to preferably be addressed in an empirical, corpus-/case-studies-based fashion or in a theoretical approach with sufficient illustration.
REFERENCES
BATEMAN, John A. (2014a) Text and Image. A Critical Introduction to the Visual/Verbal Divide. London/New York: Routledge.
BATEMAN, John (2014b) ‘Multimodal Coherence Research and its Applications’. In Gruber, Helmut and Redeker, Gisela (eds.) The Pragmatics of Discourse Coherence. Amsterdam: Benjamins, pp. 145–177.
CAPLE, Helen (2013) Photojournalism: A Social Semiotic Approach. Basingstoke and New York: Palgrave Macmillan.
JEWITT, Carey (2014) ‘An Introduction to Multimodality’. In Jewitt, Carey (ed.) The Routledge Handbook of Multi-modal Analysis. London/New York: Routledge, pp. 15–30.
KRESS, Gunther (2011): ‘Multimodal Discourse Analysis’. In Gee, James P. and Handford, Michael (eds.) The Routledge Handbook of Discourse Analysis. London/New York: Routledge, pp. 35–50.
NORRIS, Sigrid (2004) Analysing Multimodal Interaction: A Methodological Framework. London/New York: Routledge.
NORRIS, Sigrid and MAIER, Carmen D. (2004) Interactions, Images and Texts: A Reader in Multimodality. Boston/Berlin: de Gruyter.
ROYCE, Terry (1998) ‘Synergy on the Page: Exploring Intersemiotic Complementarity in Page-Based Multimodal Text’. Japan Association for Systemic Functional Linguistics (JASFL) Occasional Papers 1(1), 25–49.
STÖCKL, Hartmut (2019) ‘Linguistic Multimodality – Multimodal Linguistics: A State-of-the-Art Sketch’. In Wildfeuer, Janina et al. (eds.) Multimodality: Disciplinary Thoughts and the Challenge of Diversity. Berlin/Boston: de Gruyter, pp. 41–68.
MATTERS OF FORM
1. The papers are to be of a maximum length of 8.000 words.
2. As the papers are digital, open-access only, FRONTIERS provides the opportunity to include extra illustrations, such as charts, diagrams, images, audio-visual data and other appended materials needed to enrich the presentation and discussion of the topic at issue.
Image credit: Max Williams
Keywords: texture, discourse coherence, genre, media, inter-semiotic relationships, multimodal (inter)action analysis, multimodal discourse analysis, multimodal coherence, modelling coherence
Important Note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.
