AUTHOR=Sandler Wendy 

TITLE=Redefining Multimodality

JOURNAL=Frontiers in Communication

VOLUME=Volume 6 - 2021

YEAR=2022

URL=https://www.frontiersin.org/journals/communication/articles/10.3389/fcomm.2021.758993

DOI=10.3389/fcomm.2021.758993

ISSN=2297-900X

ABSTRACT=REDEFINING MODALITY

Abstract.  The term “multimodality” is meant to incorporate gesture as part of language, a goal first put forward by Kendon (1980, 2004), which revolutionized the scope of linguistic inquiry. But the term “multimodality” itself is rife with ambiguity.  We now know enough about sign languages, transmitted solely in the visual channel, to see that they provide the ideal testing ground for an explanatory and unambiguous definition of multimodality. The notion of ‘modality’ is very often associated with the physical channel of transmission, so that ‘multimodal’ is taken to mean the auditory modality (the voice) plus the visual modality (the hands and face), the latter considered the primary domain of gesture. Alternatively, McNeill (1992) made a distinction between the gestural and the linguistic in terms of the type of organization, independent of the physical channel of transmission. Sign languages can resolve the ambiguity because it is now clear that they have full linguistic structuring, (Sandler & Lillo-Martin 2006), but are conveyed exclusively in the visual channel. And yet signers do gesture (e.g, Emmorey 1998, Sandler 2009). The ambiguity of the term “modality” is resolved by defining modality as a type of organization and showing that it is not isomorphic with the physical channel of transmission. Novel evidence comes from the signing of actors in the Ebisu deaf theatre group, where the two types of organization and their intimate interaction emerge, and from spoken language expressions in the visual channel that are part and parcel of linguistic structure. The result is a more nuanced understanding of multimodality and a more dynamic and flexible model of human language.