About this Research Topic
This Research Topic contributes to the germinating renaissance of understanding languages as multimodal interaction systems which, beyond grammatical structure, make rich use of interfaces between symbolic, deictic, and depicting structures and representations. To further this understanding, the research topic issue will collect contributions from the fields of linguistics, computational linguistics, and computer science to shed light on how linguistic and non-linguistic structures form multimodal ensembles as the basis of language production and understanding. We, therefore, expect contributions from all areas of linguistics, computational linguistics, and related disciplines to shed light on this paradigm shift.
The Frontiers Research Topic invites contributions to all aspects of the interface of Multimodal Communication and Multimodal Computing. We seek for research covering the span from computational-oriented studies on the form and function of multimodal signaling and behavior to novel algorithmic approaches in computational hardware or software, from theoretical essays to industrial applications. The challenges that the topic of Multimodal Communication and Multimodal Computing has to face include:
- multimodal representations for representation learning
- multimodal active learning
- memory and multimodal integration
- 4E-cognition
- neural-to-symbolic AI
- spatial representation and reasoning
- the creation and exploitation of multimodal datasets
- the design and computational use of experimental studies on verbal and non-verbal communication
- representation formats and tools for annotating multimodal data
- visual reasoning
- diagrammatic reasoning
- applications: question--answering, multimodal distributional semantics, and others
- cognitive architectures for multimodal processing
- neural models and multimodal input
- multimodal generation and fission, or binding and disentanglement
- text2scene
Keywords: Multimodal Communication, Multimodal Computing, multimodal integration, multimodal representations, multimodal learning, speech, gesture, facial expressions, reasoning, memory, emotion, cognitive models, neural models
Important Note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.