Research Topic

Gesture-Speech Integration: Combining Gesture and Speech to Create Understanding

About this Research Topic

Gesture and speech were viewed historically as separate processes, involving two distinct communication systems. However, such a viewpoint has quickly come to be seen as inaccurate, with a growing body of research suggesting that gesture and speech integrate to form one communication system. Gesture and speech can interact with each other during production and comprehension across a variety of domains.

Listeners learn from speakers’ gestures. Across mathematics instruction, word and narrative descriptions, spatial route directions, conservation tasks, and foreign language instruction, studies have repeatedly shown that observing gestures enhances task performance and facilitates interactions. Similarly, speakers are better communicators and thinkers when they gesture than when they do not. Individuals performing mental rotation tasks, mathematical problem solving, and spatial tasks can all benefit from producing gestures while carrying out the task.

In some ways, the seamless integration of gesture and speech by both speakers and listeners is paradoxical, as it suggests that doing two things (producing or processing both gesture and speech) is easier than doing one thing (producing or processing speech alone). Indeed, gestures are not uniformly beneficial, and there are situations where gestures seem to be more or less difficult to integrate with speech. For example, the age of the learner, the difficulty of the task, the overlap between the content in gesture and speech, and the type of gesture may all influence the extent to which gestures and speech are successfully integrated by speakers during production, and by listeners during comprehension.

There are several possible mechanisms for how gesture and speech may be successfully integrated. Observing gestures may draw listeners’ attention to a spoken message, they may provide additional or supporting information to a message, or they may spread the cognitive load across visual and verbal resources. Producing gestures may occur automatically as a simulation of actions that is well-coordinated with accompanying speech, and producing gestures may reduce the load on the speaker’s working memory or may draw a speaker’s attention to a particular aspect of their thinking that is highlighted in gesture.

The aim of this Research Topic is to further investigate the relationship between gesture and speech. How do speakers decide what information to present in gesture and what information to present in speech and how do they coordinate the timing and form of each channel during real-time speech production? Further, how do the production of gesture change speakers’ thinking and speaking compared to gesture inhibition? How do listeners integrate the two modalities to form a single mental representation and what are the boundary conditions of when gestures are helpful?

Areas covered by this Research Topic include:
• The effects of observing different types of gestures on task performance
• The context where observers can or cannot benefit from gestures in speech comprehension
• Integration processes between gesture and speech in understanding a speaker’s message
• The effects of producing different types of gestures on task performance
• Interactions between observing or producing gesture and characteristics of specific tasks
• Theoretical proposals regarding the mechanisms underlying the integration of gesture and speech

We welcome original research articles, reviews, mini-reviews, hypothesis and theory articles, and brief research reports.


Keywords: Gesture, Learning, Actions, Speech, Comprehension


Important Note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Gesture and speech were viewed historically as separate processes, involving two distinct communication systems. However, such a viewpoint has quickly come to be seen as inaccurate, with a growing body of research suggesting that gesture and speech integrate to form one communication system. Gesture and speech can interact with each other during production and comprehension across a variety of domains.

Listeners learn from speakers’ gestures. Across mathematics instruction, word and narrative descriptions, spatial route directions, conservation tasks, and foreign language instruction, studies have repeatedly shown that observing gestures enhances task performance and facilitates interactions. Similarly, speakers are better communicators and thinkers when they gesture than when they do not. Individuals performing mental rotation tasks, mathematical problem solving, and spatial tasks can all benefit from producing gestures while carrying out the task.

In some ways, the seamless integration of gesture and speech by both speakers and listeners is paradoxical, as it suggests that doing two things (producing or processing both gesture and speech) is easier than doing one thing (producing or processing speech alone). Indeed, gestures are not uniformly beneficial, and there are situations where gestures seem to be more or less difficult to integrate with speech. For example, the age of the learner, the difficulty of the task, the overlap between the content in gesture and speech, and the type of gesture may all influence the extent to which gestures and speech are successfully integrated by speakers during production, and by listeners during comprehension.

There are several possible mechanisms for how gesture and speech may be successfully integrated. Observing gestures may draw listeners’ attention to a spoken message, they may provide additional or supporting information to a message, or they may spread the cognitive load across visual and verbal resources. Producing gestures may occur automatically as a simulation of actions that is well-coordinated with accompanying speech, and producing gestures may reduce the load on the speaker’s working memory or may draw a speaker’s attention to a particular aspect of their thinking that is highlighted in gesture.

The aim of this Research Topic is to further investigate the relationship between gesture and speech. How do speakers decide what information to present in gesture and what information to present in speech and how do they coordinate the timing and form of each channel during real-time speech production? Further, how do the production of gesture change speakers’ thinking and speaking compared to gesture inhibition? How do listeners integrate the two modalities to form a single mental representation and what are the boundary conditions of when gestures are helpful?

Areas covered by this Research Topic include:
• The effects of observing different types of gestures on task performance
• The context where observers can or cannot benefit from gestures in speech comprehension
• Integration processes between gesture and speech in understanding a speaker’s message
• The effects of producing different types of gestures on task performance
• Interactions between observing or producing gesture and characteristics of specific tasks
• Theoretical proposals regarding the mechanisms underlying the integration of gesture and speech

We welcome original research articles, reviews, mini-reviews, hypothesis and theory articles, and brief research reports.


Keywords: Gesture, Learning, Actions, Speech, Comprehension


Important Note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

About Frontiers Research Topics

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

Topic Editors

Loading..

Submission Deadlines

24 June 2020 Manuscript

Participating Journals

Manuscripts can be submitted to this Research Topic via the following journals:

Loading..

Topic Editors

Loading..

Submission Deadlines

24 June 2020 Manuscript

Participating Journals

Manuscripts can be submitted to this Research Topic via the following journals:

Loading..
Loading..

total views article views article downloads topic views

}
 
Top countries
Top referring sites
Loading..