MINI REVIEW article

Front. Hum. Neurosci., 09 May 2018

Sec. Sensory Perceptual Neuroscience

Volume 12 - 2018 | https://doi.org/10.3389/fnhum.2018.00192

Audiovisual Temporal Perception in Aging: The Role of Multisensory Integration and Age-Related Sensory Loss

  • Department of Optometry and Vision Sciences, The University of Melbourne, Melbourne, VIC, Australia

Abstract

Within each sensory modality, age-related deficits in temporal perception contribute to the difficulties older adults experience when performing everyday tasks. Since perceptual experience is inherently multisensory, older adults also face the added challenge of appropriately integrating or segregating the auditory and visual cues present in our dynamic environment into coherent representations of distinct objects. As such, many studies have investigated how older adults perform when integrating temporal information across audition and vision. This review covers both direct judgments about temporal information (the sound-induced flash illusion, temporal order, perceived synchrony, and temporal rate discrimination) and judgments regarding stimuli containing temporal information (the audiovisual bounce effect and speech perception). Although an age-related increase in integration has been demonstrated on a variety of tasks, research specifically investigating the ability of older adults to integrate temporal auditory and visual cues has produced disparate results. In this short review, we explore what factors could underlie these divergent findings. We conclude that both task-specific differences and age-related sensory loss play a role in the reported disparity in age-related effects on the integration of auditory and visual temporal information.

Introduction

The normal aging process degrades both auditory and visual temporal perception, contributing to the difficulties older adults encounter with everyday tasks. For example, age-related impairments in speech comprehension and driving performance are associated with temporal processing deficits within audition (; ; ) and vision (Wood, 2002; ; ), respectively. However, many tasks stimulate both audition and vision. Therefore, this review targets how older adults combine these two sources of temporal information.

Integration binds auditory and visual stimuli into a unified percept, altering their perception relative to how they are perceived in isolation (; Stein et al., 2009). Correctly inferring whether audiovisual integration is appropriate poses a behaviorally relevant challenge. Integration occurs when stimuli seem to share the same origin and one factor influencing this is close temporal correspondence between auditory and visual stimuli, such as similar onset () and correlated temporal structure (; ). Recent reviews indicate that aging increases integration across a range of perceptual domains and behaviors (; ). An age-related increase in integration of congruent auditory and visual cues may be beneficial, for example improving driving performance (Ramkhalawansingh et al., 2016). Conversely, integration of incongruent cues achieves a coherent percept at the cost of veridical perception and so greater integration could hinder everyday function in older adults. Interestingly, review of the literature specifically concerning the integration of temporal information indicates varied age-related effects, with reports of unaltered (e.g., Sommers et al., 2005; ; ) and decreased integration in older adults (e.g., Tye-Murray et al., 2011; Roudaia et al., 2013; ). The perceptual change induced by integration depends on unisensory reliability (the precision of a sensory estimate of a property, as given by the inverse of its variance) and the brain’s tendency to integrate auditory and visual cues (; ). In this minireview, we consider whether age-related sensory loss (which affects unisensory reliability) and task-related differences (which may impact the integration process) explain the diversity of study findings.

Age-Related Changes in Unisensory Temporal Perception

Physiological aging affects the auditory and visual pathways from the sensory periphery to the cortex, resulting in a decline in many aspects of temporal perception. Within audition, older age reduces sensitivity to temporal fine structure (; ; ; ), amplitude modulation (Takahashi and Bacon, 1992; ; ; ; Wallaert et al., 2016) and frequency modulation (; ; Wallaert et al., 2016). Older adults exhibit impaired auditory gap detection (Snell, 1997; Strouse et al., 1998; ; ; ) and duration discrimination (; ; ; ), as well as impaired temporal order judgments (; ; Ulbrich et al., 2009) and temporal sequencing (Trainor and Trehub, 1989).

Within vision, many age-related temporal processing deficits have been documented, including reduced flicker sensitivity (; Tyler, 1989; ; ), reduced critical flicker frequency (, ; ; ; ) and some impairments in motion perception (Trick and Silverman, 1991; Tran et al., 1998; ; Snowden and Kavanagh, 2006; ; ). Similar to deficits in auditory processing, there is an age-related decline in visual gap detection () and temporal order judgments (Ulbrich et al., 2009; ; ).

Furthermore, sensory decline is typically uneven across vision and audition, though the more adversely affected sensory modality varies between tasks (; ; ; ; ; ). For example, older age impairs visual duration judgments more than auditory ones (). In contrast, auditory but not visual temporal rate discriminability is poorer in older adults (). Consequently, for a given audiovisual task, auditory and visual cues are not expected to be equally degraded by age-related sensory changes.

Age-Related Changes in Audiovisual Temporal Perception

Studies have investigated age-related changes to various measures of audiovisual temporal perception presumed to reflect multisensory integration, with disparate results for transient stimuli like flashes and beeps. Older adults are more susceptible to the sound-induced flash illusion (Setti et al., 2011a; ; ). In this illusory fission effect, the integration of two beeps and a solitary flash results in the perception of two flashes (Shams et al., 2000). Older adults also exhibit an enhanced temporal ventriloquist effect (), in which the presentation of a click before and after a sequence of two flashes increases visual temporal order sensitivity (). Conversely, the integration of two flashes with a single beep results in the perception of a single flash in an illusory fusion effect () that does not change in aging (). In the audiovisual bounce effect, an auditory beep influences a visual spatiotemporal illusion, in which two disks moving toward each other appear either to stream past one another, or less frequently, to bounce (Sekuler and Sekuler, 1999). The beep, sounding as the disks overlap, causes an increase in the frequency of perceived bouncing (Sekuler et al., 1997). This effect is reduced in older adults, suggestive of decreased integration (Roudaia et al., 2013).

Age-related effects on the integration of temporally modulated stimuli depend on audiovisual congruency. Partial integration of non-identical auditory and visual temporal rates distorts perceived rate such that the auditory or visual rate subjectively equivalent to a reference is non-veridical (Roach et al., 2006). This effect is equivalent in younger and older adults (). However, integration of identical auditory and visual temporal rates improves temporal rate discrimination in a statistically optimal fashion in younger (; ) but not older adults, who fail to show the √ 2 improvement predicted by maximum likelihood estimation ().

Research on audiovisual speech perception in aging is also relevant to this review, since the temporal relationships between auditory and visual speech cues ()facilitate both auditory speech detection () and recognition (ten Oever et al., 2013; ). Visual facilitation of auditory speech detection is reduced in older adults (Tye-Murray et al., 2011). The effect of aging on audiovisual gains in speech recognition is more complex, with no age-related changes evident when auditory and visual speech cues are clear and congruent (; Sommers et al., 2005; Spehar et al., 2008; ; ; Tye-Murray et al., 2010; Winneke and Phillips, 2011; ; Smayda et al., 2016; Sommers and Phelps, 2016). However, audiovisual gains are reduced in older adults when auditory and visual speech cues are degraded (Tye-Murray et al., 2008, 2010, 2011; ; ; Stevenson et al., 2015) or asynchronous ().

In the McGurk effect, integration of incongruent auditory and visual speech results in a fused percept that matches neither cue (). Most studies report a similar proportion of fused responses in younger and older adult groups for syllables (; ; ; Stothart and Kazanina, 2016). Discrepant findings reported in the literature likely reflect differences in study design, with an age-related increase in fused responses found when speech stimuli were words (Setti et al., 2013) and an age-related decrease in fused responses found when auditory syllables were presented in modulated noise (). Comparison between studies using different experimental stimuli is also complicated by known variability in the McGurk effect with different speakers (; ) and different auditory and visual syllable pairings ().

Altogether, research indicates an age-related increase in the integration of temporally offset auditory and visual events. Older adults perceive the sound-induced flash illusion at larger temporal offsets between auditory and visual stimuli than younger adults (Setti et al., 2011a; ). When judging synchrony, older adults perceive temporally offset auditory and visual events as simultaneous over a broader (; ), narrower () or similar (; ) temporal window compared to younger adults. However, the temporal window of perceived synchrony is broader in older adults when individual differences in stimulus detectability and response criterion are normalized, indicative of enhanced integration (). Similarly, in judgments of temporal order, older adults need a larger temporal gap between an auditory and visual stimulus to distinguish which appeared first (Virsu et al., 2003; Setti et al., 2011b; ; ), but one study found no age-related differences in audiovisual temporal order judgments (). All of these temporal order studies used stimuli that were not individually scaled to detectability, and Fiacconi et al tested observers of at least 70 years of age, which is older than the inclusion criteria of the other above cited studies (at least 60 years).

In summary, integration of temporal auditory and visual information may be increased, decreased or unchanged by physiological aging (see Tables 1, 2). This suggests that age-related effects on integration may be task-specific, as discussed in the next section.

Table 1

Audiovisual taskAge-related changes in multisensory integration:

DecreasedUnchangedIncreased
Sound-induced flash fissionSetti et al., 2011a; ;
Sound-induced flash fusion
Temporal ventriloquist effect
Audiovisual bounce effectRoudaia et al., 2013
Identical temporal rates
Non-identical temporal rates
Speech detectionTye-Murray et al., 2011
Speech recognitionTye-Murray et al., 2008, 2010, 2011; ; ; Stevenson et al., 2015; ; Sommers et al., 2005; Spehar et al., 2008; ; ; Tye-Murray et al., 2010; Winneke and Phillips, 2011; ; Smayda et al., 2016; Sommers and Phelps, 2016
McGurk effect (fused responses); ; ; Stothart and Kazanina, 2016Setti et al., 2013

Age-related changes in the perception of audiovisual temporal information consistent with decreased, unchanged, or increased integration in older relative to younger adults.

See text for discussion of how closely changes in perceptual measures reflect the underlying ability to integrate.

Table 2

Audiovisual taskAge-related changes in the temporal binding window:

NarrowerUnchangedWider
Temporal order judgmentsVirsu et al., 2003; Setti et al., 2011b; ;
Synchrony judgments; ; ;

Age-related changes in the temporal binding window of auditory and visual events, indicating whether older adults were found to have narrower, unchanged, or wider temporal binding windows relative to younger adults.

Task-Specific Integration Processes

An outstanding question is how closely audiovisual integration of temporal information is linked across different tasks. On the one hand, performance across different temporal audiovisual tasks is often correlated within individuals (Tremblay et al., 2007; Stevenson et al., 2012; Stevenson and Wallace, 2013), suggesting that different tasks can index common processes. Accordingly, older adults exhibit both increased susceptibility to the sound induced flash illusion (Setti et al., 2011a; ; ) and broader temporal binding windows (; ), consistent with the correlation between performance on these tasks in younger adults (Stevenson et al., 2012). However, within-subject correlations in younger adults have a limited potential to connect age-related effects on integration across different tasks. Correlations between tasks may reflect unisensory reliability, rather than a shared tendency to integrate, as both factors contribute to observed audiovisual interactions (). Preliminary evidence suggests that the brain’s tendency to integrate auditory and visual cues is task-specific, though it remains unclear how tightly integration tendency is linked across tasks within the same perceptual domain (). Furthermore, while low level stimulus characteristics such as temporal correspondence and unisensory reliability modulate the strength of audiovisual interactions, the integration process is also influenced by contextually-driven factors such as task goal, attention, and learned audiovisual associations (for review see van Atteveldt et al., 2014; Tye-Murray et al., 2016).

Different tasks likely index different perceptual processes. For example, research suggests distinctions between the perception of audiovisual temporal rate and relative timing (). Moreover, a complex task like audiovisual speech perception is influenced by higher level factors, such as semantic congruence and linguistic meaning, not just lower level stimulus characteristics like temporal structure (; ; Stekelenburg and Vroomen, 2012; ten Oever et al., 2013; Stevenson et al., 2014). In older adults, audiovisual gains in speech recognition decrease with task complexity, such as lexical difficulty () or whole-word compared to phoneme recognition (Stevenson et al., 2015).

Additionally, audiovisual temporal tasks may not index the same perceptual processes even with comparable stimuli, such as audiovisual speech detection and recognition() and audiovisual temporal order and synchrony judgments (). In fact, estimates of the temporal binding window from temporal order and synchrony tasks are not correlated in older adults (). The McGurk effect is experienced at temporal offsets between auditory and visual syllables where observers are aware of the asynchrony (Soto-Faraco and Alsius, 2009), in line with differences in the neural substrate processing asynchrony and perceptual fusion for audiovisual speech (Stevenson et al., 2011). Furthermore, evidence points toward distinct cortical mechanisms of illusory flash fusion and fission (, ) that may be differentially vulnerable to age-related effects given differences in older adult susceptibility to each illusion type (). Likewise, older adults exhibit impaired integration of identical but not conflicting auditory and visual temporal rates, suggesting the presence of separate integration mechanisms that differ in their susceptibility to age-related decline ().

Lastly, there are concerns regarding the adequacy of some study measures as indices of integration. Though some argue that decisional processes drive the audiovisual bounce effect (; Zeljko and Grove, 2016), most studies indicate that the effect is perceptual (Watanabe and Shimojo, 2001; Sanabria et al., 2004; ; ). Multiple studies suggest that the McGurk effect is an unreliable measure of integration, due to substantial individual variability (), underestimation of visual modulation of the auditory cue (; Tiippana, 2014) and the contribution of individual differences in sensitivity to incongruity and lipreading skill to illusion susceptibility (Strand et al., 2014). As for audiovisual gains in speech recognition, recent analysis argues improved performance does not result from integration and that different integration measures indicate opposing age-related effects (Tye-Murray et al., 2016). Greater understanding of the relationships between audiovisual tasks and how closely perceptual changes reflect integration is needed to clarify current research.

The Role of Age-Related Changes in Unisensory Processing

Age-related unisensory changes are a potential confounding factor in the interpretation of age-related changes in audiovisual perception. According to the principle of inverse effectiveness, if auditory and visual cues are less salient, there will be a greater proportionate change in neural responses upon their integration (). Since this principle can also apply to perception (Stein et al., 1988), apparently enhanced integration in older age may reflect reduced stimulus saliency due to age-related sensory loss (; ). However, a relationship between sensory reliability and integration does not necessarily mean that age-related sensory loss will explain age-related differences. While lower stimulus intensities broaden temporal binding windows in younger adults (), older adults demonstrated broader temporal binding windows than younger adults when stimulus intensity was scaled to individual detection thresholds, indicating the effect was independent of age-related declines in sensory sensitivity (). Furthermore, counter to the predictions of inverse effectiveness, there is an age-related increase in audiovisual enhancement of word recognition at intermediate auditory signal to noise ratios (Stevenson et al., 2015) and an age-related decrease in audiovisual enhancement of degraded sentences (Tye-Murray et al., 2010). This is consistent with evidence that audiovisual gains in speech recognition are greatest for intermediate auditory clarity (Ross et al., 2007; cf. Stevenson et al., 2015).

The reliability of auditory relative to visual sensory estimates of a property (where reliability is defined as the estimate’s precision, as given by the inverse of its variance) also influences audiovisual interactions (). In contrast to the historical view that audition dominates temporal perception due to its superior temporal resolution (Welch and Warren, 1980), current theory holds that the brain weights a pair of sensory cues according to their relative reliability to derive the most precise multisensory representation possible from available information (; Witten and Knudsen, 2005; ). Unisensory reliability plays a role in susceptibility to audiovisual illusions, in which information from one sensory modality modulates perception in another (e.g., Sekiyama and Tohkura, 1991; ; Roach et al., 2006; ; Strand et al., 2014). Consequently, uneven age-related sensory decline could alter the relative contribution of audition and vision to the audiovisual percept, resulting in an apparent change in illusion susceptibility. Age-related shifts in the weighting of auditory and visual cues have been documented for audiovisual speech. When studies employed audiovisual speech cues that gave rise to age-related differences in unisensory performance, older adults gave more weight than younger adults to auditory () or visual information (; Sekiyama et al., 2014; ), in accordance with the more reliable sensory modality. However, it is possible to probe age-related differences in integration capacity by individually balancing the relative reliability of auditory and visual cues, as was shown for temporal rate perception ().

Unfortunately, studies of audiovisual integration in older adults have not employed a consistent definition of what constitutes normal hearing and vision, and may not screen both sensory modalities for age-abnormal changes. Even so, screening measures commonly used in older adults such as audiometric thresholds and visual acuity are poor indicators of other aspects of auditory (e.g., ; ) or visual perception (e.g., ), respectively. The issue is further complicated by the often indistinct boundary between age-related and pathological changes (). Older age increases the prevalence of ocular diseases such as glaucoma () and age-related macular degeneration () that can further impair temporal perception even in early stages (e.g., ; ; Spry et al., 2005; ; ). Age-related sensorineural hearing loss contributes to impaired auditory temporal perception independently from physiological aging () and for stimuli at low frequencies despite normal audiometric thresholds (). Age-related high frequency hearing loss also increases the intrinsic functional connectivity between auditory and visual cortical regions, which may increase audiovisual interactions in older adults (). Before drawing conclusions on how integration may change with age, studies should carefully account for age-related changes in both auditory and visual saliency on an individual basis (e.g., ; ).

Conclusion

Audiovisual integration may not necessarily provide older adults with an effective compensatory mechanism for age-related unisensory decline in temporal information. While older adults often benefit from the provision of complimentary auditory and visual cues, it can be to a reduced extent compared to younger adults. Older adults sometimes demonstrate an increased tendency to synthesize conflicting temporal auditory and visual cues into a unified percept but this is not a universal finding. Further research elucidating how closely integration is linked across different audiovisual tasks is needed to clarify this pattern of results. Currently, it remains unclear to what extent age-related changes in audiovisual temporal perception reflect true changes in integration rather than sequential effects of age-related sensory loss. Future research should account for individual differences in unisensory reliability to distinguish age-related sensory loss from age-related effects on integration.

Statements

Author contributions

CB: conception and planning of review and wrote the manuscript. YC: planning of review and contributed to the writing of Section “Age-Related Changes in Audiovisual Temporal Perception.” AM and AA: conception of review and revision of the manuscript. All authors approved the manuscript.

Funding

This study was supported by Australian Research Council (FT0990930 to AM).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Summary

Keywords

temporal perception, multisensory, audiovisual, integration, audition, vision, aging

Citation

Brooks CJ, Chan YM, Anderson AJ and McKendrick AM (2018) Audiovisual Temporal Perception in Aging: The Role of Multisensory Integration and Age-Related Sensory Loss. Front. Hum. Neurosci. 12:192. doi: 10.3389/fnhum.2018.00192

Received

27 October 2017

Accepted

20 April 2018

Published

09 May 2018

Volume

12 - 2018

Edited by

Camillo Porcaro, Istituto di Scienze e Tecnologie della Cognizione (ISTC), Italy

Reviewed by

Eugenie Roudaia, Université de Montréal, Canada; Annalisa Setti, University College Cork, Ireland

Updates

Copyright

*Correspondence: Allison M. McKendrick,

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics