Edited by: Carryl L. Baldwin, George Mason University, USA
Reviewed by: Ramesh Kandimalla, Texas Tech University, USA; Jingwen Niu, Temple University, USA
*Correspondence: Arthur Wingfield
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution and reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
In spite of the rapidity of everyday speech, older adults tend to keep up relatively well in day-to-day listening. In laboratory settings older adults do not respond as quickly as younger adults in off-line tests of sentence comprehension, but the question is whether comprehension itself is actually slower. Two unique features of the human eye were used to address this question. First, we tracked eye-movements as 20 young adults and 20 healthy older adults listened to sentences that referred to one of four objects pictured on a computer screen. Although the older adults took longer to indicate the referenced object with a cursor-pointing response, their gaze moved to the correct object as rapidly as that of the younger adults. Second, we concurrently measured dilation of the pupil of the eye as a physiological index of effort. This measure revealed that although poorer hearing acuity did not slow processing, success came at the cost of greater processing effort.
The early literature on mental performance in adult aging was largely one of cataloging age-related deficits—most notably, ineffective learning and poor memory retrieval for recent events. It is the case that aging brings changes to the neural structures and network dynamics that carry cognition (Burke and Barnes,
Given the well-documented cognitive changes that accompany adult aging, surely, understanding spoken language should be among the hardest hit of human skills. Yet, barring significant neuropathology or serious hearing impairment, comprehension of spoken language remains one of the best-preserved of our cognitive functions (Wingfield and Stine-Morrow,
A common approach to addressing the first of these questions has been to measure the relative speed with which younger and older adults can indicate the answer to a comprehension or semantic plausibility question after a sentence has been heard. These studies have typically employed a verbal or manual response, such as a key press, to indicate the moment the meaning of the sentence has been understood. Such measures have uniformly implied that older adults are slower in processing speech input than younger adults (e.g., Wingfield et al.,
To address this question, we took advantage of the finding that an individual’s eye-gaze to a picture of an object on a computer screen can be closely time-locked to its reference in a spoken sentence, such that eye-tracking can serve as a useful technique for studying real-time (in-the-moment) speech comprehension (Cooper,
Since our question pertains to age differences, it is also fortunate that there are only minimal age differences in the velocity of saccadic eye movements (Pratt et al.,
Our research strategy was to present younger and older adults recorded sentences that referred to a particular object, with their task being to select, as quickly as possible, the correct one of four pictured objects displayed on a computer screen. Our contrast would be the potential age difference in the time to indicate the referenced object with an overt, off-line selection response, vs. the moment the participants’ eyes fixated on the referenced object as an on-line measure of when the referenced object was actually understood.
In the original “visual world” eye-tracking paradigm participants viewed objects on a computer screen with instructions such as “put the apple that is on the towel in the box”. Using an eye-tracking apparatus that recorded where the eye was fixated on the computer screen, it was found that the participants’ eye gaze moved from object to object as the sentence was being understood as it unfolded in real time (Tanenhaus et al.,
Pertaining to our second question, a number of behavioral methods have been proposed to measure processing effort. One may, for example, assess the degree of effort by the degree to which conducting a speech task interferes with a concurrent non-speech task (e.g., Naveh-Benjamin et al.,
To avoid these pitfalls we took advantage of an unusual feature of the pupil of the human eye. Beyond the reflexive change in pupil diameter in response to changes in ambient light, and the discovery that the pupil enlarges with a state of emotional arousal (Kim et al.,
Participants were 20 younger adults (6 men, 14 women) ranging in age from 18 to 26 years (
All participants were screened using the Shipley vocabulary test (Zachary,
Audiometric evaluation was carried out for all participants using a Grason-Stadler AudioStar Pro clinical audiometer (Grason-Stadler, Inc., Madison, WI, USA) by way of standard audiometric techniques in a sound-attenuated testing room. The younger adults had a mean better-ear pure tone threshold average (PTA) of 7.6 dB HL (
Vision screening was conducted using a Snellen eye chart (Hetherington,
This study was carried out in accordance with the approval of the Brandeis University Committee for the Protection of Human Subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki.
The stimuli consisted of 44 sentences recorded by a female speaker of American English. The sentences were spoken with natural prosody and speech rate. The spoken sentences were recorded on computer sound files using Sound Studio v2.2.4 (Macromedia, Inc., San Francisco, CA, USA) that digitized (16-bit) at a sampling rate of 44.1 kHz. Root-mean-square (RMS) amplitude was equated across sentences. Each of the sentences made reference to a picturable object that always formed the last word of the sentence. The waveform of an example sentence is shown in Figure
Because listeners may continually update their understanding of a sentence as it is being heard, it is possible for the referent of a sentence to be understood before the sentence has been fully completed (Huettig,
The KP was determined for each sentence using a cloze procedure with a separate group of participants (27 younger adults, 9 males and 18 females;
For each trial the participants were presented with an array of four pictures of objects displayed in the four corners of a 1280 × 1040-pixel computer screen. Each object was surrounded by a 100-pixel diameter black ring to indicate the area within which the participant would be asked to place the computer cursor to indicate his or her selection. A 50-pixel red fixation circle was centered on the computer screen. Pictures were selected predominantly from the normed color image set of Rossion and Pourtois (
In all cases, one of the pictures corresponded to the final word of the sentence that would be heard (
Participants were seated 60 cm from the computer screen with their head placed in a custom-built chin rest to stabilize head movement. Each trial began with the participant positioning the computer cursor on the red fixation circle. This was followed by a 2 s display of the particular four-picture array for that trial to allow the participant to familiarize himself or herself with the pictures and their positions on the computer screen. After the 2 s familiarization period the fixation circle turned blue. This signaled the participant to click on the fixation circle to initiate the sentence presentation. The participant’s instructions were to listen carefully to the sentence and to choose the picture that they believed corresponded to the last word of the sentence as soon as they believed they knew the word. They were to indicate this by using the computer mouse to move the cursor from the fixation circle to the target object and clicking on the mouse to confirm the selection. The computer recorded the moment in time that the participant “clicked” on the correct picture with the mouse (
Throughout the course of each trial the participant’s moment-to-moment eye-gaze position on the computer screen and changes in pupil size were recorded via an EyeTrac 6000 (Model 6 series, Applied Science Laboratories, Bedford, MA, USA) eye-tracker that was situated below the computer screen and calibrated using EyeTrac software. These data as well as computer mouse movements and response-selection mouse-clicks were recorded via Gaze Tracker software (Eye Response Technologies, Inc., Charlottesville, VA, USA) at a rate of 60 Hz. The sentences and pictures were presented via a custom MATLAB (MathWorks, Natick, MA, USA) program.
The sentences were presented binaurally over Eartone 3A (E-A-R Auditory Systems, Aero Company, Indianapolis, IN, USA) insert earphones. To insure audibility sentences were presented at 25 dB above each individual’s better-ear SRT. The main experiment was preceded by three practice trials using the same procedures as used in the experiment. None of these sentences or pictures was used in the main experiment.
With our procedures we thus had two measures for each sentence presentation: the
The EFTs and the ORTs were measured from the word representing the KP for that sentence. This measure was taken from the midpoint of the KP word to take into account the finding that word recognition often occurs before the full duration of a word has been heard, especially when heard within a sentence context (Grosjean,
The waveform of the example sentence in Figure
Figure
This dissociation between knowing the identity of the referenced object, as evidenced by the participant’s eye movements to the target picture, and indicating this knowledge by an overt response, was supported by a 2 (Response type: EFT, ORT) × 2 (Age: Younger, Older) mixed-design analysis of variance (ANOVA), with response type as a within-participants factor and age as a between-participants factor. This confirmed a significant main effect of response type (
As previously noted, stimuli were presented at a loudness level relative to each individual’s SRT (25 dB above SRT). This procedure was followed to ensure that the stimuli would be audible for all participants. Following the above-cited ANOVA, we conducted an analysis of covariance (ANCOVA) with better-ear PTA as a covariate. This analysis confirmed the same pattern of main effects and the Response type × Age interaction with these effects uninfluenced by hearing acuity. Although confirming that our presentation of the speech stimuli at an equivalent suprathreshold level for each participant was successful in ensuring audibility of the stimuli, this should not necessarily imply that those with better and poorer hearing acuity accomplished their success with equivalent listening effort.
To explore the possibility that hearing acuity differences among the older adults may have affected processing effort, we separated the older adult participants into two subgroups based on a median split of hearing acuity.
The
The
The left, middle, and right panels of Figure
The normal-hearing and hearing-impaired older adults were similar in age, with the normal-hearing older adults ranging in age from 65 to 88 years (
Pupil size was continuously recorded at a rate of 60 times per second using the previously cited ASL eye tracker (Model 6 series, Applied Science Laboratories, Bedford, MA, USA). routed through the presentation software (GazeTracker, Applied Science Laboratories, Bedford, MA, USA) to allow for pupil size measurements to be synchronized in time with the speech input. Measures of pupil diameter were processed with software written with Matlab 7 (Mathworks, Natick, MA, USA).
Eye blinks were determined by a sudden drop in vertical pupil diameter and were removed from the recorded data prior to data analysis. As is common in pupillometry studies, blinks were defined by a change in the ratio between the vertical and the horizontal pupil diameter. For an essentially circular pupil, the ratio would be approximately 1.0. During a blink or semi-blink the ratio quickly drops toward 0. All samples with a ratio differing more than 1
When comparing relative changes in pupil sizes across age groups it is necessary to adjust for
To adjust for this potential age difference in the pupillary response, pupil sizes were normalized by measuring, for each individual prior to the experiment, the range of pupil size change as the participant viewed a dark screen (0.05 fL) for 10 s followed by a white screen (30.0 fL) for 10 s. Based on the individual participant’s minimum pupil constriction and maximum pupil dilation, we scaled his or her pupil diameter according to the equation: (
Figure
A one-way ANOVA conducted on the data shown in Figure
It has been well documented that older adults are on average slower than their younger adult counterparts on a range of perceptual and cognitive tasks (Cerella,
The observed dissociation in this experiment between knowing and the speed of expressing this knowledge in sentence comprehension is consistent with the previously cited distinction suggested by Caplan and Waters (
In this regard, we present our processing-speed data with two caveats. The first is that the sentences used in this study were heard in quiet, were presented at individually adjusted suprathreshold levels, and that they were intentionally, like most of the sentences we hear on a daily basis (e.g., Goldman-Eisler,
Although hearing impairment is known to interact with syntactic complexity when off-line measures of comprehension are employed (e.g., Wingfield et al.,
It may thus be that an absence of age or hearing acuity effects on on-line comprehension speed as demonstrated in the present experiment for syntactically simple sentences might appear when listeners are presented with syntactically complex sentences that place a heavy demand on working memory for their resolution, and perhaps further affected by more challenging listening conditions such as the presence of background noise or especially rapid input rates that are known to place older adults and those with hearing loss at a special disadvantage (see Wingfield and Lash,
The second caveat is that, in addition to our use of sentences with non-complex syntactic constructions not expected to place significant demands on working memory (see for example Carpenter et al.,
An alternative to eye-gaze as a measure of on-line sentence processing has been to measure electrical brain activity using event-related potentials (ERPs) as a marker of sentence comprehension. Such studies have primarily focused on the finding that an N400 component of the ERP responds to a semantic violation in a sentence while a P600 component responds to a syntactic violation (see the review in Kutas and Federmeier,
As we saw, steps were taken to insure that the speech materials were presented at an audible sound level for all participants, such that differences in hearing acuity did not affect either the EFTs to the correct object pictures or the ORTs. This should not imply, however, that this success was achieved with equivalent effort for those older adults with normal hearing or impaired hearing acuity. Indeed, using the pupillary response as a physiological index of processing effort (Piquado et al.,
The underlying connection between effortful processing and the task-evoked pupillary response (TEPR) remains a topic of active investigation. Current evidence suggests that task-related increases in pupil diameter are associated with activity of the
At the behavioral level, however, increases in pupil size relative to baseline have been shown to serve as a reliable index of effortful processing, whether in response to listening effort attendant to a degraded speech signal (Zekveld et al.,
The present study revealed larger adjusted pupil sizes in older adults with impaired hearing, relative to those with normal hearing acuity, in the time period just prior to the point where eye-fixations indicated knowledge of the object being referred to in the sentence. We take these data to support the likelihood that the hearing-impaired participants’ successful comprehension was accomplished with greater effort than the equivalent success of the older adults with better hearing acuity.
This latter finding is especially important in the face of mounting evidence that successful perception of degraded speech can come at the cost of resources that would otherwise be available for encoding what has been heard in memory (Rabbitt,
Taken together, our results show that although general slowing may be a hallmark of adult aging, its effects do not apply uniformly across all linguistic operations. Specifically, we found that eye fixations on a referenced object in a spoken sentence occurred as rapidly for older adults as for younger adults, although the older adults were slower in indicating the referenced object with an overt response. As we have indicated, this observed dissociation is consistent with Caplan and Waters (
The prevalence of hearing impairment among older adults has led to an almost exponential increase in studies of listening effort; how it can be defined and measured (McGarrigle et al.,
We have cited studies showing that listening effort attendant to mild hearing loss can affect speech comprehension and effectiveness of encoding what has been heard in memory. Although many older adults may be unaware of this “hidden effect” of hearing impairment on comprehension and immediate memory, there is one consequence of hearing loss that many older adults do recognize. That is, even with a relatively mild hearing loss, many older adults report a sense of stress and end-of-the-day fatigue consequent to the continual effort needed to understanding daily conversational speech (Pichora-Fuller,
In this latter regard, we emphasize the importance of maintaining task engagement by the older adult with or without hearing impairment, even at the cost of cognitive effort. The alternative would be to avoid all difficult tasks that would lead to a potential downward spiral to a general sense of lowered expectations and reduced self-efficacy. We suggest that this was not the case with the hearing-impaired older adults in our study.
An early finding in studies of digit- and word-list recall was that the progressive increase in pupil size as the size of a to-be-recalled list was increased may cease, or reverse, at the point where a list becomes so long as to lead to a memory overload (Kahneman and Beatty,
Although our focus is on aging, and age-related hearing loss, it should be noted that mental fatigue due to the continual effort required to successfully understand others’ speech, and its potential effects on cognitive effectiveness, is no less a concern for young adults with hearing impairment (Hicks and Tharpe,
The frequency-selective amplification and signal processing algorithms available in modern hearing aids can not only improve speech intelligibility, they can also reduce the resource drain associated with effortful listening (Sarampalis et al.,
Numerous studies have been conducted to discover why this most straightforward of interventions has such a low adoption rate. Although cost is certainly a factor, adoption rates remain low even in those countries where hearing aids are available at no cost (Hougaard and Ruf,
It should also be acknowledged that there are “higher-level” auditory processing deficits (Humes,
Among the interventions available for age-associated performance declines, addressing the immediate (Wingfield and Lash,
NDA contributed to experimental design, data collection and analysis, data interpretation, and drafting and revisions of this manuscript. AL contributed to the conception of this work and experimental design, as well as revisions of this manuscript. AW contributed to experimental design, data interpretation, and drafting and revisions of this manuscript.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was supported by the National Institutes of Health under award R01 AG019714 from the National Institute on Aging (AW). NDA and AL acknowledge support from NIH training grants T32 GM084907 and T32 AG000204, respectively. We also gratefully acknowledge support from the WM Keck Foundation. AL is now at Whittier College, Whittier, CA, USA.