Ryan M. O’Leary
Nataniela Zavlun
Arthur Wingfield- Department of Psychology, Volen National Center for Complex Systems, Brandeis University, Waltham, MA, United States
Pupil dilation has become a well-established physiological index of cognitive effort. Here we examined the often-cited finding of an inverted U-shaped function of pupil size across the continuum from highly degraded to clear speech. Unlike the short sentences with simple syntax as used in the original inverted-U demonstrations, 32 young adults recalled sentences with subject-relative embedded clause structures and more syntactically complex object-relative embedded clause structures. Sentences were degraded using time-compression and presented at 10, 25, 35 and 100% of their normal playing time selected from a preliminary calibration study. Subject-relative sentences showed an approximately symmetrical inverted U-shaped function across speech rates, but object-relative sentences failed to show the drop in pupil size with normal-rate speech. We conclude that the symmetric inverted U-shaped function of pupil size with increasing speech quality is limited to sentences with simple syntax. We thus offer a more complete picture of the inverted-U than currently in the literature.
Introduction
Although varying in detail from one language to another, all societies on earth have developed a rich speech system to communicate complex thoughts and intentions. In the everyday world, however, it is rare for spoken language to be heard under ideal listening conditions. In addition to the rapidity of natural speech that can challenge limits on processing speed (Chodorow, 1979; Foulke, 1971), speech quality is frequently degraded by extrinsic sources such as the presence of background noise (Peelle and Wingfield, 2022; Zekveld et al., 2011), informational masking from competing speakers (Ohlenforst et al., 2018), or intrinsic sources, such as hearing impairment (Kuchinsky et al., 2014; Zekveld et al., 2011).
In spite of these challenges, comprehension of spoken languages is noted for its resilience. However, even successful comprehension of rapid or degraded speech requires cognitive effort (recruitment of resources) on the part of the listener (Rönnberg et al., 2013; see also Ayasse et al., 2021; Peelle, 2017; Pichora-Fuller et al., 2016). A fundamental question central to speech communication is thus how best to characterize the relationship between speech quality, cognitive effort, and performance.
It should be noted that the term, effort, is often modified as listening effort, cognitive effort, or more generally, processing effort, depending on context or author. In the remainder of this article, we adopt a generic definition of effort as offered in the Framework for Understanding Effortful Listening (FUEL; Pichora-Fuller et al., 2016); “The deliberate allocation of mental resources to overcome obstacles in goal pursuit when carrying out a task.” We use the terms, cognitive effort, or processing effort, as inclusive of listening effort attendant to the recruitment of resources required for successful processing of a degraded acoustic signal at the perceptual level (Pichora-Fuller et al., 2016) and processing at the linguistic level when the stimuli consist of meaningful sentences (Carpenter et al., 1994; DeCaro et al., 2016).
Pupillometry (the measurement of task-related dilation of the pupil of the eye) has become a well-established physiological index of cognitive effort (Kuchinsky and Milvae, 2023). For example, one can observe dilation of the pupil while listeners are attempting to understand speech degraded by acoustic masking (Ohlenforst et al., 2018), degraded by a mild-to-moderate hearing loss (Kuchinsky et al., 2014; Zekveld et al., 2011), and processing especially rapid speech (O’Leary et al., 2023, 2025a). An increase in pupil dilation also appears while listeners are processing sentences that express their meaning with complex syntax (Ayasse and Wingfield, 2018; Just and Carpenter, 1993; Piquado et al., 2010).
Although a monotonic relationship between pupil dilation and increasing perceptual or cognitive demands has been reliably demonstrated, it has been shown that this relationship holds only within limits. Norman and Bobrow (1975) (see also Knight et al., 2023) offered two terms that are useful in an attempt to characterize this non-linear relationship between task difficulty and the effort likely to be allocated to a task.
At one extreme are occasions where speech quality is so impoverished by background noise, accelerated speech rates, or other acoustic perturbations, that successful recognition appears nearly impossible (a data-limited process). In the framework of behavioral economics, individuals are unlikely to expend effort on a task when there is an unlikely return on investment (Eckert et al., 2016). This would result in, or approach, a floor effect in performance, and a relatively smaller pupillary response as an indicator of this reduced effort.
As the quality of the speech improves, one sees the often-cited monotonic relationship between speech quality, effort, and performance; in Norman and Bobrow’s terms, a resource-limited process. As the quality of the speech further improves, the pupillary response becomes smaller due to the task becoming easier, hence requiring less effort.
The above principles are illustrated in an idealized form in Figure 1. One curve shows a typical sinusoidal psychometric function representing performance accuracy as a function of speech clarity. Superimposed on this performance curve is a second curve that represents cognitive effort, showing an inverted U-shaped function as speech quality improves (see Kuchinsky and Milvae, 2023, p. 237 for a similar function).
Figure 1. Idealized representation of postulated effort as a function of speech clarity (black curve) superimposed on the proportion of speech materials reported correctly (gray curve).
Figure 1 is an idealized schematic as it excludes effects of task demands that impact cognitive resources such as speed/accuracy tradeoffs (O’Leary et al., 2025b), and pupillary dynamics beyond the acoustic clarity of the stimulus, such as attention to the task and the listener’s motivation to succeed (cf., Pichora-Fuller et al., 2016; Richter, 2016).
The postulated inverted U-shaped function in pupil dilation nevertheless appears as a reliable finding (Ohlenforst et al., 2017, 2018; Wendt et al., 2018) and has been frequently cited in cognitive hearing science (see, for example, Kuchinsky and Milvae, 2023; Keur-Huizinga et al., 2024).
The focus of the present study is on the region of asymptotic performance, where Ohlenforst et al. (2018) and Wendt et al. (2018) have shown a drop in pupillary-indexed processing effort that approximates the level observed for a data-limited process. Because the interest in such studies has been on perceptual effort in the face of acoustic degradation, the stimuli have focused on short, syntactically simple sentences such as the 5-word active-declarative sentences from Nielsen and Dau’s (2009) Danish Hearing in Noise Test (HINT) (Ohlenforst et al., 2017, 2018; Wendt et al., 2018). It is possible, however, that the observed drop in pupillary response in the asymptotic region of task performance is limited to the use of syntactically simple sentences with which the drop has been observed. That is, it may be that additional difficulty introduced at the linguistic level may carry over to influence the shape of the inverted-U function. The following experiment was designed to test this hypothesis, offering a more complete picture of the performance-effort relationship than currently represented in the literature.
Two sentence types were chosen for this test: sentences with a subject-relative center-embedded clause (SR sentences) versus syntactically more complex object-relative center-embedded clauses (OR sentences).
Sentences with an SR structure (e.g., “Girls with big black backpacks that leave boys are harsh”) take a canonical form in which the first noun in the sentence (girls) is the agent that performs an action (leave boys), with the action the first verb encountered, and the recipient of the action (boys) is the second noun encountered. Sentences with an SR structure were contrasted with syntactically more complex OR sentences (e.g., “Boys that girls with big black backpacks leave are harsh”). It can be seen that the meaning has not changed (girls still leave boys), but the structure departs from the canonical form of an SR sentence.
A critical feature of the SR versus OR contrast is that they can be constructed to contain exactly the same words, albeit in a different order. In this way, factors such as word frequency and the number of phonological competitors for each word are automatically equated. Importantly, it is already well established in the psycholinguistic literature that OR sentences produce more errors in comprehension and recall than SR sentences (Carpenter et al., 1994; DeCaro et al., 2016; Obler et al., 1991; Stewart and Wingfield, 2009; Wingfield et al., 2006). Pertinent to the present study, it has also been shown that even with clear speech in quiet conditions, processing OR sentences results in relatively larger pupil dilations than SR sentences (Ayasse et al., 2021; Just and Carpenter, 1993; Piquado et al., 2010).
To examine whether the well-documented inverted-U pattern of effort generalizes beyond acoustic masking, we incorporated time-compressed speech to parametrically vary processing time and perceptual demands, allowing us to examine how speech rate and syntactic challenge jointly influence the effort–performance relationship.
The present experiment
The following experiment was conducted in two stages using time-compression to control speech rate. The first was a calibration study to establish the psychometric function representing a continuum of recall accuracy for SR and OR sentences as a function of decreasing degrees of time-compression. These data would be used to select compression ratios of interest for pupillometric analysis in the main experiment.
Manipulating performance accuracy with time-compression has the interesting feature that at moderate compression ratios, the difficulty is primarily due the loss of ordinarily available processing time, while at very high compression ratios a loss of richness of the speech signal is a limiting factor in phonological and lexical identification (Chodorow, 1979; Heiman et al., 1986; Wingfield et al., 1999).
The main experiment was conducted with a larger group of participants who were tested for recall of SR and OR sentences at four compression ratios selected to represent low, moderate, and high recall performance as determined in the calibration study. In this main experiment pupil dilation was recorded as an index of processing effort. Reducing the number of compression ratios allowed an increase in the number of trials per speech rate to increase power without lengthening the duration of the experiment and risking participant fatigue, a concern when using pupillometry (Winn et al., 2018).
Based on the previously cited studies that have demonstrated an inverted U-shaped function (e.g., Ohlenforst et al., 2017, 2018; Wendt et al., 2018), we expect to see a smaller pupil size in the lower end of the performance curve, larger pupil sizes when performance was at approximately 50%, and, for the simpler SR sentences, a drop in pupil size when accuracy has reached a performance asymptote. In the case of OR sentences, the cognitive effort attendant to processing these more complex sentences may counter the traditional drop in processing effort at the asymptotic performance level.
Methods
Calibration study: determining the performance curves for SR and OR sentences
Participants
Ten young adults, 4 males and 6 females, aged 18–26 years (M = 19.4, SD = 1.58) took part in the calibration study. Participants’ hearing was tested using a Grason-Stadler AudioStar Pro clinical audiometer (Grason-Stadler, Inc., Madison, WI) by way of standard audiometric procedures in a sound-attenuated testing room. All participants had age-normal hearing, with a pure tone average (PTA) across 0.5, 1, 2, and 4 kHz of 5.00 dB HL (hearing level) (SD = 4.04). All participants reported English as their first language. Written informed consent was obtained for a protocol approved by the Brandeis University Institutional Review Board (IRB).
Stimuli
Creation of the stimuli began with the construction of 140, 9-word grammatical English sentences with an SR structure (e.g., “Girls with coffee mugs that approach guys are social”). These are referred to as base sentences. For each SR base sentence, an OR sentence was created by changing the word order to yield a sentence with the same meaning, but expressed with an OR construction (e.g., “Girls with coffee mugs that approach guys are social”). This process produced a total of 280 test sentences. In addition to the test sentences, 50 filler sentences without a relative clause were included to discourage participants from potentially recognizing patterns within the stimuli. Performance on the 50 filler sentences was not analyzed.
Both test sentences and fillers were recorded by a male speaker of American English using natural prosody onto computer sound files using Sound Studio v2.2.4 (Felt Tip, Inc., New York, NY) that digitized (16-bit) at a sampling rate of 44.1 kHz. Recordings were equalized across sentences for root-mean-square (RMS) intensity using Praat (Boersma and Weenink, 2022).
Recordings of each of the 280 test sentences (140 SR sentences; 140 OR sentences) and the 50 filler sentences were time-compressed to be reproduced in 10, 15, 20, 25, 30, 35, 40, 60, 80, and 100% (not compressed) of the original playing time, corresponding to 1,486, 977, 743, 586, 495, 419, 371, 244, 186, and 146 words per minute (wpm). We note here that the increments were intentionally uneven, as we wished to sample critical regions along the speech rate continuum. To put these figures in perspective, everyday conversational speech averages between 140 to 180 wpm, while a radio or TV newsreader working from a prepared script may reach 210 wpm.
Time-compression was conducted using Praat (Boersma and Weenink, 2022) employing the pitch-synchronous overlap-and-add (PSOLA) algorithm. This compression algorithm uses uniform linear compression such that both words and silent pauses were reduced equivalently to maintain the relative temporal pattern of the original recordings. The algorithm leaves the original pitch contour intact.
Procedure
The calibration study used a within-participant design, with each participant hearing 190 sentences. Of those, 70 were SR sentences and 70 were OR sentences, plus 50 filler sentences. No participant heard the same base sentence in both its SR and OR forms, with sentences and structures counterbalanced across participants, such that, after all participants had been tested, each base sentence had been heard in its SR and OR forms. Participants received 19 sentences at each of the compression ratios (7 SR sentences and 7 OR sentences, plus 5 filler sentences).
Instructions in all cases were to listen carefully to the sentence and, when it finished, attempt to recall the sentence as accurately and completely as possible. Speech rates and sentence types were intermixed in presentation. Stimuli were presented binaurally at 65 dB HL (hearing level) over Eartone 3A insert earphones (E-A-R Auditory systems, Aero Company, Indianapolis, IN). Prior to the experiment, participants repeated two-syllable common nouns presented at 65 dB HL to confirm audibility. This was followed by a two-part familiarization and practice session. In the first part, participants were instructed to listen to a multi-sentence passage that had been compressed to each of the compression ratios used in the present study to familiarize them with the rate of the materials and the speaker’s voice. In the second part, participants were asked to listen to and repeat one sentence at the various compression ratios. For practice for the main pupillometry experiment there were two sentences presented at a compression ratio of 100%, one sentence at both 25 and 35%, and one sentence presented at 10%. None of the familiarization stimuli were used in the study.
Main experiment
Pupillary responses at critical regions of the performance curve
The main experiment measured pupil dilation to assess the level of effort participants committed to task performance. To increase power for the pupillometric portion of the study, four of the 10 compression ratios tested during the calibration study were selected to sample critical regions in the performance curve (10, 25, 35, and 100%). These ratios were chosen to sample points in the low, medium, and high range of performance to capture the hypothesized “inverted-U” pattern.
Participants
The participants were 32 young adults, 12 males and 20 females, aged 18–26 years. (M = 19.69, SD = 1.62). All had age-normal hearing, with a mean PTA of 7.6 dB HL (SD = 4.59). None of the participants from the calibration study served in the main experiment. All participants in the main experiment reported normal or corrected-to-normal vision. All participants reported English as their first language. Written informed consent was obtained from a protocol approved by the Brandeis University IRB.
Stimuli
The stimuli consisted of 142 SR sentences, 142 OR sentences. and 52 filler sentences as described in the calibration study. Two additional SR, OR, and filler sentences with the same characteristics were added to aid counterbalancing. Each of the SR sentences, its OR version, and the 52 filler sentences were time-compressed to 10, 25, 35 and 100% of the original playing time. The same male speaker of American English, recording procedures, and time-compression methods were as described for the calibration study.
Procedures
Each participant heard 192 sentences. Of those, 72 were SR sentences and 72 were OR sentences, plus the 52 filler sentences. Eighteen SR, 18 OR, and 13 filler sentences were presented at each of the four compression ratios. No participant heard the same base sentence in both its SR and OR forms, with sentences and structures counterbalanced across participants, such that, after all participants had been tested, each base sentence had been heard in its SR and OR forms and compression ratio an equal number of times. Sentence types and compression ratios were intermixed in presentation. To mitigate the potential impact of fatigue on the pupillometry data, SR and OR and speech rate trials were evenly distributed throughout the duration of the experiment. As in the calibration study, participants were instructed to listen to each sentence and then recall the sentence as accurately and completely as possible. Stimuli were presented binaurally over EAR insert earphones at 65 dB HL. The duration of the entire procedure was approximately between one and one and a half hours.
Pupillometry
Relative pupil dilation was measured as an index of processing effort associated with effects of syntax and compression ratio. Monocular pupil size was measured via an Eyelink 1,000 Plus eye-tracker (SR Research, Mississauga, Ontario, Canada) using a standard nine-point calibration procedure using a sampling rate of 1,000 Hz. The Eyelink camera was positioned 60 cm, zero azimuth, from the participants’ eyes, with a chinrest used to minimize participants’ head movements and to maintain the 60 cm distance between the participant’s eye and the Eye-Link camera. To reduce gaze position-dependent measurement errors (Gagl et al., 2011), eye movements were minimized by instructing participants to keep their eyes fixated on a continuously displayed 2 cm2 black cross centrally located on a computer screen placed above the EyeLink camera.
The computer screen was filled with a medium gray color to avoid ceiling or floor effects on the pupil size at baseline (Winn et al., 2018). Eye blinks were detected and removed using GazeR functions (Geller et al., 2020). Trials in which more than 40% of the data were missing due to blinks were removed from analyses, resulting in the exclusion of 9.07% of the trials (cf., Burg et al., 2021). Bink interpolation and signal filtering were not performed due to concerns of increasing type 1 error. The pupillary response for each sentence was baseline corrected to account for non-task changes in pupil size as can occur across trials. This was accomplished by subtracting the mean pupil size over the last 1-s of a 3-s silent period that preceded each sentence from the task-related pupil size measures (see Reilly et al., 2019, for data and a discussion of linear versus proportional baseline scaling). Although not without debate (e.g., McLaughlin et al., 2022), in cases of comparative populations or individual differences, pupil size is often normalized with respect to the pupillometric response to light range by varying screen luminance. In the present case this method was not used as the focus was within subject comparisons. Pupillary responses were aligned to the offset of each sentence.
Statistical analyses were based on the average pupil size of the final 2.5 s within a 3-s window that followed the sentence presentation. This was used to capture pupil dilation as the participant was processing the sentence and preparing their response, which is known to be sensitive to effects of cognitive effort (cf., O’Leary et al., 2025c; Piquado et al., 2010). Importantly, this region is less impacted by the potential temporal confound of stimulus length that is introduced by the use of time-compression.
Statistical analysis
Performance data from both the calibration and main studies, as well as pupillary responses from the main study, were analyzed using linear mixed-effects models in R version 4.3.1 (LMEM’s; Bates et al., 2015) with participants and items/stimuli included as random intercepts. We note that the items/stimuli term refers to the individual sentence items used in the experiment, not the experimental conditions. These random intercept terms were included to account for any idiosyncratic baseline difference among sentence items or participants that may be independent of the manipulated factors. For the calibration study, Compression ratios (ordered factor: 10, 15, 20, 25, 30, 35, 40, 60, 80, and 100%), Sentence type (SR vs. OR), and their interaction were included as fixed effects. The main study had the same structure of fixed and random effects, except Compression ratio had a reduced number of levels (10, 25, 35 and 100%). In all cases, a reverse selection approach was used to analyze models, starting with a maximal model including all variables and interactions. From the maximal model, likelihood ratio tests were used to contrast models and remove nonsignificant effects to find the most parsimonious model (Bates et al., 2015). Reported p-values are the result of likelihood ratio tests.
Results
Figure 2 shows the time series pupillary data for the SR and OR sentences for four speech rates selected on the basis of the results of the calibration on study to be described. One sees the traditional incremental increase in pupil size as the words of the sentences unfold in time. Pupil size is also shown for 3 s following the sentence presentation.
Figure 2. Time series of pupillometry data for sentences that were heard at each of the four compression ratios (separated by panel) for both SR (blue) and OR (red) sentences. Sentences are aligned to stimulus offset. The post-sentence region used to extract mean pupil dilation for analyses is indicated by shading in yellow. Error ribbons are one standard error.
Figure 3 shows two sets of curves. The two psychometric functions in Figure 3 show the mean proportion of words correctly recalled from SR and OR sentences at each of the ten compression ratios tested in the calibration study. As would be expected, recall performance yields a sinusoidal function (a progressive increase in performance accuracy with two inflection points), with recall accuracy ranging from a data-limited process when speech was heard at 10% of its original duration, rising to an asymptotic level of performance as the compression ratio was reduced (fixed effect of time-compression: χ2(9) = 2398.90, p < 0.001).
Figure 3. The dotted lines plot recall accuracy for SR (red) an OR (blue) sentences as a function of compression ratio based on results from the calibration study. Superimposed on these psychometric functions are the mean baseline-adjusted pupillary response during the post sentence interest regions for each of four compression ratios tested for both SR (red curve) and OR (blue curve) sentences. Error bars around the mean pupil sizes are one standard error.
Although the shape of the functions for SR and OR sentences was similar, a distinction between SR and OR accuracy began to appear when 50% accuracy was achieved, with SR sentences reaching a performance asymptote at a higher level of accuracy than the OR sentences. This pattern was reflected by a significant main effect of sentence type (χ2(1) = 85.22, p < 0.001) and a significant Compression ratio X Sentence type interaction (χ2(9) = 47.77, p < 0.001).
Recall accuracy for the four selected compression ratios were similar to those obtained at the same compression ratios in the calibration study (see Supplementary Figure S1). Consistent with the calibration study, there was a significant effect of Compression ratio (χ2(3) = 2664.5, p < 0.001), a significant effect of Sentence type (χ2(1) = 95.85, p < 0.001), and a significant Compression ratio X Sentence type interaction (χ2(3) = 38.32, p < 0.001).
Figure 3 shows the mean pupillary response from the post-sentence region of interest for the SR and OR sentences at the four selected compression ratios superimposed on the performance curves from the calibration study. To visualize the inverted U-shaped pupillary response, the four sampled points were connected using the geom_smooth function of ggplot2 (Wickham, 2016). We note that, as there are only four points sampled, the smooth lines should be interpreted with caution as there are many points along the continuum that were not sampled. Results of the linear mixed effects model comparison procedure confirmed a significant main effect of Compression ratio on pupil size (χ2(3) = 85.78, p < 0.001), indicating that pupil size changed significantly across compression ratios. The main effect of Sentence type on pupil size was also significant (χ2(1) = 9.40, p = 0.002), but a significant interaction between Compression ratio and Sentence type (χ2(3) = 22.31, p < 0.001) indicated that the effect of syntax was not uniform at all levels of time compression. This interaction reflects the separation in pupil size between SR and OR sentences, with only the SR sentences showing the often-cited inverted U. Follow up paired t-tests were conducted at each of the compression ratios to confirm the location of this effect. As expected, mean pupil size between SR and OR sentences were not significantly different at speech rates of 10% (t(31) = 1.32, p = 0.196), 25% (t(31) = 0.28, p = 0.783), or 35% (t(31) = 1.22, p = 0.230). Supporting the interpretation that a difference in pupil size emerges between SR and OR sentences when they are heard at normal rate, the paired contrast between pupil size for SR and OR sentences at the compression rate of 100% was statistically significant (t(31) = 3.90, p < 0.001).
Discussion
The calibration study yielded the expected sigmoidal performance functions across 10 compression ratios. From the psychometric functions, four regions of interest were sampled for the main pupillometry experiment. Recall accuracy in the main experiment closely matched the recall performance at these points in the calibration study, confirming the selection of these four compression ratios for pupillometric examination.
An equivalent floor effect in recall performance for both subject relative (SR) and object-relative (OR) sentences when speech was compressed to 10% of its normal rate followed the definition of a data-limited condition (Norman and Bobrow, 1975). The small size of the pupillary response at this point offers independent evidence of participants’ minimal commitment of effort to what was essentially an impossible task. To the extent that the size of the pupillary response is a valid index of effort, the minimal pupillary response is consistent with arguments that listeners will invest effort only to the extent that they believe this investment is likely to yield a return on this investment (Eckert et al., 2016; see also Westbrook and Braver, 2015). As postulated by Kahneman (1973), and as applied specifically to degraded speech in FUEL (Pichora-Fuller et al., 2016), there is thus a motivational element to the commitment of effort (see, for example, motivation intensity theory; Brehm and Self, 1989; Gendolla and Richter, 2013; Richter, 2013; see also Aston-Jones and Cohen, 2005).
Decreasing degrees of compression corresponding to slower speech rates resulted in a region in which an additional commitment of effort begins to bring returns; the region in which we observed an increase in mean pupil dilation accompanied by a monotonic increase in recall performance. This progressive increase in pupil dilation and performance held in a similar manner for both SR and OR sentences. When performance exceeded a level of approximately 50% correct recall for the SR sentences, a figure close to that reported by Wendt et al. (2018), pupillary responses began a decrease, reflecting the reduced effort needed as processing the slower speech required fewer resource demands. This pattern yielded the inverted U-shaped function as described in past studies (Ohlenforst et al., 2017, 2018; Wendt et al., 2018) and as appears in literature reviews (e.g., Keur-Huizinga et al., 2024; Kuchinsky and Milvae, 2023).
A different picture appears for the syntactically complex OR sentences. That is, even with a normal speech rate and asymptotic performance, mean pupil dilation fails to drop to a level approximating that for the data-limited condition that would complete the inverted U. This suggests that even though perceptual processing has become less demanding in terms of speech rate, one is still left with the resource demands associated with processing syntactically complex OR sentences (e.g., Carpenter et al., 1994; DeCaro et al., 2016).
In the time series data for the uncompressed condition, we note that differences seem to arise between the OR and SR sentence before the sentence has been completed. This pattern is not unexpected given the noncanonical word order of OR sentences. The early appearance of this sensitivity in the uncompressed condition can be attributed to the clarity of the acoustic signal and the unfolding of words at a natural rate such that listeners had sufficient time to follow the added syntactic complexity of the sentence and integrate linguistic content as it is heard, albiet at the cost of additional effort. Although likely, we note that pupillary differences within the sentences should be interpreted with caution due to the substantial variability of sentence rates, and the physiological delay of the pupillary response.
There are several accounts of why processing OR sentences is more demanding than for OR sentences, including violated agency expectations (Warren and Gibson, 2002), to the rarity of such noncanonical sentences in everyday conversation (Gibson et al., 2013; Goldman-Eisler, 1968), and to the heavier working-memory demands imposed by OR sentences compared with their SR counterparts (Cooke et al., 2002). An important finding in the present study is that the effect of syntactic difficulty is most apparent when the speech signal is clear and presented at a natural speech rate. The finding that differences in effort between OR and SR sentences appear predominantly in high signal quality conditions is consistent with the Data-Resource-Language (DRL) framework proposed by Mattys et al. (2025). The DRL framework postulates a scaffolding of operations that shift from perceptual to cognitive to linguistic as signal quality is increased. When signal quality is high, listeners have access to the highest degree of linguistic information in the sentence, and thus challenges imposed by the linguistic difficulty are reliably reflected in cognitive effort measurements.
In this study, signal quality was manipulated time-compression, which is one of many methods to perturb the speech signal. Future studies should confirm that the pattern of cognitive effort observed here generalizes to paradigms using background noise, competing speech, or noise-band vocoding. Additionally, extending this work to listeners with varying degrees of hearing loss, as well as users of cochlear implants, would help determine how individual differences in auditory perception modulates the interaction between syntactic complexity and listening effort.
We conclude that the symmetrical inverted U-shaped function in the pupillary response is limited to syntactically simple sentences, such as the 5-word active declarative sentences used in its previous demonstrations. In support of this conclusion, it can be seen that SR sentences, which are less complex than the OR sentences, but more complex than the sentences used in the demonstrations of a symmetrical inverted U, failed to yield a reduction in the pupillary response at asymptote to the level observed for the data-limited condition. With the present data we offer a more complete picture of the dynamic relationship between cognitive effort, task difficulty, and sentence complexity than currently represented in the literature.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by Brandeis University Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
RO’L: Conceptualization, Formal analysis, Visualization, Methodology, Investigation, Validation, Writing – original draft, Writing – review & editing. NZ: Conceptualization, Investigation, Methodology, Writing – original draft, Data curation. AK: Conceptualization, Methodology, Data curation, Writing – original draft, Investigation, Software, Visualization. AW: Writing – original draft, Conceptualization, Writing – review & editing, Project administration, Supervision.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. We are grateful for financial support from the Stephen J. Cloobeck Research Fund and the National Institute on Deafness and Other Communication Disorders R01 DC016834.
Acknowledgments
The authors would like to acknowledge Adetolani Odogiyon for a preliminary unpublished proof of concept pilot study that offered valuable scaffolding for the present experiment. We are also grateful to Sven Mattys and Ronan McGarrigle for many discussions on the “inverted-U.”
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The authors declare that no Gen AI was used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1685938/full#supplementary-material
References
Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604.135709
Ayasse, N. D., Hodson, A., and Wingfield, A. (2021). The principle of least effort and comprehension of spoken sentences by younger and older adults. Front. Psychol. 12:629464. doi: 10.3389/fpsyg.2021.629464
Ayasse, N. D., and Wingfield, A. (2018). A tipping point in listening effort: effects of linguistic complexity and age-related hearing loss on sentence comprehension. Trends Hear. 22, 1–14. doi: 10.1177/2331216518790907
Bates, D., Kliegl, R., Vasishth, S., and Baayen, H. (2015). Parsimonious mixed models. arXiv. doi: 10.48550/arXiv.1506.04967
Boersma, P., and Weenink, D. (2022). Praat: Doing phonetics by computer [computer program]. Version 6.2.05. Available online at: http://www.praat.org/ (Accessed January 5, 2022).
Brehm, J. W., and Self, E. A. (1989). The intensity of motivation. Annu. Rev. Psychol. 40, 109–131. doi: 10.1146/annurev.ps.40.020189.000545
Burg, E. A., Thakkar, T., Fields, T., Misurelli, S. M., Kuchinsky, S. E., Roche, J., et al. (2021). Systematic comparison of trial exclusion criteria for pupillometry data analysis in individuals with single-sided deafness and normal hearing. Trends Hear. 25:23312165211013256. doi: 10.1177/23312165211013256
Carpenter, P. A., Miyake, A., and Just, M. A. (1994). “Working memory constraints in comprehension: evidence from individual differences, aphasia, and aging” in Handbook of psycholinguistics. ed. M. Gernsbacher (San Diego, CA: Academic Press), 1075–1122.
Chodorow, M. S. (1979). “Time-compressed speech and the study of lexical and syntactic processing” in Sentence processing: psycholinguistics studies presented to Merrill Garrett. eds. W. E. Cooper and E. C. T. Walker (Hillsdale, NJ: Erlbaum Academic Press), 87–111.
Cooke, A., Zurif, E. B., DeVita, C., Alsop, D., Koenig, P., Detre, J., et al. (2002). Neural basis for sentence comprehension: grammatical and short-term memory components. Hum. Brain Mapp. 15, 80–94. doi: 10.1002/hbm.10006
DeCaro, R., Peelle, J. E., Grossman, M., and Wingfield, A. (2016). The two sides of sensory-cognitive interactions: effects of age, hearing acuity, and working memory span on sentence comprehension. Front. Psychol. 7:236. doi: 10.3389/fpsyg.2016.00236
Eckert, M. A., Rhodes-Teubner, S., and Vaden, K. I. Jr. (2016). Is listening in noise worth it? The neurobiology of speech recognition in challenging listening conditions. Ear Hear. 37, 101S–110S. doi: 10.1097/AUD.0000000000000300
Foulke, E. (1971). “The perception of time compressed speech” in The perception of language. eds. D. L. Horton and J. Jenkins (Columbus, Ohio: Merrill), 79–107.
Gagl, B., Hawelka, S., and Hutzler, F. (2011). Systematic influence of gaze position on pupil size measurement: analysis and correction. Behav. Res. Methods 43, 1171–1181. doi: 10.3758/s13428-011-0109-5
Geller, J., Winn, M. B., Mahr, T., and Mirman, D. (2020). GazeR: a package for processing gaze position and pupil size data. Behav. Res. Methods 52, 2232–2255. doi: 10.3758/s13428-020-01374-8
Gendolla, G. H. E., and Richter, M. (2013). Opportunity cost calculations only determine justified effort–or, what happened to the resource conservation principle? Behav. Brain Sci. 36, 686–687. doi: 10.1017/S0140525X13000988
Gibson, E., Bergen, L., and Piantadosi, S. T. (2013). Rational integration of noisy evidence and prior semantic expectations in sentence interpretation. Proc. Natl. Acad. Sci. 110, 8051–8056. doi: 10.1073/pnas.1216438110
Goldman-Eisler, F. (1968). Psycholinguistics: experiments in spontaneous speech. London and New York: Academic Press.
Heiman, G. W., Leo, R. J., Leighbody, G., and Bowler, K. (1986). Word intelligibility decrements and the comprehension of time-compressed speech. Percept. Psychophys. 40, 407–411. doi: 10.3758/bf03208200
Just, M. A., and Carpenter, P. A. (1993). The intensity dimension of thought: pupillometric indices of sentence processing. Can. J. Exp. Psychol. 47, 310–339. doi: 10.1037/h0078820
Keur-Huizinga, L., Kramer, S. E., de Geus, E. J., and Zekveld, A. A. (2024). A multimodal approach to measuring listening effort: a systematic review on the effects of auditory task demand on physiological measures and their relationship. Ear Hear. 45, 1089–1106. doi: 10.1097/AUD.0000000000001508
Knight, S., Rakusen, L., and Mattys, S. (2023). Conceptualising acoustic and cognitive contributions to divided-attention listening within a data-limit versus resource-limit framework. J. Mem. Lang. 131:104427. doi: 10.1016/j.jml.2023.104427
Kuchinsky, S. E., Ahlstrom, J. B., Cute, S. L., Humes, L. E., Dubno, J. R., and Eckert, M. A. (2014). Speech-perception training for older adults with hearing loss impacts word recognition and effort. Psychophysiology 51, 1046–1057. doi: 10.1111/psyp.12242
Kuchinsky, S. E., and Milvae, L. F. (2023). “Pupillometry studies of listening effort: implications for clinical audiology” in Modern pupillometry: cognition, neuroscience, and practical applications. eds. M. L. Papesh and S. D. Goldinger (Cham, Switzerland: Springer), 229–225.
Mattys, S., O’Leary, R. O., McGarrigle, R., and Wingfield, A. (2025). Reconceptualizing cognitive listening. Trends Cogn. Sci. doi: 10.1016/j.tics.2025.09.014
McLaughlin, D. J., Zink, M. E., Gaunt, L., Spehar, B., Van Engen, K. J., Sommers, M. S., et al. (2022). Pupillometry reveals cognitive demands of lexical competition during spoken word recognition in young and older adults. Psychon. Bull. Rev. 29, 268–280. doi: 10.3758/s13423-021-01991-0
Nielsen, J. B., and Dau, T. (2009). Development of a Danish speech intelligibility test. Int. J. Audiol. 48, 729–741. doi: 10.1080/14992020903019312
Norman, D. A., and Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cogn. Psychol. 7, 44–64. doi: 10.1016/0010-0285(75)90004-3
O’Leary, R. M., Amichetti, N. M., Brown, Z., Kinney, A. J., and Wingfield, A. (2025a). Congruent prosody reduces cognitive effort in memory for spoken sentences: a pupillometric study with young and older adults. Exp. Aging Res. 51, 35–58. doi: 10.1080/0361073X.2023.2286872
O’Leary, R. M., Carlson, E. L., and Wingfield, A. (2025b). Hearing loss impacts the temporal patterning of pupillary dilation in processing spoken stories. J. Speech Lang. Hear. Res. 1–11.
O’Leary, R. M., Neukam, J., Hansen, T. A., Kinney, A. J., Capach, N., Svirsky, M. A., et al. (2023). Strategic pauses relieve listeners from the effort of listening to fast speech: data limited, and resource limited processes in narrative recall by adult users of cochlear implants. Trends Hear. 27:23312165231203514. doi: 10.1177/23312165231203514
O’Leary, R. M., Omori-Hoffe, N., Dugam, G., and Wingfield, A. (2025c). A drift-diffusion decomposition of conditions that influence shallow (‘good enough’) processing of spoken sentences. Mem. Cogn. 68, 4518–4528. doi: 10.3758/s13421-025-01748-3
Obler, L. K., Fein, D., Nicholas, M., and Albert, M. L. (1991). Auditory comprehension and aging: decline in syntactic processing. Appl. Psycholinguist. 12, 433–452. doi: 10.1017/S0142716400005865
Ohlenforst, B., Wendt, D., Kramer, S. E., Naylor, G., Zekvel, A. A., and Lunner, T. (2018). Impact of SNR, masker type and noise reduction processing on sentence recognition performance and listening effort as indicated by the pupil dilation response. Hear. Res. 365, 90–99. doi: 10.1016/j.heares.2018.05.003
Ohlenforst, B., Zekveld, A., Lunner, T., Wendt, D., Naylor, G., Wang, Y., et al. (2017). Impact of stimulus-related factors and hearing impairment on listening effort as indicated by pupil dilation. Hear. Res. 351, 68–79. doi: 10.1016/j.heares.2017.05.012
Peelle, J. E. (2017). Listening effort: how the cognitive consequences of acoustic challenge are reflected in brain and behavior. Ear Hear. 39, 204–214. doi: 10.1097/AUD.0000000000000494
Peelle, J. E., and Wingfield, A. (2022). How our brains make sense of noisy speech. Acoust. Today 18, 40–48. doi: 10.1121/AT.2022.18.3.40
Pichora-Fuller, M. K., Kramer, S. E., Eckert, M., Edwards, B., Hornsby, B., Humes, L. E., et al. (2016). Hearing impairment and cognitive energy: the framework for understanding effortful listening (FUEL). Ear Hear. 37, 5S–27S. doi: 10.1097/AUD.0000000000000312
Piquado, T., Isaacowitz, D., and Wingfield, A. (2010). Pupillometry as a measure of cognitive effort in younger and older adults. Psychophysiology 47, 560–569. doi: 10.1111/j.1469-8986.2009.00947.x
Reilly, J., Kelly, A., Kim, S. H., Jett, S., and Zuckerman, B. (2019). The human task-evoked pupillary response function is linear: implications for baseline response scaling in pupillometry. Behav. Res. Methods 51, 865–878. doi: 10.3758/s13428-018-1134-4
Richter, M. (2013). A closer look into the multi-layer structure of motivational intensity theory. Soc. Personal. Psychol. Compass 7, 1–12. doi: 10.1111/spc3.12007
Richter, M. (2016). The moderating effect of success importance on the relationship between listening demand and listening effort. Ear Hear. 37, 111S–117S. doi: 10.1097/AUD.0000000000000295
Rönnberg, J., Lunner, T., Zekveld, A., Sörqvist, P., Danielsson, H., Lyxell, B., et al. (2013). The ease of language understanding (ELU) model: theoretical, empirical, and clinical advances. Front. Syst. Neurosci. 7:31. doi: 10.3389/fnsys.2013.00031
Stewart, R., and Wingfield, A. (2009). Hearing loss and cognitive effort in older adults’ report accuracy for verbal materials. J. Am. Acad. Audiol. 20, 147–154. doi: 10.3766/jaaa.20.2.7
Warren, T., and Gibson, E. (2002). The influence of referential processing on sentence complexity. Cognition 85, 79–112. doi: 10.1016/S0010-0277(02)00087-2
Wendt, D., Koelewijn, T., Książek, P., Kramer, S. E., and Lunner, T. (2018). Toward a more comprehensive understanding of the impact of masker type and signal-to-noise ratio on the pupillary response while performing a speech-in-noise test. Hear. Res. 369, 67–78. doi: 10.1016/j.heares.2018.05.006
Westbrook, A., and Braver, T. S. (2015). Cognitive effort: a neuroeconomic approach. Cogn. Affect. Behav. Neurosci. 15, 395–415. doi: 10.3758/s13415-015-0334-y
Wickham, H. (2016). Data analysis. In ggplot2: elegant graphics for data analysis. Cham: Springer international publishing, pp. 189–201.
Wingfield, A., McCoy, S. L., Peelle, J. E., Tun, P. A., and Cox, L. C. (2006). Effects of adult aging and hearing loss on comprehension of rapid speech varying in syntactic complexity. J. Am. Acad. Audiol. 17, 487–497. doi: 10.3766/jaaa.17.7.4
Wingfield, A., Tun, P. A., Koh, C. K., and Rosen, M. J. (1999). Regaining lost time: adult aging and the effect of time restoration on recall of time-compressed speech. Psychol. Aging 14, 380–389. doi: 10.1037/0882-7974.14.3.380
Winn, M. B., Wendt, D., Koelewijn, T., and Kuchinsky, S. E. (2018). Best practices and advice for using pupillometry to measure listening effort: an introduction for those who want to get started. Trends Hear. 22, 1–32. doi: 10.1177/2331216518800869
Keywords: pupillometry, effort, time-compression, inverted-U, syntactic complexity
Citation: O’Leary RM, Zavlun N, Kinney A and Wingfield A (2025) The “inverted-U” of cognitive effort across speech rates holds for simple but not complex sentence structures. Front. Psychol. 16:1685938. doi: 10.3389/fpsyg.2025.1685938
Edited by:
Jerker Rönnberg, Linköping University, SwedenReviewed by:
Khaled H. A. Abdel-Latif, University Hospital of Cologne, GermanyNicholas Giuliani, United States Department of Veterans Affairs, United States
Copyright © 2025 O’Leary, Zavlun, Kinney and Wingfield. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ryan M. O’Leary, ci5vbGVhcnlAbm9ydGhlYXN0ZXJuLmVkdQ==
†Present address: Ryan M. O’Leary, Department of Communication Sciences and Disorders, Institute for Cognitive and Brain Health, Northeastern University, Boston, MA, United States