Yasunori Yamada1,2*
Kaoru Shinkawa1
Masatomo Kobayashi1,2
Miyuki Nemoto2
Miho Ota2
Kiyotaka Nemoto2
Tetsuaki Arai2- 1Digital Health, IBM Research, Tokyo, Japan
- 2Department of Psychiatry, Division of Clinical Medicine, Institute of Medicine, University of Tsukuba, Tsukuba, Ibaraki, Japan
Background: Alzheimer’s disease (AD) and Lewy body disease (LBD), the two most common causes of neurodegenerative dementia with similar clinical manifestations, both show impaired visual attention and altered eye movements. However, prior studies have used structured tasks or restricted stimuli, limiting the insights into how eye movements alter and differ between AD and LBD in daily life.
Objective: We aimed to comprehensively characterize eye movements of AD and LBD patients on naturalistic complex scenes with broad categories of objects, which would provide a context closer to real-world free viewing, and to identify disease-specific patterns of altered eye movements.
Methods: We collected spontaneous viewing behaviors to 200 naturalistic complex scenes from patients with AD or LBD at the prodromal or dementia stage, as well as matched control participants. We then investigated eye movement patterns using a computational visual attention model with high-level image features of object properties and semantic information.
Results: Compared with matched controls, we identified two disease-specific altered patterns of eye movements: diminished visual exploration, which differentially correlates with cognitive impairment in AD and with motor impairment in LBD; and reduced gaze allocation to objects, attributed to a weaker attention bias toward high-level image features in AD and attributed to a greater image-center bias in LBD.
Conclusion: Our findings may help differentiate AD and LBD patients and comprehend their real-world visual behaviors to mitigate the widespread impact of impaired visual attention on daily activities.
1 Introduction
Visual attention and eye movements involve a widely distributed network of brain regions (Corbetta and Shulman, 2002; Miller and Buschman, 2013), and altered eye movements can reflect cognitive and motor impairments in neurodegenerative disorders (Anderson and MacAskill, 2013; Leigh and Zee, 2015). Quantifying complex patterns of these alterations may help differentiate disorders as behavioral biomarkers (Anderson and MacAskill, 2013; Tseng et al., 2013; Itti, 2015) and enhance our comprehension of disease-associated alterations in patients’ real-world visual behaviors (Molitor et al., 2015; Ramzaoui et al., 2018). Alzheimer’s disease (AD) and dementia with Lewy bodies (DLB), one form of the Lewy body diseases (LBDs), are the two most common subtypes of late-onset neurodegenerative dementias (Walker et al., 2015; Crous-Bou et al., 2017; McKeith et al., 2017) and exhibit comparable clinical manifestations, posing challenges in their differential diagnosis (Walker et al., 2015; McKeith et al., 2017; Chatterjee et al., 2021). Patients with AD and DLB often exhibit impaired visual attention (Collerton et al., 2003; McGuinness et al., 2010; Revie et al., 2019) and altered eye movements (Anderson and MacAskill, 2013; Das et al., 2022). Studies on single-feature and feature-conjunction visual search tasks suggest impaired top-down attentional guidance in DLB compared with AD, with bottom-up attentional guidance preserved (Cormack et al., 2004; Landy et al., 2015; Ramzaoui et al., 2018). In particular, DLB patients tend to have larger alterations of eye movements compared with AD patients (Cormack et al., 2004; Mosimann et al., 2005), corroborated by greater impairments in attention-relevant neuropsychological examinations (Collerton et al., 2003; Wood et al., 2013; Revie et al., 2019) and prominent changes in neural networks related to visual attention (Taylor et al., 2013; Landy et al., 2015). However, all these prior studies have used artificial stimuli like simple arrays of geometrical shapes (e.g., dots and squares), which may not represent real-world viewing behaviors. Consequently, research utilizing naturalistic complex stimuli representing those that individuals may encounter in everyday life is increasingly gaining importance owing to its ecological validity (Wang et al., 2015; Ramzaoui et al., 2018).
Several studies on complex scene viewing in AD patients, but not DLB patients, have reported two remarkable alterations in top-down attentional guidance and visual exploration. Studies using simple stimuli or complex scenes (photographs of streets) have reported reduced task-appropriate modulations of eye movements (Redel et al., 2012; Shakespeare et al., 2015) and fewer fixations on task-related informative locations (Mosimann et al., 2004; Tokushige et al., 2023), suggesting impaired top-down attentional guidance in AD patients (Redel et al., 2012; Shakespeare et al., 2015; Ramzaoui et al., 2018). Other studies have investigated free viewing of complex scenes with an incongruous element (Daffner et al., 1992) or non-social scenes (environments without animate objects like humans or animals) (Coco et al., 2021), reporting diminished visual exploration with fewer fixated regions in AD or mild cognitive impairment (MCI), possibly related to apathy (a lack of motivation and interest) or cognitive impairments such as slowing processing speed and difficulties in disengaging and shifting attention (Daffner et al., 1992, 1999; Molitor et al., 2015; Seligman and Giovannetti, 2015). Although DLB patients have not been studied in regards to visual exploration patterns, such patients might show a similar reduction in visual exploration since apathy and cognitive impairments, particularly in attention, have also been observed in DLB patients (Collerton et al., 2003; McGuinness et al., 2010; Bjoerke-Bertheussen et al., 2012; Revie et al., 2019). However, the prior studies still used restricted stimulus sets in terms of their numbers (typically less than ten) and categories (only non-social scenes). Therefore, a systematic characterization of altered eye movements on complex scenes containing broader categories of objects that could influence attention in AD and DLB patients has yet to be determined. Furthermore, although top-down attentional guidance can be influenced by both task demands and high-level image features such as faces, texts, and emotions even during free viewing (Mannan et al., 2009; Borji et al., 2012, 2013; Wang et al., 2015), the extent to which attention bias toward such high-level image features differs in AD or DLB patients remains uninvestigated.
Here, we aimed to assess spontaneous gaze allocation in patients with AD and LBD at the MCI and dementia stages to naturalistic complex scenes with broad categories of objects, assuming that it would further reflect real-world viewing behaviors. The stimulus set consisted of 200 different scene images labeled with high-level image features of object-based semantic attributes, such as object properties (e.g., shape) and semantic information (e.g., faces), annotated on a total of 1,533 manually-segmented objects. This enabled us to investigate the differences in attention bias toward these high-level features using a data-driven approach with a computational visual attention model (Xu et al., 2014; Wang et al., 2015). By comparing with matched controls, we tested the following hypotheses: (i) both AD and LBD patients would have diminished visual exploration and weaker attention bias to high-level image features, and (ii) these reductions would be larger in LBD than in AD. Furthermore, we aimed to explore the associations of these alterations in eye movements with disease severity and brain structural changes, as well as the feasibility of using gaze allocation patterns to differentiate AD and LBD.
2 Materials and methods
2.1 Study participants
We recruited community-dwelling older adults in Ibaraki, Japan, using consecutive sampling. The participants comprised outpatients from the Department of Psychiatry at the University of Tsukuba Hospital, the spouses of the patients, and other participants recruited via local agencies or community advertisements. All participants underwent cognitive, physical, and clinical assessments as listed in Table 1.
Table 1. Participant characteristics.
The participants were categorized into three clinical diagnostic groups: AD, LBD, and cognitively unimpaired (CU). Participants in the AD group met the National Institute on Aging and Alzheimer’s Association core clinical criteria for probable AD dementia (McKhann et al., 2011) or MCI (Albert et al., 2011), as well as the AD Neuroimaging Initiative criteria for AD or MCI (Petersen et al., 2010). Those in the LBD group fulfilled McKeith et al.’s 2017, 2020 clinical diagnostic criteria for probable/possible DLB or MCI with Lewy bodies. Consequently, patients in the AD and LBD groups included those with MCI (MCI-AD and MCI-LB, respectively), and ranged from MCI to moderate dementia (Morris, 1993). The CU participants were matched for age, sex, and years of education, and did not meet any of the aforementioned criteria. Participants were excluded if they had diagnoses of other types of dementia (e.g., frontotemporal dementia, vascular dementia, and Parkinson’s disease dementia) or mental illness (e.g., major depression, bipolar disorder, and schizophrenia) at the time of the experiment, evidence of stroke, or other serious diseases or disabilities that would interfere with the eye movement data collection. All diagnoses were confirmed by three psychiatrists (authors TA, KN, and MO), who are dementia specialists and were blind to the results of the eye movement analysis, in terms of cognitive and clinical measures as well as clinical records.
A power analysis was conducted to determine the appropriate sample size for comparing the three groups using Cohen’s f effect size measure. On the basis of a previous study (Mosimann et al., 2005) that analyzed eye movement characteristics for AD, DLB, and controls and yielded an f of 0.52 to 1.00, we used f = 0.5 as the target effect size for between-group differences. With a significance level of 0.05 and statistical power of 0.8, the sample size of 14 for each group was considered to be sufficient to detect meaningful differences between the groups.
The study protocol was approved by the Ethics Committee, University of Tsukuba Hospital (H29-065), and was in accordance with the Declaration of Helsinki. All participants gave written informed consent to participate in the study.
2.2 Assessments
Cognitive assessments included the Mini-Mental State Examination (MMSE), immediate and delayed recall of the logical memory–story A of the Wechsler Memory Scale–Revised, the Frontal Assessment Battery, Trail Making Test part A and part B, and the Clock Drawing Test, all of which were conducted by experienced neuropsychologists. Clinical assessments included the Clinical Dementia Rating and Geriatric Depression Scale. Patients in the LBD group were also administered the Unified Parkinson’s Disease Rating Scale (UPDRS). For the UPDRS, we used the five-item version (Thomas et al., 2017; Thomas, 2022), which has been validated for the assessment of parkinsonism in dementia (Ballard et al., 1997) and MCI (Fernando et al., 2024), and recommended for use due to its robustness against the severity of cognitive impairment (Ballard et al., 1997; Thomas et al., 2017). The physical assessment was a nine-meter walk test, where the participants were instructed to walk nine meters at their usual pace in an unobstructed space. Gait speed was measured using an optical motion capture system (OptiTrack Flex13; NaturalPoint, Inc., Corvallis, OR, USA sample rate of 120 Hz), and the first and last two meters were excluded from analyses to discard the increase and decrease speed effect. Detailed equipment and procedural information were previously outlined (Yamada et al., 2021). We used the average gait speed across three trials as the physical assessment score for each participant.
To assess structural brain changes, we used a 1.5-T Avanto scanner (Siemens Medical System, Inc., Erlangen, Germany) with a three-dimensional magnetization-prepared rapid gradient-echo sequence yielding 160 contiguous T1-weighted slices with a 1.2-mm thickness in the sagittal plane. The imaging parameters were repetition time = 2,400 ms, echo time = 3.52 ms, flip angle = 8°, field of view = 240 mm, and matrix size = 192 × 192 pixels. The voxel size was 1.25 × 1.25 × 1.25 mm3. Regions of interest (ROIs) were selected on the basis of previous studies on AD (Dickerson et al., 2009; Schwarz et al., 2016) since structural imaging studies on DLB yield mixed results but report similar patterns of cortical thinning to AD (Watson et al., 2015; Ye et al., 2020). Specifically, we analyzed the following ten brain ROIs: hippocampus volume and cortical thickness of the middle temporal, inferior temporal, inferior parietal, superior parietal, supramarginal, superior frontal, entorhinal, fusiform, and precuneus regions. The magnetic resonance images were reconstructed using FreeSurfer software version 7.3.2 (Fischl, 2012) running on Ubuntu 20.04 based Lin4Neuro (Nemoto et al., 2011). The surface-based pipeline consisted of several stages, including removal of non-brain tissue, segmentation of the subcortical white matter and deep gray matter volumetric structures, intensity normalization, tessellation of the gray matter-white matter boundary, surface deformation following intensity gradients to optimally place the gray/white and gray/cerebrospinal fluid borders at the location, registration to a spherical atlas, and parcellation of the cerebral cortex into units on the basis of gyral and sulcal structures. The hippocampus volume was calculated using the Aseg atlas (Fischl et al., 2002) and normalized to intracranial volume, while the vertex-wise mean cortical thickness values were calculated for each cortical ROI using the Desikan-Killiany-Tourville atlas (Klein and Tourville, 2012).
2.3 Eye-tracking data collection
The participants were presented with a total of 200 naturalistic complex scene images for three seconds each in two sessions (100 images for each session; see Figure 1 and Supplementary Figure 1 for examples). The stimulus duration (i.e., three seconds) was determined on the basis of previous studies investigating how neurological disorders affect eye movements and visual saliency in free-viewing conditions (Tseng et al., 2013; Wang et al., 2015). Each session was thus five minutes (three seconds × 100 images), and participants rested between the sessions. The participants were instructed to simply look at the images throughout a session. The images were randomly selected from the publicly-available Object and Semantic Images and Eye-tracking dataset (Xu et al., 2014). Briefly, the dataset comprises 700 different scenes representing a range of social and non-social situations typical of daily life. Each scene includes multiple dominant objects rather than a central one (median number of objects was 7 for the images we used; for the distribution, see Supplementary Figure 2). Each image was manually segmented into a collection of objects and was quantified in accordance with both low-level image features of pixel-level attributes (color, intensity, and orientation) and high-level image features of object-based semantic attributes that represent object properties (complexity, convexity, solidity, and eccentricity) and semantic information (face, emotion, touched, gazed, motion, sound, smell, taste, touch, text, watchability, and operability) annotated on each segmented object. Such hand-labeled stimuli have demonstrated advantages in investigating the relative contribution of multiple feature types to visual attention (Xu et al., 2014; Wang et al., 2015). The images were shown on a 20-inch computer screen (resolution = 1600 × 1200 pixels), resulting in 28.5° × 21.6° visual angles as participants were seated ∼80 cm away from the screen.
Figure 1. Examples of complex scene images with gaze densities derived from the CU, AD, and LBD groups. Results show that AD and LBD patients focused on fewer locations (A–D), had different attention biases toward high-level semantic features (E–H), and looked more at the image center (I–L), compared with CU participants. The images of visual stimuli were taken from the Object and Semantic Images and Eye-tracking dataset (Xu et al., 2014).
Eye-tracking data were recorded using an EMR ACTUS infrared eye tracker (nac Image Technology Inc., Tokyo, Japan; sampling rate = 60 Hz; spatial resolution for eye movements < 0.5°). Following previous studies (Kennedy et al., 2017; Yamada and Kobayashi, 2018; Wang, 2019), binocular gaze trajectories were analyzed in raw form with minimal preprocessing of eye-tracking data. Specifically, the eye tracker outputs binocular gaze locations when both eyes were confidently identified, i.e., data were not recorded when either eye was undetectable, for example, due to blinking. Among these data, we excluded samples falling outside the image boundaries and where eye tracker estimation failed. We then included all remaining data of gaze locations in raw form for subsequent analyses. We excluded images from the analysis when gaze data were missing for more than 10% of the total viewing duration for the image. At the beginning of each recording session, we calibrated the eye tracker using a nine-point calibration and validation procedure, repeated up to three times if necessary.
2.4 Eye movement analysis
Characteristics of eye movements were quantified and compared through three types of measures: similarities in gaze allocations between participants, degree of visual exploration, and attention biases to specific image features. These eye movement features were calculated using the aforementioned binocular gaze location data. Mixed-model repeated-measures analyses of variance (ANOVAs) with Bonferroni post-hoc tests were performed to compare the effect of the diagnostic group on eye movement measurements, with the image as a random factor. We then further investigated the results through exploratory statistical and machine-learning analyses.
Similarities in spatial gaze allocations were assessed by correlating two gaze density maps (Kennedy et al., 2017). These maps were constructed by smoothing gaze locations using a fovea-sized 2D Gaussian kernel (i.e., 55 pixels in the 1600 × 1200 image) and were then normalized within each image. These gaze density maps represent the amount of gaze data looking at a particular location of the image. Each gaze density map was then flattened into a one-dimensional form, and the similarity between two gaze density maps was calculated using the Pearson correlation, which is thought to align closely with human intuition, despite its simplicity (Kennedy et al., 2017; Wang, 2019). This similarity metric was then used to assess whether each patient group had different gaze allocation compared with controls (CU participants). Specifically, the gaze similarity to controls was calculated for each participant and each image between the two groups, and then averaged for each of the three diagnostic groups. For the control group, we calculated the gaze similarity between all possible pairs within the control participants.
The degree of visual exploration was measured by the Shannon entropy of the gaze density map (Xu et al., 2014; Kennedy et al., 2017; Mikhailova et al., 2021). A higher entropy indicates a spatially diffused gaze to many different locations, while a lower entropy indicates a tightly focused gaze to only certain locations of the image. The Shannon entropy is defined as ∑i(−pilog2pi), where piindicates i-th pixel values of the gaze density map after dividing by the sum.
Attention bias to specific image features was evaluated using a computational visual attention model, namely a saliency model, to estimate the relative contribution of the following four feature types to gaze allocations (Xu et al., 2014; Wang et al., 2015): (i) image center, (ii) background (i.e., regions without labeled objects), (iii) low-level features (three pixel-level attributes), and (iv) high-level features (17 object-based semantic attributes). See Figure 2A for a schematic overview and the aforementioned subsection for the full list of attributes. The model has been described in detail in previous works (Xu et al., 2014; Wang et al., 2015). In summary, a binary classifier was trained to determine whether each location was looked at by linearly combining saliency values of each feature map extracted from the images. Feature weights resulting from the trained classifier were then used as the relative contribution of each feature type in predicting gaze allocation. The classifiers were trained for each diagnostic group, and the saliency weights between the groups were compared as group-level effects (Xu et al., 2014; Wang et al., 2015). For the ground-truth data (i.e., whether each location was looked at), positive samples were randomly selected from the pixels with the top 10% values in the gaze density maps, whereas negative samples were randomly selected from those with the lowest 30% values. The pixel-level feature maps were extracted using the most standard saliency model proposed by Itti et al. (1998), and the object- and semantic-level feature maps were calculated by placing a two-dimensional Gaussian kernel (σ = 2°) at each object’s center. The image-center map was calculated as a Gaussian map (σ = 1°) around the image center, and the background map was modeled as a binary map indicating regions without any labeled objects in the image. Before training the model, all samples extracted from each feature map were normalized to have a zero mean and unit variance. For the binary classifier, we used a support vector machine (SVM) classifier with a linear kernel (Boser et al., 1992) implemented with the scikit-learn package (version 0.23.2) in Python, leaving all parameters to their default values. All these parameters for the saliency-model-based analysis were the same as those in the previous study (Wang et al., 2015).
Figure 2. Altered patterns of spatial gaze allocations in the AD and LBD groups. (A) Overview of analysis using computational visual attention model. The images of visual stimuli were taken from the Object and Semantic Images and Eye-tracking dataset (Xu et al., 2014). (B) Similarity of gaze density maps with CU (one-way repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001). (C) Shannon entropy of gaze density maps (one-way repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001). (D) Shannon entropy of gaze density maps for images with fewer or more objects (two-way mixed-design ANOVA with Bonferroni post-hoc test, interaction: P = 0.021). (E) Relative contributions of four feature types to gaze allocations with 95% confidence intervals, derived from the analysis using the computational visual attention model (two-way repeated-measures ANOVA with Bonferroni post-hoc test, interaction: P < 0.001). Asterisks indicate significant differences between diagnostic groups: *P < 0.05, **P < 0.01, ***P < 0.001.
As an exploratory analysis regarding clinical-instrumental correlation, we investigated the associations of gaze allocation features with disease severity as well as with brain structural changes in AD and LBD patients. Disease severity measures included the Clinical Dementia Rating–Sum of Boxes (CDR-SB) score for dementia severity, MMSE score for assessing cognitive impairment, and gait speed for measuring motor impairments. Correlation and multiple linear regression analyses were used for disease severity and brain structural changes, respectively, after adjusting for age, sex, and years of education. For the multiple regression analysis, we applied a forward stepwise variable selection procedure based on the Bayesian information criterion to reduce the model complexity.
We then built machine learning-based classifiers to discriminate the three diagnostic groups on the basis of quantitative features characterizing gaze allocations. The features for the inputs were calculated for each participant and consisted of four feature types: the weights from the saliency model analysis; Shannon entropy; and proportion of gaze density and mean duration of gaze allocations on scene ROIs. To investigate whether the eye movement features could be used to differentiate AD and LBD patients at the individual level, the saliency models were trained for each participant, and four saliency weights for the image center, low- and high-level image features, and background were used. The Shannon entropy was calculated for each image, and its mean value across the images was used. The last two features related to the proportion of gaze density and mean duration of gaze allocations were calculated for each of the 15 scene ROIs consisting of objects with twelve types of semantic information; semantic objects (i.e., objects labeled with at least one type of semantic information); other objects (i.e., objects without any labeled semantic information); and background. Then, we used their means and standard deviations. For the classification algorithm, we used the same SVM classifier with a linear kernel and the default parameters. The model performance was evaluated with the area under the receiver operating characteristic curve (AUC), through a stratified 20-fold cross-validation procedure with 20 iterations with different training and test partitions. Three-class classifiers were built with a one-vs-one scheme and their AUC was computed as defined by Hand and Till (2001). To reduce model complexity and overfitting, feature selection was applied by using the least absolute shrinkage and selection operator penalized logistic regression model before the model training through the same 20-fold cross-validation procedure. Thus, the dataset was split into training (19/20) and test (1/20) partitions, where each training partition was used for the feature selection and model training. In total, 65 features of eye movements were used as inputs for the feature selection procedure, and selected features were fed into each classifier. The statistical significance of the classification performance was estimated by permutation testing. Specifically, we performed the aforementioned cross-validation procedure 100 times with randomly-permuted diagnosis labels to obtain a null distribution. We also compared performance against a baseline model incorporating clinical and demographic variables (i.e., the use of antipsychotic medication, MMSE score, age, sex, and years of education).
3 Results
3.1 Sample characteristics
Among the 142 participants who met the inclusion criteria, 36 participants were excluded from the present study (15, 17, and 4 participants from the CU, AD, and LBD groups, respectively) due to a calibration failure related to eye diseases (e.g., cataract) and failure to detect pupils caused by age-related reduction of pupil diameter and lax/redundant upper eyelid skin. Table 1 summarizes the characteristics of the participants included in the analysis (N = 106). Supplementary Tables 1–3 provide additional information related to clinical assessments and structural brain changes. The sample size of our dataset was comparable with those used in previous studies (Tseng et al., 2013; Wang et al., 2015).
The CU, AD, and LBD groups comprised 37, 49, and 20 participants, respectively, which satisfied the requirement of the power analysis. The AD and LBD groups included 31 and 13 MCI patients, respectively, and their proportions did not show any statistical difference (P = 0.892). For demographics, neither the age, sex, nor years of education were statistically different among the three diagnostic groups (all P > 0.05). None of the CU participants received antipsychotic medication, and the proportion of those on antipsychotic medication was the highest in the LBD group among the diagnostic groups (P = 0.001). All clinical, cognitive, and physical measures differed statistically among the groups except the Geriatric Depression Scale and Clock Drawing Test (see Table 1 for details).
3.2 Altered gaze allocations in AD and LBD patients
First, we directly investigated spatial gaze allocations by using gaze density maps aggregated for each diagnostic group. As qualitatively illustrated in Figure 1 (with additional examples in Supplementary Figure 1), compared with CU, AD and LBD patients focused on fewer locations (Figures 1A–D), exhibited different attention biases toward high-level semantic features (Figures 1E–H), and looked more at the image center (Figures 1I–L).
We then quantified the degree to which gaze allocations in AD and LBD patients differed from those in CU participants by calculating the similarity of the gaze density maps for each image. Consequently, the similarity in gaze allocations between CU participants and patients with AD or LBD was lower than that within CU participants (Figure 2B; repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001; AD vs. CU: P < 0.001, Hedges’ g = −0.43; LBD vs. CU: P < 0.001, Hedges’ g = −0.74), and the similarity was lower in LBD than in AD (P < 0.001, Hedges’ g = −0.34), indicating that AD and LBD patients exhibited different patterns of gaze allocations compared with CU participants, with the differences being more pronounced in LBD than in AD. These tendencies were consistently observed even at shorter timescales (Supplementary Figure 3A).
3.3 Diminished visual exploration
We investigated the degree of visual exploration by comparing the entropy of the gaze density maps. As hypothesized, LBD patients exhibited the lowest degree of entropy, followed by AD patients, and finally by CU participants (Figure 2C; repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001; LBD vs. AD: P < 0.001, Hedges’ g = −1.35; AD vs. CU: P < 0.001, Hedges’ g = −0.87). The result indicates that LBD and AD patients tended to focus on fewer locations in the scene and had a diminished visual exploration tendency compared with CU participants.
We further examined the associations between the degree of visual exploration and the characteristics of stimulus content, i.e., the number of objects. We first split the images into two groups with more and fewer objects by using their median value and then performed a two-way mixed-design ANOVA with a Bonferroni post-hoc test with the image group (more objects vs. fewer objects) as the between-subject factor and the diagnostic group as the within-subject factor. The result showed that AD patients and CU participants exhibited a greater degree of visual exploration in images with more objects (Figure 2D; interaction of diagnostic group × number of objects: P = 0.021; CU: P < 0.001, Hedges’ g = 0.61; AD: P = 0.010, Hedges’ g = 0.42), but LBD patients showed no statistical difference between images with more and fewer objects (P = 0.107, Hedges’ g = 0.23). This result suggests larger differences in the degree of visual exploration between the diagnostic groups with more objects in the scene. Note that the between-group differences measured by partial eta-squared increased with the number of objects in the scene (Supplementary Figure 3B; Pearson’s correlation: r = 0.17, P = 0.015), showing that the difference in the degree of visual exploration across the LBD, AD, and CU groups tended to be larger as the number of objects increased, partially because the degree of visual exploration in the LBD group was less affected by the number of objects than in the AD and CU groups.
We also examined whether the difference in visual exploration between the diagnostic groups could be related to that in participant fatigue or alertness. To this end, we examined changes in pupil diameter, a representative measure indicating increased fatigue (Yamada and Kobayashi, 2018), within the eye-tracking session and compared their differences between the diagnostic groups. Specifically, we split the images into the initial and latter halves of the session and then compared changes in pupil diameters using mixed-model repeated-measures ANOVA with the group (AD, LBD, and CU) and time (initial and latter halves) as independent factors. The result showed that neither the main effect of group (P = 0.916) nor the group × time interaction (P = 0.590) was statistically significant, suggesting that the diminished visual exploration in LBD and AD patients could not be attributed to the difference in participant fatigue or alertness.
3.4 Weaker attention bias to high-level image features
We next applied a computational visual attention model to evaluate the relative contribution of the four different factors in gaze allocation. Consequently, LBD patients, followed by AD patients, had a significantly weaker attention bias toward high-level image features, as well as greater image-center and background biases, whereas there was no statistical difference in the attention bias toward low-level image features of the pixel-level attributes (Figure 2E; two-way repeated-measures ANOVA with Bonferroni post-hoc test, interaction of diagnostic group × feature type: P < 0.001).
We conducted two additional analyses to further confirm the aforementioned results regarding reduced attention bias to high-level image features annotated on objects and increased center bias. First, we directly assessed gaze allocations to the objects in the scene. The results consistently aligned with those derived from the computational attention model analysis: AD and LBD patients showed a lower proportion of gaze allocations to the objects and a higher proportion to the background compared with CU participants (Supplementary Figure 4A; one-way repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001; AD vs. CU: P < 0.001, Hedges’ g = −0.16; LBD vs. CU: P < 0.001, Hedges’ g = −0.22). Additionally, there was an increase in the mean durations of gaze allocation to each object (Supplementary Figure 5A; P < 0.001; AD vs. CU: P = 0.045, Hedges’ g = 0.08; LBD vs. CU: P < 0.001, Hedges’ g = 0.43). In particular, the proportions of gaze allocations to text and human-made objects designed to be watched (i.e., watchability-related attributes e.g., a picture, display screen, and traffic sign) were lower in both AD and LBD patients (text: P < 0.001; AD vs. CU: P = 0.007, Hedges’ g = −0.12; LBD vs. CU: P < 0.001, Hedges’ g = −0.23; watchability: P < 0.001; AD vs. CU: P = 0.010, Hedges’ g = −0.07; LBD vs. CU: P < 0.001, Hedges’ g = −0.15; see Supplementary Figure 5B for full results). Second, we examined whether the tendency of AD and LBD patients to look at the image center could be explained by the distribution of objects in the images. Specifically, we investigated whether this center bias could still be observed in images with no objects in the center 2°circular area [the parameter was determined on the basis of the literature (Wang et al., 2015)]. Comparing the gaze density across the 34 images with no objects in the center, we found that the gaze density in the center was 1.3 times higher in the AD group and 2.6 times higher in the LBD group compared with the CU group (Supplementary Figure 4B; one-way repeated-measures ANOVA with Bonferroni post-hoc test: P < 0.001; AD vs. CU: P = 0.020, Hedges’ g = 0.38; LBD vs. CU: P < 0.001, Hedges’ g = 1.39).
Taken together, the results suggest that AD and LBD patients looked less at objects, along with a weaker attention bias toward high-level image features; while they looked more at the image center, even when there was no object present at the center.
As hypothesized, we confirmed that the LBD group, followed by the AD group, tended to have diminished visual exploration and a weaker attention bias toward high-level image features of object properties and semantic information. These tendencies persisted when the AD and LBD groups were subdivided in accordance with disease stage (i.e., MCI and dementia stages), with dementia stage-wise reductions in each patient group (Supplementary Figures 3C–E).
3.5 Associations of gaze allocation features with disease severity and brain structural changes
For the exploratory analyses regarding clinical-instrumental correlation, we further investigated the associations of altered patterns of visual exploration and attention biases with measures related to disease severity. These measures included the CDR-SB score for dementia severity, MMSE score for cognitive impairments, and gait speed for motor impairments. After adjusting for the covariates, correlation analyses revealed different associations within the AD and LBD groups.
First, the degrees of diminished visual exploration in the AD group correlated with lower MMSE scores (Spearman’s correlation: ρ = 0.34, P = 0.022), while those in the LBD group correlated mainly with slower gait speed (ρ = 0.70, P = 0.025). Note that neither MMSE scores in the LBD group nor gait speed in the AD group showed any statistical correlation with the degree of visual exploration (see Supplementary Table 4 for complete results). Second, the attention bias metrics derived from the computational attention model analysis also exhibited different association patterns between the diagnostic groups. In the AD group, a weaker attention bias toward high-level features correlated with CDR-SB, MMSE, and gait speed (CDR-SB: ρ = −0.33, P = 0.026; MMSE: ρ = 0.44, P = 0.002; gait speed: ρ = 0.33, P = 0.032), while the degree of center bias did not statistically correlate with any of the three measures investigated in this study (P > 0.05). In contrast, in the LBD group, the degree of center bias correlated with CDR-SB and MMSE scores (CDR-SB: ρ = 0.50, P = 0.042; MMSE: ρ = −0.53, P = 0.029), while the degree of attention bias toward high-level features showed no statistical correlations with any of the measures (P > 0.05). The correlation between the weaker attention bias and MMSE remained statistically significant even after correcting for multiple testing (Benjamini-Hochberg-corrected P = 0.036).
The different associations between eye movement patterns and disease severity measures between the AD and LBD groups suggest that these disease-specific alterations of eye movements may reflect distinct neuropathological changes in AD and LBD. Therefore, we conducted additional analyses on the associations with structural brain atrophies using stepwise multiple linear regression controlling for the covariates.
Consequently, in the AD group, the diminished visual exploration was associated with a reduced hippocampus volume (β = 2.3 × 102, P = 0.037; Supplementary Table 5), and the reduced attention bias toward high-level features was associated with a thinning of the superior frontal cortex (β = 7.7 × 10–1, P = 0.005; Supplementary Table 5). In contrast, in the LBD group, neither the degree of visual exploration nor attention bias to high-level features was associated with structural changes in any of the ten brain ROIs, but the increased center bias was associated with the thinning of the superior parietal lobule (β = −1.1, P = 0.007; Supplementary Table 6).
These associations should be considered exploratory because of the small sample sizes and further validation is needed. However, these results suggest the following. (i) Diminished visual exploration may reflect cognitive impairment in AD patients and motor impairment in LBD patients, respectively. (ii) For the atypical bias in gaze allocations, the reduction in attention bias toward high-level image features may reflect disease severity in AD, while increased center bias may reflect severity in LBD. (iii) These alterations of eye movements in AD and LBD may reflect different patterns of neuropathological changes in each disease.
3.6 Classifier using eye movement features to identify/differentiate AD and LBD
Finally, we explored the extent to which these altered patterns of eye movements could differentiate AD and LBD patients. Through the cross-validation procedure, the models using features derived from eye movements achieved an AUC of 0.76 (95% confidence interval [CI]: 0.75 to 0.77, P < 0.01) for AD versus CU; an AUC of 0.87 (95% CI: 0.87 to 0.88, P < 0.01) for LBD versus CU; and an AUC of 0.82 (95% CI: 0.81 to 0.82, P < 0.01) for AD versus LBD (Figure 3; Supplementary Table 7 for full results). From the results of feature selection, each classification model mainly used fourteen eye movement features for AD versus CU; four features for LBD versus CU; and twelve features for AD versus LBD (Supplementary Table 8 for full results). The three-class AUC was 0.82 (95% CI: 0.81 to 0.82), which was much higher than the value of 0.59 (95% CI: 0.57 to 0.61) for the baseline model using clinical and demographic variables. When the performance was calculated for MCI stages, the models held reasonable performance, particularly for MCI-LB patients, with an AUC of 0.74 (95% CI: 0.73 to 0.75, P < 0.01) for MCI-AD versus CU; an AUC of 0.83 (95% CI: 0.82 to 0.83, P < 0.01) for MCI-LB versus CU; and an AUC of 0.75 (95% CI: 0.74 to 0.76, P < 0.01) for MCI-AD versus MCI-LB [3-class AUC of 0.77 (95% CI: 0.77 to 0.78), which was higher than the baseline model’s AUC of 0.56 (95% CI: 0.54 to 0.57)].
Figure 3. Receiver operating characteristic plot for the binary classifiers using eye movement features. Shades represent standard deviations.
4 Discussion
The current study demonstrates two remarkable disease-specific altered patterns of gaze allocation to naturalistic complex scenes during free viewing in AD and LBD patients. First, both AD and LBD patients showed diminished visual exploration and tended to focus on fewer locations rather than exploring the entire image. AD patients tended to increase visual exploration in scenes with more objects in a similar way to CU participants, but the degree itself seems to be lower than CU regardless of the number of objects. In contrast, the degree of visual exploration in LBD patients was less affected by the number of objects in the scenes and held the lowest degree compared with CU participants and AD patients regardless of the number of objects. Furthermore, the degree of diminished visual exploration was differentially correlated with cognitive impairment in AD patients and motor impairments in LBD patients, respectively. Second, both AD and LBD patients exhibited a weaker attention bias toward high-level semantic features, looking less at objects, particularly texts and human-made objects designed to be watched. They also showed a stronger center bias even when there was no object at the center. These altered patterns of attention biases also showed different associations with dementia severity measures between the AD and LBD groups. Specifically, a reduction in the attention bias to high-level features was associated with dementia severity in the AD group, while an increase in the center bias was associated with dementia severity in the LBD group. Finally, these two distinct altered patterns in gaze allocations were consistently larger in LBD than in AD, enabling differentiation of the three diagnostic groups with an AUC of 0.82.
Our findings regarding diminished visual exploration are consistent with prior studies on AD patients (Daffner et al., 1992; Coco et al., 2021) and provide further support using a larger stimulus set that includes broader categories of objects. Additionally, we provide new insights into disease-specific associations between the degree of visual exploration and the number of objects in the scene, as well as cognitive/motor measures. According to the literature, diminished visual exploration could be attributed to at least three factors (Daffner et al., 1992, 1999; Anderson and MacAskill, 2013; Molitor et al., 2015; Seligman and Giovannetti, 2015): cognitive impairments (e.g., slowed processing speed and difficulties in disengaging and shifting attention), oculomotor dysfunctions (e.g., slowed velocity and increased latency of saccades), and apathy (e.g., a lack of motivation and interest). In the present study, the reduction of visual exploration in AD patients correlated with cognitive impairments and atrophy in the hippocampus, one of the regions associated with apathy (Johansson et al., 2020; Matuskova et al., 2021; Eggins et al., 2022), while that in LBD patients correlated with motor impairments. Our results indicate that the primary factors causing a reduction in visual exploration may differ between AD and LBD. Future studies that assess neuropsychiatric symptoms and oculomotor functions will be worthwhile to further elucidate this hypothesis.
Our results, derived from the computational attention model, demonstrate no statistical difference between diagnostic groups in the attention bias toward low-level features such as color and intensity. This suggests preserved bottom-up attentional guidance, which is consistent with prior studies (Landy et al., 2015; Shakespeare et al., 2015; Ramzaoui et al., 2018). Furthermore, our results newly show reduced attention bias toward high-level features of object-based semantic attributes, the extent of which correlates with dementia severity and the thinning of the superior frontal cortex in the AD group. Previous studies on AD patients reported impaired feature-conjunction search (Cormack et al., 2004; Landy et al., 2015) and reduced modulations of gaze allocations on the basis of task demands (Mosimann et al., 2004; Redel et al., 2012; Shakespeare et al., 2015; Tokushige et al., 2023), and they suggest impaired top-down attentional guidance in AD patients (Molitor et al., 2015; Ramzaoui et al., 2018). Moreover, high-level features in the scenes may affect top-down attentional control, along with numerous other factors including context and task-related factors (Mannan et al., 2009; Borji et al., 2012, 2013; Wang et al., 2015). The superior frontal cortex has also been implicated in top-down attentional control (Hopfinger et al., 2000; Baluch and Itti, 2011). Therefore, in alignment with previous studies, our findings may suggest impaired top-down attentional control in AD patients. Additionally, because high-level features related to object properties and semantic information play important roles in gaze allocations during specific tasks such as object search and memorization (Xu et al., 2014; Ramzaoui et al., 2018), our findings about the reduction in attention bias toward high-level features may provide another key difference for understanding deficits in activities of daily living observed in AD and LBD patients.
We observed that LBD patients looked more at the image center, even when there was no object, and the extent of this center bias was related to dementia severity. Although we believe no study in DLB exists, increased center bias has been reported in diseases with similar pathological changes (i.e., alpha-synucleinopathies) including Parkinson’s disease and multiple-system atrophy (Habibi et al., 2022). Furthermore, increased center bias has also been reported in multiple different neurological diseases such as autism spectrum disorder (Wang et al., 2015) and schizophrenia (Silberg et al., 2019), which we show in AD patients (although the extent was smaller than LBD). Since multiple, different neuropathological changes may affect the tendency to look at the image center, systematic comparisons across diseases will help further elucidate this complex association with neurological diseases.
In addition to greater alterations in LBD compared with AD, aligning with prior studies on eye movements using simple arrays (Cormack et al., 2004; Mosimann et al., 2005), we provide initial evidence for disease-specific alterations of eye movements to naturalistic complex scenes, suggesting that these altered patterns may reflect pathological changes of AD and LBD. This notion was corroborated by our results, including (i) the dementia stage-wise reductions in characteristics of altered gaze allocation patterns investigated in this study; and (ii) different associations of the altered eye movement patterns with structural brain atrophies in AD and LBD. Notably, with extensive knowledge of brain regions that control attention and gaze (Corbetta and Shulman, 2002; Anderson and MacAskill, 2013; Miller and Buschman, 2013; Leigh and Zee, 2015), altered patterns of eye movements could potentially identify particular brain regions affected by neurological disorders (Anderson and MacAskill, 2013; Itti, 2015; Terao et al., 2017). Compared with eye movements under structured tasks with simple artificial stimuli, spontaneous, voluntary eye movements can involve a wider range of brain regions, including higher cortical areas such as the temporal cortex, which is involved in processing high-level stimulus structures such as objects or gist, as well as the frontal cortex, which is involved in top-down attentional guidance (Miller and Buschman, 2013; Itti, 2015). From this perspective, our approach to multifacetedly characterize spontaneous gaze allocations to naturalistic complex scenes may provide an understanding of how changes in spontaneous gaze allocations relate to the pathophysiology of neurodegenerative dementias.
Our findings have at least two important clinical implications. First, eye movements not only reflect bottom-up and top-down attentional controls but also provide scaffolding for subsequent cognitive processing and influence activities in daily life. For example, eye movement patterns play an important functional role in a wide variety of daily activities, including making meals (Land et al., 1999; Hayhoe et al., 2003), reading the clock (Mosimann et al., 2004), understanding spoken language (Tanenhaus et al., 1995), identifying objects (Seligman and Giovannetti, 2015; Ramzaoui et al., 2018), and memorizing scenes (Coco et al., 2021) and faces (Henderson et al., 2005). In particular, recall performance for scenes and faces has been suggested to be influenced by the eye movement patterns during memory encoding (Olsen et al., 2016; Damiano and Walther, 2019; Coco et al., 2021), which include those reported as the altered patterns in AD and LBD patients in this study (i.e., diminished visual exploration, reduced fixations to objects, and increased center bias). Our findings of altered patterns of eye movements to complex, real-world scenes could offer a critical basis for understanding and mitigating the widespread impact of altered visual attention on activities in daily living. Examples may include studies on environmental modifications for enhancing attentional processing and improving daily-task performance, such as the provision of visually salient cues in object identification and wayfinding (Davis et al., 2017; Ramzaoui et al., 2022). Second, quantifying complex patterns of altered eye movements can support screening and differentiation for dementia subtypes, as has already been demonstrated for several other disorders (Anderson and MacAskill, 2013; Tseng et al., 2013). Note that behavioral measures collected using digital health technologies have become increasingly used in clinical trials of neurological disorders (Masanneck et al., 2023) and are expected to be used as non-invasive, relatively inexpensive, and easily measurable behavioral biomarkers (Kourtis et al., 2019; Schütz et al., 2022; Chen et al., 2023). In this context, along with other behavioral data such as speech, gait, and drawing (Mc Ardle et al., 2019; Pieruccini-Faria et al., 2021; Yamada et al., 2021, 2022a,b), eye-tracking data a prospective example (Anderson and MacAskill, 2013; Tseng et al., 2013; Itti, 2015; Kourtis et al., 2019; Valliappan et al., 2020). Additionally, compared with studies on eye movements under particular tasks (e.g., visual search, memory), the free-viewing paradigm used in this study has a methodological advantage in probing a wider range of individuals, including those with different languages or cognitive impairments that may affect the understanding of task instructions. Thus, it may provide a promising tool for a global approach to dementia research involving diverse populations (Gauthier et al., 2021; Dementia, 2023).
This study has several limitations. First, our dataset did not include neuropathological information regarding amyloid/tau pathology or postmortem follow-up. Because AD pathology often coincides with Lewy body pathology (Chatterjee et al., 2021), our samples may include patients with AD and Lewy body co-pathologies. Furthermore, motor impairments including bradykinesia have been recognized in non-parkinsonian neurodegenerative disorders including AD and have been proposed to signify the extent of widespread neurodegeneration regardless of associated pathology (Bologna et al., 2020; Paparella et al., 2021; Marsili et al., 2023). As eye movements and visual attention also require coordination between widespread neural circuits (Corbetta and Shulman, 2002; Anderson and MacAskill, 2013; Miller and Buschman, 2013; Leigh and Zee, 2015), the altered patterns of eye movements and their degrees may potentially be elucidated more directly from the perspective of network dysfunction of multiple brain regions affected by neurodegenerative diseases rather than the intrinsic neuropathology. Further studies with neuropathological assessments for AD, LBD, and co-pathology would enhance our comprehension of the neural mechanisms underlying altered patterns of eye movements reported in this study. Second, although we show altered patterns of eye movements in AD and LBD patients in a context closer to real-world viewing compared with prior studies using restricted stimuli, it raises an important further question to elucidate how these alterations affect eye movements during active participation in a real scene, rather than during passive observation like in this study. Future studies on eye movements during natural behaviors, such as activities of daily living (Land et al., 1999; Hayhoe et al., 2003; Davis et al., 2017), will further elucidate this issue and may help deepen our understanding of how altered eye movements affect everyday functioning. Third, we did not perform individual assessments of specific neuropsychological symptoms that could affect eye movements. For example, oculomotor abnormalities including saccadic hypometria and oculomotor apraxia have been identified in laboratory tests in various neurodegenerative dementias including AD and LBD (Boxer et al., 2012; Anderson and MacAskill, 2013), which could affect voluntary viewing behaviors as investigated in this study. In addition, although extensive attention dysfunctions, including sustained attention, that could influence oculomotor performance are known in DLB (Mosimann et al., 2005; Revie et al., 2019), we did not conduct attention-specific assessments such as the Stroop task. Further investigation may help explain the mechanisms of eye movement alterations in AD and LBD from neuropsychological viewpoints. Fourth, we assumed relatively large effect sizes for between-group differences on the basis of the literature and confirmed this assumption in our dataset. However, small but meaningful effects might be undetectable particularly in the exploratory analysis on clinical-instrumental correlations. Moreover, in the analysis, several correlations lost statistical significance after correction for multiple testing, even though we mainly reported the results without correction given the exploratory nature of the analysis. Our findings need to be validated with larger samples. Fifth, our main analysis combined MCI and dementia as one group due to small sample sizes, and disease stage-wise alterations were only investigated exploratorily. Further stage-wise analysis may reveal detailed patterns of altered eye movements at the prodromal stage, which would facilitate early dementia identification and differentiation. Sixth, although several confounding factors such as fatigue and antipsychotic medication were considered in the analysis, we did not adjust our analyses for non-antipsychotic medication and eye-related comorbidities that could also affect oculomotor functions and eye movement features. Seventh, our results on the classification models showed small CIs, but this may be due to the small sample size. Specifically, although we calculated the CIs through 20 repetitions of 20-fold cross validations, the small sample size made training and test sets largely overlapped across the repetitions, which might result in small CIs that did not properly represent the certainty of the results. To show generalizability of the classification models using eye movement features, we need to validate with larger samples and those collected in independent, different institutions.
In this study on free viewing of naturalistic complex scenes, we show distinct altered patterns of eye movements associated with disease severity in AD and LBD patients at the prodromal and dementia stages: (i) diminished visual exploration, differentially associated with cognitive impairment in AD and with motor impairment in LBD; and (ii) reduced gaze allocations to objects attributed to a weaker attention bias toward object-based semantic attributes in AD and attributed to a greater center bias in LBD. These distinct altered patterns may help differentiate AD and LBD as behavioral markers and deepen our understanding of how they manifest in the real world, ultimately leading to the mitigation of the widespread impact of impaired visual attention on activities of daily living.
Data availability statement
The datasets presented in this article are not readily available because they contain information that could compromise research participant privacy/consent. Anonymized data that support the findings of this study may be made available upon reasonable request for academic purposes. Requests to access the datasets should be directed to YY, ysnr@jp.ibm.com.
Ethics statement
The studies involving humans were approved by the Ethics Committee, University of Tsukuba Hospital (H29-065). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
YY: Conceptualization, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. KS: Investigation, Visualization, Writing – review & editing. MK: Formal analysis, Writing – review & editing. MN: Data curation, Funding acquisition, Investigation, Methodology, Writing – review & editing. MO: Investigation, Writing – review & editing. KN: Formal analysis, Funding acquisition, Investigation, Methodology, Writing – review & editing. TA: Funding acquisition, Investigation, Methodology, Project administration, Writing – review & editing.
Funding
The authors declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Japan Society for the Promotion of Science, KAKENHI [grant number 19H01084] (MN, KN, and TA) and [grant number 21K12153] (KN). The funders did not play any active role in the scientific investigation and reporting of the study.
Conflict of interest
YY, MK, and KS were employed by IBM Corporation.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2024.1333894/full#supplementary-material
References
Albert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C., et al. (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 270–279. doi: 10.1016/j.jalz.2011.03.008
Anderson, T. J., and MacAskill, M. R. (2013). Eye movements in patients with neurodegenerative disorders. Nat. Rev. Neurol. 9, 74–85. doi: 10.1038/nrneurol.2012.273
Ballard, C., McKeith, I., Burn, D., Harrison, R., O’Brien, J., Lowery, K., et al. (1997). The UPDRS scale as a means of identifying extrapyramidal signs in patients suffering from dementia with Lewy bodies. Acta Neurol Scand. 96, 366–371. doi: 10.1111/j.1600-0404.1997.tb00299.x
Baluch, F., and Itti, L. (2011). Mechanisms of top-down attention. Trends Neurosci. 34, 210–224. doi: 10.1016/j.tins.2011.02.003
Bjoerke-Bertheussen, J., Ehrt, U., Rongve, A., Ballard, C., and Aarsland, D. (2012). Neuropsychiatric symptoms in mild dementia with Lewy bodies and Alzheimer’s disease. Dement. Geriatr. Cogn. Disord. 34, 1–6. doi: 10.1159/000339590
Bologna, M., Guerra, A., Colella, D., Cioffi, E., Paparella, G., Di Vita, A., et al. (2020). Bradykinesia in Alzheimer’s disease and its neurophysiological substrates. Clin. Neurophysiol. 131, 850–858. doi: 10.1016/j.clinph.2019.12.413
Borji, A., Sihite, D. N., and Itti, L. (2012). Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Trans. Image Process 22, 55–69. doi: 10.1109/TIP.2012.2210727
Borji, A., Sihite, D. N., and Itti, L. (2013). What stands out in a scene? A study of human explicit saliency judgment. Vis. Res. 91, 62–77. doi: 10.1016/j.visres.2013.07.016
Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). “A training algorithm for optimal margin classifiers,” in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, (Berkeley: University of California), 144–152. doi: 10.1145/130385.130401
Boxer, A. L., Garbutt, S., Seeley, W. W., Jafari, A., Heuer, H. W., Mirsky, J., et al. (2012). Saccade abnormalities in autopsy-confirmed frontotemporal lobar degeneration and Alzheimer disease. Arch. Neurol. 69, 509–517. doi: 10.1001/archneurol.2011.1021
Chatterjee, A., Hirsch-Reinshagen, V., Moussavi, S. A., Ducharme, B., Mackenzie, I. R., and Hsiung, G.-Y. R. (2021). Clinico-pathological comparison of patients with autopsy-confirmed Alzheimer’s disease, dementia with Lewy bodies, and mixed pathology. Alzheimers Dement. 13:e12189. doi: 10.1002/dad2.12189
Chen, C., Ding, S., and Wang, J. (2023). Digital health for aging populations. Nat. Med. 29, 1623–1630. doi: 10.1038/s41591-023-02391-8
Coco, M. I., Merendino, G., Zappalà, G., and Della Sala, S. (2021). Semantic interference mechanisms on long-term visual memory and their eye-movement signatures in mild cognitive impairment. Neuropsychology 35:498. doi: 10.1037/neu0000734
Collerton, D., Burn, D., McKeith, I., and O’Brien, J. (2003). Systematic review and meta-analysis show that dementia with Lewy bodies is a visual-perceptual and attentional-executive dementia. Dement. Geriatr. Cogn. Disord. 16, 229–237. doi: 10.1159/000072807
Corbetta, M., and Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215. doi: 10.1038/nrn755
Cormack, F., Gray, A., Ballard, C., and Tovée, M. J. (2004). A failure of ‘pop-out’ in visual search tasks in dementia with Lewy bodies as compared to Alzheimer’s and parkinson’s disease. Int. J. Geriatr. Psychiatry 19, 763–772. doi: 10.1002/gps.1159
Crous-Bou, M., Minguillo′n, C., Gramunt, N., and Molinuevo, J. L. (2017). Alzheimer’s disease prevention: From risk factors to early intervention. Alzheimers Res. Ther. 9, 1–9. doi: 10.1186/s13195-017-0297-z
Daffner, K. R., Mesulam, M. M., Cohen, L. G., and Scinto, L. F. (1999). Mechanisms underlying diminished novelty-seeking behavior in patients with probable Alzheimer’s disease. Cogn. Behav. Neurol. 12, 58–66.
Daffner, K. R., Scinto, L., Weintraub, S., Guinessey, J., and Mesulam, M. (1992). Diminished curiosity in patients with probable Alzheimer’s disease as measured by exploratory eye movements. Neurology 42, 320–320. doi: 10.1212/wnl.42.2.320
Damiano, C., and Walther, D. B. (2019). Distinct roles of eye movements during memory encoding and retrieval. Cognition 184, 119–129. doi: 10.1016/j.cognition.2018.12.014
Das, J., Graham, L., Morris, R., Barry, G., Godfrey, A., Walker, R., et al. (2022). “Eye movement in neurological disorders,” in Eye Tracking: Background, Methods, and Applications, ed. S. Stuart (New York, NY: Humana), doi: 10.1007/978-1-0716-2391-6
Davis, R., Ohman, J. M., and Weisbeck, C. (2017). Salient cues and wayfinding in Alzheimer’s disease within a virtual senior residence. Environ. Behav. 49, 1038–1065. doi: 10.1177/0013916516677341
Dementia. (2023). Dementia research needs a global approach. Nat. Med. 29:279. doi: 10.1038/s41591-023-02249-z
Dickerson, B. C., Bakkour, A., Salat, D. H., Feczko, E., Pacheco, J., Greve, D. N., et al. (2009). The cortical signature of Alzheimer’s disease: Regionally specific cortical thinning relates to symptom severity in very mild to mild ad dementia and is detectable in asymptomatic amyloid-positive individuals. Cereb. Cortex 19, 497–510. doi: 10.1093/cercor/bhn113
Eggins, P., Wong, S., Wei, G., Hodges, J. R., Husain, M., Piguet, O., et al. (2022). A shared cognitive and neural basis underpinning cognitive apathy and planning in behavioural-variant frontotemporal dementia and Alzheimer’s disease. Cortex 154, 241–253. doi: 10.1016/j.cortex.2022.05.012
Fernando, R., Thomas, A. J., Hamilton, C. A., Durcan, R., Barker, S., Ciafone, J., et al. (2024). Identifying parkinsonism in mild cognitive impairment. J. Neurol. Sci. 458:122941. doi: 10.1016/j.jns.2024.122941
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., et al. (2002). Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain. Neuron 33, 341–355. doi: 10.1016/s0896-6273(02)00569-x
Gauthier, S., Rosa-Neto, P., Morais, J. A., and Webster, C. (2021). World Alzheimer Report 2021: Journey Through the Diagnosis of Dementia. London: Alzheimer’s Disease International, 17–29.
Habibi, M., Oertel, W. H., White, B. J., Brien, D. C., Coe, B. C., Riek, H. C., et al. (2022). Eye tracking identifies biomarkers in α-synucleinopathies versus progressive supranuclear palsy. J. Neurol. 269, 4920–4938. doi: 10.1007/s00415-022-11136-5
Hand, D. J., and Till, R. J. (2001). A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach. Learn. 45, 171–186. doi: 10.1023/a:1010920819831
Hayhoe, M. M., Shrivastava, A., Mruczek, R., and Pelz, J. B. (2003). Visual memory and motor planning in a natural task. J. Vis. 3:6. doi: 10.1167/3.1.6
Henderson, J. M., Williams, C. C., and Falk, R. J. (2005). Eye movements are functional during face learning. Mem. Cognit. 33, 98–106. doi: 10.3758/bf03195300
Hopfinger, J. B., Buonocore, M. H., and Mangun, G. R. (2000). The neural mechanisms of top-down attentional control. Nat. Neurosci. 3, 284–291. doi: 10.1038/72999
Itti, L. (2015). New eye-tracking techniques may revolutionize mental health screening. Neuron 88, 442–444. doi: 10.1016/j.neuron.2015.10.033
Itti, L., Koch, C., and Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259. doi: 10.1109/34.730558
Johansson, M., Stomrud, E., Lindberg, O., Westman, E., Johansson, P. M., van Westen, D., et al. (2020). Apathy and anxiety are early markers of Alzheimer’s disease. Neurobiol. Aging 85, 74–82. doi: 10.1016/j.neurobiolaging.2019.10.008
Kennedy, D. P., D’Onofrio, B. M., Quinn, P. D., Bölte, S., Lichtenstein, P., and Falck-Ytter, T. (2017). Genetic influence on eye movements to complex scenes at short timescales. Curr. Biol. 27, 3554–3560. doi: 10.1016/j.cub.2017.10.007
Klein, A., and Tourville, J. (2012). 101 labeled brain images and a consistent human cortical labeling protocol. Front. Neurosci. 6:171. doi: 10.3389/fnins.2012.00171
Kourtis, L. C., Regele, O. B., Wright, J. M., and Jones, G. B. (2019). Digital biomarkers for Alzheimer’s disease: The mobile/wearable devices opportunity. NPJ Digit. Med. 2:9. doi: 10.1038/s41746-019-0084-2
Land, M., Mennie, N., and Rusted, J. (1999). The roles of vision and eye movements in the control of activities of daily living. Perception 28, 1311–1328. doi: 10.1068/p2935
Landy, K. M., Salmon, D. P., Filoteo, J. V., Heindel, W. C., Galasko, D., and Hamilton, J. M. (2015). Visual search in dementia with Lewy bodies and Alzheimer’s disease. Cortex 73, 228–239. doi: 10.1016/j.cortex.2015.08.020
Leigh, R. J., and Zee, D. S. (2015). The Neurology of Eye Movements (Contemporary Neurology Series). Oxford: Oxford University Press.
Mannan, S. K., Kennard, C., and Husain, M. (2009). The role of visual salience in directing eye movements in visual object agnosia. Curr. Biol. 19, R247–R248. doi: 10.1016/j.cub.2009.02.020
Marsili, L., Duque, K. R., Gregor, N., Abdelghany, E., Abanto, J., Duker, A. P., et al. (2023). Bradykinesia in neurodegenerative disorders: A blinded video analysis of pathology-proven cases. Mov. Disord. 38, 496–501. doi: 10.1002/mds.29330
Masanneck, L., Gieseler, P., Gordon, W. J., Meuth, S. G., and Stern, A. D. (2023). Evidence from ClinicalTrials.gov on the growth of Digital Health Technologies in neurology trials. npj Digit. Med. 6:23. doi: 10.1038/s41746-023-00767-1
Matuskova, V., Ismail, Z., Nikolai, T., Markova, H., Cechova, K., Nedelska, Z., et al. (2021). Mild behavioral impairment is associated with atrophy of entorhinal cortex and hippocampus in a memory clinic cohort. Front. Aging Neurosci. 13:643271. doi: 10.3389/fnagi.2021.643271
Mc Ardle, R., Galna, B., Donaghy, P., Thomas, A., and Rochester, L. (2019). Do Alzheimer’s and Lewy body disease have discrete pathological signatures of gait? Alzheimers Dement. 15, 1367–1377. doi: 10.1016/j.jalz.2019.06.4953
McGuinness, B., Barrett, S. L., Craig, D., Lawson, J., and Passmore, A. P. (2010). Attention deficits in Alzheimer’s disease and vascular dementia. J. Neurol. Neurosurg. Psychiatry 81, 157–159. doi: 10.1136/jnnp.2008.164483
McKeith, I. G., Boeve, B. F., Dickson, D. W., Halliday, G., Taylor, J.-P., Weintraub, D., et al. (2017). Diagnosis and management of dementia with Lewy bodies: Fourth consensus report of the DLB Consortium. Neurology 89, 88–100. doi: 10.1212/WNL.0000000000004058
McKeith, I. G., Ferman, T. J., Thomas, A. J., Blanc, F., Boeve, B. F., Fujishiro, H., et al. (2020). Research criteria for the diagnosis of prodromal dementia with Lewy bodies. Neurology 94, 743–755.
McKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T., Jack, C. R. Jr., Kawas, C. H., et al. (2011). The diagnosis of dementia due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 7, 263–269. doi: 10.1016/j.jalz.2011.03.005
Mikhailova, A., Raposo, A., Sala, S. D., and Coco, M. I. (2021). Eye-movements reveal semantic interference effects during the encoding of naturalistic scenes in long-term memory. Psychon. Bull. Rev. 28, 1601–1614. doi: 10.3758/s13423-021-01920-1
Miller, E. K., and Buschman, T. J. (2013). Cortical circuits for the control of attention. Curr. Opin. Neurobiol. 23, 216–222. doi: 10.1016/j.conb.2012.11.011
Molitor, R. J., Ko, P. C., and Ally, B. A. (2015). Eye movements in Alzheimer’s disease. J. Alzheimers Dis. 44, 1–12. doi: 10.3233/JAD-141173
Morris, J. C. (1993). The clinical dementia rating (CDR): Current version and scoring rules. Neurology 43, 2412–2414. doi: 10.1212/wnl.43.11.2412-a
Mosimann, U., Felblinger, J., Ballinari, P., Hess, C. W., and Müri, R. M. (2004). Visual exploration behaviour during clock reading in Alzheimer’s disease. Brain 127, 431–438. doi: 10.1093/brain/awh051
Mosimann, U. P., Müri, R. M., Burn, D. J., Felblinger, J., O’Brien, J. T., and McKeith, I. G. (2005). Saccadic eye movement changes in Parkinson’s disease dementia and dementia with Lewy bodies. Brain 128, 1267–1276. doi: 10.1093/brain/awh484
Nemoto, K., Dan, I., Rorden, C., Ohnishi, T., Tsuzuki, D., Okamoto, M., et al. (2011). Lin4neuro: A customized Linux distribution ready for neuroimaging analysis. BMC Med. Imaging 11:3. doi: 10.1186/1471-2342-11-3
Olsen, R. K., Sebanayagam, V., Lee, Y., Moscovitch, M., Grady, C. L., Rosenbaum, R. S., et al. (2016). The relationship between eye movements and subsequent recognition: Evidence from individual differences and amnesia. Cortex 85, 182–193. doi: 10.1016/j.cortex.2016.10.007
Paparella, G., Fasano, A., Hallett, M., Berardelli, A., and Bologna, M. (2021). Emerging concepts on bradykinesia in non-parkinsonian conditions. Eur. J. Neurol. 28, 2403–2422. doi: 10.1111/ene.14851
Petersen, R. C., Aisen, P., Beckett, L. A., Donohue, M., Gamst, A., Harvey, D. J., et al. (2010). Alzheimer’s disease neuroimaging initiative (ADNI): Clinical characterization. Neurology 74, 201–209. doi: 10.1212/WNL.0b013e3181cb3e2
Pieruccini-Faria, F., Black, S. E., Masellis, M., Smith, E. E., Almeida, Q. J., Li, K. Z. H., et al. (2021). Gait variability across neurodegenerative and cognitive disorders: Results from the Canadian Consortium of Neurodegeneration in Aging (CCNA) and the Gait and Brain Study. Alzheimers Dement. 17, 1317–1328. doi: 10.1002/alz.12298
Ramzaoui, H., Faure, S., David, R., and Spotorno, S. (2022). Top-down and bottom-up sources of eye- movement guidance during realistic scene search in Alzheimer’s disease. Neuropsychology 36, 597–613. doi: 10.1037/neu0000797
Ramzaoui, H., Faure, S., and Spotorno, S. (2018). Alzheimer’s disease, visual search, and instrumental activities of daily living: A review and a new perspective on attention and eye movements. J. Alzheimers Dis. 66, 901–925. doi: 10.3233/JAD-180043
Redel, P., Bublak, P., Sorg, C., Kurz, A., Fo¨rstl, H., Müller, H., et al. (2012). Deficits of spatial and task- related attentional selection in mild cognitive impairment and Alzheimer’s disease. Neurobiol. Aging 33, 195.e27–42. doi: 10.1016/j.neurobiolaging.2010.05.014.
Revie, L., Bayer, A., Teufel, C., and Metzler-Baddeley, C. (2019). A review of the perceptual and attentional-executive characteristics of dementia with Lewy bodies relative to Alzheimer’s and Parkinson’s disease. PsyArXiv [Preprint]. doi: 10.31234/osf.io/cq4m6
Schütz, N., Knobel, S. E. J., Botros, A., Single, M., Pais, B., Santschi, V., et al. (2022). A systems approach towards remote health-monitoring in older adults: Introducing a zero-interaction digital exhaust. npj Dig. Med. 5, 1–13. doi: 10.1038/s41746-022-00657-y
Schwarz, C. G., Gunter, J. L., Wiste, H. J., Przybelski, S. A., Weigand, S. D., Ward, C. P., et al. (2016). A large-scale comparison of cortical thickness and volume methods for measuring Alzheimer’s disease severity. Neuroimage Clin. 11, 802–812. doi: 10.1016/j.nicl.2016.05.017
Seligman, S. C., and Giovannetti, T. (2015). The potential utility of eye movements in the detection and characterization of everyday functional difficulties in mild cognitive impairment. Neuropsychol. Rev. 25, 199–215. doi: 10.1007/s11065-015-9283-z
Shakespeare, T. J., Pertzov, Y., Yong, K. X., Nicholas, J., and Crutch, S. J. (2015). Reduced modulation of scanpaths in response to task demands in posterior cortical atrophy. Neuropsychologia 68, 190–200. doi: 10.1016/j.neuropsychologia.2015.01.020
Silberg, J. E., Agtzidis, I., Startsev, M., Fasshauer, T., Silling, K., Sprenger, A., et al. (2019). Free visual exploration of natural movies in schizophrenia. Eur. Arch. Psychiatry Clin. Neurosci. 269, 407–418. doi: 10.1007/s00406-017-0863-1
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632–1634. doi: 10.1126/science.7777863
Taylor, J.-P., Colloby, S. J., McKeith, I. G., and O’Brien, J. T. (2013). Covariant perfusion patterns provide clues to the origin of cognitive fluctuations and attentional dysfunction in dementia with Lewy bodies. Int. Psychogeriatr. 25, 1917–1928. doi: 10.1017/S1041610213001488
Terao, Y., Fukuda, H., and Hikosaka, O. (2017). What do eye movements tell us about patients with neurological disorders? — An introduction to saccade recording in the clinical setting —. Proc. Jpn. Acad. Series B 93, 772–801. doi: 10.2183/pjab.93.049
Thomas, A. (2022). Corrigendum: Revision of assessment toolkits for improving the diagnosis of Lewy body dementia: The DIAMOND Lewy study. Int. J. Geriatr. Psychiatry [Online ahead of print]. doi: 10.1002/gps.5663
Thomas, A. J., Taylor, J. P., McKeith, I., Bamford, C., Burn, D., Allan, L., et al. (2017). Development of assessment toolkits for improving the diagnosis of the Lewy body dementias: Feasibility study within the DIAMOND Lewy study. Int. J. Geriatr. Psychiatry 32, 1280–1304. doi: 10.1002/gps.4609
Tokushige, S., Matsumoto, H., Matsuda, S., Inomata-Terada, S., Kotsuki, N., Hamada, M., et al. (2023). Early detection of cognitive decline in Alzheimer’s disease using eye tracking. Front. Aging Neurosci. 15:1123456. doi: 10.3389/fnagi.2023.1123456
Tseng, P.-H., Cameron, I. G., Pari, G., Reynolds, J. N., Munoz, D. P., and Itti, L. (2013). High-throughput classification of clinical populations from natural viewing eye movements. J. Neurol. 260, 275–284. doi: 10.1007/s00415-012-6631-2
Valliappan, N., Dai, N., Steinberg, E., He, J., Rogers, K., Ramachandran, V., et al. (2020). Accelerating eye movement research via accurate and affordable smartphone eye tracking. Nat. Commun. 11:4553. doi: 10.1038/s41467-020-18360-5
Walker, Z., Possin, K. L., Boeve, B. F., and Aarsland, D. (2015). Lewy body dementias. Lancet 386, 1683–1697. doi: 10.1016/S0140-6736(15)00462-6
Wang, S. (2019). Brief report: Atypical visual exploration in autism spectrum disorder cannot be attributed to the amygdala. J. Autism Dev. Disord. 49, 2605–2611. doi: 10.1007/s10803-019-04009-w
Wang, S., Jiang, M., Duchesne, X. M., Laugeson, E. A., Kennedy, D. P., Adolphs, R., et al. (2015). Atypical visual saliency in autism spectrum disorder quantified through model-based eye tracking. Neuron 88, 604–616. doi: 10.1016/j.neuron.2015.09.042
Watson, R., Colloby, S. J., Blamire, A. M., and O’Brien, J. T. (2015). Assessment of regional gray matter loss in dementia with Lewy bodies: A surface-based MRI analysis. Am. J. Geriatr. Psychiatry 23, 38–46. doi: 10.1016/j.jagp.2014.07.005
Wood, J. S., Firbank, M. J., Mosimann, U. P., Watson, R., Barber, R., Blamire, A. M., et al. (2013). Testing visual perception in dementia with Lewy bodies and Alzheimer disease. Am. J. Geriatr. Psychiatry 21, 501–508. doi: 10.1016/j.jagp.2012.11.015
Xu, J., Jiang, M., Wang, S., Kankanhalli, M. S., and Zhao, Q. (2014). Predicting human gaze beyond pixels. J. Vis. 14, 28–28. doi: 10.1167/14.1.28
Yamada, Y., and Kobayashi, M. (2018). Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults. Artif. Intell. Med. 91, 39–48. doi: 10.1016/j.artmed.2018.06.005
Yamada, Y., Shinkawa, K., Kobayashi, M., Caggiano, V., Nemoto, M., Nemoto, K., et al. (2021). Combining multimodal behavioral data of gait, speech, and drawing for classification of Alzheimer’s disease and mild cognitive impairment. J. Alzheimers Dis. 84, 315–327. doi: 10.3233/jad-210684
Yamada, Y., Shinkawa, K., Nemoto, M., Ota, M., Nemoto, K., and Arai, T. (2022a). Speech and language characteristics differentiate Alzheimer’s disease and dementia with Lewy bodies. Alzheimers Dement. 14:e12364. doi: 10.1002/dad2.12364
Yamada, Y., Kobayashi, M., Shinkawa, K., Nemoto, M., Ota, M., Nemoto, K., et al. (2022b). Characteristics of drawing process differentiate Alzheimer’s disease and dementia with lewy bodies. J. Alzheimers Dis. 90, 693–704. doi: 10.3233/jad-220546
Keywords: saliency model, mild cognitive impairment, dementia, free viewing, differential diagnosis, brain atrophy, Alzheimer’s disease, dementia with Lewy bodies
Citation: Yamada Y, Shinkawa K, Kobayashi M, Nemoto M, Ota M, Nemoto K and Arai T (2024) Distinct eye movement patterns to complex scenes in Alzheimer’s disease and Lewy body disease. Front. Neurosci. 18:1333894. doi: 10.3389/fnins.2024.1333894
Received: 06 November 2023; Accepted: 22 March 2024;
Published: 05 April 2024.
Edited by:
Hantao Liu, Cardiff University, United KingdomReviewed by:
Kathleen Fraser, National Research Council Canada (NRC), CanadaFrancesco Asci, Sapienza University of Rome, Italy
Luca Marsili, University of Cincinnati, United States
Etienne De Villers-Sidani, McGill University, Canada
Copyright © 2024 Yamada, Shinkawa, Kobayashi, Nemoto, Ota, Nemoto and Arai. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yasunori Yamada, ysnr@jp.ibm.com