Edited by: David Saldaña, Seville University, Spain
Reviewed by: Ernesto Guerra, University of Chile, Chile; Lesya Ganushchak, Erasmus University Rotterdam, Netherlands
This article was submitted to Language Sciences, a section of the journal Frontiers in Psychology
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
We investigated the effects of everyday language exposure on the prediction of orthographic and phonological forms of a highly predictable word during listening comprehension. Native Japanese speakers in Tokyo (Experiment 1) and Berlin (Experiment 2) listened to sentences that contained a predictable word and viewed four objects. The critical object represented the target word (e.g.,
During language comprehension, people sometimes predict a word that is likely to come up and pre-activate representations of the predictable word before it is mentioned (
Research on bilingual children suggests that both the quantity and quality of language exposure has a significant impact on their language acquisition (
Effects of reduced language exposure may impact the efficiency of orthographic processing especially in languages that use many orthographically complex characters such as Japanese and Chinese, because people can forget an orthographic form of a word that they had acquired. In a survey on the Japanese language conducted by the Agency for Cultural Affairs in Japan in 2012, 66.5% of the respondents indicated that their ability to correctly write kanji characters (logogram in Japanese) had deteriorated due to the increased use of electronic communication methods, and 87% of them believed that the writing ability of the Japanese people had been deteriorating (retrieved from:
According to word recognition models that support an interactive activation of phonology and orthography during listening and reading comprehension (
Many studies have found that people can predict upcoming information during language comprehension. For example, in a visual world eye-tracking study by
When the sentence context is highly predictive, people can predict information about a specific word that is likely to occur. For example,
ERP studies have also found evidence that people predict a specific phonological or orthographic word form of a highly predictable word (e.g.,
Probabilistic models of prediction (e.g.,
While there is evidence that people predict various types of information during comprehension, there is also evidence that not everyone predicts to the same extent (
The current study is based on
Participants were more likely to look at the target and the orthographic competitor before the target word was mentioned. Critically, the orthographic competitor effect (i.e., the difference between the orthographic competitor and unrelated conditions) was larger when the target and orthographic competitor words were more similar versus less similar in the orthographic form, as expected, if the effect was due to the prediction of the orthographic form. However, when the same set of words were presented in hiragana (phonogram in Japanese; Experiment 2), there was no orthographic competitor effect. Since the orthographic competitor was not orthographically similar to the target word in hiragana (e.g.,
However, it is unclear whether people pre-activate the orthographic form of a predictable word only when the visual context provides orthographically relevant information, as there are alternative explanations for the lack of an orthographic competitor effect in the hiragana presentation. For example, the critical words used in this study are usually written in kanji, so the use of hiragana in Experiment 2 may have artificially reduced the activation of the kanji form. Another possibility is that the predictive looks to the orthographic competitor in Experiment 1 were driven by orthographic pre-activation via orthographic priming from the orthographic competitor. A related possibility is that participants may have initially mistook the orthographic competitor word for the target word. If one of these possibilities was true, we expect to find no orthographic effect in the current study, which used objects instead of printed words.
The current study is a replication of
We discussed in the introduction that multiple factors related to language proficiency seem to modulate prediction independently. To explore which factors affect prediction most significantly, we additionally tested the effects of language proficiency by testing correlations between the degree of predictive eye movements and language proficiency measured in verbal fluency tests and kanji reading and writing tests. Studies that found a relationship between verbal fluency and the prediction (
The investigation of the effects of kanji reading and writing skills was motivated by
Fifty-seven native Japanese speakers studying at Waseda University in Tokyo, Japan (24 males, age
The auditory stimuli were identical to those used in
“
The mean cloze probability for the target words was 87% (SD = 9%). The mean cloze probability including only target responses in kanji was 77% (SD = 15%). Thus, there was a preference to write the target words in kanji (rather than other Japanese scripts). The sentences were recorded by a native male Japanese speaker using a Marantz PCM recorder. The mean duration of the target words was 261 ms. The experiment used an additional 20 filler sentences that were not predictive toward a specific word. These sentences mentioned one of the depicted objects 75% of the time, so the sentences in the entire experiment mentioned one of the depicted objects 50% of the time.
The visual stimuli were created by replacing the words in
An example of the visual stimuli for each condition. The object on the top right was the critical object for this item:
Kanji and lexical characteristics of critical words in each condition.
Condition | Stroke | Grade of kanji acquisition | Log kanji frequency | Mora count | Orthographic similarity |
Target | 7.4 (3.1) | 3.5 (2.5) | 4.2 (0.78) | 2.1 (0.51) | — |
Orthographic | 7.5 (3.3) | 3.5 (2.5) | 4.1 (0.84) | 2.1 (0.55) | 3.9 (2.8–5.9) |
Phonological | 8.9 (4.2) | 4.6 (2.4) | 4.2 (0.90) | 2.2 (0.41) | 1.4 (1.1–1.8) |
Unrelated | 7.5 (4.0) | 4.0 (2.5) | 4.2 (0.90) | 2.1 (0.61) | 1.4 (1.1–1.8) |
The critical objects were never presented together to make the orthographic or phonological relatedness less obvious and to prevent fixations on the competitor objects from being swamped by fixations on the target object. In all conditions except for the target condition, the critical object was implausible to be mentioned in the target word position. Care was taken to ensure that the critical word in the orthographic/phonological/unrelated conditions was not semantically related to the target word. All critical and distractor words were one-kanji character words. The characteristics of the critical words were obtained from
All critical objects were presented again on filler trials with a non-predictive sentence and distractors from another item to test whether there is any fixation bias toward a particular object (e.g., due to their visual features). We tested whether the critical object in each condition was similarly likely to be fixated when the sentence does not mention them (cf. “Results” section).
The stimuli were pseudorandomized and divided into four lists with two versions. Each list contained the same number of trials per condition and contained only one condition per item. The two versions were created by swapping the first half and the second half. The critical object appeared in each of the quadrants equally frequently. Due to the limited number of one-character words, some of the critical objects were used more than once, but they were never used in the same condition or presented in succession.
Before the eye-tracking experiment, participants were familiarized with the objects. In the training phase, all objects (including the distractor objects) were presented with their name one by one, and participants were instructed to memorize them so that they could name them later. In the testing phase, participants saw only the object and named it. Incorrectly named objects were repeated until they were named correctly. The mean accuracy of naming in the first instance was 99%, suggesting that it was easy for the participants to associate the objects with their intended name.
In the eye-tracking experiment, participants were instructed to listen to the sentences via headphones and click on an object that was mentioned in the sentence or click on the background if none of the objects was mentioned. Participants’ eye movements were recorded using an EyeLink 1000 Desktop mount eye-tracker sampling at 500 Hz. Each trial began with a drift check (i.e., participants fixated at the center of the screen at the beginning of the trial). Participants then heard the sentence, and the objects appeared on the screen 1000 ms before the target word. The mouse pointer appeared in the center of the screen when the objects appeared, and it disappeared when participants clicked on an object or the background. The objects remained on the screen until 3000 ms after the sentence offset. The experiment began with four practice trials, and the main experiment was divided into two blocks. Calibration (using a five-point grid) and validation were performed before the practice session as well as before the main experiment and before the second block, if necessary. The visual scenes were presented on a monitor at a resolution of 1280 × 1024 pixels.
After the eye-tracking experiment, participants performed verbal fluency tests (letter fluency and category fluency) and kanji reading and writing tests adapted from kanken (
We analyzed the eye movement data using mixed-effects logistic regression models including linear and quadratic time (using orthogonal polynomials) to capture both overall differences between the conditions and effects over the time-course (
The mean accuracy for the clicking task was very high (
Mean fixation probabilities with standard error (shaded area around each line) for each condition in Experiment 1. The results for all items (top panel), high orthographic similarity items (bottom left panel), and low orthographic similarity items (bottom right panel). The gray shaded area indicates the analyzed time window. Time 0 indicates the onset of the target word.
Following
We further tested whether participants’ predictive eye movements were mediated by their language proficiency test scores. The mean scores for the letter fluency test and category fluency test were 12.4 (SD = 2.8) and 16.6 (SD = 3.0), respectively. For kanji reading and writing tests, the scores represent the proportion of correct answers. The mean scores for kanji reading and writing tests were 76.7 (SD = 12.2) and 43.5 (SD = 16.7), respectively. Each of these scores was correlated with the fixation probability differences between the target and unrelated conditions, the orthographic and unrelated conditions, and the phonological and unrelated conditions in -800–200 ms relative to the target word onset.
The correlation matrix among the fixation probability differences and language proficiency measures in Experiment 1. The top-right values are
Experiment 1 found that participants were more likely to fixate the target object over the unrelated object before the target word was mentioned, suggesting that participants predicted some information about the target word. Critically, they were also more likely to fixate the orthographic competitor over the unrelated object, suggesting that participants predicted the orthographic form of the target word. We did not find evidence for the prediction of phonological form. In the individual difference analysis, we did not find any clear relationship between predictive eye movements and language proficiency measures, except that participants with higher kanji writing scores were more likely to predictively fixate target objects over unrelated objects (However, this effect was not found in Experiment 2).
Experiment 2 asked whether regular exposure to the language affects prediction by replicating Experiment 1 in native Japanese speakers in Berlin. To quantify the participants’ language background, we asked participants to fill in a LEAP Questionnaire before the experiment (
Fifty-six native Japanese speakers who were resident in Berlin, Germany at the time of testing (10 males, age
The stimuli and procedure were identical to Experiment 1, except that participants filled in a LEAP Questionnaire (
The mean accuracy for the clicking task was very high (
Mean fixation probabilities with standard error (shaded area around each line) for each condition in Experiment 2. The results for all items (top panel), high orthographic similarity items (bottom left panel), and low orthographic similarity items (bottom right panel). The gray shaded area indicates the analyzed time window. Time 0 indicates the onset of the target word.
Similar to Experiment 1, we tested whether the fixation bias toward the orthographic competitors was affected by the orthographic similarity between the target word and the orthographic competitor word. As can be seen in
We tested individual differences in participants’ predictive eye movements the same as in Experiment 1. The mean scores for the letter fluency test and category fluency test were 12.2 (SD = 3.1) and 17.1 (SD = 2.7), respectively. These scores did not differ from participants in Experiment 1,
The correlation matrix among the fixation probability differences and language proficiency measures in Experiment 2. The top-right values are
We tested the effect of group (Tokyo, Berlin) on predictive eye movements by including the factor group into the model that tested an interaction of condition by orthographic similarity with linear and quadratic terms. We dropped the phonological condition because neither group showed a phonological competitor effect. Thus, the model tested an interaction of condition (target vs. unrelated, orthographic vs. unrelated) by orthographic similarity by group. The factor group was deviation-coded. The model revealed more overall fixations on the target objects than the unrelated objects (54.2% vs. 15.3%), β = 2.5, SE = 0.23,
In two experiments, we found that participants showed a fixation bias toward the target object and the orthographic competitor relative to the unrelated object before the target word was mentioned. The predictive orthographic competitor effect suggests that people can pre-activate the orthographic form of a highly predictable word during listening comprehension. Interestingly, the Berlin group showed fewer predictive looks to the target than the Tokyo group. Additionally, the Berlin group did not show the orthographic competitor effect similar to the Tokyo group, although they showed a tendency to fixate the orthographic competitor when the orthographic competitor was highly similar to the target word. These group differences suggest that everyday language exposure affects language prediction. We found no evidence for the prediction of phonological form. The predictive looks to the target, the orthographic competitor, and the phonological competitor did not correlate with participants’ verbal fluency or kanji reading/writing scores.
These findings extend
The results suggest that native speakers in Tokyo made stronger predictions about the target word and the orthographic form of the target word than native speakers in Berlin. This finding is consistent with probabilistic models of prediction (e.g.,
The lack of a phonological competitor effect in our study seems inconsistent with
In both experiments, we found that participants who showed a stronger bias toward the orthographic competitors (over the unrelated objects) tended to show a stronger bias toward the phonological competitors. The orthographic competitors were not phonologically related to the target, and the phonological competitors were not orthographically related to the target (when they were written in the most preferred script – kanji). Thus, we did not expect to find this correlation. One possibility is that participants who predicted orthographic form also predicted phonological form. Participants who showed an orthographic competitor effect arguably predicted the specific word (e.g., the lexical item
Finally, we did not find any effect of language proficiency (letter/category fluency, kanji reading/writing scores) on predictive eye movements. The null effects contrast with studies that found individual differences on predictive eye movements (
The preregistration for Experiment 2, the analysis scripts, and the data (Experiments 1–2) are available at Open Science Framework (
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Open Science Framework (
The studies involving human participants were reviewed and approved by Office of Research Ethics, Waseda University, and die Ethikkommission der Deutschen Gesellschaft für Sprachwissenschaft. The patients/participants provided their written informed consent to participate in this study.
AI performed experiment design, data collection, data analysis, data interpretation, write-up (original draft). HS performed experiment design, data interpretation, and write-up (review). Both authors contributed to the article and approved the submitted version.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We thank Yohei Oseki for his voice for the auditory stimuli, and Angela Schmidt for proofreading the manuscript. We acknowledge support by the German Research Foundation (DFG) and the Open Access Publication Fund of Humboldt-Universität zu Berlin.
Critical sentences with approximate English translations and critical objects for each condition. Target words are in bold. Cloze probabilities for target words are shown after each sentence in brackets.
Participants in Experiment 1 additionally performed a lexical decision test for another study that is not reported here.
Considering the claim that a growth curve analysis may produce a high rate of false positives due to over-fitting of the model and auto-correlation in eye movement data (
The ANOVA and