The temporal dynamics of implicit processing of non-letter, letter, and word-forms in the human visual cortex
1
Center for Cognitive Neuroscience, Duke University, Durham, NC, USA
2
Department of Psychology, Simon Fraser University, Vancouver, BC, Canada
3
Research Imaging Center, UTHSCSA, San Antonio, TX, USA
4
Department of Psychiatry, Duke University, Durham, NC, USA
The decoding of visually presented line segments into letters, and letters into words, is critical to fluent reading abilities. Here we investigate the temporal dynamics of visual orthographic processes, focusing specifically on right hemisphere contributions and interactions between the hemispheres involved in the implicit processing of visually presented words, consonants, false fonts, and symbolic strings. High-density EEG was recorded while participants detected infrequent, simple, perceptual targets (dot strings) embedded amongst a of character strings. Beginning at 130 ms, orthographic and non-orthographic stimuli were distinguished by a sequence of ERP effects over occipital recording sites. These early latency occipital effects were dominated by enhanced right-sided negative-polarity activation for non-orthographic stimuli that peaked at around 180 ms. This right-sided effect was followed by bilateral positive occipital activity for false-fonts, but not symbol strings. Moreover the size of components of this later positive occipital wave was inversely correlated with the right-sided ROcc180 wave, suggesting that subjects who had larger early right-sided activation for non-orthographic stimuli had less need for more extended bilateral (e.g., interhemispheric) processing of those stimuli shortly later. Additional early (130–150 ms) negative-polarity activity over left occipital cortex and longer-latency centrally distributed responses (>300 ms) were present, likely reflecting implicit activation of the previously reported ‘visual-word-form’ area and N400-related responses, respectively. Collectively, these results provide a close look at some relatively unexplored portions of the temporal flow of information processing in the brain related to the implicit processing of potentially linguistic information and provide valuable information about the interactions between hemispheres supporting visual orthographic processing.
Written words are among the most pervasive visual stimuli that modern humans encounter over the course of their lives. The vast majority of individuals become adept at the effortless perception and decoding of visual word forms as they develop fluent reading abilities, typically at a young age. Skilled readers can process, at remarkable speeds, the complicated visual stimuli that constitute written text, distinguishing them from subtly different non-orthographic visual stimuli in order to rapidly access linguistic representations. A large body of research in the last 25 years has shown that reading abilities are supported by a high-degree of functional specialization, initiating relatively early in the hierarchy of cortical processes, and exhibiting a high degree of hemispheric lateralization.
Previous studies of the processing of visually presented word and word-like stimuli using positron emission tomography (PET) (Petersen et al., 1988
, 1990
; Petersen and Fiez, 1993
; Liotti et al., 1994
; Price et al., 1996
) and functional magnetic resonance imaging (fMRI) (Puce et al., 1996
; Indefrey et al., 1997
; Cohen et al., 2000
, 2002
; Tagamets et al., 2000
; Baker et al., 2007
), have emphasized the left-lateralized nature of word-form processing in occipital cortex (reviewed in Bastiaansen and Hagoort, 2006
; Dien, 2009b
). In general, these studies have found that visually presented real words and pseudowords (pronounceable non-words), relative to non-word control stimuli (e.g., consonant strings and digits), selectively elicit enhanced activity in portions of the left medial occipital cortex. Additional, studies have provided converging evidence for the identification of this area, located in the left midfusiform gyrus, as a putative ‘visual word form’ area (WFA) (Cohen et al., 2000
, 2002
; Dehaene et al., 2001
; McCandliss et al., 2003
; Brem et al., 2009
), although other studies have also suggested that the degree of left-lateralization depends on the linguistic familiarity of the characters (Tagamets et al., 2000
), the task demands (Ruz et al., 2005a
,b
; Ruz and Nobre, 2008
), or other linguistic and developmental factors (Maurer et al., 2005b
; Hauk et al., 2006
; Schlaggar and McCandliss, 2007
). While these hemodynamically based neuroimaging studies have been highly valuable in identifying the anatomical substrates of visual orthographic processing, they are, however, notably limited in their ability to reveal much about the temporal dynamics of the underlying processing.
An understanding of the temporal dynamics of these processes has begun to emerge through direct (Nobre et al., 1994
; Matsumoto et al., 2004
; Canolty et al., 2007
) and indirect recordings of the brain’s electrical activity. Intracranial recordings of electrical activity in the inferior temporal lobe have demonstrated high-amplitude negative potentials, peaking around 200 ms (N200) for orthographic stimuli (words, pseudowords and letter-strings) and non-orthographic stimuli (faces, checkerboards, etc). While both kinds of stimuli elicited activity in the posterior fusiform gyrus, the N200 elicited by all word-like stimuli were greater in the left hemisphere, while those elicited by faces were either bilaterally symmetric or greater in the right hemisphere (Nobre et al., 1994
).
Similarly, scalp recordings of electric brain activity (EEG; reviewed in Maurer and McCandliss, 2007
) or their magnetic counterpart (MEG; Helenius et al., 1999
; Tarkiainen et al., 1999
) of evoked response to graphemic characters have revealed patterns of laterality that are broadly consistent with intracranial recording reports. In particular, the N170 component is an ERP wave that has been reported as showing prelexical sensitivity to orthographic versus non-orthographic strings (Ziegler et al., 1997
; Bentin et al., 1999
; Gros et al., 2002
; Rossion et al., 2003
; Simon et al., 2004
; Maurer et al., 2005a
, 2008a
), or versus individual characters (Wong et al., 2005
). This component, which is elicited in the latency range of ∼120–200 ms following stimulus presentation, typically shows greater activity over the left hemisphere for orthographic stimuli, and over the right hemisphere for non-orthographic visual stimuli, especially faces and objects (Bentin et al., 1996
; Schendan et al., 1998
).
While a convergence of neuroimaging and lesion-correlational studies have indicated the importance of left-hemisphere function in visual word recognition, a much less explored aspect of orthographic processing involves the notion of a distinct right-lateralized visual word-form system that may more generally process perceptual features of visual words, rather than their specific orthographic characteristics (Marsolek et al., 1992
; Price et al., 1996
; Schendan et al., 1998
; Tagamets et al., 2000
; Rossion et al., 2003
). In their seminal PET study, Petersen et al. (1990)
observed bilateral activations in lateral extrastriate cortex for all word and word-like stimuli relative to visual fixation. However, that report focused mainly on the left medial extrastriate activity. In a follow-up report by Liotti et al. (1994)
on aspects of the data from the Petersen et al. study, it was noted that while all word-like stimuli contrasted to visual fixation activated lateral extrastriate cortex bilaterally, false fonts relative to fixation had the greatest right-lateralized extrastriate activation. In contrast, left medial extrastriate activation was only present for real words and pseudowords. Further support for a right visual word-form area comes from behavioral studies using stem-completion word priming. In particular, Marsolek et al. (1992)
found that priming of words based on their visual structure (case-specificity) was greater for words presented initially to the left-visual field, and thus the right hemisphere. Subsequent PET studies found that neural measures of perceptual priming of previously exposed visual words resulted in significant blood flow decrease in a location remarkably similar to the right lateral extrastriate area reported by Petersen (Squire et al., 1992
; Buckner et al., 1995
; Schacter et al., 1999
; Badgaiyan et al., 2003
).
Thus on the one hand hemodynamically based neuroimaging studies cannot provide much insight into the temporal dynamics of the stimulus processing, being intrinsically limited in temporal resolution. On the other hand, studies of the right-sided orthography-related occipital effects with ERPs or other methods with high temporal resolution have been rather limited, most having focused on the left-side enhancements for orthographic stimuli. Moreover, most ERP studies of orthographic versus non-orthographic processing have focused on varying the linguistic nature of the task in a variety of ways with specific goals in mind for assessing the top-down influence of these tasks (e.g., Ziegler et al., 1997
; Bentin et al., 1999
; Maurer et al., 2008b
). While some of these studies have included tasks that were mostly orthogonal to any linguistic processing (e.g., letter size judgment by Bentin et al., 1999
), and therefore presumably activated the implicit processing of words and characters, the right-sided visual form activity was not a focus and was little discussed. In addition, these studies have typically also examined the activation patterns in separated and distinct time periods as a function of the manipulations, without much examination of the processing flow between them. Lastly, few of the ERP studies have performed source analyses of the observed right-sided activations in order to help relate them to the reported locations of previous relevant neuroimaging studies.
The main goal of the present study was thus to study the processing of non-orthographic and orthographic visual stimuli using ERP recordings and a fully implicit-processing task, with a focus on extending the finding of greater right-sided activity in the N170 time range for non-orthographic stimuli and more deeply examining it under circumstances of implicit, automatized, stimulus processing. Our analyses include source modeling of the locations of the underlying brain generators of this right-sided activity, a closer examination of the temporal dynamics around and following it, and functional correlations with other activations in the processing time course, including from an interhemispheric standpoint. To focus as fully as possible on relatively automatic, implicit processing of visual-form stimuli, we used a more stringent implicit word and character processing paradigm, in which participants viewed a series of character strings of varying linguistic structure and familiarity as they performed an orthogonal detection task for occasional target stimuli that was completely unrelated to any linguistic content of the character strings.
Subjects
Eleven right-handed, native English speaking, neurologically intact, college-age subjects, with normal or corrected-to-normal visual acuity, served as subjects in this experiment. Informed consent was obtained prior to experimentation under a protocol that was approved by the Institutional Review Board of the University of Texas, Health Science Center at San Antonio. Prior to the main EEG recording, subjects were instructed on the task demands and given a practice experimental run that lasted about 3 min. Reaction times and error rates were measured while whole head 64-channel EEG was recorded. Subjects were paid $10 per hour for their participation in the study.
Stimuli and Task
This paper investigates the implicit processing of character strings with varying levels of orthographic and lexical structure. To ensure that focused spatial attention was directed to the stimuli, an intermixed target detection task was imposed. Subjects viewed character strings like those illustrated in Figure 1
that were presented for 150 ms, just above fixation, at stimulus onset asynchronies (SOAs) varying from 1000–1600 ms. There were four main stimulus types presented in randomized order: words, consonant strings, symbols strings (pound signs and parentheses), and ‘false-font’ strings (strings of meaningless characters physically similar to letters). These stimuli were styled after Petersen et al. (1990) and consisted of strings that varied between three and seven characters in length (mean length of 5). Word strings were randomly selected from a list of 256 common nouns. Both word strings and consonant strings consisted of all capital letters. The false font characters were composed of contiguous line segments designed to be dissimilar from any of the alphabetical symbols, but equivalent in the content of primitive visual features.
Figure 1. Stimulus schematic showing example character strings for the four stimulus types and the infrequent dot-string targets.
In addition to these four character string types, 15% of the stimulus trials consisted of strings of square 4-dot patterns, matched in mean and range of string length with the other character strings. Subjects were asked to fixate on a central fixation cross, attend to all stimuli, and press a button on detecting the infrequent dot-string targets. The need to detect the occurrence of the dot patterns, therefore, necessitated that subjects maintain focused attention to the stimuli, while also providing an index by which to assess performance.
ERP Recordings and Analysis
The electroencephalogram (EEG) was recorded continuously from 64 channels mounted in a customized elastic cap (Electro-Cap International, Eaton, OH) using a bandpass filter of 0.01–100 Hz, gain of 1000, and sampling rate of 400 Hz (SynAmps, Neuroscan). All channels were referenced to the right mastoid during recording. The positions of all 64 channels were equally spaced across the customized cap and covered the whole head from slightly above the eyebrows to below the inion posteriorly (Woldorff et al., 2002
). Impedances of all channels were maintained to be below 5 kΩ, and fixation and eye movements were monitored with both electro-oculogram (EOG) recordings and a zoom-lens camera. Recordings took place in an electrically shielded, sound-attenuated, dimly lit, experimental chamber.
ERP processing included the re-referencing of all channels to the algebraic mean of the two mastoid electrodes and the application of a digital, non-causal, seven-point running average filter. This filter greatly reduces frequencies at 56 Hz and above at the sampling frequency of 400 Hz that was used. Artifact rejection was performed off-line before averaging by using a computer algorithm that discarded epochs of the EEG that exceeded a pre-specified threshold in the window from −200 to 600 ms around the presentation of the string stimulus. The artifact rejection thresholds were set individually for each subject. On average, ∼12% of trials were rejected.
To assess the differential implicit processing of the character strings, we carried out analyses of variance (ANOVAs) on the ERP waveforms from sets of occipital and central-parietal electrode sites corresponding to the peak activity for the components of interest. The selection of electrode sites and latency time-windows, indicated below for each specific analysis, was made mainly based on pilot data collected on subjects performing a related variant of this task in which trial types were presented in a block design, rather than the present intermixed design. These site selections also fit well with those locations used in previous studies of orthographic processing. Repeated-measure ANOVAs with stimulus type, hemisphere, and in some cases inferior/superior electrode location, were performed on specific stimulus-type contrasts, as described in the relevant results sections for specific latency ranges. Unless otherwise specified, the ERP measures were mean amplitudes of the waveforms across specified time windows or components (relative to a 200-ms pre-stimulus baseline), which were then analyzed for significant differences between stimulus conditions. The significance threshold for all analyses was set to a p-value of 0.05, and all ANOVA analyses were corrected for violations of sphericity using the Greenhouse-Geisser correction.
In addition, Pearson’s product-moment correlations were calculated between specific component amplitudes to study the relationship between brain areas involved in processing visually presented character strings. Spherical-spline-interpolated topographic voltage maps of the grand-averaged ERP differences were derived for a series of consecutive 20 or 40 ms windows to visualize how the scalp distribution changed over time (see Figures 3 and 4
).
ERP Source Analysis
Source estimates were derived using the BESA (Brain Electrical Source Analysis) dipole modeling software package (Scherg, 1992
). This approach estimates the orientation and location of multiple equivalent dipolar sources by calculating the scalp distribution that would be obtained for a given dipole model (forward solution) and comparing it to the original ERP distribution. Iterative changes in the orientation and location of the dipole sources leads to minimization of the residual variance (RV) between the model and the observed scalp voltage potential distribution of the ERP. For models involving more than one dipole, the energy criterion of the BESA estimates was set at 12% and the separation criterion was set at 10%. These multi-dipole parameter selections were made in order to reduce interactions between dipoles and optimize the separation of the source waveforms attributed to the estimated sources. The three-shell spherical head was modeled with a radius of 80 mm and scalp and skull thicknesses of 6 mm each.
To perform the dipole source modeling, the grand averaged waveform differences over specific latency ranges correspond to the effects of interest were fed into BESA analysis along with the grand averaged, spatially normalized electrode locations in Talairach space (Talairach and Tournoux, 1988
). The effects were modeled relative to the 100 ms pre-stimulus baseline yielding Talairach coordinates and orientation estimates for each source dipole. Single spatio-temporal dipole sources were modeled for the lateralized occipital activations occurring in the ROcc180 effect range, while location and orientation of mirror-symmetrical dipole pairs were modeled for the bilateral occipital P2 effect range. These single and dual dipole selections were chosen based on the scalp voltage and current density distributions of the modeled effects, and the dipole models obtained were little affected by using different symmetry constraints or different starting locations in the dipole fitting.
Behavioral Results
To ensure and monitor that an adequate level of attention was deployed during the viewing of the character strings, subjects were instructed to detect target dot arrays that were randomly presented on 15% of the trials. These target dot arrays consisted of strings of square 4-dot patterns (see Figure 1
) that covered the same location and approximate spatial extent as the character strings, ensuring that subjects maintained attention to the presented stimuli. In this task, subjects demonstrated a high level of accuracy, detecting targets 93% of the time with a mean reaction time of 646 ms (SD = 118 ms). It can therefore be inferred that subjects were vigilant and attentive during the presentation of the interspersed character strings.
ERP Results
Robust ERPs were elicited by the four non-target stimulus types; false fonts, symbols, consonants, and words. The first ERP response component of all of these visual stimuli consisted of a bilateral occipital deflection with a positive peak amplitude at around 100 ms. (See Supplementary Materials for spatio-temporal distribution of the ERP sensory waves). This component differed little between the stimulus types and closely corresponded to the P100 component typically found in visual stimulus ERP studies (Hillyard et al., 1998
; Luck et al., 2000
).
The orthographic and non-orthographic stimuli showed a cascade of implicit-processing effects across time beginning at around 130 ms. The largest orthography-related effect seen at early latency ranges for the implicit processing of these visual stimuli was a robustly enhanced negative-polarity wave over right occipital scalp, peaking at 180 ms poststimulus, for non-orthographic stimuli relative to orthographic ones, an effect we will term here the right-occipital N180 non-orthography effect (Rocc180). Shortly preceding this robust right-occipital effect was a very brief reversed polarity effect over left occipital scalp consisting of a slightly larger negative wave for orthographic versus non-orthographic stimuli, although this was substantially smaller than the ROcc180 effect. Following the ROcc180 effect an extended enhancement of a bilateral positive-polarity component also over both left and right occipital scalp regions from 220–300 ms that was also greater for non-orthographic stimuli. Finally, late effects (300–500 ms) included modulations of a centrally distributed negative wave (peaking at 400 ms, similar to the N400) with greater amplitude for word-like than non-orthographic strings. This sequence of effects is described below in greater detail in the chronological order of their appearance.
Word-form ‘LOcc140’ effects (130–150 ms)
Words and consonants elicited a slightly greater negative amplitude than false-font strings over left occipital scalp sites in the 130–150 ms latency range, a finding that is consistent with numerous previous reports of left-occipital visual word-form selectivity (reviewed in McCandliss et al., 2003
). This effect (which we will refer to here as the ‘LOcc140’), illustrated in Figure 2
, peaked at ∼140 ms over left occipital sensors and was more pronounced for word than for consonants. A 2 × 2 ANOVA on the peak amplitudes at occipital sensors (TO1 and TO2) for words versus false fonts revealed a significant main effect [F(1,10) = 4.76, p = 0.05] of stimulus type, and a significant stimulus by hemisphere interaction [F(1,10) = 10.36, p = 0.01] in the 130–150 ms latency range. For consonants versus false fonts over this same latency range, the main effect of stimulus type did not reach significance [F(1,10) = 2.65, p = 0.13] while the stimulus by hemisphere interaction was nearly significant [F(1,10) = 4.07, p = 0.07].
Figure 2. ‘LOcc140’ results. Topographic distributions, shown from a posterior perspective, of the mean activity from 130–150 ms for Words (top left), Consonants (top right), False Fonts (middle), and their Differences (Words minus False Fonts on bottom left and Consonants minus False Fonts on bottom right).
Non-orthographic-related ‘ROcc180’ effects (170–190 ms)
In the 170–190 ms latency range, false fonts and symbols elicited substantially greater negative-polarity amplitude than words and consonants over right occipital scalp. To evaluate the differential processing of orthographic and non-orthographic character strings, two sets of comparisons are illustrated in Figure 3
. On the top row, evoked responses to false fonts and consonant strings are shown at left and right occipital electrode sites (left side of figure), along with the topographic distribution of their difference (middle). In addition, the BESA-reconstructed source estimate for this difference is also shown in Figure 3
(on the right). On the bottom row, the same data are shown for symbol strings versus consonant strings.
Figure 3. ‘ROcc180’ results are displays for false fonts (cyan) versus consonants (magenta) on top and symbols (orange) versus consonants (magenta) on the bottom. Left panels: Grand average ERPs at a pair of left and right occipital sensors [Note that slightly different sensor locations are used for the two different contrasts.] Middle panels: Difference maps computed between conditions from 170–190 ms are shown from a posterior perspective. Right panels: BESA-computed equivalent current dipole sources for these effects (single-source solutions) and the associated source waveforms are shown on the right, along with the residual variance values for these solutions.
In both cases, stimulus selectivity between non-orthographic strings and consonants strings emerged as a robust focal effect over right occipital cortex, peaking at 180 ms poststimulus (here termed the ‘ROcc180’). As seen in the ERP traces, the non-orthographic waveforms show enhanced negative-going activity relative to the consonants waveform beginning just before the peak of the N1 component over the right, but not the left, occipital electrode sites. Initial statistical assessment of this effect was performed by way of a 4 × 2 repeated measures ANOVA performed on the mean amplitude of the ERP responses for all four stimulus types at the left (mean at sites TO1 and P3i) and right (mean at sites TO2 and P4i) sets of electrodes from 170–190 ms. This test revealed a main effect of stimulus [F(3,10) = 5.36, p < 0.001, larger for non-orthographic] and a stimulus by hemisphere interaction [F(3,10) = 3.11, p = 0.042, effect larger on the right].
To further assess these effects, separate 3 × 2 ANOVAs were performed for false fonts versus consonant strings, and for symbols versus consonant strings, in the same latency window, with the inferior/superior-electrode-location (inferior [TO1-TO2] versus superior [P3i-P4i]) added as a third factor to capture slight differences observed between the response distributions for the stimulus types. A main effect of stimulus type [F(1,10) = 7.57, p = 0.022], a stimulus by hemisphere interaction [F(1,10) = 9.08, p = 0.015], and a stimulus by inferior/superior-electrode-site-location interaction [F(1,10) = 8.56, p = 0.017] were all present for the false-font contrast. Similarly, for the symbol vs. consonant strings, a main effect of stimulus type [F(1,10) = 19.61, p < 0.001] and a stimulus by hemisphere interaction [F(1,10) = 8.15, p = 0.019] were present, whereas the stimulus by electrode-site-level interaction did not quite reach significance at the p < 0.05 level [F(1,10) = 3.72, p = 0.086]. The position of these effects can be seen in Figure 3
where the right-sided activation for was bigger at the more superior site for the symbol versus consonant difference, and at the more inferior site for the false font versus consonant difference.
The right-dominant nature of these ‘false-font’ and ‘symbol’ effects can be clearly seen in the difference maps over the 170–190 ms latency range in the middle panels of Figure 3
. While the distributions of the symbol and false-font effects (relative to consonants) differed slightly, each was modeled well by a single dipole in right occipital cortex. The estimated dipole location for false-font effect (Talairach coordinates [mm]: x = 40, y = −58, z = 4; Residual Variance (RV) = 4.3%; nAm = 22.6) was within a centimeter of previously identified right-occipital effects shown with PET (Petersen et al., 1990
), while the estimated dipole location for the symbol effect was more dorsal and slightly more medial (x = 27, y = −65, z = 25; RV = 4.1%, nAm = 24.7).
In addition to the significant ROcc180 effects reported above, the contrast of false fonts and words also elicited a right occipital difference in the 170–190 ms range. This difference (see top row of Figure 4
) showed a significant main effect of stimulus [F(1,10) = 8.39, p = 0.017] and a significant stimulus by hemisphere interaction [F(1,10) = 6.71, p = 0.03], illustrating that differential processing is elicited in right occipital cortex at this latency range by non-orthographic strings relative to either form of orthographic ones (consonants or words).
Figure 4. Sequential topographic maps (posterior view) across time of the grand average voltage distributions for the different contrasts, as labeled.
‘P2’ selectivity for familiar versus non-familiar non-orthographic forms (240–300 ms)
To better characterize and illustrate the temporal unfolding of orthography-related processing differences between stimulus types, sequential 20 ms topographical maps of the difference-wave activity from 100–300 ms is shown in Figure 4
, for four contrasts of interest. For false-fonts minus words or consonants (rows 1 and 2), the ROcc180 was followed by substantial additional focal positive activity extending from ∼240–300 ms bilaterally over the occipital scalp. The ROcc180 elicited by symbols minus consonants was not followed by this protracted bilateral activity however, but rather was followed by only a brief left-lateralized positivity (row 3 of Figure 4
). It should be noted here that due to the direction of the subtraction across time (i.e., non-orthographic versus orthographic), the early left-side LOcc140 word-form effect for words versus false fonts shows up as a positive difference here (row 1).
Based on the inspection of the sequential topographical maps shown in Figure 4
, 2 × 2 ANOVAs with stimulus type and hemisphere (averaging over left channels TO1 and P3i, and right channels TO2 and P4i) were also computed from 240–300 ms to further investigate effects occurring over this latency range. False fonts versus words (top row) elicited a significant main effect of stimulus type [F(1,10) = 46.51, p < 0.01], and a significant stimulus-by-hemisphere interaction [F(1,10) = 5.87, p = 0.04], due to there being slightly more differential activity on the left. False fonts versus consonants (second row) also resulted in a main effect of stimulus (p < 0.05), although the stimulus by hemisphere interaction did not reach significance. Due to the focal bilateral distribution of the activity difference in this latency range, mirrored dipole pairs were used to model the activity sources. These analyses yielded fairly good models for the false font versus words effect (right source [x = 37, y = −71, z = −10], nAm = 15.3; left source [x = −39, y = −67, z = −11], nAm 19.1; RV = 9.93) and the false font versus consonants effect (right source [x = 35, y = −80, z = −8], nAm = 16.1; left source [x = −39, y = −77, z = −10], nAm 16.4; RV = 8.5%). An illustration of these dipole models is provided in Figure S2 in Supplementary Material. Neither symbol versus consonants, nor words versus consonants, produced a significant main effect, nor did they elicit any interaction in this latency range.
Longer-latency effects (300–500 ms)
At latencies between 300 and 500 ms, the ERPs elicited by the four stimuli showed modulations of a slow negative deflection with a broad midline-central distribution, which differed substantially in amplitude by stimulus type. This negativity modulation corresponded to a well-ordered, hierarchical pattern in which more word-like stimuli produced greater negative amplitudes than less word-like stimuli.
Figure 5
presents ERPs for the four stimulus types at a central midline (Cz), a left parietal (PO1), and a right parietal (PO2) sensor, along with the topographic maps of the ERP differences for each stimulus types relative to the symbols. These maps show that the amplitude of the differential activity increased with the level of potentially linguistic content of the stimuli. Single factor ANOVAs on the mean amplitude from 360–440 ms of the midline central-parietal channel CPz reveal a main effect of condition [F(3,30) = 5.34, p = 0.005]. Pair-wise post-hoc tests between stimulus types all showed significant differences (all p values < 0.05) in this latency range, with increasing negativity from Symbols to False Fonts, Consonants, and finally Words.
Figure 5. Late-latency effects. ERPs elicited by each stimulus type are shown at three central-parietal sites. Top: The amplitude of the negative-going wave in the 300–500 ms range showed a hierarchical pattern, with more word-like stimuli eliciting greater negative deflections (i.e., words > consonants > false fonts > symbols), activity differences likely corresponding to modulations of the linguistically related N400 component. Bottom: The topographic distributions of the difference waves, relative to symbol strings, for each stimulus type, shown as spline-interpolated flat maps.
Correlational analyses between ROcc180, P2, and Longer-latency effects
To help further understand the relationship between brain areas involved in processing visually presented character strings, Pearson’s correlation coefficients were computed between the ROcc180, P2, and longer-latency effects isolated in the contrasts described above. Pearson’s correlation coefficients computed between the right-sided ROcc180 amplitudes (170–190 ms at channel TO2) for the false fonts versus words effect, as well as for the false-fonts versus consonant effect, with the respective P2-latency difference-wave amplitudes on the left (240–300 channel TO1) for the same contrasts, both revealed significant correlations (FF-Words, r = 0.632, p = 0.037; FF-Consonants r = 0.615, p = 0.044), such that greater Rocc180 amplitude on the right correlated with less amplitude in the following P2 over the left hemisphere. These same right-sided ROCc180 effects however, did not correlate significantly with the right hemisphere P2 effects at the same locations (FF-Words_right, r = 0.005, p = 0.987; FF-Consonants_right r = 0.392, p = 0.234), suggesting that ROcc180-P2 covariations between the two hemispheres were not simply due to spatial overlap where larger negative early effects could have possibly ‘dragged down’ or systematically diluted the later positive effects.
As a check to rule out the possibility that these correlations could have been due simply to covariations in the overall sizes of ERP activations that were not specific to orthographic processing (e.g., due to subject-wise difference in volume conduction), ROcc180 amplitudes for the false font versus words, and for false fonts versus consonants, were also correlated with other brain activations in the functional time course. These tests revealed that the ROcc180 amplitudes did not covary significantly with the longer-latency (i.e., N400-range) medial-central words-minus symbol effect (FF-Words vs. Words-Symbols, r = −0.278, p = 0.408; FF-Consonants vs. Words-Symbols, r = −0.430, p = 0.186), nor with the earlier P1 amplitudes (FF-Words vs. P1, r = 0.182, p = 0.593; FF-Consonants vs. Words-Symbols, r = −0.31, p = 0.391). These correlational results therefore suggest a possible functional relationship between right-sided activation in the 180 ms latency range and the extent of subsequent analysis of the same stimuli in the left visual areas in the 240–300 ms time interval, rather than being simply due to spurious covariations (e.g., volume conduction, skull thickness) across subjects.
The present study aimed at investigating the temporal dynamics of hemisphere-specific processes and interactions involved in the implicit processing of visually presented words, consonants, false fonts, and symbolic strings. Early evoked responses in the present design most notably identify strong right-occipital enhancement for non-orthographic stimuli (170–190 ms) that was followed by a bilateral occipital modulation (240–300 ms). These activations appear to work in a interactive fashion, such that the size of components the later positive occipital wave was inversely correlated with the right-sided ROcc180 wave, suggesting that subjects who had larger early right-sided activation for non-orthographic stimuli may have had less need of a more extended bilateral processing shortly later. In addition to the novel characterization of this tight temporal cascade of activation related to the implicit processing of non-orthographic stimuli, the present results also confirm the presence of well established early, left-occipital enhancements for orthographic stimuli (130–150 ms), although rather small under the completely non-linguistic, implicit processing task used here, and longer-latency (300–500 ms) centrally distributed N400-type modulations related to the degree of semantic content present in the stimulus. These results therefore clarify the temporal flow of information processing between the hemispheres related to the implicit processing of linguistic or potentially linguistic visual information.
Early Latency, Left-Lateralized Effects: LOcc140 (130–150 Ms)
Left hemisphere ERP effects in the 140–200 ms range have been reported as a common hallmark of visual word-form selectivity (Schendan et al., 1998
; Bentin et al., 1999
; Gros et al., 2002
; Proverbio et al., 2004
; Simon et al., 2004
; Maurer et al., 2005a
,b
, 2008a
; Wong et al., 2005
; Ruz and Nobre, 2008
; Dien, 2009a
) and are believed to reflect preferential processing of orthographic forms due to long-term experience with such stimuli (Schendan et al., 1998
; Brem et al., 2009
). The LOcc140 effects reported here occurred slightly earlier and were somewhat less robust than the commonly reported 170 ms range (N170) and may be more akin to the perhaps more conceptually similar N150 component ( Spironelli and Angrilli, 2007
; Mondini et al., 2008
; reviewed in Dien, 2009a
) that has been reported to initiate as early as 116 ms in response to orthographically, phonologically, or semantically judged word pairs (Spironelli and Angrilli, 2007
). Such variability in early orthographic evoked responses has seemed to be fairly ubiquitous, and the present results are consistent with the findings of Bentin et al. (1999)
, who reported that robust word-consonant differences (in the N170 range) were only present during explicit lexical and semantic tasks and not during implicit processing of the same stimuli (although see Cohen et al., 2002
). Therefore, considering that the degree of left-lateralization of such effects has been shown to vary considerably and due to differing task demands (Ruz and Nobre, 2008
), stimulus familiarity (Tagamets et al., 2000
) and presentation modes between the particular experiments (Gros et al., 2002
; Maurer et al., 2005a
), the present LOcc140 results are consistent with the notion of an early left-lateralized occipital ‘wordform area’ (e.g., Cohen et al., 2000
; McCandliss et al., 2003
).
Early Latency, Right-Lateralized Effects: ROcc180 (170–190 Ms)
In the present event-related potential study of implicit character-string processing, particularly robust right-sided effects were observed for non-orthographic stimuli relative to orthographic stimuli. This ROcc180 effect appears as a robust, focal, negative-polarity effect peaking at ∼180 ms poststimulus that was well modeled as arising from the right extrastriate cortex, near locations implicated previously in PET and fMRI studies. We interpret this enhanced right-occipital processing for non-orthographic stimuli as reflecting the need for additional sensory analysis for unfamiliar – or less familiar – visual forms.
The somewhat differing scalp distributions and estimated dipole locations for the symbol and false-font ROcc180 effects, and their somewhat differing source analysis estimates, suggest there may be several nearby areas in right extrastriate cortex that provide early visual-form differentiation between letter and non-letter character strings. This possibility is also underscored by the presence of later (240–300 ms), prolonged, bilateral occipital differences for false fonts, but not symbols, versus consonants or words. Such a pattern suggests the possibility that visual stimuli that do not match an existing stored representation by the Rocc180 processing stage trigger additional prolonged activity in both hemispheres to resolve the meaning of the input. Such an interpretation is consistent with recent developments extending the dual-route cascade model of visual word recognition (Coltheart et al., 2001
). Specifically it has been proposed under the Janus model of laterality (Dien, 2008
) that the left hemisphere has the forward-oriented role of generating predictions while the right hemisphere is principally involved in detecting and making sense of unexpected events. In addition, it may also represent a more general substrate for visual priming/encoding for other categories of complex visual stimuli (such as objects and faces Kanwisher, 2000
; Reddy and Kanwisher, 2006
), (reviews by Cabeza and Nyberg, 1997
, 2000
) or perhaps a counterpart to the left ventral (fusiform) visual stream system specialized for the recognition of familiar/nameable objects (Haxby et al., 1991
).
Bilateral Positive Polarity, ‘Familiarity’ Effects (240–300 Ms)
The robust orthographic ERP distinctions seen in the right-sided ROcc180 were followed by a more protracted, bilateral, positive-polarity activity difference between false-font stimuli and both types of orthographic forms (consonants or words) from 240–300 ms. This effect was not seen for the familiar pound-sign and parenthesis symbols, which showed only a brief positive-polarity difference that ended by 240 ms. In line with reports of mid-latency (N2-P3) component selectivity to orthographic familiarity (Rudell, 1991
; Martin-Loeches et al., 1999
; Dien et al., 2003
) we suggest that this bilateral positive-polarity activation, seen only for the false fonts, are the result of extended processes related to the brain attempting to match the current unfamiliar stimulus with a pre-existing stored representation, based either on orthography (the set of possible written words an individual might know) or visual perceptual features (visual word form or other word forms not unique to orthographic stimuli). Accordingly, because a stored representational structure exists for the symbols, contrasts of the symbols strings with orthographic forms (consonant strings or words here) resulted in only a relatively short-lived left-sided effect that was completed by 240 ms. We believe that the extended bilateral activation occurring between 240–300 ms for the false-fonts reflects the electrophysiological signature of comparison processes, which are checking the form of the visual stimulus in relation to the various stored orthographic representations in the visual extrastriate cortices.
Longer-Latency Effects
In the 300–500 ms latency range, differences in orthographic content affected the size of a slow-wave ERP component with a broad, centro-parietal scalp distribution. The modulation of this late-wave activity elicited by the non-target character strings displayed a well-ordered hierarchical pattern, with greater negativity for more word-like stimuli (or more positivity for less word-like ones). A similar progression of long-latency activity was observed during the implicit processing of some of the same stimulus types in Experiment 1 in Bentin et al. (1999)
.
Two major components are often observed in this latency range that might underlie the data pattern seen in the present study: The N400 (or N4) and the P300b (or P3b). The N400 is observed in the case of linguistic stimuli that violate semantic expectations, with a graded amplitude response based on degree of semantic closeness to the previous word or context (van Petten, 1995
; Federmeier et al., 2000
; Kutas and Federmeier, 2000
), and closely resembles the pattern of effects seen in the present results. Alternately, posterior P300 responses are typically elicited by detected target stimuli in a stream of non-targets, such as by the dot targets here (Hillyard and Picton, 1987
). However, given the substantial dissimilarity between the non-target character-string and the target dot-string targets in the present study, it seems unlikely this longer-latency activity variation would be related to a modulation of the P300. Indeed, the current paradigm, which engaged implicit processing of character strings during a task to detect dot-string targets, was designed to keep the character strings from eliciting the large P3 components associated with target detection that would have spatially overlapped with other late activity of potential interest. Accordingly, we conclude that the centro-parietal, longer-latency difference observed in the present study is better explained by modulations of the linguistically related N400-type response.
Orthographic Processing Dynamics Revealed by Cross-Component Analysis
While a number of studies have suggested right hemisphere involvement in orthographic processing (Petersen et al., 1990
; Marsolek et al., 1996
; Bentin et al., 1999
; Tagamets et al., 2000
; Dehaene et al., 2001
; Simon et al., 2004
), especially under conditions of perceptual degradation (Sergent, 1983
; Sergent and Hellige, 1986
), masking (Polich, 1978
; Holcomb and Grainger, 2006
; Petit et al., 2006
), and short stimulus durations (Marsolek and Hudson, 1999
; Ellis et al., 2007
), few studies have systematically explored the role of right hemisphere function in orthographic processing and the dynamic interactions between the two hemispheres that support visual linguistic and non-linguistic function.
The present analyses include subject-wise comparisons of the covariations between components associated with different aspects of processing visually presented character strings in order to infer how hemisphere-specific processes interact in the decoding of visual orthography. These correlations reveal a relationship in which the size of the right-sided ROcc180 effect was inversely correlated with later contralateralized differential activity in the P2 range, but not ipsilateral activity within the same contrast. These results therefore suggest that subjects who had larger early right-sided activation for non-orthographic stimuli had less need of a more extended interhemispheric processing of those stimuli. Additional correlations ruled out spurious correlations unrelated to orthographic processing (volume conduction, overall ERP amplitudes). In addition, these results are in broad agreement with notion that the processing asymmetries are resolved in a hierarchical fashion to resolve distinctions between feature representations, feature relations, and the controlled selection of identified features where each hemispheres is relatively dominant for different aspects of feature processing (Resenhuber and Poggio, 1999
; Haxby et al., 2000
; Dien, 2009b
).
Experimental Task Considerations
A broad range of relevant studies have addressed distinctions between perceptual and lexical effects in the evoked response (Bentin et al., 1999
; Liu and Perfetti, 2003
; Hauk et al., 2006
; Martin et al., 2006
; Mariol et al., 2008
). Results from these studies have suggested that the brain initially processes physical characteristics of the stimuli, progressively accessing orthographic, phonological, and then semantic representations of the words. However, it is generally appreciated that word and word-like stimuli tend to be automatically processed, requiring neither focused attention (Ruz et al., 2005a
,b
; Ruz and Nobre, 2008
) nor cognitive discrimination to elicit a differential neural response.
While some of the previous relevant studies have included some form of implicit task, most of these studies have been primarily focused on the top-down manipulations of linguistic tasks. For example, Bentin et al. (1999)
had participants perform several tasks, each designed to promote activity at different levels of processing under the assumption that the demand characteristics of each task would intensify the activity at the respective levels and that across-task comparisons in the timing and scalp distribution of the ERPs would help disentangle one process from another. As the primary focus of their design was to relate task differences in the ERP, they provided a rather limited account of their most baseline condition (i.e., the implicit processing of orthographic differences associated with the character-string size judgment task).
Accordingly, it is of note that our right-sided early-occipital effect for non-orthographic relative to orthographic stimuli was considerably stronger than has been described. Part of the reason for this may be because of the perhaps natural but perhaps selectively guided tendency to look for activations that are larger for orthographic than for non-orthographic, leading to a focus on the left-sided effect in the 160 ms latency. In addition, however, our early right-sided effect was by far the most robust early effect (i.e., between 100 and 200 ms), much more robust than the left-sided ‘word-form’ effect previously described, which was very small in our data. We speculate that this is most likely related to the highly orthogonal nature of our task where subjects were instructed to pick out the infrequent dot-string targets in the stream. This task seems to have oriented the subjects away from the tendency for perform linguistic analysis (which still could have been the case in, for example, in the font-size judgment task of Bentin et al., 1999
), perhaps more toward a more basic, more default visual-form analysis, oriented toward trying to match the stimulus to any previous visual-form representations that may be in place in our occipital cortices. Regardless, the large and dominant size of the right-sided effect is clear and robust in the present study, which we believe is best explained as being due to our choice of a highly orthogonal, implicit-visual-processing task.
Conclusion
Due to their high temporal resolution, ERPs provide an excellent tool for studying the temporal dynamics of the functional selectivity related to linguistic processing in the brain. The present study examined the tight cascade of activity related to the implicit processing of the visual orthography of character strings. These effects begin with a sequence of several brief, but distinct, phases of activity in both the left and right occipital cortices from 130–280 ms poststimulus, followed by longer-latency differential processing by higher-order brain regions related to the possible semantic or linguistic content of the strings. The early latency occipital effects were dominated by an enhanced right-sided negative-polarity ERP activation for all non-orthographic, versus orthographic, stimuli, peaking at around 180 ms. This right-sided effect was then followed by a bilateral positive occipital wave for false-font strings (that were very similar to letters) but not symbol strings. Moreover the size of components of this later positive occipital wave was inversely correlated with the right-sided ROcc180 wave, suggesting that subjects who had larger early right-sided activation for non-orthographic stimuli had less need of a more extended bilateral (i.e., interhemispheric) processing of those stimuli shortly later). More generally considered, these findings concerning the temporal cascade of processing of character strings and other letter-like stimuli may be applicable towards the understanding and assessment of the development of reading skills in children.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was supported by funds from the Research Imaging Center at the University of Texas Health Science Center at San Antonio and by NIMH grant R01-060415 to M.G.W.
The Supplementary Material for this article can be found online at http://www.frontiersin.org/humanneuroscience/paper/10.3389/neuro.09/056.2009/