Disentangling Genuine Semantic Stroop Effects in Reading from Contingency Effects: On the Need for Two Neutral Baselines

Lorentz, Eric; McKibben, Tessa; Ekstrand, Chelsea; Gould, Layla; Anton, Kathryn; Borowsky, Ron

doi:10.3389/fpsyg.2016.00386

ORIGINAL RESEARCH article

Front. Psychol., 17 March 2016

Sec. Cognitive Science

Volume 7 - 2016 | https://doi.org/10.3389/fpsyg.2016.00386

This article is part of the Research Topic Meaning in mind: Semantic richness effects in language processing, 2nd edition View all 15 articles

Disentangling Genuine Semantic Stroop Effects in Reading from Contingency Effects: On the Need for Two Neutral Baselines

$\r\nEric Lorentz$ Eric Lorentz

Tessa McKibben

Chelsea Ekstrand

Layla Gould

Kathryn Anton

Ron Borowsky^*

Psychology, University of Saskatchewan, Saskatoon, SK, Canada

The automaticity of reading is often explored through the Stroop effect, whereby color-naming is affected by color words. Color associates (e.g., “sky”) also produce a Stroop effect, suggesting that automatic reading occurs through to the level of semantics, even when reading sub-lexically (e.g., the pseudohomophone “skigh”). However, several previous experiments have confounded congruency with contingency learning, whereby faster responding occurs for more frequent stimuli. Contingency effects reflect a higher frequency-pairing of the word with a font color in the congruent condition than in the incongruent condition due to the limited set of congruent pairings. To determine the extent to which the Stroop effect can be attributed to contingency learning of font colors paired with lexical (word-level) and sub-lexical (phonetically decoded) letter strings, as well as assess facilitation and interference relative to contingency effects, we developed two neutral baselines: each one matched on pair-frequency for congruent and incongruent color words. In Experiments 1 and 3, color words (e.g., “blue”) and their pseudohomophones (e.g., “bloo”) produced significant facilitation and interference relative to neutral baselines, regardless of whether the onset (i.e., first phoneme) was matched to the color words. Color associates (e.g., “ocean”) and their pseudohomophones (e.g., “oshin”), however, showed no significant facilitation or interference relative to onset matched neutral baselines (Experiment 2). When onsets were unmatched, color associate words produced consistent facilitation on RT (e.g., “ocean” vs. “dozen”), but pseudohomophones (e.g., “oshin” vs. “duhzen”) failed to produce facilitation or interference. Our findings suggest that the Stroop effects for color and associated stimuli are sensitive to the type of neutral baseline used, as well as stimulus type (word vs. pseudohomophone). In general, contingency learning plays a large role when repeating congruent items more than incongruent items, but appropriate pair-frequency matched neutral baselines allow for the assessment of genuine facilitation and interference. Using such baselines, we found reading processes proceed to a semantic level for familiar words, but not pseudohomophones (i.e., phonetic decoding). Such assessment is critical for separating the effects of genuine congruency from contingency during automatic word reading in the Stroop task, and when used with color associates, isolates the semantic contribution.

Introduction

Since its instantiation over 80 years ago, the Stroop effect (Stroop, 1935) has been widely investigated as a basic and robust phenomenon in cognitive psychology, and yet the source of this effect is still in contention (Melara and Algom, 2003; Levin and Tzelgov, 2014; see MacLeod, 1991 for a comprehensive review). The Stroop effect refers to the phenomenon wherein naming the font color of a congruent color word (CW; e.g., “blue” in the font color blue) shows speeded responding relative to an incongruent CW (e.g., “red” in the font color blue), despite the instructions to not read the words (i.e., unintentional, automatic word reading). One can also investigate, relative to a neutral baseline, facilitation in the congruent condition, and/or interference in the incongruent condition. Typically, greater interference is found than facilitation (Cheesman and Merikle, 1984; MacLeod, 1991; Lindsay and Jacoby, 1994).

Among the hypotheses proposed to account for Stroop effects, many have argued that both semantic conflict and response conflict make up the basis of Stroop effects (MacLeod, 1991; Botvinick et al., 2001; De Houwer, 2003; Kane and Engle, 2003; Roelofs, 2010; Anton et al., 2014). In the semantic processing account, multiple meanings from word and font color sources compete for processing resources in incongruent trials and converge to facilitate processing in congruent trials at a semantic input locus. In the response conflict account, the word stimulus evokes a learned response that is immediately brought to the output stage, which facilitates responding when congruent with the correct response, and interferes with responding when incongruent with the correct response. As semantic and response conflict models are the prevailing theoretical accounts of the Stroop effect, a significant literature has been developed around stimuli that evoke semantic conflict, but not response conflict, in order to tease apart the influences of the two. Demonstrating evidence for the semantic account, Klein (1964) first used color associates (CAs), words that are strongly associated with a particular color, (e.g., “sky” for the color blue) as stimuli in the Stroop task and found greater interference with incongruent CAs relative to common words unrelated to color (e.g., put). Soon after, Dalrymple-Alford (1972) extended these effects by including a congruent condition, presenting each word twice in each color, which revealed facilitation with congruent CAs in addition to interference with incongruent CAs, but that facilitation and interference with CAs was not as large as with CWs. Arguably, this was because CAs do not elicit response competition (see also Risko et al., 2006, on response set effects with CAs). As CAs appears to demonstrate significant facilitation and interference, there is strong support for semantic processing playing a significant role in the Stroop effect.

The automaticity with which word meaning is accessed during color-naming has also been investigated using CW pseudohomophones (CWPHs; e.g., green spelled “ghrean”), which require phonetic decoding instead of whole word retrieval for semantic access (Anton et al., 2014). For instance, Dennis and Newstead (1981) had participants name the color of CWs, CWPHs and common word PHs matched to CWs for first letter and onset (i.e., first phoneme, e.g., grief spelled “greef”) when the font color was incongruent with the word (i.e., “greef” was never presented in green) and found significant interference for both CWs and CWPHs relative to neutral PHs. A second experiment included congruent stimuli (i.e., “green,” “grean,” and “greef” presented in green font) shown three times in their congruent color and three times in different incongruent colors. The results showed faster responding for congruent CWs, CWPHs, and neutral word PHs relative to incongruent stimuli, supporting Stroop effects even when forcing the use of phonetic decoding. Moreover, while incongruent CWs showed more interference than CWPHs, congruent CWs did not show more facilitation than congruent CWPHs, which the authors attribute to fast recognition of onset matching, and the slow process of phonetic decoding. That is, onset matching slowed down incongruent stimuli enough that phonetic decoding would further interfere in the incongruent condition, but fast recognition of onset in the congruent condition would show identical results between CWs and CWPHs.

A noteworthy aspect of the methodology used by Dennis and Newstead (1981) is that each congruent letter string was presented three times, whereas incongruent letter strings were only presented once in three different colors. When the congruent stimuli, including the neutral PHs, were responded to faster than incongruent stimuli, the authors attributed this finding to the rapid recognition of each word's onset (which matched the first letter of the color response in the congruent condition). An equally plausible explanation, however, is that participants learned the correlation created between stimulus and response (also described as covariation; see Melara and Algom, 2003), given that congruent trials are repeated more often than incongruent trials (i.e., contingency learning). That is, “grean” and “greef” elicited the response “green” 50% of the time (high pair-frequency) and all other possible responses ~17% of the time (low pair-frequency). This alternative explanation may, in fact, be more plausible as color-naming shows faster responding with high pair-frequency stimuli than low pair-frequency stimuli, even when using neutral words without onset matching (e.g., Schmidt et al., 2007).

In the context of the classic Stroop task, certain stimulus-response frequencies occur when equating the proportion of congruent and incongruent stimuli. Consider balancing the number of congruent and incongruent items in an experiment with nine colors: for each congruent item (e.g., “blue” in the font color blue), there are eight incongruent items (i.e., “blue” in the eight remaining colors). Thus, the word blue would be paired with the font color blue eight times more often than any other color in order to equate the number of congruent and incongruent trials, and this could lead to significant contingency learning, which may wrongly be interpreted as genuine Stroop effects. In light of this methodological concern, Schmidt and Besner (2008) studied both contingency and congruency effects together by pairing each font color with exactly one congruent and one incongruent color word. Each color was presented with a proportion of either 75, 50, or 25% congruent trials. With this design, color word, and font color associations could be learned such that the word would predict the font color response (contingency) with either 75 or 50% accuracy for both congruent and incongruent items. Their results indicated slowed responses for incongruent trials relative to congruent trials (congruency effect), but more interestingly that the predictive nature of color words produced faster RTs for 75% predictive color words than 50% predictive color words (contingency learning). These results suggest that the Stroop effect may be attributable to both congruency effects and contingency effects, and that contingency learning plays a significant role in the Stroop task.

Contingency effects reveal the tendency to detect covariation between both relevant and irrelevant dimensions of the Stroop task (Melara and Algom, 2003). When including both congruent and incongruent trials, prior research has shown a proclivity to equate the number of congruent and incongruent trials in an orthogonal design (see MacLeod, 1991, for review), which tends to inflate the frequency of congruent color/word pairings relative to the frequency of incongruent color/word less pairings. Detecting this covariation encourages attention to word identity, and may benefit higher frequency congruent trials and hinder frequent incongruent trials, enhancing the magnitude of the measured Stroop effect that is attributed to congruency.

A number of research programmes have omitted congruent pairings altogether to avoid this artifact, comparing only equally frequent incongruent and neutral color/word pairings (e.g., Manwell et al., 2004; Risko et al., 2006; Augustinova et al., 2014). These studies provide a measure of the interference produced by incongruent color words and CAs. However, it is worth noting that they also create a negative covariation, whereby word identity still predicts responding (i.e., the word blue may be predictive of a “red” response), and can still encourage attention to the word. This design also foregoes the study of facilitation in the Stroop task, and how the inclusion of congruent stimuli can influence the context of the Stroop task as a whole (e.g., Dennis and Newstead, 1981). One solution includes presenting all congruent and incongruent color/word pairings equally frequently, but at the cost of equating the proportion of congruent and incongruent trials (e.g., Dishon-Berkovits and Algom, 2000; Sabri et al., 2001). However, a more common approach includes accounting for these differences with neutral baselines.

In order to assess facilitative and interfering effects in the Stroop paradigm, a neutral baseline is necessary (Tzelgov et al., 1992; Neely and Kahan, 2001; Augustinova and Ferrand, 2012). For instance, Tzelgov et al. (1992) manipulated the proportion of neutral trials relative to congruent and incongruent color words, and found that a larger proportion of neutral words correlated with a larger proportion of interference comparing neutral and incongruent conditions, although facilitation was not impacted by the proportion of neutral trials. In consideration of the growing literature describing contingency effects and covariation in Stroop task design, it is important to ask what constitutes an appropriate neutral baseline (see Brown, 2011, for a recent discussion). We argue that in a context where covariation exists, a neutral baseline should also incorporate this covariation, as well as match the characteristics of the condition(s) it is being compared to (i.e., the congruent or incongruent condition). Thus, if there are more repetitions of congruent trials to equate the congruent and incongruent conditions, the neutral words should follow this covariation, whereby the congruent condition is only compared to a repeated neutral counterpart, and the incongruent condition is only compared to its less frequent neutral counterpart.

Dennis and Newstead (1981) provided a starting point in matching the contingency of neutral PHs with congruent and incongruent CWs and CWPHs, but omitted neutral words (e.g., “grief” serving as a counterpart to “green”), which does not allow for direct comparison of CWs and neutral baseline while matching on reading processes like whole-word recognition. In addition, contingency effects have not been studied with CA and CAPH congruency effects, where both congruent and incongruent stimuli are included. Color associate congruency effects are taken to reflect a purely semantic contribution (i.e., associates preclude the response level conflict that color words are vulnerable to), however, one must still control for a contingency learning confound.

In order to account for both congruency and contingency effects on facilitation and interference, we used two neutral baselines: one matched for the pair-frequency of congruent stimuli and one matched for the pair-frequency of incongruent stimuli. Experiment 1 utilized typical Stroop CWs (e.g., “blue”; matched neutral “blow”) and CWPHs (e.g., “bloo”; matched neutral “bloe”), whereas Experiment 2 utilized typical CAs (e.g., “ocean”; matched neutral “odor”) and CAPHs (e.g., “oshin”; matched neutral “ohder”). Experiments 3 and 4 were identical to Experiments 1 and 2, while also examining the effects of neutral baseline onset. For these first two experiments, we hypothesized that there should be significant Stroop effects over and above contingency effects (as assessed by the two neutral conditions). The extent of facilitation vs. inhibition will also be assessed, and we note although the extant literature points to greater inhibition than facilitation effects (e.g., Cheesman and Merikle, 1984; MacLeod, 1991; Tzelgov et al., 1992) these effects have never been compared before in the present design using color associates to isolate the semantic contribution to the Stroop effects.

Experiment 1

Methods

Participants

Twenty-five undergraduate students from the University of Saskatchewan received partial course credit for their participation. All were fluent speakers of English who reported normal or corrected-to-normal vision. The data from one participant was excluded from the analysis as the individual did not learn English as their first language, and was not familiar with all of the Standard English color names used in this experiment (i.e., red, orange, yellow, green, blue, purple, pink, white, gray). The experiment (and subsequent experiments) were approved by the University of Saskatchewan Research Ethics Board with written informed consent from all participants and performed in accordance with the Helsinki Declaration.

Stimuli

Following Anton et al. (2014), stimuli appeared in nine different font colors one-at-a-time, across four types of letter strings (CWs: e.g., “blue”; CWPHs: e.g., “bloo”; neutral words: e.g., “blow”; and neutral pseudohomophones: e.g., “bloe”; see Appendix in Supplementary Materials) in the center of a black screen. Stimuli were presented individually in bold, lowercase, 32-point Arial font. The longest stimulus, “perpull,” subtended a visual angle 2.864° in height × 10.570° in width. Each stimulus appeared in the following colors: red (E-Prime's red), orange (RGB: 210, 105, 0), yellow (E-Prime's yellow), green (E-Prime's green), blue (E-Prime's blue), purple (RGB: 159, 0, 159), pink (RGB: 255, 128, 192), white (E-Prime's white), and gray (RGB: 141, 141, 141). All font color and letter string pairings were used. The neutral letter strings were matched to the color letter strings for onset (same first phoneme), log word frequency HAL, length, number of orthographic neighbors, and number of phonographic neighbors [all ts₍₈₎ ≤ 0.815, ps ≥ 0.439] based on values from the E-Lexicon database (http://elexicon.wustl.edu/; Balota et al., 2007). The pseudohomophones were generated to perfectly match their corresponding words in number of syllables and number of phonemes, see Anton et al. (2014).

The pairing of each letter string and font color was either congruent or incongruent. Stimuli were congruent if the font color matched what the letter string spelled (e.g., “blue” or “bloo” in blue font) and were incongruent if the two did not match (e.g., “blue” or “bloo” in orange font). Neutral stimuli were yoked to color-related stimuli in that they had a “congruent” color that they appeared in eight times and eight “incongruent” colors that they appeared in once (e.g., the letter strings “blue” and “blow” were presented eight times in the font color blue and once in each other color. For the purposes of referring to neutral conditions, the word congruent will be used to refer to any color/word combination that were paired frequently (i.e., eight times), and incongruent will be used to refer to color/word pairs that occurred infrequently (i.e., once), such that a congruent neutral refers to higher frequency color/word pairs and incongruent neutral refers to lower frequency color/word pairs. Thus, there were 72 congruent and 72 incongruent trials for each type of stimulus (color and neutral words), resulting in 576 trials in the entire experiment.

Apparatus and Procedure

The stimuli were presented on a standard PC computer monitor through the use of E-Prime 2.0 software (Psychology Software Tools, Inc., http://www.pstnet.com), and vocal responses were obtained using a LabTec AM-22 microphone interfaced with the E-Prime voice key serial response box. Participants were tested individually in a quiet room. Participants were instructed to press the “Go” button on the manual response box to view each stimulus and to vocally name the font color of the stimulus as quickly and as accurately as possible, while ignoring what the letter string said.

Participants first completed a series of 18 practice trials, which were designed to familiarize them with the stimuli and allow the researcher to provide feedback about incorrect responses. They were then presented with two blocks of trials, one for words and one for pseudohomophones, with block order counterbalanced across participants. Within each block, stimulus presentation was random. In each trial, participants pressed the “Go” button to view the stimulus and named the font color of the letter string as quickly as possible and the experimenter coded their response. Participants did not receive feedback during the experiment and testing took ~20 min.

Results

The response times (RTs) were measured in milliseconds (ms) and the median RT for each experimental condition was calculated for each participant's correct responses. The mean accuracy rate was also calculated for each participant by dividing the number of correct responses by the total number of unspoiled responses.

The median RTs for each condition were compared using an omnibus 2 Letter String Type (word, PH) × 4 Congruency (congruent, neutral congruent, neutral incongruent, incongruent) repeated-measures general linear model (GLM) analysis of variance (ANOVA). As predicted, there was a main effect of Congruency (Congruent Color M = 660 ms, Congruent Neutral M = 699 ms, Incongruent Neutral M = 764 ms, Incongruent Color M = 810 ms), F_{(1, 23)} = 109.921, MSE = 1932.704, p < 0.001. There was no main effect of Letter String Type (Word M = 743 ms, PH M = 723 ms), F_{(1, 23)} = 2.698, MSE = 7550.225, p = 0.114, and therefore no significant difference between the RTs for pseudohomophones and words. There was also a significant interaction between Letter String Type and Congruency, F_{(1, 23)} = 5.484, MSE = 755.225, p = 0.002. The median RTs from each cell of the 2 × 4 ANOVA are shown in Figure 1. The 95% confidence intervals in this figure were calculated using the method of Loftus and Masson (1994). We also conducted a parallel analysis by-items (see Appendix in Supplementary Materials), where matched color and neutral words were analyzed within-items. There was a main effect of Congruency (Congruent Color M = 655 ms, Congruent Neutral M = 692 ms, Incongruent Neutral M = 750 ms, Incongruent Color M = 793 ms), F_{(3, 24)} = 73.93, MSE = 905, p < 0.001, and Letter String Type (Word M = 717 ms, PH M = 728 ms), F_{(1, 8)} = 11.073, MSE = 218.64, p = 0.010. The interaction was also significant between Letter String Type and Congruency, F_{(3, 24)} = 5.484, MSE = 285.464, p = 0.020, supporting the pattern of results by subjects, although the main effect of Letter String Type emerged significant.

FIGURE 1

Figure 1. Experiment 1—Color name stimuli: Median response time (RT) and percent errors (above) in naming the font color as a function of Letter String Type (word, PHs) and Congruency (congruent, neutral congruent, neutral incongruent, incongruent). The 95% confidence intervals (shown here by the error bars attached to each data point) indicated that, for both words and PHs, there were significant differences in response time between each specific congruency condition (i.e., the RTs for congruent color, congruent neutral, incongruent neutral, and incongruent color stimuli were significantly different from each other). The percent error confidence interval was ± 2.67%. Paired-samples t-tests support the 95% CIs for Experiment 1, whereby if the CIs reflected a significant difference, so too did the t-tests, all t's₍₂₃₎ > 3.152, p's < 0.005.

The participants' percent errors were also analyzed in a 2 × 4 within-subjects GLM ANOVA. There was a main effect of Congruency (Congruent Color M = 0.010, Congruent Neutral M = 0.029, Incongruent Neutral M = 0.070, Incongruent Color M = 0.151), F_{(1, 23)} = 30.933, MSE = 0.006, p < 0.001, but no main effect of Letter String Type (Word M = 0.067, PH M = 0.063), F_{(1, 23)} = 0.218, MSE = 0.004, p = 0.645. As was found with the RT data, there was a significant interaction between Letter String Type and Congruency, F_{(1, 23)} = 12.336, MSE = 0.002, p < 0.001.

Discussion

Experiment 1 investigated the separate contributions of congruency and contingency effects with CWs and CWPHs in the Stroop task. The faster responding to higher pair-frequency neutral stimuli reflects contingency learning, which occurred for both words and pseudohomophones. Although the difference between congruent and incongruent stimuli is typically shown as the Stroop effect, the difference between the pair-frequency matched neutral conditions reflects the proportion of the Stroop effect that can be accounted for by contingency learning. These results suggest that contingency learning can occur equally for PHs and actual words (similar to (Dennis and Newstead, 1981), although actual-word neutrals were omitted from their design), even though PHs require phonetic decoding to be meaningful. The genuine Stroop effect for CWs and CWPHs can also be seen above and beyond contingency effects. That is, relative to pair-frequency matched neutral words, both facilitation and interference were found with congruent and incongruent CWs and CWPHs, respectively. Overall, while CWs and CWPHs generate a genuine Stroop effect, a substantial portion of these significant effects (46/156 ms or ~30% for words, 83/143 ms or ~58% for PHs) is attributable to contingency learning in our experiment. In addition, examining the 95% confidence intervals of Experiment 1 (see Figure 1), our results also showed that incongruent CWs showed more interference than incongruent CWPHs relative to their respective neutral baselines, whereas congruent CWs did not show more facilitation than congruent CWPHs, which paralleled the findings of Dennis and Newstead (1981).

The results of Experiment 1 support the notion that contingency learning occurs in the Stroop task. However, color name stimuli do not address the contribution of semantic processing as such effects can reflect response competition. Given that Experiment 1 addresses the main question of what amount of interference and facilitation remains in the Stroop task when controlling for contingency effects, this design also lends itself to addressing the parallel question for color associate stimuli, which allows for the isolation of the semantic component of the Stroop effect. Although the Stroop effects found with color associate stimuli are assumed to reflect semantic processing (e.g., Klein, 1964; Augustinova and Ferrand, 2012; Anton et al., 2014), their effects in the classic design Stroop task (employing all combinations of font colors and words) may conflate semantic and contingency effects. Thus, Experiment 2 aimed to estimate the magnitude of semantic Stroop interference and facilitation in font color naming for CAs and CAPHs when contingency effects are controlled using pair-frequency matched neutral words.