Original Research ARTICLE
How much of the “unconscious” is just pre – threshold?
- 1 Human Neurobiology, Centre of Cognitive Science, Bremen University, Bremen, Germany
- 2 The Henry Wellcome Laboratories of Vision Sciences, City University, London, UK
Visual awareness is a specific form of consciousness. Binocular rivalry, the alternation of visual consciousness resulting when the two eyes view differing stimuli, allows one to experimentally investigate visual awareness. Observers usually indicate the gradual changes of conscious perception in binocular rivalry by a binary measure: pressing a button. However, in our experiments we used gradual measures such as pupil and joystick movements and found reactions to start around 590 ms before observers press a button, apparently accessing even pre-conscious processes. Our gradual measures permit monitoring the somewhat gradual built-up of decision processes. Therefore these decision processes should not be considered as abrupt events. This is best illustrated by the fact that the process to take a decision may start but then stop before an action has been taken – which we will call an abandoned decision process here. Changes in analog measures occurring before button presses by which observers have to communicate that a decision process has taken place do not prove that these decisions are taken by a force other than the observer – hence eliminating “free will” – but just that they are prepared “pre-thresholdly,” before the observer considers the decision as taken.
Visual awareness, a specific form of consciousness, is challenging to approach experimentally (Myerson et al., 1981; Crick and Koch, 1995; Bhardwaj et al., 2008). One of the few suitable paradigms is binocular rivalry, the alternation of visual consciousness resulting when the two eyes view differing stimuli (Blake and Logothetis, 2002; Alais and Blake, 2005; Kim and Blake, 2005). If a grating presented to the left eye is oriented perpendicularly to that shown to the right eye as in the present study conscious experience alternates between the two orientations (O’Shea and Crassini, 1981; Fahle, 1982) though the stimulus stays constant (Figure 1A). Observers usually have to indicate these gradual changes of conscious perception by a binary measure: pressing one of two buttons, one for the emergence of each grating. Here we argue that analog, or gradual measures better reflect the gradual changes in awareness (and decision processes) than button presses (Naber et al., 2011). We used three measures of visual awareness – button presses, pupil size, and joystick movements. In our experiment, the grating to one eye differed in orientation (provoking rivalry) and luminance (eliciting pupil responses) from that in the other eye (Figure 1A). Differences in stimulus luminance cause differences in pupil size. Because pupil size is similar in both eyes (Ettinger et al., 1991; Miller et al., 2005), we expected pupil size to change depending on which of the stimuli was consciously perceived (Barany and Hallden, 1948). That is to say that both pupils should constrict when observers perceive the brighter grating and enlarge when observers perceive the dimmer grating (Harms, 1937; Lowe and Ogle, 1966; Fahle et al., 2010; Naber and Einhäuser, 2010). This change could serve as an objective correlate of the internal choice between two stimuli both represented in (early) visual cortices (Kovacs et al., 1996; Fang and He, 2005; Tong et al., 2006). And indeed, pupils not only reacted to the transitions between perceived orientations (Fahle et al., 2010; Naber and Einhäuser, 2010), but pupil sizes predicted which stimulus was perceived (Figure 1B). These earlier studies, however, did not discuss the temporal lead of the pupil response and neither did they relate it to decision processes in general.
Figure 1. Stimuli used and pupil reactions obtained during binocular rivalry and between stimuli of differing luminances. (A) Stimuli used to elicit binocular rivalry. An oblique darker grating was projected to one eye, a perpendicular brighter one to the other eye, resulting in alternation of perception between the two stimuli. (B) Pupil reactions (means and SEM for four observers) to a change in subjective perception between the two gratings relative to time of button presses during binocular rivalry. The black line shows the relative pupil response for transitions from bright to dark and the gray line transitions from dark to bright. Pupil constrictions start on average about 590 ms before the button presses (gray arrow); pupil dilations start about 120 ms before the button presses (for movement onset estimates see Bergamin and Kardon, 2003). Hence the pupil reacts to internally triggered transitions between percepts much earlier than button presses do. (C) Pupil responses to a change in physical stimulus properties in both eyes from dark to bright or vice versa. The bar indicates the time of change of physical stimulus properties. It is an interval rather than a fixed point in time because data were averaged relative to button presses and reaction times vary slightly (both intra-individually and between observers). The constriction starts about 120 ms before the button is pressed (gray arrow); dilation starts only marginally before the button press. Hence reaction times for pupil responses and button presses are quite similar for externally caused changes of perception. The lead of pupil responses in (B) is not mainly due to a faster reaction time of the pupil.
The pupil starts to change around 590 ms before observers signal changes in conscious perception by pressing a button, not just in our data, but also to be found – at least for dilations – for other types of bi-stable stimuli (Einhauser et al., 2008). That is, the pupil seems to access even sub-threshold, or pre-conscious processes. However, the lag of the behavioral response relative to the pupil response disappears if observers move a joystick rather than press a button. Hence decision processes seem to require some processing time, building up over time rather than being all-or-none events and hence require gradual measurements rather than binary ones such as button presses (Soon et al., 2008). Averaging analog measures such as pupil diameter or the EEG identifies even pre-threshold portions during the built-up of decision processes. This insight may prevent the misinterpretation of data demonstrating changes in analog measures such as the EEG occurring before binary decisions are consciously taken (Libet, 1985). These data were interpreted by some as indicating that humans do not have a “free will” since the changes in (analog) EEG potentials preceding (binary) button presses were interpreted as produced by a force independent from the observer proper, while the observer seemed to be “informed” about the decision only after a decision had been taken by this independent force (whatever this force may be; e.g., Libet et al., 1999).
The pupil response for a subjective switch to a brighter target started around 590 ms (±30 ms SEM) before the button presses by which observers indicated this change in subjective percept – even though observers were instructed to react as fast as possible (Figure 1B; Einhauser et al., 2008; Hupe et al., 2009; Alais et al., 2010). To rule out the possibility that the pupil responses are faster than button presses, we performed a first control experiment. Both eyes viewed the same grating that changed orientation and luminance simultaneously in both eyes at pseudo-random intervals (Figure 1C). Then, pupil constrictions and dilations occurred with latencies around 265 and 305 ms, respectively, after the change in stimulus orientation and luminance – only marginally before the button presses. Hence, the “lead” of pupil response in the first experiment is not primarily due to a faster response-time of the pupil as compared to the finger. Since the pupil reactions during binocular rivalry are about one fourth of the ones elicited by switching physically between the same stimuli. Therefore, as with the visually evoked potentials (VEP), averaging is required to obtain clear results. A prediction of which eye dominates during binocular rivalry, based on online pupil size, yields only between around 60% (Crouzet et al., 2011) and 70% correct responses (Naber et al., 2011 and our own data), depending on exact experimental conditions as well as on subjects (cf. also Kreiman et al., 2002; Fried et al., 2011).
We were tempted to conclude that the pupil knows something about the unconscious planning of cognitive events – in this case the internally generated decision to switch conscious perception between stimuli – that the owner of the brain does not know yet (Fahle et al., 2010). However, the apparent temporal lead of analog measures such as brain potentials and pupil size relative to button presses may rather be an artifact caused by the comparison between averaged continuous versus discontinuous signals (button presses or precise clock position; Libet, 1985). Such a comparison is in a way unfair. To press a button, a discontinuous (yes/no) decision is made on the basis of quite noisy (internal) processes which require that the signal has to pass a threshold. If the internal process fails to reach threshold, it fails to leave any trace. Pupil responses and brain potentials, on the other hand, are retained even if they fail to reach a threshold and can be averaged over time. For a fairer comparison between pupil and behavioral responses, we asked subjects in a second control experiment to move a joystick between left (one orientation dominates completely) and right (the other orientation dominates completely) with all possible in-betweens. This measure captures early parts of transitions as well as incomplete transitions. The results show a gradual transition in visual awareness that requires, on average, almost 1000 ms (shaded area in Figure 2A1). In this second control experiment the pupil constricts with a time course very similar to the joystick response (while the dilation is somewhat slower; Figure 2A2), and very similar to the main experiment (Figure 1B). This similarity in time courses of pupil responses under different experimental conditions allows one to compare reaction times between these conditions, and especially between button versus joystick responses. Button presses occurred, on average, at about the middle of the joystick transition time. In other words, observers pressed the buttons in the main experiment at about the time when they had used half of the transition time between the outer joystick positions in the control experiment (compare Figure 1B with Figure 2A2). This interpretation receives further support from the comparison between button presses and joystick responses to physical stimulus changes (compare Figure 2B1 with Figure 2B2). The joystick transitions for these physical changes of both stimuli had latencies comparable to those of button presses and pupil responses [compare Figure 2B1 (time to mid-interval) with Figure 1C (time to button press)].
Figure 2. Joystick and pupil responses obtained during binocular rivalry and between stimuli of differing luminances. (A) Joystick position and pupil responses to subjective changes in perceived grating orientation relative to joystick responses. Time zero is defined as half of the movement duration (not the mid position of the joystick which occurs earlier). This midpoint corresponds rather well with the time of button presses. (A1) Joystick position. The transition between the two percepts requires on average 928 ms in both directions (shaded area), and joystick responses start about 460 ms (left side of shaded area) before the joystick reaches its midpoint, mirroring the relative slowness of the perceptual transition. (A2) Pupil constrictions (which are known to be faster than dilations, Miller et al., 2005) start at about the same time as joystick responses [see (A1)]. Hence the apparent lead of pupil responses over behavioral responses disappears if a continuous measure is taken rather than a discontinuous one (button presses). (B) Joystick and pupil responses to physical changes of stimuli. (B1) Joystick responses relative to physical stimulus changes which took place within the shaded area. Latencies when expressed as midpoints of the joystick movement are very similar to those for button presses. Joystick movements are much faster here than for rivalrous transitions, reflecting the fact that the transition here is instantaneous (external) rather than gradual (internal; rivalrous). (B2) Pupil constrictions caused by physical stimulus transitions start at the same time as joystick movements [see also (B1)].
To push the button or to move the joystick several internal thresholds must be passed. First a change in stimulus must be detected. Secondly, an internal decision criterion must be reached and third the motor threshold must be passed to initiate the movement. To cross these three thresholds and to move the hand requires about 200 ms (initial Joystick movement) or 400 ms (Button) for physical stimulus changes. We assume that reaching the third, the motor threshold; will follow the same time course also during rivalry. Comparison between the data for button presses versus joystick movements shows that the delay of responses in the initial button press experiment is not due to the fact that the stimulus change stays undetected. Quite to the contrary, the change is detected and indicated by a joystick movement, i.e., the first threshold is crossed fast. It is the second threshold, a cognitive one, which produces the delay: participants push the button not before the perceived stimulus change crosses an internal decision criterion, or threshold, which corresponds to a relative dominance (50 or more percent) of the new stimulus.
We infer from these results that the internal decision process during binocular rivalry – switching between the input of one eye to the input of the partner eye, clearly is not an abrupt one, occurring within a few milliseconds, but one that gradually builds up over a time course of about a second – possibly due to the piecemeal nature of the rivalry process and due to the incomplete inhibition between the two eyes or stimuli during the gradual transition time. The completely endogenously generated switching process during binocular rivalry may be an example of decision processes in general, with the advantage of being relatively slow and directly observable, since it relates to the decision between two different stimuli. The time difference between the start of the pupil response and the pressing of the button is not due to pupils having access to signals predating the conscious switch from one percept to its alternative. Rather, averaged analog signals allow one to detect imminent internal decisions earlier than a binary decision that has to be taken on the basis of a noisy trial-by-trial signal (Soon et al., 2008). This interpretation relates to the results of Libet (1985) who investigated a different type of internally generated decision processes. In his experiments, subjects were asked to press a button at irregular intervals, performing what Libet calls “freely voluntary, fully endogenous motor acts.” During the experiments, subjects watched a revolving spot and were asked to recall the spatial “clock position” of this spot at the time when they became first aware of their decision or intention to move their finger. Libet found cortical potentials starting around 300–500 ms before the time at which subjects had consciously made the decision to press the button – i.e., these potentials were pre-conscious. Libet and others hypothesized that the brain makes a decision before the owner of the brain actually becomes aware of this decision (van de Grind, 2002; Wegner, 2003; Haggard, 2005). This interpretation would have significant consequences for theories of decision making including, as some argue, the concept of free will. But based on our own results, we would not jump to such conclusions, as outlined above. We would rather argue that also in the case of Libet’s experiments likewise a certain proportion of decision processing are started (“Maybe I should press the button now?”), but are abandoned before the button is actually pressed (“I’d rather wait a little longer”). Under these circumstances, subjects would wait, in a way analogous to the situation during binocular rivalry, until they were sufficiently sure that the decision process just started would, indeed, lead to a button-press and, hence, press the button clearly after the decision process started. Some indicators of actions to be taken can be detected at much longer lead times than the ones found in Libet‘s as well as our experiments (Soon et al., 2008), up to 10 s. These indicators presumably reflect activity in high level control areas of the cortex that prepare actions in a way even more basic (and possibly completely unaware for the subject), and are (therefore?) far less reliable than the ones we measured here.
For a quantitative comparison between the binary versus analog response times, we measured the mean transition times of the joystick response of all observers to be 928 ms (± 51 ms SEM), and the rate of incomplete or interrupted joystick moves (i.e., those not even reaching the mid position, see movement “3” in Figure 3A to be 24% (± 3% SEM). The earlier the subjective percept moves back to the initial orientation or the shorter the interval between subsequent physical stimulus changes, the smaller becomes the joystick movement. As can be seen in Figure 3, the relative probability of all these partial movements does not differ much between all possible intervals. This is time both for purely perceptual changes (Figure 3B) as well as for physical changes (Figure 3C). However, in the latter case, the overall probability is much reduced since intervals below 1 s were relatively rare. From the results above, one can conclude that it takes on average 464 ms to complete half of the transition between percepts, and to perceive as dominant the competing stimulus. This interval corresponds nicely to the time difference between the beginning of pupil and joystick response on one hand and the button press on the other hand. Around 24% of incomplete transitions obviously prevent the subjects from signaling, by button press, the very start of the transition, since they cannot be sure whether this beginning transition will indeed lead to a dominance of the competing stimulus. This uncertainty results in very similar latencies for button presses and the middle of joystick transition time: observers press the button when the competing stimulus becomes dominant, not when the “previous” one starts to fade. Incidentally the speed of change in incomplete decision processes does not differ from those of complete ones and is not related to the frequency of switches in individual observers. Our results are in good agreement with single cell and field potential studies in monkeys that found neurons in cortical areas on several levels of the visual pathway reflecting the perceptual switches of binocular rivalry (Logothetis and Schall, 1989; Leopold and Logothetis, 1996) which in turn may influence the subcortical centers regulating pupil size (Barbur, 2004).
Figure 3. Complete versus incomplete reversals. (A) Example for interrupted joystick movements, for an arbitrary 50 s period (x-axis). Joystick position “1” signals a complete percept of a dark grating and position “−1” that of a bright grating (y-axis). Movement “1” is considered as a sufficient transition (>60%), “2” is still considered as a sufficient reversal (>50%) but not included in pupil analysis while “3” is considered as an incomplete reversal (<50%), i.e., it does not qualify as a transition. (B) Percentages of complete versus incomplete joystick movements for changes during rivalry. Movements of less than half amplitude are considered as “incomplete” here since they indicate that percepts did not switch sufficiently to the competing stimulus or else the dominance of the competing stimulus was too short lived. (C) Percentages of complete versus incomplete joystick movements for physical stimulus changes. Note that the Poisson distribution determining physical changes contained dominance times down to 20 ms, leading observers to change joystick-direction before the reached the endpoint.
We conclude that (a) the pupil is a valid objective correlate of subjective perceptual changes in binocular rivalry (Naber and Einhäuser, 2010; Crouzet et al., 2011), (b) the start of pupil reactions predates the button presses of subjects by about 590 ms, and (c) this time difference is not due to different motor response times, but (d) is due to averaged analog responses starting before all – or none (binary decisions) are taken – the latter requiring a certain threshold to be reached. Therefore, perceptual decisions during binocular rivalry require almost 1000 ms to develop fully (Wilson et al., 2001; van Ee et al., 2005). These results bear consequences for the interpretation of a number of similar experiments that compare analog responses (such as averaged brain responses; Morgan, 2005) with binary ones such as button presses or memorizing the exact position of a clock’s hand. We conclude that extreme care should be taken not to over-interpret such comparisons between continuous and discontinuous indicators. In addition, we conjecture that the pupil seems to be a promising candidate for an objective measure of subjective phenomena such as binocular rivalry.
Materials and Methods
All observers had normal or corrected-to-normal vision and gave written informed consent to participate. All procedures conformed with national and institutional guidelines and the Declaration of Helsinki. Observers saw stimuli at a distance of 0.6 m on a LED monitor (Zalman Trimon 2D/3D 22″) with polarizing filters of opposite circular polarization for odd and even pixel lines and matched filters in front of both eyes. Heads were stabilized with a headrest, the room was darkened and special care was taken to prevent scattered light. Stimuli had a diameter of 4°, a spatial frequency of 2 cycle/° and a Weber-contrast of eight relative to the background (luminances of 2.8 versus 108 cd/m2). Observers (n = 4) looked at a central fixation point and indicated the change of the prevailing orientation of their percept as fast as possible by pressing the corresponding button in the first experiment and by moving a joystick between left and right in the control experiments. Transitions between grating orientations and the correlated luminance differences were caused either by internal decision processes (binocular rivalry) or by changing the stimulus orientation and luminance of the stimulus at random intervals (Poisson distribution) between one frame and the next. The start of pupil reactions and joystick movements was determined by the positive peak of the second derivative, the end of joystick movement by the negative peak of this derivative (Bergamin and Kardon, 2003). Observers were tested for 10 min twice for each experiment. Each data point in the graphs relies on a least 200 reports from each observer. Details regarding the recording and analysis of the data as well single subject results are to be found under Sections “Recording” and “Analysis.” A separate pilot experiment with six additional observers reproduced the main results, i.e., reaction of pupils before button presses and simultaneous pupil and joystick reaction during rivalry. (Results not shown, since not all observers participated in all conditions). Data analysis was implemented in Matlab (MathWorks, Natick, MA, USA).
We used the standard joystick for the pilot experiments but preferred the “throttle” for the experiment proper since it has a linear mechanical characteristic without favoring the middle position. Resolution of the joystick movement was 8 bit (256 levels).
The right eye was illuminated by means of two infrared LEDs and recorded through a CCD camera (Watec 902 H3 supreme) at a rate of 50 Hz and a spatial resolution of 752 × 582. A computer program developed in house fitted the pupil by an ellipsoid and calculated its center and diameter as well as the positions of the Purkinje reflexes.
Data were smoothed by Gaussian filtering (half-width = 120 ms) to reduce noise. Start and end of the joystick movement were defined, for constrictions, as the negative and positive peak of the second derivative, respectively; hence both start and end of the movement were objectively determined. The slope midpoint is defined as half the time between start and end of the movement. This midpoint is supposed to correspond to the time of the button press. Indeed, these two measures correspond nicely to each other if they are compared on the basis of the corresponding pupil responses. Joystick movements that did not reach at least 50% of the maximum joystick amplitude were counted as partial or incomplete transitions and those below 60% were not included in the analysis of pupil responses (see Figure S5 in Supplementary Material). Transitions with small discontinuities of movement but without a change in direction were considered as one (slower) movement. The data were epoch-based z-transformed and averaged. The following analysis was the same as for the pupil data.
Data points lost for example due to eye blinks were extrapolated by a polynomial function. No data points were discarded. Subsequently the whole data set was smoothed with a 7 point median filter. Averaging into “epochs” was relative either to the button response or the midpoint of the joystick movement, supplying means, and SEs of the means of pupil size for the two types of pupil transitions caused by rivalry or stimulus changes. Each epoch was normalized via z-transformation (z-score), see Figure S1a in Supplementary Material. A polynomial function was fitted to the averaged data, separately for transitions from dark to bright versus bright to dark. The second derivative of the fit function served to identify the exact start of the pupil response. z-Scores of pupil diameter were finally transformed to arbitrary units for better comparability in the graphs by setting the difference between maximum and minimum values for the averaged epoch to “unity.”
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We would like to thank Dennis Trenner and Sven Eberhardt for help with programming and data analysis and members of the group for discussions. Supported by German Research Council.
The Supplementary Material for this article can be found online at http://www.frontiersin.org/human_neuroscience/10.3389/fnhum.2011.00120/abstract
Barbur, J. L. (2004). “Learning from the pupil: studies of basic mechanisms and clinical applications,” in The Visual Neurosciences, eds L. M. Chalupa, and J. S. Werner (Cambridge, MA: MIT Press), 641–656.
Einhauser, W., Stout, J., Koch, C., and Carter, O. (2008). Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc. Natl. Acad. Sci. U.S.A. 105, 1704–1709.
Ettinger, E. R., Wyatt, H. J., and London, R. (1991). Anisocoria. Variation and clinical observation with different conditions of illumination and accommodation. Invest. Ophthalmol. Vis. Sci. 32, 501–509.
Keywords: binocular rivalry, decision making, pupil, conscious visual perception
Citation: Fahle MW, Stemmler T and Spang KM (2011) How much of the “unconscious” is just pre – threshold? Front. Hum. Neurosci. 5:120. doi: 10.3389/fnhum.2011.00120
Received: 15 July 2011; Accepted: 04 October 2011;
Published online: 21 October 2011.
Edited by:Alexander Maier, Vanderbilt University, USA
Reviewed by:Olivia Carter, University of Melbourne, Australia
Gabriel Kreiman, Harvard Medical School, USA
Copyright: © 2011 Fahle, Stemmler and Spang. This is an open-access article subject to a non-exclusive license between the authors and Frontiers Media SA, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and other Frontiers conditions are complied with.
*Correspondence: Manfred W. Fahle, Human Neurobiology, Centre of Cognitive Science, Bremen University, Hochschulring 18, 28359 Bremen, Germany. e-mail: firstname.lastname@example.org