Original Research ARTICLE
The Neural Basis of Individual Face and Object Perception
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, Netherlands
We routinely need to process the identity of many faces around us, and how the brain achieves this is still the subject of much research in cognitive neuroscience. To date, insights on face identity processing have come from both healthy and clinical populations. However, in order to directly compare results across and within participant groups, and across different studies, it is crucial that a standard task is utilized which includes different exemplars (for example, non-face stimuli along with faces), is memory-neutral, and taps into identity matching across orientation and across viewpoint change. The goal of this study was to test a previously behaviourally tested face and object identity matching design in a healthy control sample whilst being scanned using fMRI. Specifically, we investigated categorical, orientation, and category-specific orientation effects while participants were focused on identity matching of simultaneously presented exemplar stimuli. Alongside observing category and orientation specific effects in a distributed set of brain regions, we also saw an interaction between stimulus category and orientation in the bilateral fusiform gyrus and bilateral middle occipital gyrus. Generally these clusters showed the pattern of a heightened response to inverted versus upright faces, and to upright, as compared to inverted shoes. These results are discussed in relation to previous studies and to potential future research within prosopagnosic individuals.
It has long been understood that faces are special. From birth we are drawn to faces and recognizing and responding to the information contained within them is something we are particularly good at. For decades now, researchers have attempted to understand the specific cognitive and neural mechanisms underscoring face perception. Combined evidence for the ‘specialness’ of faces has come from a number of different experimental sources, including cognitive psychology, and cognitive and clinical neuroscience.
Cognitive psychology experiments have revealed phenomena such as the part-whole effect, which shows that it is easier to recognize a face part when it is presented as part of a whole face, rather than on its own; the composite effect, which shows that people have difficulties recognizing the top half of a face if it is aligned with a non-corresponding bottom half; and also the face-inversion effect (FIE), where recognition of inverted faces is less accurate than recognition of upright faces. These effects have been used to demonstrate the holistic nature of face processing, which appears to be more marked for faces as compared to other objects (e.g., Yin, 1969). The FIE is of particular interest as compared to the composite and part-whole effects, it appears to tap into a more basic level of configural processing. To explain the effect in more detail, it has been proposed that faces are processed and stored as perceptual wholes, as opposed to a configuration of different individual parts. Thus, when faces are rotated away from an upright orientation holistic processing is impaired, resulting in an inversion effect (e.g., Tanaka and Farah, 1993; Farah et al., 1995a).
Many studies using neuroimaging techniques have highlighted regions with pronounced selectivity for faces brain. Particularly well-documented is an area in the lateral fusiform gyrus where the activity in response to faces is consistently found to be greater than that evoked by non-face objects, which was named the ‘fusiform face area’ (FFA; Kanwisher et al., 1997). Furthermore, some neuroimaging studies have found this region to be affected by orientation manipulations, reporting that the FFA exhibits a greater response to images of upright faces than to images of inverted faces (e.g., Kanwisher et al., 1998; Yovel and Kanwisher, 2005; Passarotti et al., 2007; but also note Aguirre et al., 1999; Haxby et al., 1999; Leube et al., 2003 who did not find such an effect). Although a number of studies have focussed predominately on the FFA, face processing appears to depend on a network consisting of several dedicated cerebral modules, including the occipital face area (OFA), superior temporal sulcus (STS), and face selective areas in the anterior temporal lobe (ATL) and prefrontal cortex (e.g., Kanwisher et al., 1997; Gauthier et al., 2000; Axelrod and Yovel, 2015). An influential model detailed by Haxby et al. (2000) proposed that different regions involved in the face processing network are engaged specifically for different aspects of face processing – for example, face identity processing and face expression processing. This is also supported by electrophysiological research in non-human primates which has highlighted neurons in distinct brain regions that are tuned to expression and orientation, and others identity (Hasselmo et al., 1989; Eifuku et al., 2004).
Furthermore, clinical neuroimaging studies have described patients with selective impairments in the identification of faces. This impairment is referred to as prosopagnosia, a deficit of familiar face recognition. Prosopagnosia can be either acquired – for example, after a brain injury – or developmental, where there is no clear underlying cause; however, in both cases the individuals can perceive a face for a face, but are specifically unable to recognize its identity. In fact, the earliest insights into the neural mechanisms underlying the ability to recognize face identity came from the study of patients with selective impairment for the recognition of faces, with the right occipito-temporal cortex emerging as a common location of the lesion in prosopagnosic patients (e.g., Damasio et al., 1982; Tranel et al., 1997; for a review, see de Gelder and Van den Stock, 2015). It should be noted that support for this region’s involvement in identity processing has also come from functional imaging studies in non-prosopagnosic participants utilizing techniques such as fMR-adaptation (fMR-A; e.g., Grill-Spector et al., 1999; Winston et al., 2004; for a review, see Anzellotti and Caramazza, 2014). A number of studies have now focussed on investigating the FIE in prosopagnosic patients. Specifically, the FIE has been used to assess whether such individuals show impairments in configural perception, and how this is related to feature processing abilities. In theory, one might expect that prosopagnosic patients should show a reduced FIE, or that it should not be present. However, a range of different outcomes have been observed, spanning from a normal effect in one patient, to a reduced/absent effect or even an “inverted” inversion effect in others (see Busigny and Rossion, 2010, for a discussion). In particular, the presence of an inverted inversion effect, or paradoxical inversion effect (specifically, a better performance for inverted faces, e.g., Farah et al., 1995b; de Gelder et al., 1998; de Gelder and Rouw, 2000; Schmalzl et al., 2009) suggests that the prosopagnosic deficit is not simply a loss of configuration perception and its replacement by feature processing, as upright and inverted faces are still being processed differently. Importantly though, Busigny and Rossion (2010) note that discrepancies in results across these studies investigating the FIE may in part be due to differences in patient selection (e.g., using patients with differing general vision ability), tasks utilized, and behavioral measurements that are acquired. Furthermore, to date the neuroimaging evidence regarding the FIE effect in prosopagnosic patients is scarce. This could provide a valuable complement to existing research.
Related to this, it is important to note that when assessing face processing ability in prosopagnosia, several aspects should be taken into consideration. First of all, prosopagnosic individuals are specifically unable to recognize the identity of individual faces. Therefore, presenting a series of single faces and objects, without any focus on identity recognition or matching, merely taps into simple face and object perception. Specifically, when used in combination with fMRI, such a design may inform us about the neural basis of face perception but not about face identity processing. Furthermore, individuals with intact face-recognition skills are usually very well able to distinguish between faces of differing identities, but also non-face stimuli such as cars, houses, shoes, and so forth. What makes these perceptual judgements instances of genuine object recognition is that we can recognize the same object when seen from different viewpoints. Indeed, a hallmark of object recognition is viewpoint invariance, where an object is recognized independently of the viewpoint under which it is presented. To take this into account it is important to prevent “physical identity matching,” which can occur when two identical images are presented together. Experiments designed for prosopagnosic individuals should thus not only present multiple stimuli at the same time, but also from different viewpoints. Furthermore, even if studies do focus on exemplar recognition, they often use one-back or delayed match-to-sample paradigms (Kanwisher et al., 1998; Haxby et al., 1999; Yovel and Kanwisher, 2005; Epstein et al., 2006). Face memory deficits in prosopagnosia have been well documented over the years, but even a short delay in stimulus presentation is disproportionally more detrimental to both the accuracy and reaction times of prosopagnosics as compared to controls (Shah et al., 2015).
In order to study face and object perception with a task suitable for both healthy and prosopagnosic participants, we previously constructed a behavioral face and object perception test battery including an face and object identity matching task, which is a match-to-sample task with upright and inverted faces and shoes (de Gelder and Bertelson, 2009; Huis in’t Veld et al., 2012). In this task, match-to-sample is tested using simultaneously presented sample, target and distractor stimuli, and faces and objects are presented both in the habitual upright and an upside-down orientation, and across varying viewpoints: the sample is seen in frontal view, the target and distractor from a three fourths profile view. This design is optimal as it is memory-neutral, and at the same time uses faces as well as objects, and targets identity matching across orientation and across viewpoint change.
The goal of the current study was to test a version of this face and object identity matching design in a healthy control sample whilst being scanned using fMRI. Here, we were particularly interested to investigate categorical and inversion effects on brain activity; specifically, whether faces and objects have a different neural substrate, whether this is sensitive to orientation, and/or there is neural evidence in one or more areas for a category specific inversion effect, while participants were focused on identity matching of simultaneously presented exemplar stimuli.
Materials and Methods
Sixteen healthy participants (eight males, age range 19–27 years) participated in the study. All had normal or corrected-to-normal vision, and no history of neuropsychiatric disorders. The experiment was approved by the Ethics committee of Maastricht University, and written informed consent was obtained before participation. Participants were screened for fMRI experimentation safety and received monetary compensation for their participants.
Stimuli and Task
The study consisted of eight experimental conditions with a 2 (category: faces, shoes) by 2 (orientation: upright, inverted) by 2 (congruency: same, different identity) design. The materials consisted of greyscale photographs of shoes (12 unique shoes) taken from the faces and objects matching test (de Gelder and Bertelson, 2009) and faces (six male, six female; all with a neutral facial expression) from a database created at Tilburg University. Each face and each shoe were photographed once in front view and once in three-quarter profile view.
A trial consisted of one frontal view picture and one three-quarter profile view picture, placed equidistant from a center fixation point, in a counterbalanced way. The span of the two visual images was 16.67 cm by 12.5 cm in total, presented at a visual angle of 12.54°. This presentation was adapted from the behavioral version of the task which consists of a triangular presentation of three stimuli. This adaptation was an attempt to limit excessive eye movement, or saccades of participants, which although cannot be eliminated via simultaneous stimuli presentation, could at least be reduced. Each trial was presented for 800 ms. Participants were asked to fixate at the black center fixation cross between the stimuli and to concentrate on whether the two stimuli were of a matching identity or not, but no response was required (see Figure 1, for examples of the stimuli used). Stimuli were presented using Presentation software (Neurobehavioural Systems, San Francisco, CA, USA).
FIGURE 1. Examples of stimuli from main experiment. Each trial within a block consisted of a simultaneous presentation of two within-category stimuli: Matching upright faces; Different upright faces; Matching inverted faces; Different inverted faces; Matching upright shoes; Different upright shoes; Matching inverted shoes; Different inverted shoes.
The experimental design was blocked, with four blocks per condition (32 blocks in total) and 12 trials per block. The order of blocks was pseudo randomized; additionally the order of the trials within each block was randomized. Within blocks, the inter-trial interval was 200 ms. Time between blocks was 12000 ms.
MRI Parameters and Functional Data Processing
MRI was performed using a 3-Tesla Siemens Trio scanner (Siemens, Erlangen, Germany). Both high-resolution anatomical [T1-weighted, flip angle (FA) = 9°, TR = 2250, TE = 2.6 ms, 192 slices, field of view (FoV) = 256 mm, isotropic voxel resolution of 1 mm × 1 mm × 1 mm] and whole-brain functional images [T2*-weighted echo-planar imaging: TR = 2000, TE = 30 ms, 35 contiguous slices, slice thickness = 3 mm, voxel resolution = 3 mm × 3 mm × 3 mm] were obtained. Participants’ hearing was protected using earplugs, and head movement was restricted using foam pads.
FMRI data were processed using BrainVoyager QX (Brain Innovation, Maastricht, The Netherlands). Pre-processing included slice acquisition time correction, temporal high-pass filtering, rigid-body transformation of data to the first acquired image to correct for motion, and spatial smoothing. Functional data were co-registered to anatomical data per subject, and further transformed to Talairach space.
Activation Data Analysis
BOLD time courses of 12 s individual blocks were regressed onto a pre-specified model in a conventional GLM. Separate predictors were implemented for four different conditions: Faces-Upright; Faces-Inverted; Shoes-Upright; Shoes-Inverted.
At the group level, we performed a 2 (Category: Face and Shoe) × 2 (Orientation: Upright and Inverted) repeated-measure RFX ANOVA. This analysis allowed us to observe main effects of all three factors, as well as interactions between the conditions. Results were thresholded at p < 0.05 FDR corrected for cluster size.
A main effect of category was found in a wide range of frontal, occipital, temporal, and parietal regions, in addition to the cerebellum, and cingulate and insular cortices. Face specificity was seen in the right fusiform gyrus, right precentral gyrus, right middle frontal gyrus, and right lingual gyrus. For interest, we further investigated these categorical effects using a direct comparison between upright faces and shoes only: here, face specificity was observed in the right inferior frontal gyrus, fusiform gyrus, and left lingual gyrus, whereas shoe (object) specificity was seen in the bilateral fusiform gyrus, extending to the parahippocampal gyrus, and left cuneus. A main effect of orientation was observed in the right postcentral gyrus, middle frontal gyrus, precentral gyrus, claustrum, fusiform gyrus/cerebellum, thalamus, midbrain, posterior cingulate cortex, cuneus, cingulate gyrus and left lentiform nucleus, insula, precentral gyrus, claustrum and precentral gyrus. Generally, all these regions showed a higher response to upright, as opposed to inverted stimuli. We additionally examined inversion effects across category: upright vs. inverted faces elicited activity in the right fusiform gyrus and lingual gyrus, whereas the reverse contrast showed activity in a more posterior region of the right fusiform gyrus. Comparing the response to upright vs. inverted shoes showed higher activation in one cluster in the middle occipital gyrus, whereas the reverse contrast showed no significant activation. Finally, an interaction between both factors emerged in four specific clusters, specifically in the bilateral fusiform gyrus and bilateral middle occipital gyrus. In all of these clusters, the general pattern was that a stronger response was observed for inverted, as opposed to upright faces; and the converse pattern for shoes. All activated clusters are detailed in Tables 1–3. Interactions are illustrated in Figure 2.
FIGURE 2. Two-way interaction between category of stimulus, and stimulus orientation, within a whole brain search. Plot shows average beta value within cluster for each point measured. Coordinates are in Talairach space.
In this study, the purpose was to investigate the neural correlates of face and object perception using a memory-neutral design utilizing both faces and objects of different identities and viewpoints; a design we proposed would be ideally suited for use not only in healthy but also prosopagnosic individuals. Specifically, we aimed to establish the correlates of face-selective effects using this task firstly in participants without any face processing impairments, with the future intention of comparing these results to those with prosopagnosia.
Category and Orientation Specificity
Regarding category specificity, in a whole brain search we observed a number of distinct regions that responded more to faces as compared to shoes, averaged across orientation: specifically, the fusiform gyrus, precentral gyrus, middle frontal gyrus, and lingual gyrus. Furthermore, when we performed a direct contrast between upright face and shoe stimuli, three clusters with peaks in the inferior frontal gyrus, lingual gyrus, and fusiform gyrus emerged. The observed regions have previously been implicated in the face processing network. For example, face-selective activation in the fusiform gyrus corresponded in location to the well-documented FFA (Kanwisher et al., 1997). Furthermore, the lingual gyrus has been linked to the very early stages of facial processing (Luks and Simpson, 2004), and the precentral gyrus has also been proposed to be part of a large brain network for face recognition (Zhen et al., 2013). Additionally, we found several areas sensitive to the orientation of the stimuli, averaged across category. These included both cortical (e.g., right middle frontal gyrus, bilateral precentral gyrus, right cingulate gyrus, left insula, and right fusiform gyrus) and subcortical (bilateral claustrum, left lentiform nucleus, and thalamus) regions. It should be noted, that interestingly in all of these regions (aside from the right middle frontal gyrus, precentral gyrus, fusiform gyrus, and thalamus) the effect appeared to be driven by a deactivation in response to inverted stimuli, whether these were faces or shoes. However, a different pattern was seen in the category specific inversion effects, as detailed below.
Category Specific Inversion Effects
In order to explore more the specific effect of stimulus orientation and its relation to stimulus category, we further computed an interaction between these two factors. Four distinct clusters emerged, specifically in the bilateral middle occipital gyrus, and bilateral posterior fusiform gyrus. All these clusters showed the general pattern of responding more to inverted, as opposed to upright faces; additionally, in all of the clusters aside from the left fusiform gyrus there was a contrasting pattern for shoes, in that there was a heightened response to upright as opposed to inverted shoes. The involvement of both purported “face-selective” and “object-selective” cerebral regions in the FIE has been the focus of a number of studies. Some studies demonstrated that the FFA exhibited a greater response to images of upright faces than to images of inverted faces (e.g., Kanwisher et al., 1998; Yovel and Kanwisher, 2005; Passarotti et al., 2007). However, these effects have often been small, and have also not been replicated other studies (e.g., Aguirre et al., 1999; Haxby et al., 1999; Leube et al. 2003), with authors failing to find differences in these two types of stimulus presentation in this region. Interestingly, amongst other clusters we did find in this study that a region within the right fusiform gyrus, appearing to correspond with the FFA, showed a heightened response for upright vs. inverted face stimuli; however, no interaction between category and orientation was observed in this region. A range of studies have now also shown that object recognition and object selective areas [for example, in the lateral occipital complex (LOC)] exhibit greater responses to inverted faces than to upright faces (Aguirre et al., 1999; Haxby et al., 1999; Yovel and Kanwisher, 2005; Epstein et al., 2006). Specifically, Haxby et al. (1999) saw that the only selective effect of face inversion was an increase in activity in extrastriate cortical regions that responded more to houses than to faces. At the same time, effects of face inversion in face-selective regions, and house inversion in house-selective regions, were both small. Similarly, Epstein et al. (2006) found that face inversion lead to greater response in LO and the right middle fusiform object area for inverted versus upright faces but observed no change in the FFA. These results suggest that the presumable additional processing that is required for inverted faces may be undertaken by object-responsive regions. Although we cannot confirm the regions found in our study as ‘face-selective’ or ‘object-selective’ using an independent functional localiser, we would propose the loci of the observed interactions as potentially more object-selective, in line with the studies above.
Single case studies of prosopagnosia following brain damage in adulthood continue to be a very important source of information about the neural basis of face perception. However, findings from different case studies are not always comparable due to differences in fMRI design. To our knowledge, there is not to date a standard task incorporating use of different exemplars of faces as well as of objects, viewpoints, and orientations used in prosopagnosia research. We believe that the present design may help adopting a common platform for such research from where to develop specific hypotheses. Regarding future research, face-selective and inversion effects within object-selective regions are of particular interest for further understanding face-processing in prosopagnosic individuals: for example, these cases could perhaps be expected to show more face-specific activation than controls in object-specific areas, as has been detailed in one study by Dricot et al. (2008) which found activation in an object-specific area when viewing different as opposed to repeated upright faces. In conclusion, we believe that the task detailed in this study provides an interesting starting point for this line of research.
BdG designed the experiment, RW collected and analysed data, BdG, RW and EHIV wrote the paper.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was funded by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement number 295673.
de Gelder, B., Bachoud-Lévi, A.-C., and Degos, J. D. (1998). Inversion superiority in visual agnosia may be common to a variety of orientation polarised objects besides faces. Vision Res. 38, 2855–2861. doi: 10.1016/S0042-6989(97)00458-6
de Gelder, B., and Bertelson, P. (2009). A comparative approach to testing face perception: face and object identification by adults in a simultaneous matching task. Psychol. Belg. 42, 177–190. doi: 10.5334/pb-49-2-3-177
Dricot, L., Sorger, B., Schiltz, C., Goebel, R., and Rossion, B. (2008). The roles of “face” and “non-face” areas during individual face perception: evidence by fMRI adaptation in a brain-damaged prosopagnosic patient. Neuroimage 40, 318–332. doi: 10.1016/j.neuroimage.2007.11.012
Eifuku, S., De Souza, W. C., Tamura, R., Nishijo, H., and Ono, T. (2004). Neuronal correlates of face identification in the monkey anterior temporal cortical areas. J. Neurophysiol. 91, 358–371. doi: 10.1152/jn.00198.2003
Epstein, R. A., Higgins, J. S., Parker, W., Aguirre, G. K., and Cooperman, S. (2006). Cortical correlates of face and scene inversion: a comparison. Neuropsychologia 44, 1145–1158. doi: 10.1016/j.neuropsychologia.2005.10.009
Farah, M. J., Wilson, K. D., Drain, H. M., and Tanaka, J. R. (1995b). The inverted face inversion effect in prosopagnosia: evidence for mandatory, face-specific perceptual mechanisms. Vision Res. 35, 2089–2093. doi: 10.1016/0042-6989(94)00273-O
Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., Itzchak, Y., and Malach, R. (1999). Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron 24, 187–203. doi: 10.1016/S0896-6273(00)80832-6
Hasselmo, M. E., Rolls, E. T., and Baylis, G. C. (1989). The role of expression and identityin the face selective response of neurons in the temporal visual cortex of the monkey. Behav. Brain Res. 32, 203–218. doi: 10.1016/S0166-4328(89)80054-3
Haxby, J. V., Ungerleider, L. G., Clark, V. P., Schouten, J. L., Hoffman, E. A., and Martin, A. (1999). The effect of face inversion on activity in human neural systems for face and object perception. Neuron 22, 189–199. doi: 10.1016/S0896-6273(00)80690-X
Huis in’t Veld, E., Van den Stock, J., and de Gelder, B. (2012). Configuration perception and face memory, and face context effects in developmental prosopagnosia. Cogn. Neuropsychol. 29, 464–481. doi: 10.1080/02643294.2012.732051
Leube, D. T., Yoon, H. W., Rapp, A., Erb, M., Grodd, W., Bartels, M., et al. (2003). Brain regions sensitive to the face inversion effect: a functional magnetic resonance imaging study in humans. Neurosci. Lett. 342, 143–146. doi: 10.1016/S0304-3940(03)00232-5
Luks, T. L., and Simpson, G. V. (2004). Preparatory deployment of attention to motion activates higher-order motion-processing brain regions. Neuroimage 22, 1515–1522. doi: 10.1016/j.neuroimage.2004.04.008
Passarotti, A. M., Smith, J., DeLano, M., and Huang, J. (2007). Developmental differences in the neural bases of the face inversion effect show progressive tuning of face-selective regions to the upright orientation. Neuroimage 34, 1708–1722. doi: 10.1016/j.neuroimage.2006.07.045
Schmalzl, L., Palermo, R., Harris, I. M., and Coltheart, M. (2009). Face inversion superiority in a case of prosopagnosia following congenital brain abnormalities: what can it tell us about the specificity and origin of face-processing mechanisms? Cogn. Neuropsychol. 26, 286–306. doi: 10.1080/02643290903086904
Winston, J. S., Henson, R. N., Fine-Goulden, M. R., and Dolan, R. J. (2004). fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. J. Neurophysiol. 92, 1830–1839. doi: 10.1152/jn.00155.2004
Keywords: face processing, identity recognition, fMRI BOLD, prosopagnosia, categorical perception
Citation: Watson R, Huis in ’t Veld EMJ and de Gelder B (2016) The Neural Basis of Individual Face and Object Perception. Front. Hum. Neurosci. 10:66. doi: 10.3389/fnhum.2016.00066
Received: 01 October 2015; Accepted: 09 February 2016;
Published: 01 March 2016.
Edited by:Nathalie Tzourio-Mazoyer, Université de Bordeaux, France
Copyright © 2016 Watson, Huis in ’t Veld and de Gelder. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Beatrice de Gelder, firstname.lastname@example.org