Visuo-haptic multisensory object recognition, categorization, and representation

Abstract

Visual and haptic unisensory object processing show many similarities in terms of categorization, recognition, and representation. In this review, we discuss how these similarities contribute to multisensory object processing. In particular, we show that similar unisensory visual and haptic representations lead to a shared multisensory representation underlying both cross-modal object recognition and view-independence. This shared representation suggests a common neural substrate and we review several candidate brain regions, previously thought to be specialized for aspects of visual processing, that are now known also to be involved in analogous haptic tasks. Finally, we lay out the evidence for a model of multisensory object recognition in which top-down and bottom-up pathways to the object-selective lateral occipital complex are modulated by object familiarity and individual differences in object and spatial imagery.

INTRODUCTION

Despite the fact that object perception and recognition are invariably multisensory processes in real life, the haptic modality was for a long time the poor relation in a field dominated by vision science, with the other senses lagging even further behind (Gallace and Spence, 2009; Gallace, 2013). Two things have happened to change this: firstly, from the 1980s, haptics has developed as a field in its own right; secondly, from the 1990s, there has been an accelerated interest in multisensory interactions. Here, we review the interactions and commonalities in visuo-haptic multisensory object processing, beginning with the capabilities and limits of haptic and visuo-haptic recognition. One way to facilitate recognition is to group like objects together: hence, we review recent work on the similarities between visual and haptic categorization and cross-modal transfer of category knowledge. Changes in orientation and size present a major challenge to within-modal object recognition. However, these obstacles seem to be absent in cross-modal recognition and we show that a shared representation underlies both cross-modal recognition and view-independence. We next compare visual and haptic representations from the point of view of individual differences in preferences for object or spatial imagery. A shared representation for vision and touch suggests shared neural processing and therefore we review a number of candidate brain regions, previously thought to be selective for visual aspects of object processing, which have subsequently been shown to be engaged by analogous haptic tasks. This reflects the growing consensus around the concept of a “metamodal” brain with a task-based organization and multisensory inputs, rather than organization around discrete unisensory inputs (Pascual-Leone and Hamilton, 2001; Lacey et al., 2009a; James et al., 2011). Finally, we draw these threads together and discuss the evidence for a model of multisensory visuo-haptic object recognition in which representations are flexibly accessible by either top-down or bottom-up pathways depending on object familiarity and individual differences in imagery preference (Lacey et al., 2009a).

HAPTIC AND VISUO-HAPTIC OBJECT RECOGNITION

The speed and accuracy of visual object recognition is well-established. Haptic recognition, albeit less well studied, is somewhat slower than visual recognition, but, at least for everyday objects, is still fairly fast and highly accurate with 96% correctly named: 68% in less than 3 s and 94% within 5 s (Klatzky et al., 1985); indeed, a “haptic glance” of less than 1 s suffices in some circumstances (Klatzky and Lederman, 1995). Longer response times in the study of Klatzky et al. (1985) likely reflect the time taken to explore some of the larger items such as a tennis racket or hairdryer. A remarkable fact about haptic processing is that it can be achieved with the feet as well as the hands, albeit more slowly and less accurately, with hand and foot performance being highly correlated across individuals (Lawson, 2014). Haptic identification proceeds, with increasing accuracy, from a “grasp and lift” stage that extracts basic low-level information about a variety of object properties to a series of hand movements that extract more precise information (Klatzky and Lederman, 1992). These hand movements, known as “exploratory procedures,” are property-specific, for example, lateral motion is used to assess texture and contour-following to precisely assess shape (Lederman and Klatzky, 1987). These properties differ in salience to haptic processing depending on the context: under neutral instructions, salience progressively decreases in this order: hardness > texture > shape; under instructions that emphasized haptic processing, the order changes to texture > shape > hardness (Klatzky et al., 1987). Note that the saliency order under neutral instructions is reversed to shape > texture > hardness/size in simultaneous visual and haptic perception, and in haptic perception under instructions to use concurrent visual imagery (Klatzky et al., 1987).

Overall, cross-modal visuo-haptic object recognition, while fairly accurate, comes at a cost compared to within-modal recognition (e.g., Bushnell and Baxt, 1999; Casey and Newell, 2007; and see Lacey et al., 2007). Cross-modal performance is generally better when visual encoding is followed by haptic retrieval than the reverse (e.g., Jones, 1981; Streri and Molina, 1994; Lacey and Campbell, 2006). This asymmetry appears to be a consistent feature of visuo-haptic cross-modal memory but has generally received little attention (e.g., Easton et al., 1997a,b; Reales and Ballesteros, 1999; Nabeta and Kawahara, 2006). One explanation for cross-modal asymmetry might be that shape information is not encoded equally well by the visual and haptic systems, because of competition from other, more salient, modality-specific object properties. Thus, in the haptic-visual cross-modal condition it might be more difficult to encode shape because of the more salient hardness and texture information, as noted above. This effect might be suppressed by the use of concurrent visual imagery in which shape information, common to vision and touch, might be brought to the fore. We should note, however, that when vision and touch are employed simultaneously, properties that are differently weighted in these modalities may be optimally combined on the basis of maximum likelihood estimates (see Ernst and Banks, 2002; Helbig and Ernst, 2007; Helbig et al., 2012; Takahashi and Watt, 2014).

Another explanation for cross-modal asymmetry could be differences in visual and haptic memory capacity. Haptic working memory capacity appears to be limited and variable, and may therefore be more error-prone than visual working memory (Bliss and Hämäläinen, 2005). Alternatively, haptic representations may simply decay faster than visual representations. Rather than a progressive decline over time, the haptic decay function appears to occur entirely in a band of 15–30 s post-stimulus (Kiphart et al., 1992). Consistent with this, a more recent study showed no decline in performance at 15 s (Craddock and Lawson, 2010) although longer intervals were not tested. Haptic-visual performance might therefore be lower because by the time visual recognition is tested, haptically encoded representations have substantially decayed. However, other cross-modal memory studies show that delays up to 30 s (Garvill and Molander, 1973; Woods et al., 2004) or even a week (Pensky et al., 2008) did not affect haptic-visual recognition more than visual-haptic recognition. Thus, an explanation in terms of a simple function of haptic memory properties is likely insufficient.

Cross-modal asymmetry is observed even in very young infants where it is ascribed to constraints imposed by different stages of motor development (Streri and Molina, 1994). But this explanation is also unsatisfactory since the asymmetry persists into maturity (Easton et al., 1997a,b; Bushnell and Baxt, 1999; Lacey and Campbell, 2006). Interestingly, implicit memory does not appear to be affected: cross-modal priming is symmetric (Easton et al., 1997a,b; Reales and Ballesteros, 1999) although verbal encoding strategies may have played a mitigating role in these studies. A recent study suggests that underlying neural activity is asymmetric between the two crossmodal conditions. Using a match-to-sample task, Kassuba et al. (2013) showed that bilateral lateral occipital complex (LOC), fusiform gyrus (FG), and anterior intraparietal sulcus (aIPS) selectively responded more strongly to crossmodal, compared to unimodal, object matching when haptic targets followed visual samples, and more strongly still when the haptic target and visual sample were congruent rather than incongruent; however, these regions showed no such increase for visual targets in either crossmodal or unimodal conditions. This asymmetric increase in activation in the visual-haptic condition may reflect multisensory binding of shape information and suggests that haptics – traditionally seen as the less reliable modality – has to integrate previously presented visual information more than vision has to integrate previous haptic information (Kassuba et al., 2013).

OBJECT CATEGORIZATION

Categorization facilitates recognition and is critical for much of higher-order cognition (Graf, 2010); hitherto, the emphasis in terms of perceptual categorization has been almost exclusively on the visual, rather than the haptic, modality. More recently, however, a series of studies has systematically compared visual and haptic categorization. Using multi-dimensional scaling analysis, these studies showed that visual and haptic similarity ratings and categorization result in perceptual spaces [i.e., topological representations of the perceived (dis)similarity along a given dimension] that are highly congruent between modalities for novel 3-D objects (Cooke et al., 2007), more realistic 3-D shell-like objects (Gaißert et al., 2008, 2010, 2011) and for natural objects, i.e., actual seashells (Gaißert and Wallraven, 2012). This was so in both unisensory and bisensory conditions (Cooke et al., 2007) and whether 2-D visual objects were compared to haptic 3-D objects (Gaißert et al., 2008, 2010) or passive viewing of 2-D objects was compared to interactive viewing and active haptic exploration of 3-D objects, i.e., such that visual and haptic exploration were more similar (Gaißert et al., 2010). These highly similar visual and haptic perceptual spaces both showed high fidelity to the physical object space [i.e., a topological representation of the actual (dis)similarity along a given dimension; Gaißert et al., 2008, 2010], retaining the category structure (the ordinal adjacency relationships within the category, i.e., the actual progression in variation along a given dimension, for example from roughest to smoothest; Cooke et al., 2007). The isomorphism between perceptual (in either modality) and physical spaces was, furthermore, task-independent, whether simple similarity rating (Gaißert et al., 2008), unconstrained (free sorting), semi-constrained (making exactly three groups) or constrained (matching to a prototype object) categorization (Gaißert et al., 2011). As in vision, haptics also exhibits categorical perception, i.e., discriminability increases sharply when objects belong to different categories and decreases when they belong to the same category (Gaißert et al., 2012).

However, visual and haptic categorization are not entirely alike and, consistent with differential perceptual salience (Klatzky et al., 1987), object properties are differentially weighted depending on the modality, whether they are controlled parametrically (Cooke et al., 2007) or vary naturally (Gaißert and Wallraven, 2012). Shape was more important than texture for visual categorization whereas in haptic and bisensory categorization, shape and texture were approximately equally weighted (Cooke et al., 2007), although in this study shape and texture varied in ways that were intuitive to vision and haptics (broadly, width for shape and smoothness for texture). Using specially manufactured shell-like objects, Gaißert et al. (2010) varied three complex shape parameters that were not intuitive to either modality. While visual and haptic perceptual spaces and the physical object space were all highly similar, the shape dimensions were weighted differently: symmetry was more important than convolutions for vision while the reverse was true for haptics; aperture-tip distance was the least important factor for both modalities (Gaißert et al., 2010). For natural objects – seashells – that varied naturally in a number of properties, similarity ratings and categorization were still driven by global and local shape parameters rather than size, texture, weight etc. (Gaißert and Wallraven, 2012).

These studies suggest a close connection between vision and haptics in terms of similarity mechanisms for categorization but do not necessarily imply a shared representation because of the differential weighting of object properties in each modality. Nonetheless, there is symmetric cross-modal transfer of category information following either visual or haptic category learning, even for complex novel 3-D objects, and furthermore this transfer generalizes to new objects from these categories (Yildirim and Jacobs, 2013). A recent study shows that not only does category membership transfer cross-modally, as shown by Yildirim and Jacobs (2013), but so does category structure (Wallraven et al., 2013), i.e., the ordinal relationships and category boundaries (see Cooke et al., 2007) transcend modality. Crossmodal transfer of category structure is interesting because the ordering of each item within the category is (at least in the studies reviewed here) perceptually driven; thus it may be that a shared multisensory representation underlies cross-modal categorization, as has been suggested for cross-modal recognition (Lacey et al., 2009a; Lacey and Sathian, 2011).

Of course, perceptual similarity is not the only basis for categorization (Smith et al., 1998) and neither vision nor haptics appear to naturally recover categories on alternative bases that are more abstract or semantic. For example, Haag (2011) used realistically textured models of familiar animals that retained real-life size relations, and required visual and haptic categorization on the basis of size (big/small in real life), domesticity (wild/domestic), and predation (carnivore/herbivore). Errors increased as the basis of categorization moved from concrete (size) to abstract (predation) and were consistently greater in haptics than vision (Haag, 2011). Similarly, neither vision nor haptics naturally recovered the taxonomic relationships between the natural seashells used by Gaißert and Wallraven (2012): participants distinguished between concrete categories such as whether the shells used were flat or convoluted, rather than between abstract categories such as gastropods (e.g., sea-snail) vs. bivalves (e.g., oyster). If biological relationships were recovered at all, this was mainly contingent on shape similarities, although vision was better than haptics in this respect (Gaißert and Wallraven, 2012) as it was for the abstract categories studied by Haag (2011).

FACES: A SPECIAL CATEGORY

Faces are a special category of object that we encounter every day and at which we are especially expert, being able to differentiate large numbers of individuals (Maurer et al., 2002). We are also able to recognize faces under conditions that would impair recognition in other categories; for example, bad lighting or changes in viewpoint (Maurer et al., 2002) – though face recognition is impaired if the face is upside-down (Yin, 1969). An important distinction is made between configural and featural processing: the former refers to processing the spatial relationships between individual facial features as well as the shapes of the features themselves, while the latter refers to the piecemeal processing of individual face parts (Maurer et al., 2002; Dopjans et al., 2012). Although sighted humans obviously recognize faces almost exclusively through vision, live faces can also be identified haptically with high levels of accuracy (over 70%), whether they are learned through touch alone or using both vision and touch (Kilgour and Lederman, 2002). Interestingly, when participants had to haptically identify clay masks produced from live faces, accuracy was significantly lower than for live faces, suggesting that natural material cues and surface properties are important for haptic face recognition (Kilgour and Lederman, 2002). Visual experience may be necessary for haptic face recognition, since the congenitally blind were significantly less accurate than both the sighted and the late-blind (Wallraven and Dopjans, 2013). Nonetheless, haptic face recognition is not as good as visual recognition in the sighted either (Dopjans et al., 2012). This may be due to basic differences between visual and haptic processing. Haptic exploration of any object is almost exclusively sequential and serial (Lederman and Klatzky, 1987; Loomis et al., 1991) whilst visual processing is massively parallel (see Nassi and Callaway, 2009). In the context of face processing, therefore, haptics might be restricted to featural processing, in which individual features are processed independently and have to be assembled into a face context, which may account for lower haptic performance compared to visual configural encoding (Dopjans et al., 2012). When visual encoding was restricted, by using a participant-controlled moving window that only revealed a small portion of the face at a time, so that it was more like haptic sequential processing, visual and haptic performance were more equal (Dopjans et al., 2012), suggesting that any differences arise from different encoding strategies ¹.

Despite these various differences in performance, visual and haptic face processing do have common aspects. For example, consistent with the shared perceptual spaces discussed above (e.g., Gaißert et al., 2008, 2010, 2011; Gaißert and Wallraven, 2012), there is evidence for similar “face-spaces” for vision and touch in which, again, different properties carry different weights depending on the modality (Wallraven, 2014). The evidence for a face-inversion effect – better recognition when faces are upright than inverted, an effect not seen for non-face categories – is clear for vision but less so for haptics. Kilgour and Lederman (2006) showed a clear haptic inversion effect for faces compared to non-face stimuli, whereas Dopjans et al. (2012) found an inversion effect for unrestricted visual, but not for haptic or restricted visual, face encoding. In “face adaptation,” a neutral face is perceived as having the opposite facial expression to a previously perceived face; for example, adaptation to a sad face leads to perception of a happy face upon subsequent presentation of a face with a neutral expression (e.g., Skinner and Benton, 2010). Such an effect is also seen in within-modal haptic adaptation to faces (Matsumiya, 2012) and transfers cross-modally both from vision to touch and vice versa, indicating that haptic face-related information and visual face processing share some common processing (Matsumiya, 2013).

Faces can also be recognized cross-modally between vision and touch (Kilgour and Lederman, 2002); this comes at a cost relative to within-modal recognition (Casey and Newell, 2007) although the cost decreases with familiarity (Casey and Newell, 2005). However, this disadvantage for cross-modal face recognition is unrelated to the encoding modality or to differences in encoding strategies, which suggests that, in contrast to object recognition (see below), vision and touch do not share a common face representation (Casey and Newell, 2007). On the other hand, visually presented faces disrupt identification of haptic faces when their facial expressions are incongruent and facilitate identification when they are congruent (Klatzky et al., 2011) which suggests a shared representation although response competition cannot be excluded as an explanation for these results. However, taken in conjunction with the finding that a visually prosopagnosic patient (i.e., a patient unable to recognize faces visually despite intact basic visual perception) was also unable to recognize faces haptically (Kilgour et al., 2004), a shared representation seems likely.

OBSTACLES TO EFFICIENT RECOGNITION

VIEW-DEPENDENCE

A change in the orientation of an object changes the related sensory input, e.g., retinal pattern, such that recognition is potentially impaired; an important goal of sensory systems is therefore to achieve perceptual constancy so that objects can be recognized independently of such changes. Visual object recognition is considered view-dependent if rotating an object away from its original orientation impairs subsequent recognition and view-independent if not (reviewed in Peissig and Tarr, 2007). During haptic exploration, the hands can contact an object from different sides simultaneously: intuitively, therefore, one might expect information about several different “views” to be acquired at the same time and that haptic recognition would be view-independent. However, numerous studies have now shown that this intuition is not correct and that haptic object recognition is also view-dependent (Newell et al., 2001; Lacey et al., 2007, 2009b; Ueda and Saiki, 2007, 2012; Craddock and Lawson, 2008, 2010; Lawson, 2009, 2011). The factors underlying haptic view-dependence are not currently known: even unlimited exploration time and orientation cuing do not reduce view-dependence (Lawson, 2011). It is interesting to examine how vision and touch are affected by different types of rotation. Visual recognition is differentially impaired by changes in orientation depending on the axis around which an object is rotated (Gauthier et al., 2002; Lacey et al., 2007). Recognition is slower and less accurate when objects are rotated about the x- and y-axes, i.e., in depth (Figure 1), than when rotated about the z-axis, i.e., in the picture plane, for both 2-D (Gauthier et al., 2002) and 3-D stimuli (Lacey et al., 2007). By contrast, haptic recognition is equally impaired by rotation about any axis (Lacey et al., 2007), suggesting that, although vision and haptics are both view-dependent, the basis for this is different in each modality. One possible explanation is that vision and haptics differ in whether or not a surface is occluded by rotation. In vision, a change in orientation can involve not only a transformation in perceptual shape but also occlusion of one or more surfaces – unless the observer physically changes position relative to the object (e.g., Pasqualotto et al., 2005; Pasqualotto and Newell, 2007). Compare, for example, Figures 1A,C – rotation about the x-axis means that the object is turned upside-down and that the former top surface becomes occluded. In haptic exploration, the hands are free to move over all surfaces of an object and to manipulate it into different orientations relative to the hand, thus in any given orientation, no surface is necessarily occluded, provided the object is small enough. If this is true, then no single axis of rotation should be more or less disruptive than another due to surface occlusion, so that haptic recognition only has to deal with a shape transformation. Further work is required to examine whether this explanation is, in fact, correct.

FIGURE 1

Lacey et al. (2007).

View-dependence mostly occurs when objects are unfamiliar. Increasing object familiarity reduces the disruptive effect of orientation changes and visual recognition tends to become view-independent (Tarr and Pinker, 1989; Bülthoff and Newell, 2006). An exception to this is when a familiar object is typically seen in one specific orientation known as a canonical view, for example the front view of a house (Palmer et al., 1981). View-independence may still occur for a limited range of orientations around the canonical view, but visual recognition is impaired for radically non-canonical views, for example, a teapot seen from directly above (Palmer et al., 1981; Tarr and Pinker, 1989; Bülthoff and Newell, 2006). Object familiarity also results in haptic view-independence and this remains so even where there is a change in the hand used to explore the object (Craddock and Lawson, 2009a). Haptic recognition also reverts to view-dependence for non-canonical orientations (Craddock and Lawson, 2008). However, vision and haptics differ in what constitutes a canonical view. The preferred view in vision is one in which the object is aligned at 45^∘ to the observer (Palmer et al., 1981) while objects are generally aligned either parallel or orthogonal to the body midline in haptic canonical views (Woods et al., 2008). Canonical views may facilitate view-independent recognition either because they provide the most structural information about an object or because they most closely match a stored representation, but the end result is the same for both vision and haptics (Craddock and Lawson, 2008; Woods et al., 2008).

In contrast to within-modal recognition, visuo-haptic cross-modal recognition is view-independent even for unfamiliar objects that are highly similar (Figure 1), whether visual study is followed by haptic test or vice versa and whatever the axis of rotation (Lacey et al., 2007, 2010b; Ueda and Saiki, 2007, 2012). Haptic-visual, but not visual-haptic, cross-modal view-independence has been shown for familiar objects (Lawson, 2009). This asymmetry might be due to the fact that the familiar objects used in this particular study were a mixture of scale models (e.g., bed, bath, and shark) and actual-size objects (e.g., jug, pencil); thus, some of these might have been more familiar visually than haptically, resulting in greater error when visually familiar objects had to be recognized by touch. Additional research on the potentially disruptive effects of differential familiarity is merited.

A strange finding is that knowledge of the test modality does not appear to help achieve view-independence. When participants knew the test modality, both visual and haptic within-modal recognition were view-dependent whereas cross-modal recognition was view-independent (Ueda and Saiki, 2007, 2012), but when the test modality was unknown both within- and cross-modal recognition were view-independent (Ueda and Saiki, 2007). At first glance this is puzzling: one would expect that knowledge of the test modality would confer an advantage. However, Ueda and Saiki (2012) showed that eye movements differed during encoding, with longer and more diffuse fixations when participants knew that they would be tested cross-modally (visual-haptic only) compared to within-modally. It is possible that, on the “principle of least commitment” (Marr, 1976), the same pattern of eye movements occurs when the test modality is not known (i.e., it is not possible to commit to an outcome), preserving as much information as possible and resulting in both within- and cross-modal view-independence. Further examination of eye movements during both cross-modal conditions would be valuable, as eye movements could serve as behavioral markers for the multisensory view-independent representation discussed next.

The simplest way in which cross-modal view-independence could arise is that the view-dependent visual and haptic unisensory representations are directly integrated into a view-independent multisensory representation (Figure 2A). An alternative explanation is that unisensory view-independence in vision and haptics is a precondition for cross-modal view-independence (Figure 2B). In a perceptual learning study, view-independence acquired by learning in one modality transferred completely and symmetrically to the other; thus, whether visual or haptic, within-modal view-independence relies on a single view-independent representation (Lacey et al., 2009b). Furthermore, both visual and haptic within-modal view-independence were acquired following cross-modal training (whether haptic-visual or visual-haptic); we therefore concluded that visuo-haptic view-independence is supported by a single multisensory representation that directly integrates the unisensory view-dependent representations (Lacey et al., 2009b; Figure 2A), similar to models that have been proposed for vision (Riesenhuber and Poggio, 1999). Thus, the same representation appears to support both cross-modal recognition and view-independence (whether within- or cross-modal).

FIGURE 2

Lacey et al. (2009b).

SIZE-DEPENDENCE

In addition to achieving object constancy across orientation changes, the visual system also has to contend with variations in the size of the retinal image that arise from changes in object-observer distance: the same object can produce retinal images that vary in size depending on whether it is near to, or far from, the observer. Presumably, this is compensated by cues arising from depth or motion perception, accounting for the fact that a change in size does not disrupt visual object identification (Biederman and Cooper, 1992; Uttl et al., 2007). However, size change does produce a cost in visual recognition for both unfamiliar (Jolicoeur, 1987) and familiar objects (Jolicoeur, 1987; Uttl et al., 2007). Interestingly, changes in retinal size due to movement of the observer result in better size-constancy than those due to movement of the object (Combe and Wexler, 2010).

Haptic size perception requires integration of both cutaneous (contact area and force) and proprioceptive (finger spread and position) information at initial contact (Berryman et al., 2006). Neither gripping an object tighter, which increases contact area, nor enlarging the spread of the fingers leads us to perceive a change in size (Berryman et al., 2006). Thus, in contrast to vision where perceived size varies with distance, in touch, physical size is perceived directly, i.e., haptic size equals physical size. It is intriguing then, that haptic (Craddock and Lawson, 2009b,c) and cross-modal (Craddock and Lawson, 2009c) recognition are apparently size-dependent and this merits further investigation. Further research should address whether haptic representations store a canonical size for familiar objects (as has recently been proposed for visual representations, Konkle and Oliva, 2011), deviations from which could impair recognition, and whether object constancy can be achieved across size changes in unfamiliar objects.

REPRESENTATIONS AND INDIVIDUAL DIFFERENCES

A crucial question for object recognition is what information is contained in the mental representations that support it. Visual shape, color, and texture are processed in different cerebral cortical areas (Cant and Goodale, 2007; Cant et al., 2009) but these structural (shape) and surface (color, texture, etc.) properties are integrated in visual object representations (Nicholson and Humphrey, 2003). Changing the color of an object or its part-color combinations between study and test impaired shape recognition, while altering the background color against which objects were presented did not (Nicholson and Humphrey, 2003). This effect could therefore be isolated to the object representation, indicating that this contains both shape and color information (Nicholson and Humphrey, 2003). Visual and haptic within-modal object discrimination are similarly impaired by a change in surface texture (Lacey et al., 2010b), showing firstly that haptic representations also integrate structural and surface properties and secondly that information about surface properties in visual representations is not limited to modality-specific properties like color. In order to investigate whether surface properties are integrated into the multisensory representation underlying cross-modal object discrimination, we tested object discrimination across changes in orientation (thus requiring access to the view-independent multisensory representation discussed above), texture or both. In line with earlier findings (Lacey et al., 2007; Ueda and Saiki, 2007, 2012), cross-modal object discrimination was view-independent when texture did not change; but if texture did change, performance was reduced to chance levels, whether orientation also changed or not (Lacey et al., 2010b). However, some participants were more affected by the texture changes than others. We wondered whether this arose from individual differences in the nature of object representations, which can be conveniently indexed by preferences for different kinds of imagery.

Two kinds of visual imagery have been described: “object imagery” (involving pictorial images that are vivid and detailed, dealing with the literal appearance of objects in terms of shape, color, brightness, etc.) and “spatial imagery” (involving schematic images more concerned with the spatial relations of objects, their component parts, and spatial transformations; Kozhevnikov et al., 2002, 2005; Blajenkova et al., 2006). An experimentally important difference is that object imagery includes surface property information while spatial imagery does not. To establish whether object and spatial imagery differences occur in touch as well as vision, we required participants to discriminate shape across changes in texture, and texture across changes in shape (Figure 3), in both visual and haptic within-modal conditions. We found that spatial imagers could discriminate shape despite changes in texture but not vice versa, presumably because their images tend not to encode surface properties. By contrast, object imagers could discriminate texture despite changes in shape, but not the reverse (Lacey et al., 2011), indicating that texture, a surface property, is integrated into their shape representations. Importantly, visual and haptic performance was not significantly different on either task and performance largely reflected both self-reports of imagery preference and scores on the Object and Spatial Imagery Questionnaire (OSIQ: Blajenkova et al., 2006). Thus, the object-spatial imagery continuum characterizes haptics as well as vision, and individual differences in imagery preference along this continuum affect the extent to which surface properties are integrated into object representations (Lacey et al., 2011). Further analysis of the texture-change condition in our earlier study (Lacey et al., 2010b) showed that performance was indeed related to imagery preference: both object and spatial imagers showed cross-modal view-independence but object imagers were impaired by texture changes whereas spatial imagers were not (Lacey et al., 2011). In addition, the extent of the impairment was correlated with OSIQ scores such that greater preference for object imagery was associated with greater impairment by texture changes; surface properties are therefore likely only integrated into the multisensory representation by object imagers (Lacey et al., 2011). Moreover, spatial imagery preference correlated with the accuracy of cross-modal object recognition (Lacey et al., 2007). It appears, then, that the multisensory representation has some features that are stable across individuals, like view-independence, and some that vary across individuals, such as integration of surface property information and individual differences in imagery preference.

FIGURE 3

Lacey et al. (2011).

THE NEURAL BASIS OF VISUO-HAPTIC OBJECT PROCESSING

SEGREGATED VENTRAL “WHAT” AND DORSAL “WHERE/HOW” PATHWAYS

At the macro-level, visual object processing divides along a ventral pathway concerned with object identity and perception for recognition, and a dorsal pathway dealing with object location and perception for action, e.g., reaching and grasping, (Ungerleider and Mishkin, 1982; Goodale and Milner, 1992). Similar ventral and dorsal pathways have been proposed for the auditory (e.g., De Santis et al., 2007a) and somatosensory domains (Dijkerman and de Haan, 2007), with divergence of the “what” and “where/how” pathways in a similar timeframe (∼200 ms after stimulus onset) (De Santis et al., 2007a,b), and thus are probably common aspects of functional architecture across modalities.

In the case of touch, an early functional magnetic resonance (fMRI) study found that haptic object recognition activated frontal cortical areas as well as inferior parietal cortex, while a haptic object location task activated superior parietal regions (Reed et al., 2005). A later study from our laboratory (Sathian et al., 2011) compared perception of haptic texture and location, reasoning that texture would be a better marker of haptic object identity, given the salience of texture to touch (Klatzky et al., 1987). This study found that, while both visual and haptic location judgments involved a similar dorsal pathway comprising large sectors of the IPS and frontal eye fields (FEFs) bilaterally, haptic texture perception engaged extensive areas of the parietal operculum (OP), which contains higher-order (i.e., non-primary), ventral regions of somatosensory cortex. In addition, shared cortical processing of texture across vision and touch was found in parts of extrastriate (i.e., non-primary) visual cortex and ventral premotor cortex (Sathian et al., 2011). For both texture and location, several of these bisensory areas showed correlations of activation magnitude between the visual and haptic tasks, indicating some commonality of cortical processing across modalities (Sathian et al., 2011). Another group extended these findings by showing that early visual cortex showed activation magnitudes that not only scaled with the interdot spacing of dot-patterns, but were also modulated by the presence of matching haptic input (Eck et al., 2013).

MULTISENSORY PROCESSING OF OBJECT SHAPE

Cortical areas in both the ventral and dorsal pathways previously identified as specialized for various aspects of visual processing are also functionally involved during the corresponding haptic tasks (for reviews see Amedi et al., 2005; Sathian and Lacey, 2007; Lacey and Sathian, 2011). In the human visual pathway even early visual areas (which project to both dorsal and ventral streams) have been found to respond to changes in haptic shape, suggesting that haptic shape perception might involve the entire ventral stream (Snow et al., 2014). If true, this might reflect cortical pathways between primary somatosensory and visual cortices previously demonstrated in the macaque (Négyessy et al., 2006); however, as with other studies (see below), it is not possible to exclude visual imagery as an explanation for the findings of Snow et al. (2014). The majority of research on visuo-haptic processing of object shape has concentrated on higher-level visual areas, in particular the LOC, an object-selective region in the ventral visual pathway (Malach et al., 1995), a sub-region of which also responds selectively to objects in both vision and touch (Amedi et al., 2001, 2002; Stilla and Sathian, 2008). The LOC responds to both haptic 3-D (Amedi et al., 2001; Zhang et al., 2004; Stilla and Sathian, 2008) and tactile 2-D stimuli (Stoesz et al., 2003; Prather et al., 2004) but does not respond during auditory object recognition cued by object-specific sounds (Amedi et al., 2002). However, when participants listened to the impact sounds made by rods and balls made of either metal or wood and categorized these sounds by the shape of the object that made them, the material of the object, or by using all the acoustic information available, the LOC was more activated when these sounds were categorized by shape than by material (James et al., 2011). Here again though, participants could have solved this matching task using visual imagery: we return to the potential role of visual imagery in a later section.

The LOC does, however, respond to auditory shape information created by a visual-auditory sensory substitution device (Amedi et al., 2007) using a specific algorithm to convert visual information into an auditory stream or “soundscape” in which the visual horizontal axis is represented by auditory duration and stereo panning, the visual vertical axis by variations in tone frequency, and pixel brightness by variations in tone loudness. Although it requires extensive training, both sighted and blind humans can learn to recognize objects by extracting shape information from such soundscapes (Amedi et al., 2007). However, the LOC only responds to soundscapes created according to the algorithm – and which therefore represent shape in a principled way – and not when participants learn soundscapes that are merely arbitrarily associated with particular objects (Amedi et al., 2007). Thus, the LOC can be regarded as processing geometric shape information independently of the sensory modality used to acquire it.

Apart from the LOC, multisensory (visuo-haptic) responses have also been observed in several parietal regions: in particular, the aIPS is involved in perception of both the shape and location of objects, with co-activation of the LOC for shape and the FEF for location (Stilla and Sathian, 2008; Sathian et al., 2011; see also Saito et al., 2003). The postcentral sulcus (PCS; Stilla and Sathian, 2008), corresponding to Brodmann’s area 2 of primary somatosensory cortex (S1; Grefkes et al., 2001), also shows visuo-haptic shape-selectivity. This area is normally considered exclusively somatosensory but the bisensory responses observed by Stilla and Sathian (2008) are consistent with earlier neurophysiological studies that suggested visual responsiveness in parts of S1 (Iriki et al., 1996; Zhou and Fuster, 1997).

Multisensory responses in the LOC and elsewhere might reflect visuo-haptic integration in neurons that process both visual and haptic input; alternatively, they might arise from separate inputs to discrete but interdigitated unisensory neuronal populations. Tal and Amedi (2009) sought to distinguish between these using fMRI adaptation (fMR-A). This technique utilizes the repetition suppression effect, i.e., when the same stimulus is repeated, the blood-oxygen level dependent (BOLD) signal is attenuated. Since repetition suppression can be observed in single neurons, fMR-A can reveal neuronal selectivity profiles (see Grill-Spector et al., 2006; Krekelberg et al., 2006 for reviews). When stimuli that had been presented visually were presented again haptically, there was a robust cross-modal adaptation effect not only in the LOC and the aIPS, but also in bilateral precentral sulcus (preCS) corresponding to ventral premotor cortex, and the right anterior insula, suggesting that these areas were integrating multisensory inputs at the neuronal level. However, a separate preCS site and posterior parts of the IPS did not show cross-modal adaptation, suggesting that their multisensory responses arise from separate unisensory populations. Because fMR-A effects may not necessarily reflect neuronal selectivity (Mur et al., 2010), it will be necessary to confirm the findings of Tal and Amedi (2009) with converging evidence using other methods.

It is critical to determine whether haptic or tactile involvement in supposedly visual cortical areas is functionally relevant, i.e., whether it is actually necessary for task performance. Although research along these lines is still relatively sparse, two lines of evidence indicate that this is indeed the case. Firstly, case studies indicate that the LOC is necessary for both haptic and visual shape perception. A lesion to the left occipito-temporal cortex, which likely included the LOC, resulted in both tactile and visual agnosia even though somatosensory cortex and basic somatosensory function were intact (Feinberg et al., 1986). Another patient with bilateral LOC lesions was unable to learn new objects either visually or haptically (James et al., 2006b). These case studies are consistent with the existence of a shared multisensory representation in the LOC.

Transcranial magnetic stimulation (TMS) is a technique used to temporarily deactivate specific, functionally defined, cortical areas, i.e., to create “virtual lesions” (Sack, 2006). TMS over a parieto-occipital region previously shown to be active during tactile grating orientation discrimination (Sathian et al., 1997) interfered with performance of this task (Zangaladze et al., 1999) indicating that it was functionally, rather than epiphenomenally, involved. This area is the probable human homolog of macaque area V6 (Pitzalis et al., 2006). Repetitive TMS (rTMS) over the left LOC impaired visual object, but not scene, categorization (Mullin and Steeves, 2011), similarly suggesting that this area is necessary for object processing. rTMS over the left aIPS impaired visual-haptic, but not haptic-visual, shape matching using the right hand (Buelte et al., 2008), but shape matching with the left hand during rTMS over the right aIPS was unaffected in either cross-modal condition. The reason for this discrepancy is unclear, and emphasizes that the precise roles of the IPS and LOC in multisensory shape processing have yet to be fully worked out.

CATEGORY-SPECIFIC REPRESENTATIONS

There has been rather limited neural study of cross-modal category-selective representations. Using multivoxel pattern analysis of fMRI data, Pietrini et al. (2004) demonstrated that selectivity for particular categories of man-made objects was correlated across vision and touch in a region of inferotemporal cortex. In the case of face perception, fMRI studies, in contrast to the behavioral studies reviewed above, tend to favor separate, rather than shared representations. For example, visual and haptic face-selectivity in ventral and inferior temporal cortex are in largely separate voxel populations (Pietrini et al., 2004). Haptic face recognition activates the left FG, whereas visual face recognition activates the right FG (Kilgour et al., 2005); furthermore, activity in the left FG increases during haptic processing of familiar, compared to unfamiliar, faces while the right FG remains relatively inactive (James et al., 2006a). A further difference in FG face responses is that imagery of visually presented faces activates the left FG more than the right FG (Ishai et al., 2002) ²; this raises the possibility that haptic face perception involves visual imagery mechanisms. Although one study found that haptic face recognition ability and imagery vividness ratings were uncorrelated (Kilgour and Lederman, 2002), the implication of visual imagery in haptic face perception is very consonant with our findings in haptic shape perception discussed below (Deshpande et al., 2010; Lacey et al., 2010a) especially as vividness ratings do not particularly index imagery ability (reviewed in Lacey and Lawson, 2013). Further studies are needed to resolve the neural basis of multisensory face perception, and its differences from multisensory object perception.

VIEW- AND SIZE-INDEPENDENCE

The cortical locus of the multisensory view-independent representation is currently not known. Evidence for visual view-independence in the LOC is mixed: as might be expected, unfamiliar objects produce view-dependent LOC responses (Gauthier et al., 2002) and familiar objects produce view-independent responses (Valyear et al., 2006; Eger et al., 2008a; Pourtois et al., 2009). By contrast, one study found view-dependence in the LOC even for familiar objects, although in this study there was position-independence (Grill-Spector et al., 1999), whereas another found view-independence for both familiar and unfamiliar objects (James et al., 2002a). A recent TMS study of 2-D shape suggests that the LOC is functionally involved in view-independent recognition (Silvanto et al., 2010) but only two rotations, 20 and 70^∘, were tested and TMS effects were only seen for the 20^∘ rotation; further work is required to substantiate this finding. Responses in the FG are also variable with the left FG less sensitive to orientation changes than the right FG (Andresen et al., 2009; Harvey and Burgund, 2012). A study of face viewpoint-selectivity showed a gradient of decreasing orientation sensitivity, from view-dependence in early visual cortex to partial view-independence in later areas including LOC (Axelrod and Yovel, 2012); this sensitivity gradient may also apply to non-face objects.

Various parietal regions show visual view-dependent responses, e.g., the IPS (James et al., 2002a) and a parieto-occipital area (Valyear et al., 2006). Superior parietal cortex is view-dependent during mental rotation but not visual object recognition (Gauthier et al., 2002; Wilson and Farah, 2006). As these regions are in the dorsal pathway, concerned with object location and perception for action, view-dependent responses in these regions are not surprising (Ungerleider and Mishkin, 1982; Goodale and Milner, 1992). Actions such as reaching and grasping adapt to changes in object orientation and consistent with this, lateral parieto-occipital cortex shows view-dependent responses for graspable, but not for non-graspable objects (Rice et al., 2007).

To date, we are not aware of neuroimaging studies of haptic or cross-modal processing of stimuli across changes in orientation. James et al. (2002b) varied object orientation, but this study concentrated on haptic-to-visual priming rather than the cross-modal response to same vs. different orientations per se. Additionally, there is much work to be done on the effect of orientation changes when shape information is derived from the auditory soundscapes produced by sensory substitution devices (SSDs) and also when the options for haptically interacting with an object are altered by a change in orientation. Similarly, there is no neuroimaging work on haptic and multisensory processing of stimuli across changes in size. However, visual size-independence has been consistently observed in the LOC (Grill-Spector et al., 1999; Ewbank et al., 2005; Eger et al., 2008a,b), with anterior regions showing more size-independence than posterior regions (Sawamura et al., 2005; Eger et al., 2008b).

FIGURE 4

Lacey et al., 2007, 2009b, 2011).

A MODEL OF VISUO-HAPTIC MULTISENSORY OBJECT REPRESENTATION

Haptic activation of the LOC might arise from direct somatosensory input. Activity in somatosensory cortex propagates to the LOC as early as 150 ms after stimulus onset during tactile discrimination of simple shapes, a timeframe consistent with “bottom-up” projections to LOC (Lucan et al., 2010; Adhikari et al., 2014). Similarly, in a tactile microspatial discrimination task, LOC activity was consistent with feedforward propagation in a beta-band oscillatory network (Adhikari et al., 2014). In addition, a patient with bilateral ventral occipito-temporal lesions, but with sparing of the dorsal part of the LOC that likely included the multisensory sub-region, showed visual agnosia but intact haptic object recognition (Allen and Humphreys, 2009). Haptic object recognition was associated with activation of the intact dorsal part of the LOC, suggesting that somatosensory input could directly activate this region (Allen and Humphreys, 2009).

Alternatively, haptic perception might evoke visual imagery of the felt object resulting in “top-down” activation of the LOC (Sathian et al., 1997) and consistent with this hypothesis, many studies show LOC activity during visual imagery. During auditorily cued mental imagery of familiar object shape, both blind and sighted participants show left LOC activation, where shape information would arise mainly from haptic experience for the blind and mainly from visual experience for the sighted (De Volder et al., 2001). The left LOC is also active when geometric and material object properties are retrieved from memory (Newman et al., 2005) and haptic shape-selective activation magnitudes in the right LOC were highly correlated with ratings of visual imagery vividness (Zhang et al., 2004). A counter-argument is that imagery plays a relatively minor role because LOC activity was substantially lower during visual imagery compared to haptic shape perception (Amedi et al., 2001). However, this study could not verify that participants engaged in imagery throughout the imaging session, so that lower imagery-related activity might have resulted from non-compliance (or irregular compliance) with the task. It has also been argued that visual imagery cannot explain haptically evoked LOC activity because early- as well as late-blind individuals show shape-related LOC activation via both touch (reviewed in Pascual-Leone et al., 2005; Sathian, 2005; Sathian and Lacey, 2007) and hearing using SSDs (Arno et al., 2001; Renier et al., 2004, 2005; Amedi et al., 2007). But this argument, while true for the early blind, does not rule out a visual imagery explanation in the sighted, given the extensive evidence for cross-modal plasticity following visual deprivation (reviewed in Pascual-Leone et al., 2005; Sathian, 2005; Sathian and Lacey, 2007).

In this section we describe a model of visuo-haptic multisensory object representation (Lacey et al., 2009a) and review the evidence for this model from studies designed to explicitly test the visual imagery hypothesis discussed above (Deshpande et al., 2010; Lacey et al., 2010a, 2014). In this model, object representations in the LOC can be flexibly accessed either bottom-up or top-down, depending on object familiarity, and independently of the input modality. There is no stored representation for unfamiliar objects so that during haptic recognition, an unfamiliar object has to be explored in its entirety in order to compute global shape and to relate component parts to one another. This, we propose, occurs in a bottom-up pathway from somatosensory cortex to the LOC, with involvement of the IPS in computing part relationships and thence global shape, facilitated by spatial imagery processes. For familiar objects, global shape can be inferred more easily, perhaps from distinctive features or one diagnostic part, and we suggest that haptic exploration rapidly acquires enough information to trigger a stored visual image and generate a hypothesis about its identity, as has been proposed for vision (e.g., Bar, 2007). This occurs in a top-down pathway from prefrontal cortex to LOC, involving primarily object imagery processes (though spatial imagery may still have a role in processing familiar objects, for example, in view-independent recognition).

We tested this model using analyses of inter-task correlations of activation magnitude between visual object imagery and haptic shape perception (Lacey et al., 2010a) and analyses of effective connectivity (Deshpande et al., 2010), reasoning that reliance on similar processes across tasks would lead to correlations of activation magnitude across participants, as well as similar patterns of effective connectivity across tasks. In contrast to previous studies, we ensured that participants engaged in visual imagery throughout each scan by using an object imagery task and recording responses. Participants also performed a haptic shape discrimination task using either familiar or unfamiliar objects. We found that object familiarity modulated inter-task correlations as predicted by our model. There were eleven regions common to visual object imagery and haptic perception of familiar shape, six of which (including bilateral LOC) showed inter-task correlations of activation magnitude. By contrast, object imagery and haptic perception of unfamiliar shape shared only four regions, only one of which (an IPS region) showed an inter-task correlation (Lacey et al., 2010a). More recently, we examined the relation between haptic shape perception and spatial imagery, using a spatial imagery task in which participants memorized a 4 × 4 lettered grid and, in response to auditory letter strings, constructed novel shapes within the imagined grid from component parts (Lacey et al., 2014); the haptic shape tasks were the same as in Lacey et al. (2010a). Contrary to the model, relatively few regions showed inter-task correlations between spatial imagery and haptic perception of either familiar or unfamiliar shape, with parietal foci featuring in both sets of correlations. This suggests that spatial imagery is relevant to haptic shape perception regardless of object familiarity, whereas our earlier finding suggested that object imagery is more strongly associated with haptic perception of familiar, than unfamiliar, shape (Lacey et al., 2010a). However, it is also possible that the parietal foci showing inter-task correlations between spatial imagery and haptic shape perception reflect spatial processing more generally, rather than spatial imagery per se (Lacey et al., 2014; and see Jäncke et al., 2001), or generic imagery processes, e.g., image generation, common to both object and spatial imagery (Lacey et al., 2014; and see Mechelli et al., 2004).

In our study of spatial imagery (Lacey et al., 2014), we also conducted effective connectivity analyses, based on the inferred neuronal activity derived from deconvolving the hemodynamic response out of the observed BOLD signals (Sathian et al., 2013). In order to make direct comparisons between the neural networks underlying object and spatial imagery in haptic shape perception, we re-analyzed our earlier data (Deshpande et al., 2010) using the newer effective connectivity methods. These analyses supported the broad architecture of the model, showing that the spatial imagery network shared much more commonality with the network associated with unfamiliar, compared to familiar, shape perception, while the object imagery network shared much more commonality with familiar, than unfamiliar, shape perception (Lacey et al., 2014). More specifically, the model proposes that the component parts of an unfamiliar object are explored in their entirety and assembled into a representation of global shape via spatial imagery processes (Lacey et al., 2009a). Consistent with this, in the parts of the network that were common to spatial imagery and unfamiliar haptic shape perception, the LOC is driven by parietal foci, with complex cross-talk between posterior parietal and somatosensory foci. These findings fit with the notion of bottom-up pathways from somatosensory cortex and a role for cortex in and around the IPS in spatial imagery (Lacey et al., 2014). The IPS and somatosensory interactions were absent from the sparse network that was shared by spatial imagery and haptic perception of familiar shape. By contrast, the relationship between object imagery and familiar shape perception is characterized by top-down pathways from prefrontal areas reflecting the involvement of object imagery, according to our model (Lacey et al., 2009a). The re-analyzed data supported this, showing the LOC driven bilaterally by the left inferior frontal gyrus in the network shared by object imagery and haptic perception of familiar shape, while these pathways were absent from the extremely sparse network common to object imagery and unfamiliar haptic shape perception (Lacey et al., 2014).

Figure 4 shows the current version of our model for haptic shape perception in which the LOC is driven bottom-up from primary somatosensory cortex as well as top-down via object imagery processes from prefrontal cortex, with additional input from the IPS involving spatial imagery processes. We propose that the bottom-up route is more important for haptic perception of unfamiliar than familiar objects, whereas the converse is true of the top-down route – more important for haptic perception of familiar than unfamiliar objects. It will be interesting to explore the impact of individual preferences for object vs. spatial imagery on these processes and paths.

SUMMARY

The research reviewed here illustrates how deeply interconnected the visual and haptic modalities are in object processing, from highly similar and transferable perceptual spaces underlying categorization, through shared representations in cross-modal and view-independent recognition and commonalities in imagery preferences, to multisensory neural substrates and complex interactions between bottom-up and top-down processes as well as between object and spatial imagery. Much, however, remains to be done in order to provide a detailed account of visuo-haptic multisensory behavior and its underlying mechanisms and how this understanding can be put to use, for example in the service of neurorehabilitation, particularly for those with sensory deprivation of various sorts.

REFERENCES

• 1

  AdhikariB. M.SathianK.EpsteinC. M.LamichhaneB.DhamalaM. (2014). Oscillatory activity in neocortical networks during tactile discrimination near the limit of spatial acuity.Neuroimage91300–310. 10.1016/j.neuroimage.2014.01.007

  • CrossRef
  • Google Scholar
  • View reference in article

• 2

  AllenH. A.HumphreysG. W. (2009). Direct tactile stimulation of dorsal occipito-temporal cortex in a visual agnosic.Curr. Biol.191044–1049. 10.1016/j.cub.2009.04.057

  • CrossRef
  • Google Scholar
  • View reference in article

• 3

  AmediA.JacobsonG.HendlerT.MalachR.ZoharyE. (2002). Convergence of visual and tactile shape processing in the human lateral occipital complex.Cereb. Cortex121202–1212. 10.1093/cercor/12.11.1202

  • CrossRef
  • Google Scholar
  • View reference in article

• 4

  AmediA.MalachR.HendlerT.PeledS.ZoharyE. (2001). Visuo-haptic object-related activation in the ventral visual pathway.Nat. Neurosci.4324–330. 10.1038/85201

  • CrossRef
  • Google Scholar
  • View reference in article

• 5

  AmediA.SternW. M.CamprodonJ. A.BermpohlF.MerabetL.RotmanS.et al (2007). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex.Nat. Neurosci.10687–689. 10.1038/nn1912

  • CrossRef
  • Google Scholar
  • View reference in article

• 6

  AmediA.von KriegsteinK.van AtteveldtN. M.BeauchampM. S.NaumerM. J. (2005). Functional imaging of human crossmodal identification and object recognition.Exp. Brain Res.166559–571. 10.1007/s00221-005-2396-5

  • CrossRef
  • Google Scholar
  • View reference in article

• 7

  AndresenD. R.VinbergJ.Grill-SpectorK. (2009). The representation of object viewpoint in human visual cortex.Neuroimage45522–536. 10.1016/j.neuroimage.2008.11.009

  • CrossRef
  • Google Scholar
  • View reference in article

• 8

  ArnoP.De VolderA. G.VanlierdeA.Wanet-DefalqueM.-C.StreelE.RobertA.et al (2001). Occipital activation by pattern recognition in the early blind using auditory substitution for vision.Neuroimage13632–645. 10.1006/nimg.2000.0731

  • CrossRef
  • Google Scholar
  • View reference in article

• 9

  AxelrodV.YovelG. (2012). Hierarchical processing of face viewpoint in human visual cortex.J. Neurosci.322442–2452. 10.1523/JNEUROSCI.4770-11.2012

  • CrossRef
  • Google Scholar
  • View reference in article

• 10

  BarM. (2007). The proactive brain: using analogies and associations to generate predictions.Trends Cogn. Sci.11280–289. 10.1016/j.tics.2007.05.005

  • CrossRef
  • Google Scholar
  • View reference in article

• 11

  BerrymanL. J.YauJ. M.HsiaoS. S. (2006). Representation of object size in the somatosensory system.J. Neurophysiol.9627–39. 10.1152/jn.01190.2005

  • CrossRef
  • Google Scholar
  • View reference in article

• 12

  BiedermanI.CooperE. E. (1992). Size invariance in visual object priming.J. Exp. Psychol. Hum. Percept. Perform.18121–133. 10.1037/0096-1523.18.1.121

  • CrossRef
  • Google Scholar
  • View reference in article

• 13

  BlajenkovaO.KozhevnikovM.MotesM. A. (2006). Object-spatial imagery: a new self-report imagery questionnaire.Appl. Cogn. Psychol.20239–263. 10.1002/acp.1182

  • CrossRef
  • Google Scholar
  • View reference in article

• 14

  BlissI.HämäläinenH. (2005). Different working memory capacity in normal young adults for visual and tactile letter recognition task.Scand. J. Psychol.46247–251. 10.1111/j.1467-9450.2005.00454.x

  • CrossRef
  • Google Scholar
  • View reference in article

• 15

  BuelteD.MeisterI. G.StaedtgenM.DambeckN.SparingR.GrefkesC.et al (2008). The role of the anterior intraparietal sulcus in crossmodal processing of object features in humans: an rTMS study.Brain Res.1217110–118. 10.1016/j.brainres.2008.03.075

  • CrossRef
  • Google Scholar
  • View reference in article

• 16

  BülthoffI.NewellF. N. (2006). The role of familiarity in the recognition of static and dynamic objects.Prog. Brain Res.154315–325. 10.1016/S0079-6123(06)54017-8

  • CrossRef
  • Google Scholar
  • View reference in article

• 17

  BushnellE. W.BaxtC. (1999). Children’s haptic and cross-modal recognition with familiar and unfamiliar objects.J. Exp. Psychol. Hum. Percept. Perform.251867–1881. 10.1037/0096-1523.25.6.1867

  • CrossRef
  • Google Scholar
  • View reference in article

• 18

  CantJ. S.ArnottS. R.GoodaleM. A. (2009). fMR-adaptation reveals separate processing regions for the perception of form and texture in the human ventral stream.Exp. Brain Res.192391–405. 10.1007/s00221-008-1573-8

  • CrossRef
  • Google Scholar
  • View reference in article

• 19

  CantJ. S.GoodaleM. A. (2007). Attention to form or surface properties modulates different regions of human occipitotemporal cortex.Cereb. Cortex17713–731. 10.1093/cercor/bhk022

  • CrossRef
  • Google Scholar
  • View reference in article

• 20

  CaseyS. J.NewellF. N. (2005). The role of long-term and short-term familiarity in visual and haptic face recognition.Exp. Brain Res.166583–591. 10.1007/s00221-005-2398-3

  • CrossRef
  • Google Scholar
  • View reference in article

• 21

  CaseyS. J.NewellF. N. (2007). Are representations of unfamiliar faces independent of encoding modality?Neuropsychologia45506–513. 10.1016/j.neuropsychologia.2006.02.011

  • CrossRef
  • Google Scholar
  • View reference in article

• 22

  CombeE.WexlerM. (2010). Observer movement and size constancy.Psychol. Sci.21667–675. 10.1177/0956797610367753

  • CrossRef
  • Google Scholar
  • View reference in article

• 23

  CookeT.JäkelF.WallravenC.BülthoffH. H. (2007). Multimodal similarity and categorization of novel, three-dimensional objects.Neuropsychologia45484–495. 10.1016/j.neuropsychologia.2006.02.009

  • CrossRef
  • Google Scholar
  • View reference in article

• 24

  CraddockM.LawsonR. (2008). Repetition priming and the haptic recognition of familiar and unfamiliar objects.Percept. Psychophys.701350–1365. 10.3758/PP.70.7.1350

  • CrossRef
  • Google Scholar
  • View reference in article

• 25

  CraddockM.LawsonR. (2009a). Do left and right matter for haptic recognition of familiar objects?Perception381355–1376. 10.1068/p6312

  • CrossRef
  • Google Scholar
  • View reference in article

• 26

  CraddockM.LawsonR. (2009b). The effect of size changes on haptic object recognition.Atten. Percept. Psychophys.71910–923. 10.3758/APP.71.4.910

  • CrossRef
  • Google Scholar
  • View reference in article

• 27

  CraddockM.LawsonR. (2009c). Size-sensitive perceptual representations underlie visual and haptic object recognition.PLoS ONE4:e8009. 10.1371/journal.pone.0008009

  • CrossRef
  • Google Scholar
  • View reference in article

• 28

  CraddockM.LawsonR. (2010). The effects of temporal delay and orientation on haptic object recognition.Atten. Percept. Psychophys.721975–1980. 10.3758/APP.72.7.1975

  • CrossRef
  • Google Scholar
  • View reference in article

• 29

  De SantisL.ClarkeS.MurrayM. M. (2007a). Automatic and intrinsic auditory ‘what’ and ‘where’ processing in humans revealed by electrical neuroimaging.Cereb. Cortex179–17. 10.1093/cercor/bhj119

  • CrossRef
  • Google Scholar
  • View reference in article

• 30

  De SantisL.SpiererL.ClarkeS.MurrayM. M. (2007b). Getting in touch: segregated somatosensory what and where pathways in humans revealed by electrical neuroimaging.Neuroimage37890–903. 10.1016/j.neuroimage.2007.05.052

  • CrossRef
  • Google Scholar
  • View reference in article

• 31

  DeshpandeG.HuX.LaceyS.StillaR.SathianK. (2010). Object familiarity modulates effective connectivity during haptic shape perception.Neuroimage491991–2000. 10.1016/j.neuroimage.2009.08.052

  • CrossRef
  • Google Scholar
  • View reference in article

• 32

  De VolderA. G.ToyamaH.KimuraY.KiyosawaM.NakanoH.VanlierdeA.et al (2001). Auditory triggered mental imagery of shape involves visual association areas in early blind humans.Neuroimage14129–139. 10.1006/nimg.2001.0782

  • CrossRef
  • Google Scholar
  • View reference in article

• 33

  DijkermanH. C.de HaanE. H. F. (2007). Somatosensory processes subserving perception and action.Behav. Brain Sci.30189–239. 10.1017/S0140525X07001392

  • CrossRef
  • Google Scholar
  • View reference in article

• 34

  DopjansL.BülthoffH. H.WallravenC. (2012). Serial exploration of faces: comparing vision and touch.J. Vis.126. 10.1167/12.1.6

  • CrossRef
  • Google Scholar
  • View reference in article

• 35

  EastonR. D.GreeneA. J.SrinivasK. (1997a). Transfer between vision and haptics: memory for 2-D patterns and 3-D objects.Psychon. Bull. Rev.4403–410. 10.3758/BF03210801

  • CrossRef
  • Google Scholar
  • View reference in article

• 36

  EastonR. D.SrinivasK.GreeneA. J. (1997b). Do vision and haptics share common representations? Implicit and explicit memory within and between modalities.J. Exp. Psychol. Learn. Mem. Cogn.23153–163. 10.1037/0278-7393.23.1.153

  • CrossRef
  • Google Scholar
  • View reference in article

• 37

  EckJ.KaasA. L.GoebelR. (2013). Crossmodal interactions of haptic and visual texture information in early sensory cortex.Neuroimage75123–135. 10.1016/j.neuroimage.2013.02.075

  • CrossRef
  • Google Scholar
  • View reference in article

• 38

  EgerE.AshburnerJ.HaynesJ-D.DolanR. J.ReesG. (2008a). fMRI activity patterns in human LOC carry information about object exemplars within category.J. Cogn. Neurosci.20356–370. 10.1162/jocn.2008.20019

  • CrossRef
  • Google Scholar
  • View reference in article

• 39

  EgerE.KellC. A.KleinschmidtA. (2008b). Graded size-sensitivity of object-exemplar-evoked activity patterns within human LOC regions.J. Neurophysiol.1002038–2047. 10.1152/jn.90305.2008

  • CrossRef
  • Google Scholar
  • View reference in article

• 40

  ErnstM. O.BanksM. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion.Nature415429–433. 10.1038/415429a

  • CrossRef
  • Google Scholar
  • View reference in article

• 41

  EwbankM. P.SchluppeckD.AndrewsT. J. (2005). fMR-adaptation reveals a distributed representation of inanimate objects and places in human visual cortex.Neuroimage28268–279. 10.1016/j.neuroimage.2005.06.036

  • CrossRef
  • Google Scholar
  • View reference in article

• 42

  FeinbergT. E.RothiL. J.HeilmanK. M. (1986). Multimodal agnosia after unilateral left hemisphere lesion.Neurology36864–867. 10.1212/WNL.36.6.864

  • CrossRef
  • Google Scholar
  • View reference in article

• 43

  GaißertN.BülthoffH. H.WallravenC. (2011). Similarity and categorization: from vision to touch.Acta Psychol.138219–230. 10.1016/j.actpsy.2011.06.007

  • CrossRef
  • Google Scholar
  • View reference in article

• 44

  GaißertN.WallravenC. (2012). Categorizing natural objects: a comparison of the visual and haptic modalities.Exp. Brain Res.216123–134. 10.1007/s00221-011-2916-4

  • CrossRef
  • Google Scholar
  • View reference in article

• 45

  GaißertN.WallravenC.BülthoffH. H. (2008). “Analyzing perceptual representations of complex, parametrically-defined shapes using MDS,” inProceedings of the Sixth International Conference EuroHaptics 2008, Lecture Notes in Computer Science – Haptics: Perception, Devices and Scenarios, Vol. 5024 (Heidelberg: Springer Berlin Heidelberg), 265–274.

  • Google Scholar
  • View reference in article

• 46

  GaißertN.WallravenC.BülthoffH. H. (2010). Visual and haptic perceptual spaces show high similarity in humans.J. Vis.102. 10.1167/10.11.2

  • CrossRef
  • Google Scholar
  • View reference in article

• 47

  GaißertN.WaterkampS.FlemingR. W.BülthoffI. (2012). Haptic categorical perception of shape.PLoS ONE7:e43062. 10.1371/journal.pone.0043062

  • CrossRef
  • Google Scholar
  • View reference in article

• 48

  GallaceA. (2013). “Somesthetic mental imagery,” inMultisensory ImageryedsLaceyS.LawsonR. (New York: Springer), 29–50. 10.1007/978-1-4614-5879-1_3

  • CrossRef
  • Google Scholar
  • View reference in article

• 49

  GallaceA.SpenceC. (2009). The cognitive and neural correlates of tactile memory.Psychol. Bull.135380–406. 10.1037/a0015325

  • CrossRef
  • Google Scholar
  • View reference in article

• 50

  GarvillJ.MolanderB. (1973). Effects of standard modality, comparison modality and retention interval on matching of form.Scand. J. Psychol.14203–206. 10.1111/j.1467-9450.1973.tb00111.x

  • CrossRef
  • Google Scholar
  • View reference in article

• 51

  GauthierI.HaywardW. G.TarrM. J.AndersonA. W.SkudlarskiP.GoreJ. C. (2002). BOLD activity during mental rotation and view-dependent object recognition.Neuron34161–171. 10.1016/S0896-6273(02)00622-0

  • CrossRef
  • Google Scholar
  • View reference in article

• 52

  GoodaleM. A.MilnerA. D. (1992). Separate visual pathways for perception and action.Trends Neurosci.1520–25. 10.1016/0166-2236(92)90344-8

  • CrossRef
  • Google Scholar
  • View reference in article

• 53

  GrafM. (2010). “Categorization and object shape,” inTowards a Theory of Thinking: Building Blocks for a Conceptual FrameworkedsGlatzederB. M.GoelV.von MüllerA. (Berlin: Springer-Verlag), 73–101. 10.1007/978-3-642-03129-8_6

  • CrossRef
  • Google Scholar
  • View reference in article

• 54

  GrefkesC.GeyerS.SchormannT.RolandP.ZillesK. (2001). Human somatosensory area 2: observer-independent cytoarchitectonic mapping, interindividual variability, and population map.Neuroimage14617–631. 10.1006/nimg.2001.0858

  • CrossRef
  • Google Scholar
  • View reference in article

• 55

  Grill-SpectorK.HensonR.MartinA. (2006). Repetition and the brain: neural models of stimulus-specific effects.Trends Cogn. Sci.1014–23. 10.1016/j.tics.2005.11.006

  • CrossRef
  • Google Scholar
  • View reference in article

• 56

  Grill-SpectorK.KushnirT.EdelmanS.AvidanG.ItzchakY.MalachR. (1999). Differential processing of objects under various viewing conditions in the human lateral occipital complex.Neuron24187–203. 10.1016/S0896-6273(00)80832-6

  • CrossRef
  • Google Scholar
  • View reference in article

• 57

  HaagS. (2011). Effects of vision and haptics on categorizing common objects.Cogn. Process.1233–39. 10.1007/s10339-010-0369-5

  • CrossRef
  • Google Scholar
  • View reference in article

• 58

  HarelA.KravitzD.BakerC. I. (2013). Beyond perceptual expertise: revisiting the neural substrates of expert object recognition.Front. Hum. Neurosci.7:885. 10.3389/fnhum.2013.00885

  • CrossRef
  • Google Scholar
  • View reference in article

• 59

  HarveyD. Y.BurgundE. D. (2012). Neural adaptation across viewpoint and exemplar in fusiform cortex.Brain Cogn.8033–44. 10.1016/j.bandc.2012.04.009

  • CrossRef
  • Google Scholar
  • View reference in article

• 60

  HelbigH. B.ErnstM. O. (2007). Optimal integration of shape information from vision and touch.Exp. Brain Res.179595–606. 10.1007/s00221-006-0814-y

  • CrossRef
  • Google Scholar
  • View reference in article

• 61

  HelbigH. B.ErnstM. O.RicciardiE.PietriniP.ThielscherA.MayerK. M.et al (2012). The neural mechanisms of reliability weighted integration of shape information from vision and touch.Neuroimage601063–1072. 10.1016/j.neuroimage.2011.09.072

  • CrossRef
  • Google Scholar
  • View reference in article

• 62

  IrikiA.TanakaM.IwamuraY. (1996). Attention-induced neuronal activity in the monkey somatosensory cortex revealed by pupillometrics.Neurosci. Res.25173–181. 10.1016/0168-0102(96)01043-7

  • CrossRef
  • Google Scholar
  • View reference in article

• 63

  IshaiA.HaxbyJ. V.UngerleiderL. G. (2002). Visual imagery of famous faces: effects of memory and attention revealed by fMRI.Neuroimage171729–1741. 10.1006/nimg.2002.1330

  • CrossRef
  • Google Scholar
  • View reference in article

• 64

  JamesT. W.HumphreyG. K.GatiJ. S.MenonR. S.GoodaleM. A. (2002a). Differential effects of view on object-driven activation in dorsal and ventral streams.Neuron35793–801. 10.1016/S0896-6273(02)00803-6

  • CrossRef
  • Google Scholar
  • View reference in article

• 65

  JamesT. W.HumphreyG. K.GatiJ. S.ServosP.MenonR. S.GoodaleM. A. (2002b). Haptic study of three-dimensional objects activates extrastriate visual areas.Neuropsychologia401706–1714. 10.1016/S0028-3932(02)00017-9

  • CrossRef
  • Google Scholar
  • View reference in article

• 66

  JamesT. W.ServosP.KilgourA. R.HuhE.LedermanS. (2006a). The influence of familiarity on brain activation during haptic exploration of 3-D facemasks.Neurosci. Lett.397269–273. 10.1016/j.neulet.2005.12.052

  • CrossRef
  • Google Scholar
  • View reference in article

• 67

  JamesT. W.JamesK. H.HumphreyG. K.GoodaleM. A. (2006b). “Do visual and tactile object representations share the same neural substrate?” inTouch and Blindness: Psychology and NeuroscienceedsHellerM. A.BallesterosS. (Mahwah, NJ: Lawrence Erlbaum Associates), 139–155.

  • Google Scholar
  • View reference in article

• 68

  JamesT. W.StevensonR. W.KimS.VanDerKlokR. M.JamesK. H. (2011). Shape from sound: evidence for a shape operator in the lateral occipital cortex.Neuropsychologia491807–1815. 10.1016/j.neuropsychologia.2011.03.004

  • CrossRef
  • Google Scholar
  • View reference in article

• 69

  JänckeL.KleinschmidtA.MirzazadeS.ShahN. J.FreundH.-J. (2001). The role of the inferior parietal cortex in linking the tactile perception and manual construction of object shapes.Cereb. Cortex11114–121. 10.1093/cercor/11.2.114

  • CrossRef
  • Google Scholar
  • View reference in article

• 70

  JolicoeurP. (1987). A size-congruency effect in memory for visual shape.Mem. Cogn.15531–543. 10.3758/BF03198388

  • CrossRef
  • Google Scholar
  • View reference in article

• 71

  JonesB. (1981). “The developmental significance of cross-modal matching,” inIntersensory Perception and Sensory Integration,edsWalkR. D.PickH. L.Jr. (New York: Plenum Press), 108–136.

  • Google Scholar
  • View reference in article

• 72

  KassubaT.KlingeC.HöligC.RöderB.SiebnerH. R. (2013). Vision holds a greater share in visuo-haptic object recognition than touch.Neuroimage6559–68. 10.1016/j.neuroimage.2012.09.054

  • CrossRef
  • Google Scholar
  • View reference in article

• 73

  KilgourA. R.de GelderB.LedermanS. (2004). Haptic face recognition and prosopagnosia.Neuropsychologia42707–712. 10.1016/j.neuropsychologia.2003.11.021

  • CrossRef
  • Google Scholar
  • View reference in article

• 74

  KilgourA. R.KitadaR.ServosP.JamesT. W.LedermanS. J. (2005). Haptic face identification activates ventral occipital and temporal areas: an fMRI study.Brain Cogn.59246–257. 10.1016/j.bandc.2005.07.004

  • CrossRef
  • Google Scholar
  • View reference in article

• 75

  KilgourA. R.LedermanS. (2002). Face recognition by hand.Percept. Psychophys.64339–352. 10.3758/BF03194708

  • CrossRef
  • Google Scholar
  • View reference in article

• 76

  KilgourA. R.LedermanS. (2006). A haptic face-inversion effect.Perception35921–931. 10.1068/p5341

  • CrossRef
  • Google Scholar
  • View reference in article

• 77

  KiphartM. J.HughesJ. L.SimmonsJ. P.CrossH. A. (1992). Short-term haptic memory for complex objects.Bull. Psychon. Soc.30212–214. 10.3758/BF03330444

  • CrossRef
  • Google Scholar
  • View reference in article

• 78

  KlatzkyR. L.AbramowiczA.HamiltonC.LedermanS. J. (2011). Irrelevant visual faces influence haptic identification of facial expressions of emotion.Atten. Percept. Psychophys.73521–530. 10.3758/s13414-010-0038-x

  • CrossRef
  • Google Scholar
  • View reference in article

• 79

  KlatzkyR. J.LedermanS. J. (1992). Stages of manual exploration in haptic object identification.Percept. Psychophys.52661–670. 10.3758/BF03211702

  • CrossRef
  • Google Scholar
  • View reference in article

• 80

  KlatzkyR. L.LedermanS. J. (1995). Identifying objects from a haptic glance.Percept. Psychophys.571111–1123. 10.3758/BF03208368

  • CrossRef
  • Google Scholar
  • View reference in article

• 81

  KlatzkyR. L.LedermanS. J.MetzgerV. A. (1985). Identifying objects by touch: an ‘expert system’.Percept. Psychophys.37299–302. 10.3758/BF03211351

  • CrossRef
  • Google Scholar
  • View reference in article

• 82

  KlatzkyR. L.LedermanS. J.ReedC. L. (1987). There’s more to touch than meets the eye: the salience of object attributes for haptics with and without vision.J. Exp. Psychol. Gen.116356–369. 10.1037/0096-3445.116.4.356

  • CrossRef
  • Google Scholar
  • View reference in article

• 83

  KonkleT.OlivaA. (2011). Canonical visual size for real-world objects.J. Exp. Psychol. Hum. Percept. Perform.3723–37. 10.1037/a0020413

  • CrossRef
  • Google Scholar
  • View reference in article

• 84

  KozhevnikovM.HegartyM.MayerR. E. (2002). Revising the visualiser-verbaliser dimension: evidence for two types of visualisers.Cogn. Instr.2047–77. 10.1207/S1532690XCI2001_3

  • CrossRef
  • Google Scholar
  • View reference in article

• 85

  KozhevnikovM.KosslynS. M.ShephardJ. (2005). Spatial versus object visualisers: a new characterisation of cognitive style.Mem. Cogn.33710–726. 10.3758/BF03195337

  • CrossRef
  • Google Scholar
  • View reference in article

• 86

  KrekelbergB.BoyntonG. M.van WezelR. J. A. (2006). Adaptation: from single cells to BOLD signals.Trends Neurosci.29250–256. 10.1016/j.tins.2006.02.008

  • CrossRef
  • Google Scholar
  • View reference in article

• 87

  LaceyS.CampbellC. (2006). Mental representation in visual/haptic crossmodal memory: evidence from interference effects.Q. J. Exp. Psychol.59361–376. 10.1080/17470210500173232

  • CrossRef
  • Google Scholar
  • View reference in article

• 88

  LaceyS.FlueckigerP.StillaR.LavaM.SathianK. (2010a). Object familiarity modulates the relationship between visual object imagery and haptic shape perception.Neuroimage491977–1990. 10.1016/j.neuroimage.2009.10.081

  • CrossRef
  • Google Scholar
  • View reference in article

• 89

  LaceyS.HallJ.SathianK. (2010b). Are surface properties integrated into visuo-haptic object representations?Eur. J. Neurosci.311882–1888. 10.1111/j.1460-9568.2010.07204.x

  • CrossRef
  • Google Scholar
  • View reference in article

• 90

  LaceyS.LawsonR. (2013). “Imagery questionnaires: vividness and beyond,” inMultisensory Imagery,edsLaceyS.LawsonR. (New York: Springer), 271–282.

  • Google Scholar
  • View reference in article

• 91

  LaceyS.LinJ. B.SathianK. (2011). Object and spatial imagery dimensions in visuo-haptic representations.Exp. Brain Res.213267–273. 10.1007/s00221-011-2623-1

  • CrossRef
  • Google Scholar
  • View reference in article

• 92

  LaceyS.PetersA.SathianK. (2007). Cross-modal object representation is viewpoint-independent.PLoS ONE2:e890. 10.1371/journal.pone0000890

  • CrossRef
  • Google Scholar
  • View reference in article

• 93

  LaceyS.SathianK. (2011). Multisensory object representation: insights from studies of vision and touch.Prog. Brain Res.191165–176. 10.1016/B978-0-444-53752-2.00006-0

  • CrossRef
  • Google Scholar
  • View reference in article

• 94

  LaceyS.StillaR.SreenivasanK.DeshpandeG.SathianK. (2014). Spatial imagery in haptic shape perception.Neuropsychologia10.1016/j.neuropsycholopia.2014.05.008[Epub ahead of print].

  • CrossRef
  • Google Scholar
  • View reference in article

• 95

  LaceyS.TalN.AmediA.SathianK. (2009a). A putative model of multisensory object representation.Brain Topogr.21269–274. 10.1007/s10548-009-0087-4

  • CrossRef
  • Google Scholar
  • View reference in article

• 96

  LaceyS.PappasM.KrepsA.LeeK.SathianK. (2009b). Perceptual learning of view-independence in visuo-haptic object representations.Exp. Brain Res.198329–337. 10.1007/s00221-009-1856-8

  • CrossRef
  • Google Scholar
  • View reference in article

• 97

  LawsonR. (2009). A comparison of the effects of depth rotation on visual and haptic three-dimensional object recognition.J. Exp. Psychol. Hum. Percept. Perform.35911–930. 10.1037/a0015025

  • CrossRef
  • Google Scholar
  • View reference in article

• 98

  LawsonR. (2011). An investigation into the cause of orientation-sensitivity in haptic object recognition.Seeing Perceiving24293–314. 10.1163/187847511X579052

  • CrossRef
  • Google Scholar
  • View reference in article

• 99

  LawsonR. (2014). Recognizing familiar objects by hand and foot: haptic shape perception generalizes to inputs from unusual locations and untrained body parts.Atten. Percept. Psychophys.76541–558. 10.3758/s13414-013-0559-1

  • CrossRef
  • Google Scholar
  • View reference in article

• 100

  LedermanS. J.KlatzkyR. L. (1987). Hand movements: a window into haptic object recognition.Cogn. Psychol.19342–368. 10.1016/0010-0285(87)90008-9

  • CrossRef
  • Google Scholar
  • View reference in article

• 101

  LoomisJ.KlatzkyR. L.LedermanS. J. (1991). Similarity of tactual and visual picture recognition with limited field of view.Perception20167–177. 10.1068/p200167

  • CrossRef
  • Google Scholar

• 102

  LucanJ. N.FoxeJ. J.Gomez-RamirezM.SathianK.MolholmS. (2010). Tactile shape discrimination recruits human lateral occipital complex during early perceptual processing.Hum. Brain Mapp.311813–1821. 10.1002/hbm.20983

  • CrossRef
  • Google Scholar

• 103

  MalachR.ReppasJ. B.BensonR. R.KwongK. K.JiangH.KennedyW. A.et al (1995). Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex.Proc. Natl. Acad. Sci. U.S.A.928135–8139. 10.1073/pnas.92.18.8135

  • CrossRef
  • Google Scholar

• 104

  MarrD. (1976). Early processing of visual information.Philos. Trans. R. Soc. Lond. B Biol. Sci.275483–524. 10.1098/rstb.1976.0090

  • CrossRef
  • Google Scholar

• 105

  MatsumiyaK. (2012). Haptic face aftereffect.Iperception397–100.

  • Google Scholar

• 106

  MatsumiyaK. (2013). Seeing a haptically explored face: visual facial-expression aftereffect from haptic adaptation to a face.Psychol. Sci.242088–2098. 10.1177/0956797613486981

  • CrossRef
  • Google Scholar

• 107

  MaurerD.Le GrandR.MondlochC. J. (2002). The many faces of configural processing.Trends Cogn. Sci.6255–260. 10.1016/S1364-6613(02)01903-4

  • CrossRef
  • Google Scholar

• 108

  MechelliA.PriceC. J.FristonK. J.IshaiA. (2004). Where bottom-up meets top-down: neuronal interactions during perception and imagery.Cereb. Cortex141256–1265. 10.1093/cercor/bhh087

  • CrossRef
  • Google Scholar

• 109

  MullinC. R.SteevesJ. K. E. (2011). TMS to the lateral occipital cortex disrupts object processing but facilitates scene processing.J. Cogn. Neurosci.234174–4184. 10.1162/jocn_a_00095

  • CrossRef
  • Google Scholar

• 110

  MurM.RuffD. A.BodurkaJ.BandettiniP. A.KriegeskorteN. (2010). Face-identity change activation outside the face system: “release from adaptation” may not always indicate neuronal selectivity.Cereb. Cortex202027–2042. 10.1093/cercor/bhp272

  • CrossRef
  • Google Scholar

• 111

  NabetaT.KawaharaJ. (2006). Congruency effect of presentation modality on false recognition of haptic and visual objects.Memory14307–315. 10.1080/09658210500277398

  • CrossRef
  • Google Scholar

• 112

  NassiJ. J.CallawayE. M. (2009). Parallel processing strategies of the primate visual system.Nat. Rev. Neurosci.10360–372. 10.1038/nrn2619

  • CrossRef
  • Google Scholar

• 113

  NégyessyL.NepuszT.KocsisL.BazsóF. (2006). Prediction of the main cortical areas and connections involved in the tactile function of the visual cortex by network analysis.Eur. J. Neurosci.231919–1930. 10.1111/j.1460-9568.2006.04678.x

  • CrossRef
  • Google Scholar

• 114

  NewellF. N.ErnstM. O.TjanB. S.BülthoffH. H. (2001). View dependence in visual and haptic object recognition.Psychol. Sci.1237–42. 10.1111/1467-9280.00307

  • CrossRef
  • Google Scholar

• 115

  NewmanS. D.KlatzkyR. L.LedermanS. J.JustM. A. (2005). Imagining material versus geometric properties of objects: an fMRI study.Cogn. Brain Res.23235–246. 10.1016/j.cogbrainres.2004.10.020

  • CrossRef
  • Google Scholar

• 116

  NicholsonK. G.HumphreyG. K. (2003). The effect of colour congruency on shape discriminations of novel objects.Perception32339–353. 10.1002/hbm.20983

  • CrossRef
  • Google Scholar

• 117

  PalmerS. E.RoschE.ChaseP. (1981). “Canonical perspective and the perception of objects,” inAttention and Performance IXedsLongJ.BaddeleyA. (Hillsdale, NJ: Lawrence Erlbaum Associates), 135–151.

  • Google Scholar

• 118

  Pascual-LeoneA.AmediA.FregniF.MerabetL. B. (2005). The plastic human brain.Annu. Rev. Neurosci.28377–401. 10.1146/annurev.neuro.27.070203.144216

  • CrossRef
  • Google Scholar

• 119

  Pascual-LeoneA.HamiltonR. H. (2001). The metamodal organization of the brain.Prog. Brain Res.134427–445. 10.1016/S0079-6123(01)34028-1

  • CrossRef
  • Google Scholar

• 120

  PasqualottoA.FinucaneC.NewellF. N. (2005). Visual and haptic representations of scenes are updated with observer movement.Exp. Brain Res.166481–488. 10.1007/s00221-005-2388-5

  • CrossRef
  • Google Scholar

• 121

  PasqualottoA.NewellF. N. (2007). The role of visual experience on the representation and updating of novel haptic scenes.Brain Cogn.65184–194. 10.1016/j.bandc.2007.07.009

  • CrossRef
  • Google Scholar

• 122

  PeissigJ. J.TarrM. J. (2007). Visual object recognition: do we know more now than we did 20 years ago?Annu. Rev. Psychol.5875–96. 10.1146/annurev.psych.58.102904.190114

  • CrossRef
  • Google Scholar

• 123

  PenskyA. E. C.JohnsonK. A.HaagS.HomaD. (2008). Delayed memory for visual-haptic exploration of objects.Psychon. Bull. Rev.15574–580. 10.3758/PBR.15.3.574

  • CrossRef
  • Google Scholar

• 124

  PetersonM. A.RhodesG. (2003). Perception of Faces, Objects, and Scenes: Analytic and Holistic Processes.Oxford: Oxford University Press.

  • Google Scholar

• 125

  PietriniP.FureyM. L.RicciardiE.GobbiniM. I.WuW.-H. C.CohenL.et al (2004). Beyond sensory images: object-based representation in the human ventral pathway.Proc. Natl. Acad. Sci. U.S.A.1015658–5663. 10.1073/pnas.0400707101

  • CrossRef
  • Google Scholar

• 126

  PitzalisS.GallettiC.HuangR. S.PatriaF.CommitteriG.GalatiG.et al (2006). Wide-field retinotopy defines human cortical visual area V6.J. Neurosci.267962–7973. 10.1523/JNEUROSCI.0178-06.2006

  • CrossRef
  • Google Scholar

• 127

  PourtoisG.SchwarzS.SpiridonM.MartuzziR.VuilleumierP. (2009). Object representations for multiple visual categories overlap in lateral occipital and medial fusiform cortex.Cereb. Cortex191806–1819. 10.1093/cercor/bhn210

  • CrossRef
  • Google Scholar

• 128

  PratherS. C.VotawJ. R.SathianK. (2004). Task-specific recruitment of dorsal and ventral visual areas during tactile perception.Neuropsychologia421079–1087. 10.1016/j.neuropsychologia.2003.12.013

  • CrossRef
  • Google Scholar

• 129

  RealesJ. M.BallesterosS. (1999). Implicit and explicit memory for visual and haptic objects: cross-modal priming depends on structural descriptions.J. Exp. Psychol. Learn. Mem. Cogn.25644–663. 10.1037/0278-7393.25.3.644

  • CrossRef
  • Google Scholar

• 130

  ReedC. L.KlatzkyR. L.HalgrenE. (2005). What vs. where in touch: an fMRI study.Neuroimage25718–726. 10.1016/j.neuroimage.2004.11.044

  • CrossRef
  • Google Scholar

• 131

  RenierL.CollignonO.TranduyD.PoirierC.VanlierdeA.VeraartC.et al (2004). Visual cortex activation in early blind and sighted subjects using an auditory visual substitution device to perceive depth.Neuroimage22 S1.

  • Google Scholar

• 132

  RenierL.CollignonO.PoirierC.TranduyD.VanlierdeA.BolA.et al (2005). Cross modal activation of visual cortex during depth perception using auditory substitution of vision.Neuroimage26573–580. 10.1016/j.neuroimage.2005.01.047

  • CrossRef
  • Google Scholar

• 133

  RiceN. J.ValyearK. F.GoodaleM. A.MilnerA. D.CulhamJ. C. (2007). Orientation sensitivity to graspable objects: an fMRI adaptation study.Neuroimage36T87–T93. 10.1016/j.neuroimage.2007.03.032

  • CrossRef
  • Google Scholar

• 134

  RiesenhuberM.PoggioT. (1999). Hierarchical models of object recognition in cortex.Nat. Neurosci.21019–1025. 10.1038/14819

  • CrossRef
  • Google Scholar

• 135

  SackA. T. (2006). Transcranial magnetic stimulation, causal structure-function mapping and networks of functional relevance.Curr. Opin. Neurobiol.16593–599. 10.1016/j.conb.2006.06.016

  • CrossRef
  • Google Scholar

• 136

  SaitoD. N.OkadaT.MoritaY.YonekuraY.SadatoN. (2003). Tactile-visual cross-modal shape matching: a functional MRI study.Cogn. Brain Res.1714–25. 10.1016/S0926-6410(03)00076-4

  • CrossRef
  • Google Scholar

• 137

  SathianK. (2005). Visual cortical activity during tactile perception in the sighted and the visually deprived.Dev. Psychobiol.46279–286. 10.1002/dev.20056

  • CrossRef
  • Google Scholar

• 138

  SathianK.DeshpandeG.StillaR. (2013). Neural changes with tactile learning reflect decision-level reweighting of perceptual readout.J. Neurosci.335387–5398. 10.1523/JNEUROSCI.3482-12.2013

  • CrossRef
  • Google Scholar

• 139

  SathianK.LaceyS. (2007). Journeying beyond classical somatosensory cortex.Can. J. Exp. Psychol.61254–264. 10.1037/cjep2007026

  • CrossRef
  • Google Scholar

• 140

  SathianK.LaceyS.StillaR.GibsonG.DeshpandeG.HuX.et al (2011). Dual pathways for somatosensory information processing.Neuroimage57462–475. 10.1016/j.neuroimage.2011.05.001

  • CrossRef
  • Google Scholar

• 141

  SathianK.ZangaladzeA.HoffmanJ. M.GraftonS. T. (1997). Feeling with the mind’s eye.Neuroreport83877–3881. 10.1097/00001756-199712220-00008

  • CrossRef
  • Google Scholar

• 142

  SawamuraH.GeorgievaS.VogelsR.VanduffelW.OrbanG. A. (2005). Using functional magnetic resonance imaging to assess adaptation and size invariance of shape processing by humans and monkeys.J. Neurosci.254294–4306. 10.1523/JNEUROSCI.0377-05.2005

  • CrossRef
  • Google Scholar

• 143

  SilvantoJ.SchwarzkopfD. S.Gilaie-DotanS.ReesG. (2010). Differing causal roles for lateral occipital complex and occipital face area in invariant shape recognition.Eur. J. Neurosci.32165–171. 10.1111/j.1460-9568.2010.07278.x

  • CrossRef
  • Google Scholar

• 144

  SkinnerA. L.BentonC. P. (2010). Anti-expression aftereffects reveal prototype-referenced coding of facial expressions.Psychol. Sci.211248–1253. 10.1177/0956797610380702

  • CrossRef
  • Google Scholar

• 145

  SmithE. E.PatalanoA. L.JonidesJ. (1998). Alternative strategies of categorization.Cognition65167–196. 10.1016/S0010-0277(97)00043-7

  • CrossRef
  • Google Scholar

• 146

  SnowJ. C.StrotherL.HumphreysG. W. (2014). Haptic shape processing in visual cortex.J. Cogn. Neurosci.261154–1167. 10.1162/jocn_a_00548

  • CrossRef
  • Google Scholar

• 147

  StillaR.SathianK. (2008). Selective visuo-haptic processing of shape and texture.Hum. Brain Mapp.291123–1138. 10.1002/hbm.20456

  • CrossRef
  • Google Scholar

• 148

  StoeszM.ZhangM.WeisserV. D.PratherS. C.MaoH.SathianK. (2003). Neural networks active during tactile form perception: common and differential activity during macrospatial and microspatial tasks.Int. J. Psychophysiol.5041–49. 10.1016/S0167-8760(03)00123-5

  • CrossRef
  • Google Scholar

• 149

  StreriA.MolinaM. (1994). “Constraints on intermodal transfer between touch and vision in infancy,” inThe Development of Intersensory Perception: Comparative PerspectivesedsLewkowiczD. J.LickliterR. (Hove: Lawrence Erlbaum Associates), 285–307.

  • Google Scholar

• 150

  TakahashiC.WattS. J. (2014). Visual-haptic integration with pliers and tongs: signal ‘weights’ take account of changes in haptic sensitivity caused by different tools.Front. Psychol.5:109. 10.3389/fpsyg.2014.00109

  • CrossRef
  • Google Scholar

• 151

  TalN.AmediA. (2009). Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach.Exp. Brain Res.198165–182. 10.1007/s00221-009-1949-4

  • CrossRef
  • Google Scholar

• 152

  TarrM. J.PinkerS. (1989). Mental rotation and orientation dependence in shape recognition.Cogn. Psychol.21233–282. 10.1016/0010-0285(89)90009-1

  • CrossRef
  • Google Scholar

• 153

  UedaY.SaikiJ. (2007). Viewpoint independence in visual and haptic object recognition.Jpn. J. Psychon. Sci.2611–19.

  • Google Scholar

• 154

  UedaY.SaikiJ. (2012). Characteristics of eye movements in 3-D object learning: comparison between within-modal and cross-modal object recognition.Perception411289–1298. 10.1068/p7257

  • CrossRef
  • Google Scholar

• 155

  UngerleiderL. G.MishkinM. (1982). “Two cortical visual systems,” inAnalysis of Visual BehavioredsIngleD. J.GoodaleM. A.MansfieldR. J. W. (Cambridge, MA: MIT Press), 549–586.

  • Google Scholar

• 156

  UttlB.GrafP.SiegenthalerA. L. (2007). Influence of object size on baseline identification, priming, and explicit memory.Scand. J. Psychol.48281–288. 10.1111/j.1467-9450.2007.00571.x

  • CrossRef
  • Google Scholar

• 157

  ValyearK. F.CulhamJ. C.SharifN.WestwoodD.GoodaleM. A. (2006). A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal streams: a human fMRI study.Neuropsychologia44218–228. 10.1016/j.neuropsychologia.2005.05.004

  • CrossRef
  • Google Scholar

• 158

  WallravenC. (2014). Touching on face space: comparing visual, and haptic processing of face shapes.Psychon. Bull. Rev.10.3758/s13423-013-0577-y[Epub ahead of print].

  • CrossRef
  • Google Scholar

• 159

  WallravenC.BülthoffH. H.WaterkampS.van DamL.GaiβertN. (2013). The eyes grasp, the hands see: metric category knowledge transfers between vision and touch.Psychon. Bull. Rev.10.3758/s13423-013-0563-4[Epub ahead of print].

  • CrossRef
  • Google Scholar

• 160

  WallravenC.DopjansL. (2013). Visual experience is necessary for efficient haptic face recognition.Neuroreport24254–258. 10.1097/WNR.0b013e32835f00c0

  • CrossRef
  • Google Scholar

• 161

  WilsonK.FarahM. J. (2006). Distinct patterns of viewpoint-dependent BOLD activity during common-object recognition and mental rotation.Perception351351–1366. 10.1068/p5571

  • CrossRef
  • Google Scholar

• 162

  WoodsA. T.MooreA.NewellF. N. (2008). Canonical views in haptic object perception.Perception371867–1878. 10.1068/p6038

  • CrossRef
  • Google Scholar

• 163

  WoodsA. T.O’ModhrainS.NewellF. N. (2004). The effect of temporal delay and spatial differences on crossmodal object recognition.Cogn. Affect. Behav. Neurosci.4260–269. 10.3758/CABN.4.2.260

  • CrossRef
  • Google Scholar

• 164

  YildirimI.JacobsR. A. (2013). Transfer of object category knowledge across visual and haptic modalities: experimental and computational studies.Cognition126135–148. 10.1016/j.cognition.2012.08.005

  • CrossRef
  • Google Scholar

• 165

  YinR. K. (1969). Looking at upside-down faces.J. Exp. Psychol.81141–145. 10.1037/h0027474

  • CrossRef
  • Google Scholar

• 166

  ZangaladzeA.EpsteinC. M.GraftonS. T.SathianK. (1999). Involvement of visual cortex in tactile discrimination of orientation.Nature401587–590. 10.1038/44139

  • CrossRef
  • Google Scholar

• 167

  ZhangM.WeisserV. D.StillaR.PratherS. C.SathianK. (2004). Multisensory cortical processing of object shape and its relation to mental imagery.Cogn. Affect. Behav. Neurosci.4251–259. 10.3758/CABN.4.2.251

  • CrossRef
  • Google Scholar

• 168

  ZhouY.-D.FusterJ. M. (1997). Neuronal activity of somatosensory cortex in a cross-modal (visuo-haptic) memory task.Exp. Brain Res.116551–555. 10.1007/PL00005783

  • CrossRef
  • Google Scholar