Impact Factor 2.089

The world's most-cited Multidisciplinary Psychology journal

Editorial ARTICLE

Front. Psychol., 22 November 2016 | https://doi.org/10.3389/fpsyg.2016.01593

Editorial: Perception–Cognition Interface and Cross-Modal Experiences: Insights into Unified Consciousness

  • Institute of Philosophy of Mind and Cognition, National Yang-Ming University, Taipei, Taiwan

The present Research Topic explores closely related aspects of mental functioning, namely an interplay between perception and cognition, interactions among various sensory modalities, and finally, more or less unified conscious experiences arising in the context of these relations. Contributions emphasize a high flexibility observed in perception and may be seen as potential challenges to the traditional modular architecture of perceptual systems. Although the articles describe different phenomena, they follow one common theme—to investigate broadly understood unified experience—by studying either perception–cognition integration or the integration between sensory modalities. These integrative processes may well apply to subpersonal unconscious representations. However, the aim here is to approach phenomenal experience and thus a straightforward way of thinking about it is in terms of conscious perception.

One of the seemingly principled divisions in the human mind is between sense perception and high-level cognition. Traditionally, perception and cognition have been viewed as distinct, encapsulated domains operating independently of each other (Fodor, 1983, 1984, 2008; Pylyshyn, 1999, 2003; Barrett, 2005; Heck, 2007; Firestone and Scholl, 2014). However, recent studies support a different view about the impact of perception on cognition (Barsalou, 2009, 2012; Goldstone and Hendrickson, 2010; Prinz, 2011; Weiskopf, 2015) as well as the various ways in which perceptual experiences can be influenced by cognitive states such as thoughts, judgments, beliefs, intuitions, expectations, desires, mental images, and emotions (Brockmole et al., 2002; Raftopoulos, 2011; Lupyan, 2012; Macpherson, 2012, 2016; Siegel, 2012; Stokes, 2012; Bannert and Bartels, 2013; Vetter and Newen, 2014; Raftopoulos and Zeimbekis, 2016). Thus, although the mentioned division between perceiving and reasoning may seem conceptually clear and unambiguous, these mental domains become closely intertwined when our beliefs, expectations, or desires affect what we see, hear, or taste, leading to complex phenomenal states of a hybrid nature. No matter whether we assume that there is no dichotomy between perception and cognition (Clark, 2013; Lupyan, 2015) or instead assume that there is a principled difference and a joint in nature between perception and cognition (Block, Submitted; cf. Firestone and Scholl, 2015), cognitive-sensory interactions can and need to be accommodated within any of these accounts.

Perception has typically been studied in a single sense, mostly in the visual or auditory modality (Haynes et al., 2005; Gutschalk et al., 2008; Bekinschtein et al., 2009; Dehaene and Changeux, 2011; De Graaf et al., 2012). However, cross-modal experiences and heterogeneous multisensory interactions, in which input in one sensory modality elicits or modulates contents in another modality, reveal that such perceptual experiences cannot be easily categorized as belonging to only one of the senses (De Gelder and Bertelson, 2003; Stein, 2012). Furthermore, recent studies suggest examining the role of multisensory signals in perceptual consciousness (Chen and Spence, 2010; De Meo et al., 2015; Deroy et al., 2016). While processing sensory information in cross-modal cases is generally multisensory, the result of that processing can be interpreted as either just a sum of coexisting modality-specific representations or an intrinsically multisensory whole. Determining whether multisensory processing results in a decomposable conjunction of independent unisensory contents or in a multimodal holistic state that cannot be parceled out into modality-specific components would provide the needed characterization of the basic units of perceptual consciousness (Bayne, 2014). Still, it is important to realize that instances of successful multisensory integration and cross-modal binding facilitated by spatio-temporal or semantic congruence are not necessarily accompanied by unified experiences of objects across the senses and that the complex relationship between multisensory integration and perceptual consciousness remains to be clarified (Deroy, 2014; c.f. Spence and Bayne, 2015).

Recent years have seen a surge of novel interdisciplinary work questioning the received view of separate sensory systems and traditional conceptions of different mental domains operating independently (Shimojo and Shams, 2001; Driver and Noesselt, 2008; Bayne, 2010; Macpherson, 2011a,b; Mroczko-Wąsowicz, 2013, 2016; Bennett and Hill, 2014; Deroy et al., 2014; de Vignemont, 2014a,b; Mroczko-Wąsowicz and Nikolić, 2014; O'Callaghan, 2014; Matthen, 2015; Stokes et al., 2015). For instance, the occurrence of cross-domain interchange going beyond the link between the sense perception and the domain of abstract, conceptually represented entities, i.e., extending to the domains of bodily, motor, and emotional states provides challenges to standard methods individuating our epistemic abilities (O'Regan and Noë, 2001; Barsalou, 2008; Tajadura-Jiménez et al., 2011, 2015; Mroczko-Wąsowicz and Werning, 2012; De Coster et al., 2013; Weiss et al., 2013; Shapiro, 2014; Goldstone et al., 2015). Results of these studies point to a significant change in our understanding of perception; they demonstrate an emerging agreement on an integrative picture of perception incorporating informational interactions. All this indicates a need for a new research methodology. A full understanding of how the mind works requires considering multifaceted links holding between various mental domains and their mutual impact. Our mental faculties should not only be studied separately. They require a more holistic approach in order to uncover their extensive capacity for interactions producing differently unified conscious experiences.

Putting together scientific and philosophical concerns, this special issue encourages extending the study of perceptual experience beyond the single sense perception to advance our understanding of the complex interdependencies between different sensory modalities, other mental domains, and various kinds of unifying relations within conscious experience. It exhibits a remarkable need and benefit to study these phenomena in tangent, and so, the articles in this Research Topic examine a variety of ways in which our perceptual experiences may be cross-modal or multisensory, integrated, embodied, synesthetic, or affected by top-down influences.

Fulkerson argues that there are many forms of sensory interaction and unity, therefore classification of sensory systems and generated experiences is a matter of particular explanatory projects.

Connolly suggests that it is not an automatic feature binding mechanism that is responsible for our multimodal perceptual experiences, but rather an associative learning process that couples features from different sensory modalities so that we experience them as part of the same event.

Liang et al. investigate experiential ownership of bodily sensation and if it is guaranteed that a subject cannot be wrong about whether it is him who feels the sensation.

Van Leeuwen et al. propose several reasons for why the phenomenon of synesthesia and related alterations of brain networks and functional connectivity can be of merit for consciousness research.

Gray and Simner consider synesthesia and release phenomena in terms of disinhibited embodiment in sensory and motor systems respectively.

The following papers explore how perceptual processes can fail to be modular. They discuss a range of questions regarding cognitive effects on perception, including the issue of cognitive penetrability of perception.

In their contribution, Masrour et al. guide readers through philosophical issues of the debate on perceptual modularity, emphasizing results from cognitive neuroscience against the encapsulation thesis.

Marchi and Newen address the possibility of cognitive penetrability of perceptual experience in the domain of social cognition, namely visual experience of the facial expressions of emotion.

Nanay argues that the attribution of aesthetically relevant properties supervenes on one's perceptual experience, i.e., if there is a difference in such an attribution, there must also be a difference in perceptual experience.

Briscoe considers whether intentions for action penetrate visual experience of an object's size by analyzing various explanations of mechanisms possibly involved in such penetration.

Brown claims that theories of consciousness that see it as cognitive in nature or as an aspect of cognitive functioning such as the higher-order thought theory of consciousness provide a reasonable working hypothesis in the explanation of conscious experience.

There is also some grouping among the contributions discussing empirical results from their own studies on cross-modal associations, sensory integration, and unified conscious experience.

Brunel et al. demonstrate that cross-modal correspondences influence cross-modal integration during perceptual learning, leading to new learned units that have different stability over time.

Montoro et al. explore cross-modal metaphorical mapping of auditory emotion words onto vertical visual space and conclude that this association is not automatically activated but requires an explicit semantic evaluation of the emotion concepts to obtain an embodied effect.

Albertazzi et al. examine the existence of cross-modal associations between highly complex stimuli (i.e., materic painting and classical guitar music) due to patterns of qualitative similarity present in stimuli of different sensory modalities.

Finally, Winkielman et al. propose that unified consciousness is constructed from cross-modal inputs via integrated processing experiences, an experiential mechanism that combines signals of processing quality.

Author Contributions

The author confirms being the sole contributor of this work and approved it for publication.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by the Ministry of Science and Technology (project: MOST104-2628-H-010-002-MY3).

References

Bayne (2014). “The multisensory nature of perceptual consciousness,” in Sensory Integration and the Unity of Consciousness, eds D. J. Bennett and C. S. Hill (Cambridge, MA: MIT Press), 15–36.

Bannert, M. M., and Bartels, A. (2013). Decoding the yellow of a gray banana. Curr. Biol. 23, 2268–2272. doi: 10.1016/j.cub.2013.09.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Barrett, H. C. (2005). Enzymatic computation and cognitive modularity. Mind Lang. 259–287. doi: 10.1111/j.0268-1064.2005.00285.x

CrossRef Full Text | Google Scholar

Barsalou, L. W. (2008). Grounded cognition. Annu. Rev. Psychol. 59, 617–645. doi: 10.1146/annurev.psych.59.103006.093639

PubMed Abstract | CrossRef Full Text | Google Scholar

Barsalou, L. W. (2009). Simulation, situated conceptualization, and prediction. Philos. Trans. R. Soc. Lond. Biol. Sci. 364, 1281–1289. doi: 10.1098/rstb.2008.0319

PubMed Abstract | CrossRef Full Text | Google Scholar

Barsalou, L. W. (2012). “The human conceptual system,” in The Cambridge Handbook of Psycholinguistics, eds M. Spivey, K. McRae, and M. Joanisse (New York, NY: Cambridge University Press), 239–258.

Google Scholar

Bayne, T. (2010). The Unity of Consciousness. New York, NY: Oxford University Press.

Bekinschtein, T. A., Dehaene, S., Rohaut, B., Tadel, F., Cohen, L., and Naccache, L. (2009). Neural signature of the conscious processing of auditory regularities. Proc. Natl Acad. Sci. U.S.A. 106, 1672–1677. doi: 10.1073/pnas.0809667106

PubMed Abstract | CrossRef Full Text | Google Scholar

Bennett, D. J., and Hill, C. S. (eds.). (2014). Sensory Integration and the Unity of Consciousness. Cambridge, MA: MIT Press.

Block, N. (Submitted). Tweaking the concepts of perception cognition. Behav. Brain Sci.

Google Scholar

Brockmole, J. R., Wang, R. F., and and, Irwin, D. E. (2002). Temporal integration between visual images and visual percepts. J. Exp. Psychol. Hum. Percept. Perform. 28, 315–334. doi: 10.1037//0096-1523.28.2.315

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Y.-C., and Spence, C. (2010). When hearing the bark helps to identify the dog: semantically-congruent sounds modulate the identification of masked pictures. Cognition 114, 389–404. doi: 10.1016/j.cognition.2009.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36, 181–204. doi: 10.1017/S0140525X12000477

PubMed Abstract | CrossRef Full Text | Google Scholar

De Coster, L., Verschuere, B., Goubert, L., Tsakiris, M., and Brass, M. (2013). I suffer more from your pain when you act like me: being imitated enhances affective responses to seeing someone else in pain. Cogn. Affect. Behav. Neurosci. 13, 519-532. doi: 10.3758/s13415-013-0168-4

PubMed Abstract | CrossRef Full Text | Google Scholar

De Gelder, B., and Bertelson, P. (2003). Multisensory integration, perception and ecological validity. Trends Cogn. Sci. 7, 460–467. doi: 10.1016/j.tics.2003.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

De Graaf, T. A., Hsieh, P. J., and Sack, A. T. (2012). The “correlates” in neural correlates of consciousness. Neurosci. Biobehav. Rev. 36, 191–197. doi: 10.1016/j.neubiorev.2011.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Dehaene, S., and Changeux, J.-P. (2011). Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227. doi: 10.1016/j.neuron.2011.03.018

PubMed Abstract | CrossRef Full Text | Google Scholar

De Meo, R., Murray, M. M., Clarke, S., and Matusz, P. J. (2015). Top-down control and early multisensory processes: chicken vs. egg. Front. Integr. Neurosci. 9:17. doi: 10.3389/fnint.2015.00017

PubMed Abstract | CrossRef Full Text | Google Scholar

Deroy, O. (2014). “The unity assumption and the many unities of consciousness,” in Sensory Integration and the Unity of Consciousness, eds D. Bennett and C. Hill (Cambridge, MA: MIT Press), 105–124.

Deroy, O., Chen, Y.-C., and Spence, C. (2014). Multisensory constraints on awareness. Phil. Trans. R. Soc. Lond. B Biol. Sci. 369:20130207. doi: 10.1098/rstb.2013.0207

PubMed Abstract | CrossRef Full Text | Google Scholar

Deroy, O., Faivre, N., Lunghi, C., Spence, C., Aller, M., and Noppeney, U. (2016). The complex interplay between multisensory integration and perceptual awareness. Multisens. Res. 29, 585–606. doi: 10.1163/22134808-00002529

PubMed Abstract | CrossRef Full Text | Google Scholar

de Vignemont, F. (2014a). “Multimodal unity and multimodal binding,” in Sensory Integration and the Unity of Consciousness, eds D. Bennett and C. Hill (Cambridge, MA: MIT Press), 125–150.

Google Scholar

de Vignemont, F. (2014b). A multimodal conception of bodily awareness. Mind 123, 989–1020. doi: 10.1093/mind/fzu089

CrossRef Full Text | Google Scholar

Driver, J., and Noesselt, T. (2008). Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments. Neuron 57, 11–23. doi: 10.1016/j.neuron.2007.12.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Firestone, C., and Scholl, B. J. (2014). “Top-down” effects where none should be found: the El Greco fallacy in perception research. Psychol. Sci. 25, 38–46. doi: 10.1177/0956797613485092

PubMed Abstract | CrossRef Full Text | Google Scholar

Firestone, C., and Scholl, B. J. (2015). Cognition does not affect perception: evaluating the evidence for “top-down” effects. Behav. Brain Sci. [Epub ahead of print]. doi: 10.1017/S0140525X15000965

PubMed Abstract | CrossRef Full Text | Google Scholar

Fodor, J. A. (1983). The Modularity of Mind: An Essay in Faculty Psychology. Cambridge, MA: MIT Press.

Google Scholar

Fodor, J. A. (1984). Observation reconsidered. Philos. Sci. 51, 23–43.

Fodor, J. A. (2008). LOT 2: The Language of Thought Revisited. Oxford: Oxford University Press.

Goldstone, R. L., de Leeuw, J. R., and Landy, D. H. (2015). Fitting perception in and to cognition. Cognition 135, 24–29. doi: 10.1016/j.cognition.2014.11.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldstone, R. L., and Hendrickson, A. T. (2010). Categorical perception. Wiley Interdiscipl. Rev. Cogn. Sci. 1, 69–78. doi: 10.1002/wcs.26

PubMed Abstract | CrossRef Full Text | Google Scholar

Gutschalk, A., Micheyl, C., and Oxenham, A. J. (2008). Neural correlates of auditory perceptual awareness under informational masking. PLoS Biol. 6:138. doi: 10.1371/journal.pbio.0060138

PubMed Abstract | CrossRef Full Text | Google Scholar

Haynes, J. D., Driver, J., and Rees, G. (2005). Visibility reflects dynamic changes of effective connectivity between V1 and fusiform cortex. Neuron 46, 811–821. doi: 10.1016/j.neuron.2005.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Heck, R. G. (2007). “Are there different kinds of content?” in Contemporary Debates in Philosophy of Mind, eds B. P.McLaughlin and J. Cohen (New York, NY: Blackwell Publishing), 117–138.

Lupyan, G. (2012). Linguistically modulated perception and cognition: the label-feedback hypothesis. Front. Psychol. 3:54. doi: 10.3389/fpsyg.2012.00054

PubMed Abstract | CrossRef Full Text | Google Scholar

Lupyan, G. (2015). Cognitive penetrability of perception in the age of prediction: predictive systems are penetrable systems. Rev. Philos. Psychol. 6, 547–569. doi: 10.1007/s13164-015-0253-4

CrossRef Full Text | Google Scholar

Macpherson, F. (2011a). Cross-modal experiences. Proc. Aristotelian Soc. 111, 429–468. doi: 10.1111/j.1467-9264.2011.00317.x

CrossRef Full Text | Google Scholar

Macpherson, F. (2011b). “Introduction: The admissible contents of experience,” in The Admissible Contents of Experience, eds F. Macpherson and K. Hawley (Chichester: Wiley-Blackwell), 1–15.

Google Scholar

Macpherson, F. (2012). Cognitive penetration of colour experience: Rethinking the issue in light of an indirect mechanism. Philos. Phenomenol. Res. 84, 24–62. doi: 10.1111/j.1933-1592.2010.00481.x

CrossRef Full Text | Google Scholar

Macpherson, F. (2016). The relationship between cognitive penetration and predictive coding. Conscious. Cogn. [Epub ahead of print] doi: 10.1016/j.concog.2016.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Matthen, M. (ed.). (2015). The Oxford Handbook of Philosophy of Perception. Oxford: Oxford University Press.

Google Scholar

Mroczko-Wąsowicz, A. (2013). The Unity of Consciousness and the Phenomenon of Synesthesia [Die Einheit des Bewusstseins und das Phänomen der Synästhesie]. Published doctoral dissertation, Johannes Gutenberg University of Mainz.

Mroczko-Wąsowicz, A. (2016). “The unity and complexity of flavour perception: multisensory phenomenology and processing in human-food interactions,” in Proceedings of ICMI 2016 Workshop on Multi-sensorial Approaches to Human-Food Interaction (New York, NY: ACM Digital Library). doi: 10.1145/3007577.3007583

CrossRef Full Text

Mroczko-Wąsowicz, A., and Nikolić, D. (2014). Semantic mechanisms may be responsible for developing synesthesia. Front. Hum. Neurosci. 8:509. doi: 10.3389/fnhum.2014.00509

PubMed Abstract | CrossRef Full Text | Google Scholar

Mroczko-Wąsowicz, A., and Werning, M. (2012). Synesthesia, sensory-motor contingency and semantic emulation: how swimming style-color synesthesia challenges the traditional view of synesthesia. Front. Psychol. 3:279. doi: 10.3389/fpsyg.2012.00279

PubMed Abstract | CrossRef Full Text | Google Scholar

O'Callaghan, C. (2014). “Intermodal binding awareness,” in Sensory Integration and the Unity of Consciousness, eds D. Bennett and C. Hill (Cambridge, MA: MIT Press), 73–104.

O'Regan, J. K., and Noë, A. (2001). A sensorimotor approach to vision and visual consciousness. Behav. Brain Sci. 24, 883–975. doi: 10.1017/S0140525X01000115

CrossRef Full Text | Google Scholar

Prinz, J. (2011). “The sensory basis of cognitive phenomenology,” in Cognitive Phenomenology, eds T. Bayne and M. Montague (Oxford: Oxford University Press), 174–196.

Google Scholar

Pylyshyn, Z. (1999). Is vision continuous with cognition? The case for cognitive impenetrability of visual perception. Behav. Brain Sci. 22, 341–423.

PubMed Abstract | Google Scholar

Pylyshyn, Z. (2003). Seeing and Visualizing: It's Not What You Think. Cambridge, MA: MIT Press.

Google Scholar

Raftopoulos, A. (2011). Late vision: processes and epistemic status. Front. Psychol. 2:382. doi: 10.3389/fpsyg.2011.00382

PubMed Abstract | CrossRef Full Text | Google Scholar

Raftopoulos, A., and Zeimbekis, J. (eds.). (2016). Cognitive Penetrability. Cambridge, MA: MIT Press.

Google Scholar

Shapiro, L. (ed.). (2014). The Routledge Handbook of Embodied Cognition. London; New York, NY: Routledge.

Google Scholar

Shimojo, S., and Shams, L. (2001). Sensory modalities are not separate modalities: plasticity and interactions. Curr. Opin. Neurobiol. 11, 505–509. doi: 10.1016/S0959-4388(00)00241-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Siegel, S. (2012). Cognitive penetrability and perceptual justification. Noûs 46, 201–222. doi: 10.1111/j.1468-0068.2010.00786.x

CrossRef Full Text | Google Scholar

Spence, C., and Bayne, T. (2015). “Is consciousness multisensory?” in Perception and its Modalities, eds D. Stokes, M. Matthen, and S. Biggs. (New York, NY: Oxford University Press), 95–132.

Stein, B. E. (ed.). (2012). The New Handbook of Multisensory Processing. Cambridge, MA: MIT Press.

Google Scholar

Stokes, D. (2012). Perceiving and desiring: a new look at the cognitive penetrability of experience. Philos. Stud. 158, 479–492. doi: 10.1007/s11098-010-9688-8

CrossRef Full Text | Google Scholar

Stokes, D., Matthen, M., and Biggs, S. (eds.). (2015). Perception and its Modalities. New York, NY: Oxford University Press.

Google Scholar

Tajadura-Jiménez, A., Bianchi-Berthouze, N., Furfaro, E., and Bevilacqua, F. (2015). Sonification of surface tapping changes behavior, surface perception, and emotion. IEEE MultiMedia 22, 48–57. doi: 10.1109/MMUL.2015.14

CrossRef Full Text | Google Scholar

Tajadura-Jiménez, A., Pantelidou, G., Rebacz, P., Västfjäll, D., and Tsakiris, M. (2011). I-space: the effects of emotional valence and source of music on interpersonal distance. PLoS ONE 6:e26083. doi: 10.1371/journal.pone.0026083

PubMed Abstract | CrossRef Full Text | Google Scholar

Vetter, P., and Newen, A. (2014). Varieties of cognitive penetration in visual perception. Conscious. Cogn. 27, 62–75. doi: 10.1016/j.concog.2014.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Weiskopf, D. (2015). “Observational concepts,” in The Conceptual Mind: New Directions in the Study of Concepts, eds E. Margolis and S. Laurence (Cambridge: MIT Press), 223–247.

Weiss, C., Tsakiris, M., Haggard, P., and Schütz-Bosbach, S. (2013). Agency in the sensorimotor system and its relation to explicit action awareness. Neuropsychologia. 52, 82–92. doi: 10.1016/j.neuropsychologia.2013.09.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: perception, cognition, concepts, cognitive penetrability of perception, cross-modal experience, multisensory integration, multimodal binding, the unity of consciousness

Citation: Mroczko-Wąsowicz A (2016) Editorial: Perception–Cognition Interface and Cross-Modal Experiences: Insights into Unified Consciousness. Front. Psychol. 7:1593. doi: 10.3389/fpsyg.2016.01593

Received: 14 June 2016; Accepted: 30 September 2016;
Published: 22 November 2016.

Edited by:

Morten Overgaard, Aarhus University, Denmark

Reviewed by:

Luis Lemus, National Autonomous University of Mexico, Mexico
Liad Mudrik, Tel Aviv University, Israel

Copyright © 2016 Mroczko-Wąsowicz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Aleksandra Mroczko-Wąsowicz, mroczko-wasowicz@hotmail.com