Marina A. Pavlova- Department of Psychiatry and Psychotherapy and Department of Biomedical Magnetic Resonance, Medical School, Eberhard Karls University of Tübingen, Tübingen, Germany
With the advent of sophisticated tools and techniques over the past three decades, brain imaging has energized the rapidly developing field of Social Neuroscience, and has sparked a wide range of research in Emotion Science. Since its launch in 2010, Frontiers in Emotion Science has published more than 400 articles, many of them utilizing brain imaging tools. However, the question arises: Are we on the way to real progress? It is reasonable, and healthy, to reflect on how brain imaging has helped to elucidate our understanding of socio-emotional functioning, and how its contributions can be enriched and improved in the future. As a neuroscientist who has conducted many brain imaging studies, I wrestle with several doubts and concerns. In this piece of work, I will discuss the critical importance of behavior, time, and sex/gender not only for Emotion Science, but also “over and above” this specific area of investigation. I will focus on the ways in which these key constructs represent a grand (and very exciting) challenge for neuroscience. I elaborate on three of these here.
Yes, Look at Behavior!
Even for the most enthusiastic colleagues, it is becoming more and more evident that functional brain imaging is not at all a “magic tool” that routinely offers desirable solutions for our understanding of the social brain and behavior, and, in particular, real-world affectively-motivated behavior. Decisive steps in understanding and explaining the multitude of subtle and sublime, emotionally meaningful, daily-life behaviors can be made only by carefully studying behavior itself (see also Beatrice de Gelder's analysis: “Behavior, what else?”; de Gelder, 2017). Of course, this does not mean that brain imaging does not provide us with valuable information about typical and aberrant affective processing. We might even think of brain imaging as providing a quantitative, reliable assay of the “behavior” of the nervous system. Yet (though it appears quite obvious, but hopefully not too banal), a relationship between behavioral performance during brain imaging and the patterns of change in brain activity should be in focus of investigation. Unfortunately, this is not always the case. This relationship is far from simple even in healthy adults, free from the rich complexities of developmental change or psychopathology: sometimes there is a strong positive correlation between behavioral measures of performance and brain activation suggesting that a higher performance level is accompanied by greater brain engagement (e.g., Grossman et al., 2004). In other cases, however, this correlation is negative (e.g., Raichle et al., 1994). This means that in those cases higher performance is accompanied by lower signal strength in the responsive brain regions. Why does this happen? Does greater functional brain activation points to greater task-related engagement of brain areas? If so, why is a more robust brain response needed for an easier or less demanding task, where participants exhibit ceiling levels of performance? These issues become even more challenging to understand when comparing performance and brain activations in individuals with atypical social cognition (e.g., in the presence of neurodevelopmental disorders such as autism and schizophrenia) to those with typical developmental pathways.
The question of how to investigate behavior, particularly, in relation to Emotion Science, is even more complicated. There is an understandable push to rapidly move to real-life situations and to leverage the possibilities afforded by virtual reality. However, by watching affective behavior of people in natural environment or “in the wild,” one confronts the ethical issues of how far and how deep to go when privacy might be violated? At the same time, simple awareness of the fact that “we-are-under-observation” leads to substantial alterations in behavior. “People-under-observation” may exhibit rather strong experimenter expectancy: they may wish to please the experimenter or behave in a socially desirable way suppressing either negative or positive (depending on cultural norms) emotional expressions. These are all well-known “ancient” problems of psychological research. But new tools reintroduce old problems and raise new ones as well: although virtual environments offer promising tools for studying more realistic affective behavior, the “question of belief” again comes into a play. Participants are aware that this reality is only virtual and this awareness potentially affects their behavior and emotional experience.
On a more positive note, we are clearly on the path to progress in studying dynamic instead of static (such as photographs, still images, depictions) affective faces and bodies (de Gelder, 2009; Kret and De Gelder, 2012; Pavlova, 2012), and (how delightful!) we are “standing up for the body” (de Gelder et al., 2010) and exploring body language reading in typical (Atkinson et al., 2004, 2012; Chouchourelou et al., 2006; Ikeda and Watanabe, 2009; Alaerts et al., 2011; Sokolov et al., 2011; Krüger et al., 2013; Actis-Grosso et al., 2015; de Gelder et al., 2015; for review, see Pavlova, 2017) and atypical development (e.g., Nackaerts et al., 2012; Strauss et al., 2015; Van den Stock et al., 2015; Vaskinn et al., 2016; Blain et al., 2017).
Time Matters
To date, brain imaging faces with a set of issues that must be addressed. Most prior work was restricted to localization of brain areas, and often referred to (in an unfortunately derisory fashion) as “blobology.” The “blobs” of activation are now recognized or believed to represent not only isolated icebergs of brain activation, but also hubs (i.e., key nodes for information processing as well as receptacles for connections to other brain regions) of the networks or neural circuits underlying complex cognitive and affective processes. Topography of these areas as well as the strength of activation vary meaningfully even with subtle alterations in the visual input, task demands, and participants' characteristics such as gender (Anderson et al., 2013; Pavlova et al., 2015), age (Ross et al., 2014), and socio-economic status (Muscatell et al., 2012). Recent work is beginning to focus on interactions between brain regions making up the social brain. One of the most desirable appeals for future work concerns re-definition of the concept of functional brain network both theoretically and experimentally (e.g., special issue in Frontiers Systems Neuroscience; New concepts in brain networks, Eds. Turner and Lohmann, 2012; Pessoa, 2014; Lohmann et al., 2016). Neuronal communication of the entire social brain should be a target of study in typical and atypical development. This goal demands a combination of advanced multimodal brain imaging tools allowing for assessment of temporal and spatial dynamics [such as magnetencephalography (MEG) and electroencephalography (EEG) along with (ultra) high field functional magnetic resonance imaging (fMRI) providing for high sensitivity and spatial resolution], and thereby, unveiling neural communication in real time. Time is a key to understanding the organization of functional brain networks, since brain topography alone do not allow us to understand neural communication as well as feed-forward and feed-back connections in the brain. Only the accurate measurement of temporal dynamics, in the context of spatial localization, provides the precise information about the formation of specialized functional brain networks, i.e., about brain areas and large-scale brain ensembles playing in unison at different time intervals.
The other equally important reason for emphasizing time is that differences (sex-, age-, and psychopathology-related) in the topography of brain responses are often subtle or negligible: topographically similar brain areas or clusters of activation may be engaged in different groups of participants. What really matters is time! Temporal dynamics often helps to differentiate individuals and groups in spatial context. Let me give just one example from our own work in which we studied sex differences in the brain response to social interaction represented by motion of simple geometric shapes in Heider-and-Simmel-type animations (Pavlova et al., 2010). The induced oscillatory MEG response of similar topography (localized to the left prefrontal cortex) peaked later in males than in females (Figure 1). These findings reflect sex differences in cortical processing of visually acquired social information. For females, anticipation of socially relevant events may be of higher biological, sociocultural and ecological value, and thus the left prefrontal cortex (a region known to be implicated in perceptual decision making) might be involved in expectation of the social environment and affective events by generating a template against which incoming sensory evidence is matched. In other words, females may anticipate social interaction predicting others' actions ahead of their occurrence, whereas males require accumulation of more sensory evidence before reaching social decisions. Without unveiling the time course of brain activity, it would be difficult to reveal these sex differences.
Figure 1. Sex effects on induced oscillatory gamma MEG activity. The graph on the top represents the outcome of t-test comparisons between females and males, i.e., the time course of the p values of the differences in the spectral amplitude in the filtered frequency bands. The solid curve shows the time course for differences in spectral amplitude between Heider-and-Simmel (HS) and control displays in females as compared with males at one of the sensors in the left prefrontal area (filled gray circle on the left map), whereas the dotted curve represents differences in males as compared with females at the sensor of similar topography (largest open circle on the right map). The culmination point of visual social interaction occurred at 1.3 s after stimulus onset. The left and right maps depict the topography of spectral amplitude differences in the 73-Hz and the 54-Hz ranges, respectively. Each circle represents one of the 275 MEG sensors projected onto a 2-dimensional cortical surface map with major anatomical landmarks (dorsal view, nose up). The size of the circle reflects the statistical strength of sex differences in induced oscillatory gamma activity. From Pavlova et al. (2010). © 2009 Elsevier Inc. with permission of Elsevier.
Sex/Gender, Age and …and… and…
Research in healthy typically developing individuals most often overlooks possible (if not likely) sex/gender differences. Indeed, sex differences appear to be the norm rather than the exception. Usually, if not explicitly directed at investigation of sex/gender differences, both behavioral and brain imaging studies enroll a roughly equal number of female and male participants. Then the data are pooled together, and as a result we deal with the averaged “sexless” outcome that is then applied equally (poorly) to real females and males “in the wild.”
Without understanding the impact of sex/gender, age, social identity, and socio-cultural differences (e.g., Tanaka et al., 2010; Spencer et al., 2016; Hausmann, 2017; Koelkebeck et al., 2017; see also special issue on aging in Frontiers Psychology: Frontiers in Emotion Science entitled Emotion and aging: evidence from brain and behavior; Eds. Ebner and Fischer, 2014), it is impossible to make any progress in understanding affective behavior and underlying brain mechanisms in health and disease. The outcome of such research sheds light on complicated, contextually-dependent interactions of different major factors influencing affective behavior, and often counters popular wisdom and stereotypes! For example, females are widely believed to be more skilled in body language reading. Yet the pattern of experimental data in both typically developing healthy individuals and patients with deficient social abilities is “beyond simplicity” (Pavlova, 2017). Sex of observers affects body language reading in point-light movies depicting knocking at a door, but the effects are modulated by emotional content of actions. Women tend to surpass in recognition of angry knocking, whereas men excel in recognition accuracy of happy actions (Sokolov et al., 2011). At the same time, females exhibit an advantage in recognition accuracy of neutral actions that suggests that females are better tuned to the lack of emotional body language. A similar pattern of results was observed for subtle body language expressed by point-light human locomotion: The sex effects are modulated by the emotional content of locomotion and opposite actor gender (Krüger et al., 2013). Males surpass females in recognition accuracy and readiness to respond to expressions of happiness performed by female actors, whereas females show a tendency to be better in recognition of angry locomotion expressed by male actors. This may reflect biological and evolutionary significance of the opposite sex appearance with higher tuning of females to possible (even subtle) signals of threat or danger in body language of males, and higher tuning of males to happiness communicated through body motion of females.
The social brain impairments have many facets playing a key role in many neuropsychiatric conditions such as autistic spectrum disorders, schizophrenia, depression, eating disorders, and many others: all of them possess profound affective components. Most of these disorders are characterized by impairments in visual social cognition, nonverbal communication, body language reading, and facial assessment of social counterparts (e.g., Pavlova, 2012, 2017; Lazar et al., 2014; Pelphrey et al., 2014; Strauss et al., 2015; Pavlova et al., 2017; Yang et al., 2017). And many of these disorders (for example, autism spectrum disorders, Hull et al., 2017) also display a skewed sex ratio: females and males are affected differently in terms of clinical picture, prevalence, and severity. Currently, we are only beginning to understand the origins of sex/gender specificity of the most psychiatric and neurologic conditions.
Author Contributions
The author confirms being the sole contributor of this work and approved it for publication.
Funding
The Else Kröner Fresenius Foundation (Grant P2013_127), the BBBank Foundation, and the Reinhold Beitlich Foundation to MAP. The funders had no role in the decision to prepare, contents of the manuscript, or decision to publish.
Conflict of Interest Statement
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
I am grateful to Kevin A. Pelphrey, Alexander N. Sokolov, and Axel Cleeremans for discussions, insightful comments and valuable suggestions that helped to improve my reasoning, and Beatrice de Gelder for encouraging me to undertake this work. Many thanks are going to my husband and my son for their generous everyday support.
References
Actis-Grosso, R., Bossi, F., and Ricciardelli, P. (2015). Emotion recognition through static faces and moving bodies: a comparison between typically developed adults and individuals with high level of autistic traits. Front. Psychol. 6:1570. doi: 10.3389/fpsyg.2015.01570
Alaerts, K., Nackaerts, E., Meyns, P., Swinnen, S. P., and Wenderoth, N. (2011). Action and emotion recognition from point light displays: an investigation of gender differences. PLoS ONE 6:e20989. doi: 10.1371/journal.pone.0020989
Anderson, L. C., Bolling, D. Z., Schelinski, S., Coffman, M. C., Pelphrey, K. A., and Kaiser, M. D. (2013). Sex differences in the development of brain mechanisms for processing biological motion. Neuroimage 83, 751–760. doi: 10.1016/j.neuroimage.2013.07.040
Atkinson, A. P., Dittrich, W. H., Gemmell, A. J., and Young, A. W. (2004). Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception 33, 717–746. doi: 10.1068/p5096
Atkinson, A. P., Vuong, Q. C., and Smithson, H. E. (2012). Modulation of the face- and body-selective visual regions by the motion and emotion of point-light face and body stimuli. Neuroimage 59, 1700–1712. doi: 10.1016/j.neuroimage.2011.08.073
Blain, S. D., Peterman, J. S., and Park, S. (2017). Subtle cues missed: impaired perception of emotion from gait in relation to schizotypy and autism spectrum traits. Schizophr Res. 183, 157–160. doi: 10.1016/j.schres.2016.11.003
Chouchourelou, A., Toshihiko, M., Harber, K., and Shiffrar, M. (2006). The visual analysis of emotional actions. Soc. Neurosci. 1, 63–74. doi: 10.1080/17470910600630599
de Gelder, B. (2009). Why bodies? Twelve reasons for including bodily expressions in affective neuroscience. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3475–3484. doi: 10.1098/rstb.2009.0190
de Gelder, B. (2017). Grand challenge: emotion science in the 21th century. Front. Psychol. doi: 10.3389/fpsyg.2017.01212
de Gelder, B., de Borst, A. W., and Watson, R. (2015). The perception of emotion in body expressions. Wiley Interdiscip. Rev. Cogn. Sci. 6, 149–158. doi: 10.1002/wcs.1335
de Gelder, B., Van den Stock, J., Meeren, H. K., Sinke, C. B., Kret, M. E., and Tamietto, M. (2010). Standing up for the body. recent progress in uncovering the networks involved in the perception of bodies and bodily expressions. Neurosci. Biobehav. Rev. 34, 513–527. doi: 10.1016/j.neubiorev.2009.10.008
Ebner, N. C., and Fischer, H. (2014). Emotion and aging: evidence from brain and behavior. Front. Psychol. 5:996. doi: 10.3389/fpsyg.2014.00996
Grossman, E. D., Blake, R., and Kim, C. Y. (2004). Learning to see biological motion: brain activity parallels behavior. J. Cogn. Neurosci. 16, 1669–1679. doi: 10.1162/0898929042568569
Hausmann, M. (2017). Why sex hormones matter for neuroscience: a very short review on sex, sex hormones, and functional brain asymmetries. J. Neurosci. Res. 95, 40–49. doi: 10.1002/jnr.23857
Hull, L., Petrides, K. V., Allison, C., Smith, P., Baron-Cohen, S., Lai, M.-C., et al. (2017). “Putting on My Best Normal”: social camouflaging in adults with Autism spectrum conditions. J. Autism Dev. Disord. 47, 2519–2534. doi: 10.1007/s10803-017-3166-5
Ikeda, H., and Watanabe, K. (2009). Anger and happiness are linked differently to the explicit detection of biological motion. Perception 38, 1002–1011. doi: 10.1068/p6250
Koelkebeck, K., Uwatoko, T., Tanaka, J., and Kret, M. E. (2017). How culture shapes social cognition deficits in mental disorders: a review. Soc. Neurosci. 12, 102–112. doi: 10.1080/17470919.2016.1155482
Kret, M. E., and De Gelder, B. (2012). A review on sex differences in processing emotional signals. Neuropsychologia 50, 1211–1221. doi: 10.1016/j.neuropsychologia.2011.12.022
Krüger, S., Sokolov, A. N., Enck, P., Krägeloh-Mann, I., and Pavlova, M. A. (2013). Emotion through locomotion: gender impact. PLoS ONE 11:e81716. doi: 10.1371/journal.pone.0081716
Lazar, S. M., Evans, D. W., Myers, S. M., Moreno-De Luca, A., and Moore, G. J. (2014). Social cognition and neural substrates of face perception: implications for neurodevelopmental and neuropsychiatric disorders. Behav. Brain Res. 263, 1–8. doi: 10.1016/j.bbr.2014.01.010
Lohmann, G., Stelzer, J., Zuber, V., Buschmann, T., Margulies, D., Bartels, A., et al. (2016). Task-Related Edge Density (TED)-a new method for revealing dynamic network formation in fMRI data of the human brain. PLoS ONE 11:e0158185. doi: 10.1371/journal.pone.0158185
Muscatell, K. A., Morelli, S. A., Falk, E. B., Way, B. M., Pfeifer, J. H., Galinsky, A. D., et al. (2012). Social status modulates neural activity in the mentalizing network. Neuroimage 60, 1771–1777. doi: 10.1016/j.neuroimage.2012.01.080
Nackaerts, E., Wagemans, J., Helsen, W., Swinnen, S. P., Wenderoth, N., and Alaerts, K. (2012). Recognizing biological motion and emotions from point-light displays in autism spectrum disorders. PLoS ONE 7:e44473. doi: 10.1371/journal.pone.0044473
Pavlova, M. A. (2012). Biological motion processing as a hallmark of social cognition. Cereb. Cortex 22, 981–995. doi: 10.1093/cercor/bhr156
Pavlova, M. A. (2017). Sex and gender affect the social brain: beyond simplicity. J. Neurosci. Res. 95, 235–250. doi: 10.1002/jnr.23871
Pavlova, M. A., Guerreschi, M., Tagliavento, L., Gitti, F., Sokolov, A. N., Fallgatter, A. J., et al. (2017). Social cognition in autism: face tuning. Sci. Rep. 7:2734. doi: 10.1038/s41598-017-02790-1
Pavlova, M. A., Sokolov, A. N., and Bidet-Ildei, C. (2015). Sex differences in the neuromagnetic response to biological motion. Cereb. Cortex 25, 3468–3474. doi: 10.1093/cercor/bhu175
Pavlova, M., Guerreschi, M., Lutzenberger, W., Sokolov, A. N., and Krägeloh-Mann, I. (2010). Cortical response to social interaction is affected by gender. Neuroimage 50, 1327–1332. doi: 10.1016/j.neuroimage.2009.12.096
Pelphrey, K. A., Yang, D. Y., and McPartland, J. C. (2014). Building a social neuroscience of autism spectrum disorder. Curr. Top. Behav. Neurosci. 16, 215–233. doi: 10.1007/978-3-662-45758-0_253
Pessoa, L. (2014). Understanding brain networks and brain organization. Phys. Life Rev. 11, 400–435. doi: 10.1016/j.plrev.2014.03.005
Raichle, M. E., Fiez, J. A., Viden, T. O., MacLeod, A. M., Pardo, J. V., Fox, P. T., et al. (1994). Practice-related changes in human brain functional anatomy during nonmotor learning. Cereb. Cortex 4, 8–26. doi: 10.1093/cercor/4.1.8
Ross, P. D., de Gelder, B., Crabbe, F., and Grosbras, M. H. (2014). Body-selective areas in the visual cortex are less active in children than in adults. Front. Hum. Neurosci. 8:941. doi: 10.3389/fnhum.2014.00941
Sokolov, A. A., Krüger, S., Enck, P., Krägeloh-Mann, I., and Pavlova, M. A. (2011). Gender affects body language reading. Front. Psychol. 2:16. doi: 10.3389/fpsyg.2011.00016
Spencer, J. M., Sekuler, A. B., Bennett, P. J., Giese, M. A., and Pilz, K. S. (2016). Effects of aging on identifying emotions conveyed by point-light walkers. Psychol. Aging 31, 126–138. doi: 10.1037/a0040009
Strauss, G. P., Keller, W. R., Koenig, J. I., Sullivan, S. K., Gold, J. M., and Buchanan, R. W. (2015). Endogenous oxytocin levels are associated with the perception of emotion in dynamic body expressions in schizophrenia. Schizophr. Res. 162, 52–56. doi: 10.1016/j.schres.2015.01.022
Tanaka, A., Koizumi, A., Imai, H., Hiramatsu, S., Hiramoto, E., and de Gelder, B. (2010). I feel your voice. Cultural differences in the multisensory perception of emotion. Psychol. Sci. 21, 1259–1262. doi: 10.1177/0956797610380698
Turner, R., and Lohmann, G. (2012). New concepts in brain networks. Front. Syst. Neurosci. 6:56. doi: 10.3389/fnsys.2012.00056
Van den Stock, J., De Winter, F. L., de Gelder, B., Rangarajan, J. R., Cypers, G., Maes, F., et al. (2015). Impaired recognition of body expressions in the behavioral variant of frontotemporal dementia. Neuropsychologia 75, 496–504. doi: 10.1016/j.neuropsychologia.2015.06.035
Vaskinn, A., Sundet, K., Østefjells, T., Nymo, K., Melle, I., and Ueland, T. (2016). Reading emotions from body movement: a generalized impairment in schizophrenia. Front. Psychol. 6:2058. doi: 10.3389/fpsyg.2015.02058
Yang, Y. J. D., Allen, T., Abdullahi, S. M., Pelphrey, K. A., Volkmar, F. R., and Chapman, S. B. (2017). Brain responses to biological motion predict treatment outcome in young adults with autism receiving virtual reality social cognition training: preliminary findings. Behav. Res. Ther. 93, 55–66. doi: 10.1016/j.brat.2017.03.014
Keywords: emotion science, brain imaging, gender/sex differences, brain networks, behavior, time course
Citation: Pavlova MA (2017) Emotion Science in the Twenty-First Century. Time, Sex, and Behavior in Emotion Science: Over and Above. Front. Psychol. 8:1211. doi: 10.3389/fpsyg.2017.01211
Received: 19 April 2017; Accepted: 03 July 2017;
Published: 21 July 2017.
Edited and reviewed by: Axel Cleeremans, Free University of Brussels, Belgium
Copyright © 2017 Pavlova. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marina A. Pavlova, marina.pavlova@uni-tuebingen.de