Abstract
Humans substantially rely on non-verbal cues in their communication and interaction with others. The eyes represent a “simultaneous input-output device”: While we observe others and obtain information about their mental states (including feelings, thoughts, and intentions-to-act), our gaze simultaneously provides information about our own attention and inner experiences. This substantiates its pivotal role for the coordination of communication. The communicative and coordinative capacities – and their phylogenetic and ontogenetic impacts – become fully apparent in triadic interactions constituted in its simplest form by two persons and an object. Technological advances have sparked renewed interest in social gaze and provide new methodological approaches. Here we introduce the ‘Social Gaze Space’ as a new conceptual framework for the systematic study of gaze behavior during social information processing. It covers all possible categorical states, namely ‘partner-oriented,’ ‘object-oriented,’ ‘introspective,’ ‘initiating joint attention,’ and ‘responding joint attention.’ Different combinations of these states explain several interpersonal phenomena. We argue that this taxonomy distinguishes the most relevant interactional states along their distinctive features, and will showcase the implications for prominent social gaze phenomena. The taxonomy allows to identify research desiderates that have been neglected so far. We argue for a systematic investigation of these phenomena and discuss some related methodological issues.
Social Gaze as Special Case of Non-verbal Communication
Non-verbal communication does not only supplement verbal utterances but constitutes a crucial part of communication in itself. Thereby, non-verbal communication must not be treated as a series of isolated and discrete signals but as a complex and dynamic process (Burgoon et al., 1989, p. 23). In addition, the production and perception of non-verbal communication behavior are often implicit and automatic (Choi et al., 2005) – i.e., unintentional, uncontrollable processes humans are unaware of (Bargh, 1994).
Among the non-verbal cues, gaze behavior plays a pivotal role. The eyes are among the first and most frequently fixated regions in humans (Yarbus, 1967; Walker-Smith et al., 1977) from early infancy on (Haith et al., 1977), serve face and emotion recognition, and allow to identify gender, age, and personality (George and Conty, 2008; Itier and Batty, 2009).
The morphology of the human eye with its white sclera significantly enhances the visibility of the eyes and facilitates gaze recognition (Kobayashi and Kohshima, 1997, 2001), suggesting evolutionary adaptation to the increased importance of gaze-based social interaction and, eventually, social cognition in humans (Emery, 2000). Ontogenetically, attending to gaze can be considered a precursor of cooperation in young children (Tomasello et al., 2007). Both phylogenetically and ontogenetically (Grossmann, 2017) social gaze opens a “window into social cognition” (Shepherd, 2010).
In addition to coordination and management of verbal conversation (Argyle and Cook, 1976), gaze mutually coordinates attention which is a hallmark of social learning, communication, social interaction, and, finally, shared intentionality (Tomasello et al., 2007) and joint action (Sebanz and Knoblich, 2009). So-called joint attention (JA) is typically defined in the gaze domain: In triadic interactions (e.g., Lee et al., 1998), two persons can jointly attend to an object by one person following another person’s gaze toward a given object or possibly a third person. JA is the basis and prerequisite of cooperation (Tomasello et al., 2007) and has been investigated in great detail (Kleinke, 1986; Emery, 2000; Frischen et al., 2007; George and Conty, 2008; Itier and Batty, 2009; Shepherd, 2010; Falck-Ytter and von Hofsten, 2011; Pfeiffer et al., 2013b; Oberwelland et al., 2016; Grossmann, 2017).
The “Social Gaze Space” (SGS)
Despite the wealth of social gaze research, a unifying taxonomy of social gaze is still lacking. For the most commonly used taxonomy Emery (2000) summarized several core processes like averted gaze, mutual gaze, gaze following and JA under the term social gaze. However, this taxonomy has two major limitations: (1) the basic processes described by Emery were not considered as extended in time. Relatedly, transitions between states have not been taken into account. The taxonomy of Emery therefore lacks the complex and dynamic character of gaze encounters between two persons, which are extended in time and are based on the continuous exchange between the interactants. (2) An additional restriction of the traditional social gaze terminology and research is that they focus on explicit interactions in which at least one person deliberately tries to interact with or respond to another (Schilbach et al., 2010; Pfeiffer et al., 2014). However, already the mere presence of another person presumably strongly affects a persons’ behavior even when the partner is not interactively engaged. Recent research about the dual function of social gaze demonstrates that the awareness of someone else watching oneself can change the own gaze behavior (Gobel et al., 2015; Jarick and Kingstone, 2015). In accordance with recent interactionist advances emphasizing the dynamical character of interactions and arguing for ecological validity (Risko et al., 2012, 2016; Pfeiffer et al., 2013a; Schilbach et al., 2013), it is therefore important to consider all possible states of triadic interactions in a holistic approach.
In the following, we propose a taxonomy of the “Social Gaze Space” (SGS) that comprises all internal states a person can possibly adopt in the most basic setup of a gaze-based triadic interaction, as constituted by two interaction partners and an object1. These states are: partner-oriented (PO), object-oriented (OO), introspective (INT), responding joint attention (RJA), and initiating joint attention (IJA). We define these states on the basis of the behavior of one interactant (Figure 1). A dynamic interaction involving two persons can be conceptualized as a combination of two out of five different states which need not necessarily be temporally aligned. All combinations of states are possible and generate different types of interactional encounters that can be represented as a two-dimensional series of social gaze states evolving in time (Figure 2). This particularly applies to the interactive states of RJA and IJA, in which a person attempts to engage another person in an interaction which can be successful or not (see below section Triadic Interaction as a Dynamic Function of a Two-Dimensional State-Space). For this conceptualization, our focus lies on overt visual attention as deducible from gaze direction, whereas covert attention and other correlates of attention (e.g., pupil diameter, eye convergence, blinking rate) will be discussed only marginally.
FIGURE 1
FIGURE 2
The Five Gaze States
Partner-Oriented (PO)
In the partner-oriented state person A focuses her attention on person B. The eyes automatically attract visual attention (Laidlaw et al., 2012) and possibly convey information about personal attributes including gender, age and identity (Schyns et al., 2002), as well as emotional and attentional states (Baron-Cohen et al., 1997; Emery, 2000).
Eyes that focus on the viewer will be preferentially looked at (Senju and Hasegawa, 2005) or evaluated much more positively (Stass and Willis, 1967), modulate attention (Senju and Hasegawa, 2005; Dalmaso et al., 2017), increase emotional empathy (Schulte-Rüther et al., 2007) and modulate cognition suggesting a substantial ‘eye-contact-effect’ for diverse aspects of socio-emotional perception (Senju and Johnson, 2009). Among distractor stimuli, viewer-directed gaze is detected easily and much faster than averted gaze (von Grünau and Anston, 1995; Conty et al., 2006; Senju et al., 2008). Profound effects of viewer-direct eye gaze on preference (Hains and Muir, 1996) and attentional modulation (Farroni et al., 2002) have also been demonstrated in infants. This is probably the most thoroughly studied gaze state.
The effect of eye contact is much stronger during dynamic interactions with real persons than when confronted with static pictures (Hietanen et al., 2008; Pönkänen et al., 2011). This requires interactive approaches with dynamic face-to-face interactions (Pfeiffer et al., 2012, 2013a; Risko et al., 2012, 2016; Schilbach et al., 2013; Schilbach, 2014; Oberwelland et al., 2016).
Object-Oriented (OO)
In the object-oriented state person A’s attention is focused more or less entirely on an object in the shared environment, but not on the other person (as opposed to joint attention states described below during which person A oscillates between objects and person B). That is B’s presence and behavior are likely to influence A to some level but merely coincidentally and probably without A’s awareness. The exploration of different objects in a visual scenery is affected by the saliency of objects and thus the probability of persons directing their attention toward the objects (Itti and Koch, 2000). However, top-down as well as bottom-up processes are actively working together or compete for attention (Egeth and Yantis, 1997). Again, our attention and behavior toward objects are altered by actions or even the mere presence of another person looking at us (Senju and Hasegawa, 2005). Gaze cueing can automatically lead the attention toward particular objects (Frischen et al., 2007), even overriding the effect of higher psychophysical saliency (Borji et al., 2014). This brief instance of social interaction might induce a lasting attentional shift from a state of OO to the state of RJA [as examined in section Responding Joint Attention (RJA)]. However, even in the absence of any active gaze cuing, the presence of another person can attract covert attention (Kuhn et al., 2016; Laidlaw et al., 2016). Furthermore, the mere knowledge of the possibility of someone else watching their gaze lets participants control their gaze behavior with respect to its social adequacy (Risko and Kingstone, 2011).
Introspective (INT)
In this state person A neither focuses on objects nor on persons in the environment but only on his inner experience. Attentional disengagement from the outside world has been shown to correlate with a decrease in saccade frequency and an increase in saccade amplitude (Benedek et al., 2017) and, accordingly, a decrease in fixation frequency and an increase in fixation duration (Reichle et al., 2010; Benedek et al., 2017). Furthermore, in these situations blinking rate can increase (Smilek et al., 2010) and blinking duration can be prolonged (Salvi et al., 2015; Benedek et al., 2017). INT seems to show more variability in pupil diameter than episodes of directed attention to outward stimuli (Smallwood et al., 2011; Benedek et al., 2017). A higher variability of eye vergence (Benedek et al., 2017) suggests a less focused gaze (Solé Puig et al., 2013).
While it is intuitively obvious that these changes are indicative of a reduced responsiveness to events in the outside world (Smallwood et al., 2011; Benedek et al., 2017), it is an open question whether the reduced responsiveness to external stimuli and the overall change in gaze behavior are both the result and an epiphenomenon of INT, or whether changes such as a decrease in the frequency of microsaccades during INT may represent active visual disengagement as a strategy to achieve reduced responsiveness (Benedek et al., 2017). Another strategy participants adopt in situations of high cognitive load is to avoid looking into the eyes of an observer because this would entail higher demands on cognitive processing (Glenberg et al., 1998; Doherty-Sneddon and Phelps, 2005; Phelps et al., 2006; Markson and Paterson, 2009). Interestingly, the additional cognitive demands of mutual gaze do not seem to originate in the physical properties of the stimulus (e.g., the eyes) but in the interactive character inherent in this situation (Markson and Paterson, 2009). It is therefore crucial to consider introspective attentional states as potentially socially influenced by the presence of another person.
Responding Joint Attention (RJA)
In the responding JA state person A waits for B to initiate and lead the interaction, e.g., B chooses an object and A follows B’s gaze toward the object. Gaze following reactions that respond to the invitation of another person thereby establishing a rudimentary form of JA appear to be deeply rooted in human behavior (Pfeiffer et al., 2011). The gaze of another person automatically cues one’s own attention even when it is uninformative (Friesen and Kingstone, 1998), and participants exhibit gaze following even for forthright counter-predictive gaze cues (Driver et al., 1999; Bayliss and Tipper, 2006).
Gaze following with the aim of establishing JA constitutes a very simple though effective mechanism allowing for the inference of the attentional focus of other persons. The ability to adopt the attentional focus of another person is a prerequisite for reinforcement learning, from infants to adults (Vernetti et al., 2017). Infants at 6 months of age are already able to follow the eyes of other persons, in particular in a communicative context (Senju and Csibra, 2008). Accordingly, early proficiency in gaze following in infants predicts the development of mentalizing and emergence of language (Morales et al., 1998; Charman et al., 2000). JA and gaze following facilitate social learning, social competence, self-regulation, intelligence, and depth of information processing (Mundy and Newell, 2007).
Initiating Joint Attention (IJA)
In this state, person A takes the lead within the interaction by initiating JA. While gaze following in RJA reflects person A’s understanding that B’s perception and actions are goal-directed or have communicative intent, the initiation of JA is considered to require elaborate processing and insight (Tomasello and Carpenter, 2005). To initiate JA, A has to acknowledge (1) the dual function of social gaze (Gobel et al., 2015; Jarick and Kingstone, 2015) i.e., that gaze does not only serves her in perceiving but also that her gaze informs B about her focus of attention and, (2) sharing of attention is a desirable aim for mutual interaction (Tomasello et al., 2005). Whereas first elements of RJA are already evident at 6 months of age, IJA does not emerge before the second year of life (Mundy and Newell, 2007; Mundy et al., 2007). Chimpanzees followed the experimenters gaze on a frequent basis but did not try to initiate JA (Tomasello and Carpenter, 2005). Interestingly, differential development of both RJA and IJA can be observed in brain systems from childhood to adulthood (Oberwelland et al., 2016), as well as during atypical development in disorders such as autism (Oberwelland et al., 2017). In autism, IJA is typically more impaired than RJA and emerges much later than in typical development (Mundy, 2003). These empirical findings clearly point toward separate underlying cognitive systems of RJA and IJA (Mundy and Newell, 2007).
The innate tendency to expect other humans to follow their gaze (Pfeiffer et al., 2011) corresponds to the perception of successful initiation of JA as rewarding (Schilbach et al., 2010; Pfeiffer et al., 2014; Oberwelland et al., 2016). A successfully initiated instance of JA alters the consecutive interaction by increasing the tendency to look at and dwell upon the partners face (Bayliss et al., 2013).
Triadic Interaction as a Dynamic Function of a Two-Dimensional State-Space
Having defined the basic states during triadic JA, the picture becomes more complex when considering that each of the two participants can adhere to any of these states during a triadic interaction unfolding in time. In theory, a dual social state may be one of 25 possible combinations (representing varying degrees of “interactivity”), spanning a two dimensional SGS (Figure 2; see McCall and Singer, 2015 for an alternative concept of a 2D gaze space). Some of these combinations might be more ephemeral than others: e.g., a person A might soon lose the motivation to initiate JA if person B does not respond to him adequately, person A might switch to PO very soon subsequently (‘stability’ of states is indicated by cell color in Figure 2, with gray cells indicating unstable and ephemeral states; red arrows represent subsequent shifts from unstable to stable states).
Furthermore, it is conceivable that mutual attention (PO/PO) might facilitate transitions from non-interactive to interactive states (indicated by green arrows in Figure 2). These transitions have yet to be empirically investigated. Only non-interactive states (blue box in Figure 2) can be understood on the basis of single persons whereas the study of interactive situations (purple box in Figure 2) requires a complex dynamic concept and experimental setup, based on the idea that the basic unit of analysis is the interaction between both interactants.
Reflections and Future Directions
It is our goal to provide a unifying taxonomy of social gaze in triadic interactions and their respective interdependencies. This complex, dynamic and holistic approach has two major achievements. First, it facilitates the integration of existing empirical findings within one unifying framework and helps to identify research desiderates. Second, it will go beyond many of the previous studies that investigated gaze behavior in isolation and it will provide a theoretical background to study the complex dynamics of dual states including their transitions, thereby increasing the ecological validity of the empirical approaches.
This approach is in accordance with a growing number of proposals that argued in favor of “embedded” interactionist or “enactive” approaches and emphasize the importance of ecological validity in non-verbal communication and social cognition research (Kingstone, 2009; Marsh et al., 2009; De Jaegher et al., 2010; Konvalinka and Roepstorff, 2012; Risko et al., 2012, 2016; Skarratt et al., 2012; Gallagher, 2013; Pfeiffer et al., 2013b; Schilbach et al., 2013). New methodological approaches due to technological advances increasingly allow for the development of paradigms which meet those demands (Pfeiffer et al., 2013b; Oberwelland et al., 2016, 2017).
This paves the way to research questions concerning the nature of gaze communication in triadic interactions. Even in triadic encounters which are not explicitly interactive interactants are still likely to exert subtle influences on each other in many reciprocal ways: In PO, dynamic interactions elicit a much stronger eye contact effect that static pictures (Hietanen et al., 2008; Pönkänen et al., 2011); In OO, the visual attention of another person will influence object processing in an observer in multiple ways (Reid et al., 2004; Bayliss et al., 2006; Becchio et al., 2008); the oculomotor changes observable in INT might be an active form of visual disengagement (Benedek et al., 2017). Therefore, a separate examination of allegedly interactive and non-interactive states in triadic interactions is not adequate. From the new unifying perspective of the SGS the very first step must be to systematically describe and identify the characteristics of gaze behavior associated with the individual gaze states. However, given the dynamic and continuous nature of non-verbal communication (Burgoon et al., 1989) our appreciation of the interactants experience of the encounter relies on our comprehension of transitions between interactional states. The consequential next step will then be the identification of potentially complex signifiers of these transitions in gaze behavior, yet unknown (e.g., gaze patterns characteristic for active attempts to catch the partners attention to reach a full-fledged state of JA), which can serve as indicators of these transitions in future studies.
We speculate that transitions between gaze states of the individual interactants are not independent, but are contingent upon each other to a changing degree. If these contingencies are crucial in the establishment of states of higher interactivity and phenomena like synchrony and rapport between interactants, then it should be possible to establish their causal role in experimental paradigms. The dual state of mutual attention (PO/PO) as a candidate state for a gate to higher degrees of interactivity (Figure 2) – as soon as its role is empirically corroborated – could be a potential starting point in these investigations.
Having established the prototypical SGS it is worth studying individual differences in the behavior and experiences in triadic gaze interactions. Questions which to the best of our knowledge have not been tackled before concern the relationship between specific personality traits and gaze behavior in triadic encounters and to which degree personality traits are ascribed on the basis of gaze behavior. Other obvious topics relate to developmental factors in the SGS and how and when children access the SGS or the effect of impairments in non-verbal communication as observable in autism have in the SGS.
Statements
Author contributions
All authors substantially contributed to the conception of the work. MJ drafted the manuscript. AH, GB, MS-R, and KV revised it critically.
Funding
This work is a part of the project ‘EMOTISK’ from the ‘Bringing technology to the people’ research program on human–machine interaction, funded by the German Federal Ministry of Education and Research (Grant Number for MS-R: 16SV7242, for KV: 16SV7244). MS-R received funding by the Excellence Initiative of the German federal and state governments.
Acknowledgments
The authors like to thank their respective work units for their continued support and constructive discussions. The authors want to express their gratitude to Joseph Walther from Nanyang Technological University for helpful and substantial feedback at the 3rd Annual International Summer School at Michigan State University and at the University Hospital Cologne.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Footnotes
1.^Although, our taxonomy explicitly comprises a set of discrete states, we use the term “Social Gaze Space” throughout the manuscript instead of the more precise term “Social Gaze State Space,” for the sake of simplicity and comprehensibility.
References
1
ArgyleM.CookM. (1976). Gaze and Mutual Gaze.Cambridge: Cambridge University Press.
2
BarghJ. A. (1994). “The four horsemen of automaticity: awareness, intention, efficiency, and control in social cognition,” inHandbook of Social Cognition: Basic Processes; Applications, edsWyerR. S.Jr.SrullT. (Hillsdale, NJ: Lawrence Erlbaum), 1–40.
3
Baron-CohenS.WheelwrightS.JolliffeT. (1997). Is there a “language of the eyes”? Evidence from normal adults, and adults with autism or Asperger Syndrome.Vis. Cogn.4311–331. 10.1080/713756761
4
BaylissA. P.MurphyE.NaughtinC. K.KritikosA.SchilbachL.BeckerS. I. (2013). “Gaze leading”: initiating simulated joint attention influences eye movements and choice behavior.J. Exp. Psychol. Gen.14276–92. 10.1037/a0029286
5
BaylissA. P.PaulM. A.CannonP. R.TipperS. P. (2006). Gaze cuing and affective judgments of objects: I like what you look at.Psychon. Bull. Rev.131061–1066. 10.3758/BF03213926
6
BaylissA. P.TipperS. P. (2006). Predictive gaze cues and personality judgments: should eye trust you?Psychol. Sci.17514–520. 10.1111/j.1467-9280.2006.01737.x
7
BecchioC.BertoneC.CastielloU. (2008). How the gaze of others influences object processing.Trends Cogn. Sci.12254–258. 10.1016/j.tics.2008.04.005
8
BenedekM.StoiserR.WalcherS.KörnerC. (2017). Eye behavior associated with internally versus externally directed cognition.Front. Psychol.8:1092. 10.3389/fpsyg.2017.01092
9
BorjiA.ParksD.IttiL. (2014). Complementary effects of gaze direction and early saliency in guiding fixations during free viewing.J. Vis.143–3. 10.1167/14.13.3
10
BurgoonJ. K.BullerD. B.WoodallW. G. (1989). Nonverbal Communication: The Unspoken Dialogue.Manhattan, NY: Harper & Row.x
11
CharmanT.Baron-CohenS.SwettenhamJ.BairdG.CoxA.DrewA. (2000). Testing joint attention, imitation, and play as infancy precursors to language and theory of mind.Cogn. Dev.15481–498. 10.1016/S0885-2014(01)00037-5
12
ChoiY. S.GrayH. M.AmbadyN. (2005). “The glimpsed world: Unintended communication and unintended perception,” inThe New Unconscious, edsHassinR. R.UlemanJ. S.BarghJ. A. (New York, NY: Oxford University Press), 309–333.
13
ContyL.TijusC.HuguevilleL.CoelhoE.GeorgeN. (2006). Searching for asymmetries in the detection of gaze contact versus averted gaze under different head views: a behavioural study.Spat. Vis.19529–545. 10.1163/156856806779194026
14
DalmasoM.CastelliL.ScatturinP.GalfanoG. (2017). Trajectories of social vision: eye contact increases saccadic curvature.Vis. Cogn.25358–365. 10.1080/13506285.2017.1290727
15
De JaegherH.Di PaoloE.GallagherS. (2010). Can social interaction constitute social cognition?Trends Cogn. Sci.14441–447. 10.1016/j.tics.2010.06.009
16
Doherty-SneddonG.PhelpsF. G. (2005). Gaze aversion: a response to cognitive or social difficulty?Mem. Cognit.33727–733. 10.3758/BF03195338
17
DriverJ.DavisG.RicciardelliP.KiddP.MaxwellE.Baron-CohenS. (1999). Gaze perception triggers reflexive visuospatial orienting.Vis. Cogn.6509–540. 10.1080/135062899394920
18
EgethH. E.YantisS. (1997). Visual attention: control, representation, and time course.Annu. Rev. Psychol.48269–297. 10.1146/annurev.psych.48.1.269
19
EmeryN. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze.Neurosci. Biobehav. Rev.24581–604. 10.1016/S0149-7634(00)00025-7
20
Falck-YtterT.von HofstenC. (2011). How special is social looking in ASD: a review.Prog. Brain Res.189209–222. 10.1016/B978-0-444-53884-0.00026-9
21
FarroniT.CsibraG.SimionF.JohnsonM. H. (2002). Eye contact detection in humans from birth.Proc. Natl. Acad. Sci. U.S.A.999602–9605. 10.1073/pnas.152159999
22
FriesenC. K.KingstoneA. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze.Psychon. Bull. Rev.5490–495. 10.3758/BF03208827
23
FrischenA.BaylissA. P.TipperS. P. (2007). Gaze cueing of attention: visual attention, social cognition, and individual differences.Psychol. Bull.133694–724. 10.1037/0033-2909.133.4.694
24
GallagherS. (2013). The socially extended mind.Cogn. Syst. Res.24–12. 10.1016/j.cogsys.2013.03.008
25
GeorgeN.ContyL. (2008). Facing the gaze of others.Neurophysiol. Clin.38197–207. 10.1016/j.neucli.2008.03.001
26
GlenbergA. M.SchroederJ. L.RobertsonD. A. (1998). Averting the gaze disengages the environment and facilitates remembering.Mem. Cognit.26651–658. 10.3758/BF03211385
27
GobelM. S.KimH. S.RichardsonD. C. (2015). The dual function of social gaze.Cognition136359–364. 10.1016/j.cognition.2014.11.040
28
GrossmannT. (2017). The eyes as windows into other minds: an integrative perspective.Perspect. Psychol. Sci.12107–121. 10.1177/1745691616654457
29
HainsS. M. J.MuirD. W. (1996). Infant sensitivity to adult eye direction.Child Dev.671940–1951. 10.2307/1131602
30
HaithM. M.BergmanT.MooreM. J. (1977). Eye contact and face scanning in early infancy.Science198853–855. 10.1126/science.918670
31
HietanenJ. K.LeppänenJ. M.PeltolaM. J.Linna-ahoK.RuuhialaH. J. (2008). Seeing direct and averted gaze activates the approach–avoidance motivational brain systems.Neuropsychologia462423–2430. 10.1016/j.neuropsychologia.2008.02.029
32
ItierR. J.BattyM. (2009). Neural bases of eye and gaze processing: the core of social cognition.Neurosci. Biobehav. Rev.33843–863. 10.1016/j.neubiorev.2009.02.004
33
IttiL.KochC. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention.Vision Res.401489–1506. 10.1016/S0042-6989(99)00163-7
34
JarickM.KingstoneA. (2015). The duality of gaze: eyes extract and signal social information during sustained cooperative and competitive dyadic gaze.Front. Psychol.6:1423. 10.3389/fpsyg.2015.01423
35
KingstoneA. (2009). Taking a real look at social attention.Curr. Opin. Neurobiol.1952–56. 10.1016/j.conb.2009.05.004
36
KleinkeC. L. (1986). Gaze and eye contact: a research review.Psychol. Bull.10078–100. 10.1037/0033-2909.100.1.78
37
KobayashiH.KohshimaS. (1997). Unique morphology of the human eye.Nature387767–768. 10.1038/42842
38
KobayashiH.KohshimaS. (2001). Unique morphology of the human eye and its adaptive meaning: comparative studies on external morphology of the primate eye.J. Hum. Evol.40419–435. 10.1006/jhev.2001.0468
39
KonvalinkaI.RoepstorffA. (2012). The two-brain approach: how can mutually interacting brains teach us something about social interaction?Front. Hum. Neurosci6:215. 10.3389/fnhum.2012.00215
40
KuhnG.TeszkaR.TenawN.KingstoneA. (2016). Don’t be fooled! Attentional responses to social cues in a face-to-face and video magic trick reveals greater top-down control for overt than covert attention.Cognition146136–142. 10.1016/j.cognition.2015.08.005
41
LaidlawK. E. W.RiskoE. F.KingstoneA. (2012). A new look at social attention: orienting to the eyes is not (entirely) under volitional control.J. Exp. Psychol. Hum. Percept. Perform.381132–1143. 10.1037/a0027075
42
LaidlawK. E. W.RothwellA.KingstoneA. (2016). Camouflaged attention: covert attention is critical to social communication in natural settings.Evol. Hum. Behav.37449–455. 10.1016/j.evolhumbehav.2016.04.004
43
LeeK.EskrittM.SymonsL. A.MuirD. (1998). Children’s use of triadic eye gaze information for “mind reading.”Dev. Psychol.34525–539.
44
MarksonL.PatersonK. B. (2009). Effects of gaze-aversion on visual-spatial imagination.Br. J. Psychol.100553–563. 10.1348/000712608X371762
45
MarshK. L.RichardsonM. J.SchmidtR. C. (2009). Social connection through joint action and interpersonal coordination.Top. Cogn. Sci.1320–339. 10.1111/j.1756-8765.2009.01022.x
46
McCallC.SingerT. (2015). Facing off with unfair others: introducing proxemic imaging as an implicit measure of approach and avoidance during social interaction.PLoS One10:e0117532. 10.1371/journal.pone.0117532
47
MoralesM.MundyP.RojasJ. (1998). Following the direction of gaze and language development in 6-month-olds.Infant Behav. Dev.21373–377. 10.1016/S0163-6383(98)90014-5
48
MundyP. (2003). Annotation: the neural basis of social impairments in autism: the role of the dorsal medial-frontal cortex and anterior cingulate system.J. Child Psychol. Psychiatry44793–809. 10.1111/1469-7610.00165
49
MundyP.BlockJ.DelgadoC.PomaresY.Van HeckeA. V.ParladeM. V. (2007). Individual differences and the development of joint attention in infancy.Child Dev.78938–954. 10.1111/j.1467-8624.2007.01042.x
50
MundyP.NewellL. (2007). Attention, joint attention, and social cognition.Curr. Dir. Psychol. Sci.16269–274. 10.1111/j.1467-8721.2007.00518.x
51
OberwellandE.SchilbachL.BarisicI.KrallS. C.VogeleyK.FinkG. R.et al (2016). Look into my eyes: investigating joint attention using interactive eye-tracking and fMRI in a developmental sample.Neuroimage130248–260. 10.1016/j.neuroimage.2016.02.026
52
OberwellandE.SchilbachL.BarisicI.KrallS. C.VogeleyK.FinkG. R.et al (2017). Young adolescents with autism show abnormal joint attention network: a gaze contingent fMRI study.Neuroimage Clin.14112–121. 10.1016/j.nicl.2017.01.006
53
PfeifferU. J.SchilbachL.JordingM.TimmermansB.BenteG.VogeleyK. (2012). Eyes on the mind: investigating the influence of gaze dynamics on the perception of others in real-time social interaction.Front. Psychol.3:537. 10.3389/fpsyg.2012.00537
54
PfeifferU. J.SchilbachL.TimmermansB.KuzmanovicB.GeorgescuA. L.BenteG.et al (2014). Why we interact: on the functional role of the striatum in the subjective experience of social interaction.Neuroimage101124–137. 10.1016/j.neuroimage.2014.06.061
55
PfeifferU. J.TimmermansB.BenteG.VogeleyK.SchilbachL. (2011). A non-verbal turing test: differentiating mind from machine in gaze-based social interaction.PLoS One6:e27591. 10.1371/journal.pone.0027591
56
PfeifferU. J.TimmermansB.VogeleyK.FrithC. D.SchilbachL. (2013a). Towards a neuroscience of social interaction.Front. Hum. Neurosci7:22. 10.3389/fnhum.2013.00022
57
PfeifferU. J.VogeleyK.SchilbachL. (2013b). From gaze cueing to dual eye-tracking: novel approaches to investigate the neural correlates of gaze in social interaction.Neurosci. Biobehav. Rev.372516–2528. 10.1016/j.neubiorev.2013.07.017
58
PhelpsF. G.Doherty-SneddonG.WarnockH. (2006). Helping children think: gaze aversion and teaching.Br. J. Dev. Psychol.24577–588. 10.1348/026151005X49872
59
PönkänenL. M.AlhoniemiA.LeppänenJ. M.HietanenJ. K. (2011). Does it make a difference if I have an eye contact with you or with your picture? An ERP study.Soc. Cogn. Affect. Neurosci.6486–494. 10.1093/scan/nsq068
60
ReichleE. D.ReinebergA. E.SchoolerJ. W. (2010). Eye movements during mindless reading.Psychol. Sci.211300–1310. 10.1177/0956797610378686
61
ReidV. M.StrianoT.KaufmanJ.JohnsonM. H. (2004). Eye gaze cueing facilitates neural processing of objects in 4-month-old infants.Neuroreport152553–2555. 10.1097/00001756-200411150-00025
62
RiskoE. F.KingstoneA. (2011). Eyes wide shut: implied social presence, eye tracking and attention.Atten. Percept. Psychophys.73291–296. 10.3758/s13414-010-0042-1
63
RiskoE. F.LaidlawK.FreethM.FoulshamT.KingstoneA. (2012). Social attention with real versus reel stimuli: toward an empirical approach to concerns about ecological validity.Front. Hum. Neurosci6:143. 10.3389/fnhum.2012.00143
64
RiskoE. F.RichardsonD. C.KingstoneA. (2016). Breaking the fourth wall of cognitive science: real-world social attention and the dual function of gaze.Curr. Dir. Psychol. Sci.2570–74. 10.1177/0963721415617806
65
SalviC.BricoloE.FranconeriS. L.KouniosJ.BeemanM. (2015). Sudden insight is associated with shutting out visual inputs.Psychon. Bull. Rev.221814–1819. 10.3758/s13423-015-0845-0
66
SchilbachL. (2014). On the relationship of online and offline social cognition.Front. Hum. Neurosci.8:278. 10.3389/fnhum.2014.00278
67
SchilbachL.TimmermansB.ReddyV.CostallA.BenteG.SchlichtT.et al (2013). Toward a second-person neuroscience.Behav. Brain Sci.36393–414. 10.1017/S0140525X12000660
68
SchilbachL.WilmsM.EickhoffS. B.RomanzettiS.TepestR.BenteG.et al (2010). Minds made for sharing: initiating joint attention recruits reward-related neurocircuitry.J. Cogn. Neurosci.222702–2715. 10.1162/jocn.2009.21401
69
Schulte-RütherM.MarkowitschH. J.FinkG. R.PiefkeM. (2007). Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: a functional magnetic resonance imaging approach to empathy.J. Cogn. Neurosci.191354–1372. 10.1162/jocn.2007.19.8.1354
70
SchynsP. G.BonnarL.GosselinF. (2002). Show me the features! Understanding recognition from the use of visual information.Psychol. Sci.13402–409. 10.1111/1467-9280.00472
71
SebanzN.KnoblichG. (2009). Prediction in joint action: what, when, and where.Top. Cogn. Sci.1353–367. 10.1111/j.1756-8765.2009.01024.x
72
SenjuA.CsibraG. (2008). Gaze following in human infants depends on communicative signals.Curr. Biol.18668–671. 10.1016/j.cub.2008.03.059
73
SenjuA.HasegawaT. (2005). Direct gaze captures visuospatial attention.Vis. Cogn.12127–144. 10.1080/13506280444000157
74
SenjuA.JohnsonM. H. (2009). The eye contact effect: mechanisms and development.Trends Cogn. Sci.13127–134. 10.1016/j.tics.2008.11.009
75
SenjuA.KikuchiY.HasegawaT.TojoY.OsanaiH. (2008). Is anyone looking at me? Direct gaze detection in children with and without autism.Brain Cogn.67127–139. 10.1016/j.bandc.2007.12.001
76
ShepherdS. V. (2010). Following gaze: gaze-following behavior as a window into social cognition.Front. Integr. Neurosci.4:5. 10.3389/fnint.2010.00005
77
SkarrattP. A.ColeG. G.KuhnG. (2012). Visual cognition during real social interaction.Front. Hum. Neurosci.6:196. 10.3389/fnhum.2012.00196
78
SmallwoodJ.BrownK. S.TipperC.GiesbrechtB.FranklinM. S.MrazekM. D.et al (2011). Pupillometric evidence for the decoupling of attention from perceptual input during offline thought.PLoS One6:e18298. 10.1371/journal.pone.0018298
79
SmilekD.CarriereJ. S. A.CheyneJ. A. (2010). Out of mind, out of sight: eye blinking as indicator and embodiment of mind wandering.Psychol. Sci.21786–789. 10.1177/0956797610368063
80
Solé PuigM.Pérez ZapataL.Aznar-CasanovaJ. A.SupèrH. (2013). A role of eye vergence in covert attention.PLoS One8:e52955. 10.1371/journal.pone.0052955
81
StassJ. W.WillisF. N. (1967). Eye contact, pupil dilation, and personal preference.Psychon. Sci.7375–376. 10.3758/BF03331131
82
TomaselloM.CarpenterM. (2005). The emergence of social cognition in three young chimpanzees.Monogr. Soc. Res. Child Dev.70 vii-132. 10.1111/j.1540-5834.2005.00324.x
83
TomaselloM.CarpenterM.CallJ.BehneT.MollH. (2005). Understanding and sharing intentions: the origins of cultural cognition.Behav. Brain Sci.28675–691. 10.1017/S0140525X05000129
84
TomaselloM.HareB.LehmannH.CallJ. (2007). Reliance on head versus eyes in the gaze following of great apes and human infants: the cooperative eye hypothesis.J. Hum. Evol.52314–320. 10.1016/j.jhevol.2006.10.001
85
VernettiA.SmithT. J.SenjuA. (2017). Gaze-contingent reinforcement learning reveals incentive value of social signals in young children and adults.Proc. R. Soc. B Biol. Sci. 284: 20162747. 10.1098/rspb.2016.2747
86
von GrünauM.AnstonC. (1995). The detection of gaze direction: a stare-in-the-crowd effect.Perception241297–1313. 10.1068/p241297
87
Walker-SmithG. J.GaleA. G.FindlayJ. M. (1977). Eye movement strategies involved in face perception.Perception6313–326. 10.1068/p060313
88
YarbusA. L. (1967). Eye Movements and Vision.Boston, MA: Springer, 10.1007/978-1-4899-5379-7
Summary
Keywords
non-verbal communication, social gaze, joint attention, triadic interaction, ecological validity, taxonomy, social psychology
Citation
Jording M, Hartz A, Bente G, Schulte-Rüther M and Vogeley K (2018) The “Social Gaze Space”: A Taxonomy for Gaze-Based Communication in Triadic Interactions. Front. Psychol. 9:226. doi: 10.3389/fpsyg.2018.00226
Received
06 October 2017
Accepted
09 February 2018
Published
26 February 2018
Volume
9 - 2018
Edited by
Mario Weick, University of Kent, United Kingdom
Reviewed by
Matthias S. Gobel, University College London, United Kingdom; Cade Andrew McCall, University of York, United Kingdom
Updates
Copyright
© 2018 Jording, Hartz, Bente, Schulte-Rüther and Vogeley.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Mathis Jording, mathis.jording@uk-koeln.de
This article was submitted to Personality and Social Psychology, a section of the journal Frontiers in Psychology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.