MINI REVIEW article

Front. Psychol., 06 January 2017

Sec. Human-Media Interaction

Volume 7 - 2016 | https://doi.org/10.3389/fpsyg.2016.02061

An Integrated Review of Emoticons in Computer-Mediated Communication

  • 1. Escuela de Psicología, Universidad de Santiago de Chile Santiago, Chile

  • 2. Departamento de Ingeniería Informática, Universidad de Santiago de Chile Santiago, Chile

Abstract

Facial expressions constitute a rich source of non-verbal cues in face-to-face communication. They provide interlocutors with resources to express and interpret verbal messages, which may affect their cognitive and emotional processing. Contrarily, computer-mediated communication (CMC), particularly text-based communication, is limited to the use of symbols to convey a message, where facial expressions cannot be transmitted naturally. In this scenario, people use emoticons as paralinguistic cues to convey emotional meaning. Research has shown that emoticons contribute to a greater social presence as a result of the enrichment of text-based communication channels. Additionally, emoticons constitute a valuable resource for language comprehension by providing expressivity to text messages. The latter findings have been supported by studies in neuroscience showing that particular brain regions involved in emotional processing are also activated when people are exposed to emoticons. To reach an integrated understanding of the influence of emoticons in human communication on both socio-cognitive and neural levels, we review the literature on emoticons in three different areas. First, we present relevant literature on emoticons in CMC. Second, we study the influence of emoticons in language comprehension. Finally, we show the incipient research in neuroscience on this topic. This mini review reveals that, while there are plenty of studies on the influence of emoticons in communication from a social psychology perspective, little is known about the neurocognitive basis of the effects of emoticons on communication dynamics.

Introduction

Emotions are one of the most important information signals in social cognition (). This information usually comes from facial expressions in face-to-face interactions. However, today people use alternative communication channels – like text-based communication – to interact with others when they are not physically present. In such cases, facial information is not always available. The lack of affective signals and pragmatic information in human communication has been associated with difficulty in understanding other people (). In computer-mediated communication (CMC), particularly in text-based communication, face-related information has been partially substituted by emoticons, which enable interlocutors to express different emotional states by using text-based representations of facial expressions. Although emoticons are unnatural, iconic, and static representations of human facial expressions, they have become a popular resource to enrich text-based communication since 1982 (). Beyond their uses and impacts, little is known about how emoticons are processed in CMC at the brain level.

In this mini review, we examine socio-cognitive and neuroscience research on the evidence of psychological effects of emoticons in CMC and their processing in human cognition. This article is structured in four parts. The first part presents a review of relevant literature on the importance of facial expressions in communication. Second, the importance of emoticons in text-based communication, social presence, and language processing is discussed. Third, research on neuroscience and emoticons is reviewed. Finally, we conclude with a discussion about future research on this topic and practical implications for cognitive sciences.

The Importance of Facial Expressions in Communication

Natural interactions in human communication are full of contextual cues, such as gestures, prosody, and facial expressions, which are processed for common understanding. Some theories about human interactions have considered that meaning comprehension requires an ecological context and the use of expressive non-verbal tools to enrich communication, which is the case of face-to-face contexts ().

Facial expressions are considered one of the most important cues in human communication. Evidence suggests that from birth, human beings prefer facial expressions over other types of stimuli (; ). The face is one of the most visible and complex sources of information about the emotional states of individuals. Faces are integrated into comprehension processes during social interactions and communication. In fact, most human beings have a special capacity to process them with specific mechanisms (). For example, the process of face perception involves a complex neural network that enables people to recognize aspects of facial structure of others and to communicate socially using the perception of facial movements (, ).

In natural conditions, facial expressions and other contextual cues inform individuals about the affective states of the participants in the communication process. However, during the last decades, alternative communication media have proliferated (e.g., desktop computers and smartphones) in developed countries (). In particular, in text-based CMC there is a physical gap that implies a loss of contextual information (e.g., facial expressions). This aspect constitutes a difficulty for CMC, especially for communication with figurative language (), which may have negative (; ) or positive (, ; ) effects on human relationships as a result of the lack of nonverbal and contextual cues.

The lack of face-to-face interactions and therefore the absence of regulating feedback, affect interactions at different levels. It also causes people to behave differently while interacting in some CMC contexts. For example, it is known that when small groups interact face-to-face in a problem-solving discussion, they are more likely to reach agreement than when they communicate in a computerized conference (). On the other hand, self-awareness has been reported to be lower in face-to-face communication than in CMC (, ). These differences are due to difficulties in communicative coordination between people without a context that provides informational feedback, cues for controlling discussion, and lack of nonverbal involvement ().

The Importance of Emoticons in Text-Based Communication

Two factors that influence the success of a communication channel are its “social presence” and its “richness.” On the one hand, social presence (; ) refers to the degree to which a channel can be used to transmit relational information that increases the perception of human interaction with non-verbal cues such as facial expressions (). For instance, text-based CMC like chats or emails, have less social presence than face-to-face communication because they lack social non-verbal cues for the interactions. On the other hand, media richness () refers to the degree of adequacy of a channel to the communication context and goals in order to reduce uncertainty and equivocality. For example, managers may prefer face-to-face group meetings over email exchange in order to avoid misinterpretations of messages when working on a task. In this scenario, face-to-face communication, which is equipped with a variety of resources, such as facial expressions and prosody, is considered to be richer than text-based communication.

Despite its levels of social presence and richness, text-based communication often has a social function; however, it needs more expressive tools to enrich and socialize interactions. One way text-based CMC users have made their conversations more dynamic, conveying more social presence and richness has been through the use of emoticons. Emoticons can be considered a humanized tool for emotional expressiveness that replaces cues that are naturally present in face-to-face and voice-to-voice communication. Emoticons provide enjoyment and more information to the interaction () and they also enhance the comprehension of messages () with a more accurate perception of emotions, attitudes, and intentions during text-based communication ().

While there is quite abundant evidence on how emoticons are used in text-based CMC, little is known about how they are processed at the cognitive and brain level. This knowledge could help to explain the effects of emoticons on text-based communication. In this sense, both cognitive science and neuroscience could contribute to address this question. On the one hand, research in cognitive science related to language processing constitutes a valuable framework for understanding how emoticons can influence the interpretation of messages. On the other hand, research in neuroscience provides further evidence on how the brain operates in CMC contexts, especially when interlocutors perceive affective information on schematic faces integrated with language messages.

Influence of Emoticons on Language Comprehension

Facial expressions are naturally present in face-to-face communication, however, in mediated communication, particularly in CMC, communication of facial expressions may be limited to the richness of the underlying communication channel (). Emoticons can be defined as facial expression surrogates, which correspond to symbolic, lexical, or graphical resources used in written communication to convey emotional expressions. In this sense, emoticons can be used as paralinguistic cues of an utterance, which in written communication can help writers and readers to elucidate meaning ().

Emoticons have been described, in some cases, as useful resources in text-based mediated communication (). They have provided people with an alternative way to express their emotions, feelings, and mood changes to others in restricted communication channels such as text-based ones. Emoticons have been important in increasing social presence in text-based interactions, where communication tends to be cold without affective cues. For example, a study has shown that in socio-emotional contexts, emoticons are more commonly used than in task-oriented ones (). Likewise, the same authors also showed that emoticons are more common in interactions between friends than between strangers, and in positive contexts than in negative ones ().

It has been stated that emoticons can play a central role in interpreting text messages in CMC (). In this context, and found that text messages can be strengthened by using emoticons. Emoticons can also change the direction of a message to the opposite direction, which is the case of sarcastic utterances. A related study by showed that emoticons are more expressive than punctuation marks in the disambiguation of sarcastic utterances. Along the same lines, and showed that emoticons in short messages like those found on microblogging platforms (e.g., Twitter) are particularly helpful for humans and machines in classifying sarcastic, non-sarcastic, positive, and negative messages. For example, consider the following written sentence: “I love the customer service of this company.” Without a context and lack of verbal cues, it is likely that its interpretation is literal. However, with the incorporation of traditional emoticons (i.e. :) and :( ) at the end of the sentence, its intentional meaning (literal and sarcastic, respectively) is clarified.

Contrary to the above studies, results from an experiment conducted by showed that the contribution of emoticons in interpreting messages is limited. However, the authors found consistency in the interpretation of negatively biased messages as a result of both negative emoticons and negative verbal content.

In addition to message interpretation, found that emoticons have the potential to reduce negative responses that are typically experienced when exposed to ironic texts. It is important to note that these findings are contextualized in an artificial setting in which electrodermal activity (for arousal) and electromyography (for facial expressions) were used to measure participants’ reactions. Overall, the authors found that the presence of emoticons increased arousal levels, reduced frowning expressions, and enhanced smiling.

The above articles illustrate the active role that emoticons can play in language comprehension, specifically in text message interpretation. Although simple in expressions, emoticons have the potential to enrich verbal expressions acting as substitutes of richer forms, such as facial expressions and prosody, which are naturally present in face-to-face interactions.

What Does Neuroscience Tell Us About Emoticons?

Recently, neuroscientists have been studying neural processing during emoticon perception. Different techniques have been chosen to study emoticon processing. One of these has been functional Magnetic Resonance Imaging (fMRI), which provides information on the brain areas involved in particular processing with a high spatial resolution. The other technique is electroencephalography (EEG), which provides high temporal resolution. In EEG studies, there is a specific technique known as event related potentials (ERPs), which provides information about the time course of the processing and its characteristics after a specific stimulation.

fMRI Studies

One question that has driven research is whether emoticons activate the same brain areas that faces do. One of the structures involved in face processing is the right fusiform gyrus, which participates in face recognition () and presents dysfunctional activation in patients with prosopagnosia, which refers to the incapacity to recognize faces ().

Initial fMRI studies indicated that isolated emoticons do not activate the same brain regions in the processing of human faces. However, they activate the same areas involved in emotional discrimination, like the right inferior frontal gyrus. showed that in an emotional valence decision task, human faces activate the right fusiform gyrus, right inferior frontal gyrus, right middle frontal gyrus, and right inferior parietal lobe. Conversely, Japanese emoticons did not activate the right fusiform gyrus, but they activated the right inferior frontal gyrus. These findings suggest that these kinds of emoticons may be associated with emotional discrimination. In a posterior study, investigated brain activations during the exposure to emoticons presented after sentences. Results showed that emoticons are associated with nonverbal information processing and that the brain areas involved in verbal and nonverbal information processing are more active when sentences are followed by emoticons than when they are not. Specifically, they found that, although the right fusiform gyrus was not activated when sentences with emoticons were presented, the right inferior frontal gyrus was activated. This finding indicates that emotional discrimination based on emoticon perception is similar to other non-verbal cues like facial expressions or speech prosody.

In another study, compared brain activity for pictures of real faces and real houses with their corresponding schematic icons to observe the neural basis for iconic symbol recognition. For this purpose, they used emoticons as face icons. The results of this study showed more cortical brain activation than subcortical for icons, involving the frontal and parietal cortex, and the temporo-occipital junction in the ventral pathway – implied in the know-what processes, which are related to object recognition. The authors suggest that this activation corresponds to neural correlates of visual concept formation, distinguishing it from the perception of concrete visual objects. Specifically, they observed a bilateral activation of the fusiform gyrus during the exposure to both emoticons and house icons. Such activation could be associated with the characteristics of the icons, which share aspects of words and objects. The authors also observed activation in a region known as Visual Word Form Area, which is a specific area of he left fusiform gyrus that responds to word’s letters () and participates in “integrating the shape elements into a meaningful whole” (; , p. 303). Unlike emoticons, faces activated the inferior occipital cortex, which participates in face information processing at the individual level (), and this activity decreased along the ventral pathway.

Recently, the activity in the fusiform gyrus for emotional words and emoticons was observed in patients with autism spectrum disorder (ASD) (). These patients have a social dysfunction due to different brain pattern alterations. For example, patients with ASD do not present activation in the fusiform gyrus when they see emotional facial expressions (; ). Moreover, patients with ASD do not show activation in the right fusiform gyrus in the discrimination of facial expressions (). In the study of , patients with ASD and healthy participants were exposed to emotional words and emoticons to observe the activation of the fusiform gyrus. They observed an increment in the activity of the fusiform gyrus for emotional words, which led the authors to hypothesize that this result could be a manifestation of a compensatory mechanism to control social information processing. Nevertheless, further analyses of brain activation during the presentation of emoticons revealed a lower activation in the fusiform gyrus. This result suggests that word descriptions are more familiar with emotional processing than facial expressions in ASD patients.

EEG Studies

Face processing has been studied with EEG showing the N170 ERP for face perception. This component is a negative-going deflection in the occipito-temporal cortex with a maximum peak around 170 ms after the presentation of a face and it is related to the configurational processing of faces (; ; ). It is known that this component appears even if faces are schematic (; ).

Today, there are very few studies about emoticon processing with EEG recording. obtained similar ERPs (N170) for typographic smiley emoticons “:-)” and faces, suggesting similarities in emoticon and face perception. A study by suggests that processing of emoticons is more privileged than that of the word to which they refer to. In particular, the authors showed a priming effect with masked emoticons in word processing with modulations in N2 and late positive component (LPC), which is not present by masked affective words. It is known that positive words with high positive arousal level influence LPC (), reflecting the activation of motivational and attentional brain systems by emotional stimuli and the initial memory storage during the processing of affective information (). It is important to note that this component is related to the stimuli’s arousal and not to its valence ().

Concluding Thoughts

At present, humans relations are rapidly evolving through the use of electronic means. This fact has involved an increasing use of alternative cues to communicate emotional states in computer-mediated contexts. Emoticon, as one of these alternative cues, can change people’s dispositions in the comprehension processes or interactions with others. Moreover, they can affect decisions, mood, or perspective of the conversation.

The use of emoticons is important in CMC because they enable greater emotional expressiveness in absence of context. Not only do emoticons provide more fun to the interaction (e.g., ; ), but they also facilitate disambiguation of messages, enabling better comprehension (), therefore they are an expressive resource. Although this is important, there is little evidence on the cognitive and brain dimensions on how these emoticons are processed in human communication. Today, studies have focused on processing structural characteristics of emoticons and their relationship with the processing of emotional information and human faces. However, further research toward understanding the processing of emoticons within the dynamics of text-based communications is required. This would enable in-depth understanding of how affectivity unfolds in CMC interactions and how this affective dimension affects language understanding.

Additionally, it is important to note that not all emoticons have the same degree of humanization. For example, emojis are emoticons that are more human than typographic in nature. Research on the effects of different types of emoticons according to their structural characteristics (e.g., emojis, typographic, Japanese, etc.) may be useful for developing communication technology to favor social presence and the affectivity that characterizes communication in social interactions.

Statements

Author contributions

All authors listed, have made substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

This study was funded by Inserción a la Academia 79140067 (PAI), Comisión Nacional de Investigación Científica y Tecnológica (CONICYT).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Summary

Keywords

emoticons, emotional expressivity, computer-mediated communication, language comprehension, neuroscience

Citation

Aldunate N and González-Ibáñez R (2017) An Integrated Review of Emoticons in Computer-Mediated Communication. Front. Psychol. 7:2061. doi: 10.3389/fpsyg.2016.02061

Received

26 September 2016

Accepted

20 December 2016

Published

06 January 2017

Volume

7 - 2016

Edited by

Dirk Heylen, University of Twente, Netherlands

Reviewed by

Andrea Kleinsmith, University of Maryland, Baltimore County, USA; Andrej Košir, University of Ljubljana, Slovenia

Updates

Copyright

*Correspondence: Nerea Aldunate,

This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Psychology

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics