Processing emotional expressions is fundamental for social interactions and communication; in fact, from a very young age, infants are able to detect visual and auditory information in faces and voices of people around them (Grossmann, 2010). Such capability would develop into the skill to recognize and discriminate emotions, thanks to the contribution of the experience and of the maturation of sensory and perceptual systems. This recognition involves a multisensory effect, evidenced by integration effects of facial and vocal information on cerebral activity, which are apparent both at the level of heteromodal cortical regions of convergence (e.g., bilateral posterior superior temporal sulcus), and at unimodal levels of sensory processing (Campanella and Belin, 2007; Davies-Thompson et al., 2019; Young et al., 2020).

In relation to such cross-sensorial and unisensorial effects, hearing impairment could compromise multisensory integration, in relation to its onset, etiology, and severity, leading the patient to rely only or predominantly on the visual modality in communication, including emotional perception and expression (Mildner and Koska, 2014). In fact, for 92% of children with cochlear implant (CI), perception was dominated by vision when visual and auditory speech information conflicted (Schorr et al., 2005). This statement is supported by the results of studies employing the McGurk effect on CI users, which requires the integration of auditory and visual sensory stimuli. For instance, children who received their CI prior to age 30 months accurately identified the incongruent auditory–visual stimuli, whereas children who received their CI after 30 months of age did not (Schorr, 2005). This evidence appears particularly worthy because differently from adults, who mainly prefer visual modality, infants and young children show auditory processing preference, but in children with congenital hearing impairment, such auditory dominance appears absent. Interestingly, in post-lingually deaf CI patients, such greater relying on visual information, indexed by higher speech-reading performances than normal-hearing (NH) individuals, led instead to an increased capacity of integrating visual and distorted speech signals, producing higher visuoauditory performances (Rouger et al., 2007). Furthermore, such evidence in post-lingual deaf patients was also supported by neurophysiological assessments, evidencing a positive correlation between visual activity and auditory speech recovery, suggesting a facilitating role for the visual modality in auditory words’ perception during communicative situations (Strelnikov et al., 2013). With respect to general processing preferences, contrary to adults, who prefer the visual modality (Scherer, 2003), infants and young children exhibit auditory processing preference. Importantly, congenital hearing-impaired children who underwent auditory–verbal therapy (a therapy limiting visual cue in order to strengthen the auditory pathway for language learning) reported a behavior similar to NH children, which is an overall auditory preference in response to audiovisual stimuli, although responses did not significantly differ from chance (Zupan and Sussman, 2009). Contrary to NH individuals, those with hearing impairments do not benefit from the addition of the auditory cues to the visual mode (e.g., Most and Aviner, 2009). Although the accuracy of emotion perception among children with hearing loss (HL) was lower than that of NH children in auditory, visual, and auditory–visual conditions, in prelingually deaf very young children (about 4–6 years old), the combined auditory–visual mode significantly surpassed the auditory or visual modes alone, as in the NH group, supporting the use of auditory information for emotion perception, probably thanks to intensive rehabilitation (Most and Michaelis, 2012) and neuroplasticity. Such results strongly support the hypothesis of a sensitive period (Kral et al., 2001; Sharma et al., 2005; Gilley et al., 2010) for the establishment of the integration of auditory and visual stimuli.

Thanks to their activity of direct stimulation of the acoustic nerve, converting the auditory stimuli into electrical signals directed to the brain, CIs can successfully restore hearing in profoundly deaf individuals. After intensive rehabilitation, most CI users can reach a good level of speech comprehension. However, the acoustic signal provided by the device is severely degraded, resulting in a poor frequency resolution and deficits in pitch patterns (Gfeller et al., 2007; eHopyan et al., 2012) and pitch changes or direction discrimination (Gfeller et al., 2002) in comparison to NH controls.

Hearing-impaired children go through an early auditory development that is different from that of NH toddlers. This condition would affect their judgment of the emotional content of a stimulus, insofar as the auditory modality resulted as particularly important for the communication of emotions in young children (Baldwin and Moses, 1996; Akhtar and Gernsbacher, 2008). The study of such mechanisms appears of great impact since about 600,000 patients world-wide are CI users (The Ear Foundation, 2017), and many of them are children who were born deaf or lost their hearing within the first few years of life. CI children are a paradigmatic model for the study of emotion recognition skills, as due to the early acquisition of deafness, they learned language through the degraded input of the CI, which greatly affects harmonic pitch perception. This ability is strongly necessary for emotion recognition in voices, and its deficiency could have implications on how child CI users learn to produce vocal emotions (Damm et al., 2019). However, a very recent study provided evidence that also deaf people can develop skills for emotional vocalizations despite the presence of some differences in comparison to NH adults (Sauter et al., 2019). Using unilateral CI (UCI) in children, due to non-physiological development of their auditory system and to their asymmetry in receiving auditory inputs, represents a powerful model of investigation of the possible modulation of the hemispheric specialization and of auditory-related emotional skills development in relation to the restored hearing condition. Additionally, such participants would provide evidence of the possible modulation of the physiological processes of emotion recognition following the restoration of the auditory capabilities, of which the exact time of beginning is due to the CI surgery time. Children, 7–13 years of age, using UCIs perform more poorly than age- and gender-matched controls on the affective speech prosody task but as well as controls in tasks of facial affect perception (Hopyan-Misakyan et al., 2009), as measured by the DANVA-2 (Nowicki and Duke, 1994).

One of the few studies that investigated both auditory recognition and vocal production of emotions did not find any consistent advantage for age-matched NH participants in comparison to three prelingually, bilaterally, profoundly deaf children aged 6–7 years who received CIs before age 2 years; however, confusion matrices among three of the investigated emotions (anger, happiness, and fear) showed that children with and without hearing impairment may rely on different cues (Mildner and Koska, 2014).

With respect to emotional skills attainment and in relation to the hemispheric specialization for emotional processing (Gainotti, 2019), it is interesting to consider that patients enrolled in the present study were UCI users, that is, single-side deaf (SSD) patients. In fact, in SSD population, it was evidenced that the occurrence of a massive reorganization of aural preference in favor of the hearing ear is greater than the precocity of unilateral HL onset, therefore supporting the importance of a short time between the first and second implantation in children (Kral et al., 2013; Gordon et al., 2015; Gordon and Papsin, 2019).

Concerning neural correlates of emotion recognition, gamma band electroencephalogram (EEG) was found to be particularly sensitive for emotion classification (Li and Lu, 2009; Yang et al., 2020). Gamma band cerebral activity has been previously linked to facial emotion recognition processes; for instance, a right hemisphere dominance in gamma activity was found during emotional processing of faces in comparison to neutral ones (e.g., Balconi and Lucchiari, 2008). Such evidences are in accord to the right hemisphere hypothesis for emotion processing, that starting from observations on patients with single hemisphere lesions states the dominance of the right hemisphere for every kind of emotional response (Gainotti, 2019). With specific regard to emotional prosody processing and brain activity lateralization, Kotz and colleagues hypothesized that (i) differentially lateralized subprocesses underlie emotional prosody processing and (ii) the lateralization of emotional prosody can be modulated by methodological factors (Kotz et al., 2006). Furthermore, concerning verbal stimuli, in adult CI users, gamma band–induced activity was found to be higher in NH than in CI users, irrespectively of the valence of the emotions investigated (Agrawal et al., 2013).

On the base of the previous issues, the following experimental questions have been approached in a population of NH and UCI children: (i) Given the non-physiological development of the auditory system in deaf children who underwent hearing restoration through CI use, are the emotional auditory stimuli processed in a similar way than NH children? (ii) Is the auditory age, meant as the age at CI surgery, crucial in the capacity of recognizing emotions? (iii) In light of the evidence that the right hemisphere has a unique contribution in emotional processing – summarized in the right hemisphere emotion hypothesis – does the side of the CI influence the processing of emotional cues in UCI children, or is the “physiological right lateralization” respected?

Behavioral results evidenced a higher percentage of correct responses provided by NH children in comparison to UCI children (F = 18.898, p < 0.001, partial η² = 0.270) (Table 2), but an effect of the emotional state was not seen (F = 1.890, p = 0.161, partial η² = 0.069), although for both groups the neutral cues were the most difficult to recognize. Neither the interaction between the variable group and emotional state (F = 0.032, p = 0.968, partial η² = 0.001) was observed (Figure 2).

ANOVA results showed higher LI values, indicating a higher activity in gamma band in the right in comparison to the left hemisphere, in UCI in comparison to NH children (F = 58.656, p < 0.001, partial η² = 0.535) (Figure 3), irrespectively of the emotional state (negative, neutral, and positive) (F = 1.686, p = 0.195, partial η² = 0.062). Additionally, any interaction between the variable groups and emotional state was not found (F = 1.121, p = 0.333, partial η² = 0.042).

A negative correlation was observed between LI gamma values and the percentage of correct responses (F = 11.801, p = 0.001, r = −0.420, partial η² = 0.177) (Figure 4).

Additionally, for the UCI group, any difference between the CI side and the deaf contralateral side in the gamma activity was not shown (F = 0.598, p = 0.212, partial η² = 0.032) (Figure 5A), but a higher gamma activity in the right in comparison to the left hemisphere was found (F = 54.552, p < 0.001, partial η² = 0.532) (Figure 5B).

For the UCI group, no correlation was found between the age of the UCI children at the moment of the experiment and the percentage of correct responses (F = 0.052, p = 0.821, r = 0.046, partial η² = 0.002), similarly to the NH children group (F = 1.130, p = 0.297, r = 0.197, partial η² = 0.039). Additionally, a negative correlation was shown between the age at the CI surgery and the percentage of correct response reported by UCI children (F = 7.030, p = 0.014, r = 0.468, partial η² = 0.219) (Figure 6). Finally, when calculating the mean of the correct responses for each participant, irrespective of the emotional states, despite the lack of significance (F = 3.056, p = 0.124, r = −0.551, partial η² = 0.304), a higher percentage of correct responses was highlighted, higher than 70%, only in early implanted children, that is, before 3.5 years of age (Figure 6, black dots).

According to literature, the lower percentage of correct responses provided by UCI children in comparison to NH children highlights their impairment in vocal emotion recognition skills (Agrawal et al., 2013; Wiefferink et al., 2013; Chatterjee et al., 2015; Jiam et al., 2017; Ahmed et al., 2018; Paquette et al., 2018). This would be strongly related to the preverbal and periverbal deafness acquisition. In fact, in a study employing emotional vocal stimuli in adult CI users, such performance difference was not shown (Deroche et al., 2019). Furthermore, there are evidences of different strategies implemented by CI and NH listeners for emotional stimuli recognition, more based on pitch range cues in the former and more relying on mean pitch in the latter group (Gilbers et al., 2015). In addition, such deficit in emotion recognition in UCI children in comparison to NH children appears strictly related to the matter of social interaction and social development (Jiam et al., 2017); in fact, a correlation between impairments in perception and production of voice emotion was found, like in the case of infant-directed speech, and in 5- to 13-year-old children who used CI (Nakata et al., 2012). It is interesting to note that a previous study employing vocal child-directed happy and sad speech stimuli reported higher performance in NH in comparison to CI using children; however, the percentage of recognition was higher than the one reported in the present study, probably due to the child-directed characteristic of the stimuli (Volkova et al., 2013).

Concerning the difference in gamma LI values observed in UCI in comparison to the NH group, it confirmed a difference in gamma band activity previously reported by Agrawal et al. (2013) in comparison between the same groups, therefore supporting the suitability of the study of gamma rhythms in the investigation of emotional messages conveyed by means of auditory stimuli. However, the previously mentioned study and the present study are not perfectly comparable because of the differences (i) in the sample – adults and children, respectively, – and therefore plausibly in the etiology of deafness; (ii) in the location of EEG activity acquisition, that is, Cz and multiple electrodes over the two hemispheres, respectively; and (iii) in the kind of emotional stimuli, that is, verbal stimuli pronounced with neutral, happy, and angry prosody in Agrawal and colleagues’ study, while vocal nonverbal stimuli belonging to 10 emotions grouped into three emotional states in the present study. Moreover, the higher LI values reported for UCI in comparison to NH children would imply a more sustained conscious processing of the stimuli for the NH group in comparison to the UCI group and a higher processing of the emotional face stimuli – employed for the matching of the auditory stimuli for the identification of the target emotion – by the UCI group (Balconi and Lucchiari, 2008). In fact, McGurk studies showed a higher relying of UCI children on the visual sensation than on the auditory one in case of uncertainty (Schorr et al., 2005).

The correlation between higher right lateralization, as indexed by higher LI values, and the percentage of correct responses could be explained by the evidence of higher activation and asymmetry levels in poorer performers in emotion-in-voice recognition tasks than those of more proficient ones (Kislova and Rusalova, 2009). This possibly also reflects the poorer performance in emotion recognition obtained by UCI children, as well as their higher LI values in comparison to NH children. In fact, it was shown by studies on single hemisphere damage that although the right hemisphere is responsible for low-level discrimination and recognition of affective prosody, in case of higher task demands in terms of associational-cognitive requirements, the left hemisphere is engaged (Tompkins and Flowers, 1985). Thus, UCI children would present deficits in such engaging of the left hemisphere for more complex emotional processing tasks. This could be explained by the neuroimaging evidence that indeed areas appearing to be primarily involved in emotional prosodic processing, that is, posterior temporal (parietal) brain regions (Kotz et al., 2006), are the same areas presumably more involved by the neuroplastic changes that occurred after CI surgery (Giraud et al., 2000; Kang et al., 2004) and the following hearing sensation restoration.

The negative correlation between age of implantation and percentage of correct responses in emotion recognition is in accordance with previous studies (Mancini et al., 2016). On the contrary, in the Deroche and colleagues’ study on adult CI users cited above, any effect of the age at implantation on the emotion recognition was not found, but this would be caused by the post-lingual acquisition of deafness in the majority of the sample (19 over 22 CI users) and by the type of emotions investigated, which is happy, sad, and neutral, whereas in the present study, 10 emotions were employed (Deroche et al., 2019). Furthermore, in Volkova et al.’ (2013) study, employing child-directed emotional speech, performance of the children CI users was positively associated with duration of implant use. Such evidence could be compared to present results, given the almost overlap between age at CI surgery and length of CI use in the enrolled sample. In addition, the trend that better performances were obtained by children implanted before 3.5 years old suggests the influence of a sensitive period, identified through P1 cortical auditory-evoked potential trajectory post-CI development (Sharma et al., 2002, 2005; Sharma and Dorman, 2006; Kral and Sharma, 2012; Kral et al., 2019) also on emotion recognition skills development. Such phenomenon could be explained by the better auditory–visual integration achieved by children implanted before 3.5 years of age as shown by Miller’s test of the race model inequality executed by early and late implanted children (Gilley et al., 2010). Such auditory–visual integration capability achievement is also witnessed by McGurk effect tests on CI children, showing that 38% of early implanted children – before the age of 2.5 years – but none of the late implanted children exhibited the bimodal fusion occurring in the McGurk effect, being instead biased toward the visual modality in contrast to the NH children who were biased toward the audio modality (Schorr et al., 2005). These evidences, with respect to the topic of emotion recognition skills development, are in accord to studies indicating that auditory and visual integration is necessary for the achievement of such capabilities (Campanella and Belin, 2007). In relation to this matter, there is also the evidence of a delay on facial emotion recognition in preschoolers using CI (and hearing aids) in comparison to NH mates, and interestingly, there was not any correlation between facial emotion recognition and language abilities (Wang et al., 2011). Differently, another study found a relation between better language skills and higher social competence, both in NH and CI children, although in the latter group, less adequate emotion-regulation strategies and less social competence than NH children were highlighted (Wiefferink et al., 2012). In addition, a study investigating both linguistic (recognition of monosyllabic words and of key words from sentences within background noise; repetition of non-words) and indexical (discrimination of across-gender and within-gender talkers; identification of emotional content from spoken sentences) properties in perceptual analysis of speech in CI children found an association between better performances in such feature recognition and a younger age at implantation (and use of more novel speech processor technology) (Geers et al., 2013).

Moreover, concerning the emotional communication, a suggestion of deficits also in the imitation of emotional (happy and sad) speech stimuli was found (Wang et al., 2013). Therefore, it sharply results in the vision and need of two targets of rehabilitation for children with CI that should be treated both conjointly and separately: language treatment and emotional intervention.

In light of the present results, in relation to the experimental questions previously declared, it is possible to conclude that (i) the processing of the emotional stimuli by deaf children using CI appears to be different from NH children, as suggested by the higher relative right hemisphere gamma band activity, possibly explained by the non-physiological development of the auditory system; (ii) on account of the inverse correlation between the age at the CI surgery and the percentage of correct responses, the precocity of performing the CI surgery for the attainment of best emotion recognition skills appears crucial, probably because of neuroplastic changes allowing a better processing and categorization of emotional stimuli; and (iii) the CI side does not appear to influence the processing of emotional stimuli, although interestingly the relative higher gamma band activity appears to be counterproductive in terms of emotion recognition performances; such aspect needs further investigation at the light of the possible particular implications of the right hemisphere hypothesis (Kotz et al., 2006).

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The studies involving human participants were reviewed and approved by the Bambino Gesù Pediatric Hospital Ethics Committee. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

GC conceived and conducted the study, performed the data analysis, and wrote the manuscript. AG and BI prepared the experimental protocol, conducted the study, and elaborated data. AS, SGi, AD, SGa, RG, CL, PL, and FF enrolled patients and organized experimental sessions. PM provided support for the organization and realization of the study. AS and FB edited the manuscript. FB supervised the entire experiment. All authors read and approved the final version of the article.

GC, AG, BI, and FB were employed by BrainSigns Srl. PM was employed by Cochlear Italia Srl. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.