ORIGINAL RESEARCH article
Sec. Cognitive Neuroscience
Hippocampal sclerosis affects fMR-adaptation of lyrics and melodies in songs
- 1Laboratoire de Neurosciences Fonctionnelles et Pathologies (EA 4559), Université Lille-Nord de France, Lille, France
- 2Epilepsy Unit, Hôpital de la Pitié-Salpêtrière, Paris, France
- 3Centre de NeuroImagerie de Recherche, Groupe Hospitalier Pitié-Salpêtrière, Paris, France
- 4Centre de Recherche de l’Institut du Cerveau et de la Moëlle Épinière, UPMC – UMR 7225 CNRS – UMRS 975 INSERM, Paris, France
- 5Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- 6Centre for Cognitive Neuroimaging, Department of Psychology, University of Glasgow, Glasgow, UK
- 7Laboratories for Brain, Music and Sound, Université de Montréal and McGill University, Montreal, QC, Canada
- 8Institut des Neurosciences de la Timone, UMR7289, CNRS-Université Aix Marseille, Marseille, France
Songs constitute a natural combination of lyrics and melodies, but it is unclear whether and how these two song components are integrated during the emergence of a memory trace. Network theories of memory suggest a prominent role of the hippocampus, together with unimodal sensory areas, in the build-up of conjunctive representations. The present study tested the modulatory influence of the hippocampus on neural adaptation to songs in lateral temporal areas. Patients with unilateral hippocampal sclerosis and healthy matched controls were presented with blocks of short songs in which lyrics and/or melodies were varied or repeated in a crossed factorial design. Neural adaptation effects were taken as correlates of incidental emergent memory traces. We hypothesized that hippocampal lesions, particularly in the left hemisphere, would weaken adaptation effects, especially the integration of lyrics and melodies. Results revealed that lateral temporal lobe regions showed weaker adaptation to repeated lyrics as well as a reduced interaction of the adaptation effects for lyrics and melodies in patients with left hippocampal sclerosis. This suggests a deficient build-up of a sensory memory trace for lyrics and a reduced integration of lyrics with melodies, compared to healthy controls. Patients with right hippocampal sclerosis showed a similar profile of results although the effects did not reach significance in this population. We highlight the finding that the integrated representation of lyrics and melodies typically shown in healthy participants is likely tied to the integrity of the left medial temporal lobe. This novel finding provides the first neuroimaging evidence for the role of the hippocampus during repetitive exposure to lyrics and melodies and their integration into a song.
As humans, we learn and enjoy songs from a very early age on. Over the course of our lives, we hear and remember thousands of songs and, most of the time, we learn them implicitly without much effort especially after repeated presentations (as with hit songs on the radio). Songs naturally combine music and language into a unique acoustic signal. However, it remains unclear whether memory traces of lyrics and melodies are built separately or in integration. Indeed, evidence from healthy participants and brain-damaged patients diverge on this question. On the one hand, several behavioral studies in healthy participants support the tight association of lyrics and melodies during the creation of a song memory trace as shown by cueing effects of one element on the other during song recognition (Serafine et al., 1984, 1986; Crowder et al., 1990; Baur et al., 2000; Peretz et al., 2004; Peynircioglu et al., 2008; Johnson and Halpern, 2012). On the other hand, neuropsychological studies in patients with lesions in the medial or lateral temporal lobes reveal dissociated recognition impairments for verbal and musical features of songs (Samson and Zatorre, 1991; Hébert and Peretz, 2001). These results suggest that the natural binding of lyrics and melodies into one unique song memory trace may be disrupted after brain damage. The present study seeks to find neural evidence for this hypothesis by investigating the effect of hippocampal damage on the emergence of integrated memory traces for lyrics and melodies during repeated exposure to songs.
Research over the last two decades testifies to a growing awareness that the hippocampus – beyond its classical role in explicit episodic memory (Scoville and Milner, 1957; Mishkin, 1982; Zola-Morgan and Squire, 1993) – plays a role in the implicit build-up of a memory trace (Chun and Phelps, 1999; Graham et al., 2010) and the bridging between perception and encoding (Bussey and Saksida, 2005; Baxter, 2009; Suzuki, 2009; Suzuki and Baxter, 2009; Olsen et al., 2012). According to the Emergent Memory Account (Graham et al., 2010) advancing a non-modular view of memory and perception, memory arises from a dynamic interaction between the perceptual representations distributed across the whole brain and a key role of the medial temporal lobe. More specifically, the hippocampus is thought to form conjunctive representations of inputs from unimodal and polymodal sensory cortices and to continuously return the processed information to the sensory cortex via feedback connections (McClelland et al., 1995; Eichenbaum, 2000; Turk-Browne et al., 2006; Bast, 2007), thus constantly updating the current representations with new experiences. This cortico-hippocampal loop of flowing information guarantees the encoding of events and its storage (Eichenbaum, 2000). Note that this mechanism not only implies a shared, anatomically distributed cerebral network for both memory and perception, but also puts the medial temporal lobe into a cardinal position between perceptual processes (Lee et al., 2005; Lee, 2006; Lee and Rudebeck, 2010a) and memory (long-term as well as short-term and working memory: Zarahn, 2004; Axmacher et al., 2007; Lee and Rudebeck, 2010b; Rose et al., 2012). Crucially, the hippocampus’ combined role in (i) memory formation and (ii) conjunction of sensory inputs (Sutherland and Rudy, 1989; Eichenbaum et al., 1994; Rudy and Sutherland, 1995; O’Reilly and Rudy, 2001; Winters, 2004; Cowell et al., 2006, 2010; Barense et al., 2007; Diana et al., 2007) makes it a potential key candidate for (i) the build-up of song memory traces, in which (ii) lyrics and melodies are integrated.
Although most of the studies on the hippocampus’ role in memory formation and binding come from the visual domain (Davachi, 2006; Diana et al., 2007; Shimamura, 2010), we hypothesize that similar processes also apply to the auditory domain (Overath et al., 2007, 2008; Buchsbaum and D’Esposito, 2009), especially to songs. It is reasonable to assume that memory formation for lyrics and melodies happens through a cortico-hippocampal loop, and that the natural combination of a verbal and a melodic component into a single song percept and memory trace requires binding mechanisms as described above. Tentative support for this comes from lesion studies in patients with anterior temporal lobectomy for treatment of pharmaco-resistant epilepsy (Samson and Zatorre, 1991). Using explicit recognition memory tasks after presentation of short unfamiliar songs, these experiments revealed a clear deficit in recognition of sung and spoken lyrics after left temporal lobe resection, and impaired recognition of melodies (without text) after right temporal lobe resection. On top of that, the data suggest a lack of integration of lyrics and melodies in patients with unilateral left (but not those with right) temporal lobe lesions. While patients with right temporal lobe resections had deficits in melody recognition when the tune was sung with new words, i.e., showing that they had bound the melody to the original lyrics, no such conjunction was observed in left-hemisphere damaged patients. In fact, their recognition of lyrics was impaired irrespective of whether these were presented with (or without) old or new melodies, suggesting an independent processing of the two song components and an isolated deficit for lyrics.
While these results lend initial support for our hypothesis of hippocampal involvement in song memory formation, they leave two important questions open: first, in how far can these deficit patterns be attributed to hippocampal dysfunctions, and second, in how far may these results depend on the use of a recognition memory task? First, the resection always included anterior temporal lobe structures beyond the hippocampus, making it difficult to pinpoint a specific hippocampal role. Furthermore, although the lesion description was based upon the surgeon’s meticulous drawings, a precise assessment of how far the resection extended into the hippocampus was not possible at that time. Moreover, although recognition tasks certainly depend on successful encoding, they also involve aspects of memory retrieval making it difficult to disentangle these effects with behavioral data. The present study seeks to address the points by first, testing patients with circumscribed unilateral hippocampal sclerosis (i.e., prior to surgery without further macroscopic lesions) and precisely describing the extent of hippocampal damage by means of volumetric analyses. Second, the incidental build-up of a song memory trace was assessed unbeknownst to the participants by examining the dynamics of neural adaptation during natural passive listening as described below.
Numerous studies have investigated the neural correlates of song processing (Samson and Zatorre, 1991; Brown et al., 2004a,b; Schön et al., 2005; Callan et al., 2006; Suarez et al., 2010; Merrill et al., 2012; Saito et al., 2012; Tierney et al., 2012), however, rarely has any study touched upon the implicit emergence of song memory. Indirect evidence can be drawn from studies using the successive presentation of changed and unchanged song stimuli (Same vs. Different) (Schön et al., 2010) and neural adaptation paradigms (Sammler et al., 2010). Adaptation is “a reduction of neural activity following prolonged or repetitive exposure to identical or at least similar stimuli” (Dobbins et al., 2004; Ganel et al., 2006; Grill-Spector et al., 2006), similar to repetition priming (Old vs. New stimuli) (Krekelberg et al., 2006). Although typically described in studies on perception, it appears that neural adaptation may also be indicative of memory trace formation. In line with the Emergent Memory Account (Graham et al., 2010), neural adaptation may reflect the emergence of a memory trace within cortical areas of perceptual representation through implicit learning during repeated exposure. Given the role of the hippocampus in memory formation (Turk-Browne et al., 2006) and according to connectionist models of memory (Damasio, 1989; McClelland et al., 1995; Rolls, 1996; Fuster, 1997), it is reasonable to suggest that cortical adaptation effects are subject to top-down modulations driven by the hippocampus (Blondin and Lepage, 2005; Goh et al., 2007), including integration of lyrics and melodies through binding (for a review on binding, see Opitz, 2010).
Of particular relevance for our research question of how lyrics and melody are bound in a conjunctive song memory trace are those studies describing the cerebral substrates underlying the integration of verbal and melodic components of songs (Sammler et al., 2010; Schön et al., 2010). These studies, which consider songs to be more than the sum of lyrics and melodies, examined modulations of brain activity to investigate how the two components interact, and how their processing is lateralized. For instance, Schön et al. (2010, Exp. 2) presented pairs of sung words that could vary or repeat in terms of the verbal and/or the melody component in a same-different task. Their results showed interactive processing in the left and the right superior temporal gyrus (STG), suggesting an integrated processing of the two components in these areas. Sammler et al. (2010) adopted a similar approach, taking advantage of neural adaptation effects. In this study, healthy participants were presented with blocks of short songs in which repetition of lyrics and/or melodies was varied in a factorial design to induce selective adaptation to lyrics, melodies, or unified songs. Consistent with Schön et al. (2010), repeated lyrics or repeated tunes evoked adaptation effects in bilateral STG. Core areas of integration were found in the left middle superior temporal sulcus (STS) and the left premotor cortex (PMC). Based on the previously reported literature, we hypothesize that these adaptation effects and the integration of lyrics and melodies are likely mediated by the hippocampus through feedback connections to STG/STS and binding of verbal and melodic information.
To investigate the modulatory effect of the hippocampus on (i) the incidental emergence of a song memory trace and (ii) the integration of the verbal and melodic components of songs, we adopted the paradigm by Sammler et al. (2010) to test patients with unilateral left or right hippocampal sclerosis and healthy controls. We compared the patterns of adaptation produced by songs in which either the lyrics, or the melodies, or both were repeated. As demonstrated by diffusion-weighted imaging studies, patients with hippocampal sclerosis present disconnections between medial and lateral temporal lobe regions (Focke et al., 2008; Bettus et al., 2009; Diehl et al., 2010; Riley et al., 2010; Liao et al., 2011). Such lesions have the capacity to prevent the hippocampus from sending feedback predictions and from updating the sensory memory trace (as expected by default after repetitions) and thus weaken adaptation effects in general and integration of lyrics and melodies in particular. More precisely, following Samson and Zatorre (1991), we hypothesized reduced adaptation for lyrics after left and for melodies after right hippocampal sclerosis. Moreover, following previous studies showing binding deficits in patients with left anterior temporal lobe resections (Samson and Zatorre, 1991) and correlates of lyrics–melody integration mainly in the left hemisphere (Sammler et al., 2010), we hypothesized that left hippocampal lesions, in particular, would have a negative impact on integration of lyrics and melodies in songs.
Materials and Methods
Twenty-four temporal lobe epilepsy patients with left (n = 12; LTLE) or right (n = 12; RTLE) hippocampal sclerosis participated in this study. They all presented with medically intractable epilepsy and were seen during pre-surgical evaluation at Pitié-Salpêtrière Hospital (Paris, France). All patients were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971), except for one LTLE (−83.33) and one RTLE patient (−75). All patients had language lateralization to the left hemisphere except for the left-handed RTLE patient with bilateral language representation. Language lateralization was assessed by means of a verbal fluency test that is part of the standard functional magnetic resonance imaging (fMRI) assessment prior to epilepsy surgery at the Pitié-Salpêtrière Hospital. In the scanner, patients are required to think as many words of a semantic category (e.g., tools) as possible. The number of activated left and right fronto-temporo-parietal voxels against baseline was used to calculate a standard language lateralization score (Lehéricy et al., 2000; Thivard et al., 2005). The control group consisted of 19 right-handed healthy participants including 12 subjects, who had already participated in a previous study (Sammler et al., 2010), and 7 new volunteers. All participants were French native speakers and reported to have normal hearing. Controls were carefully selected to match the patient groups in terms of age, mean years of education, and musical expertise (Ehrlé musical expertise questionnaire, unpublished). A verbal memory deficit was present in the LTLE as opposed to the RTLE patients, as assessed with the Rey Auditory Verbal Learning Test (RAVLT) (Rey, 1964; Sziklas and Jones-Gotman, 2008) in accordance with the usual neuropsychological profile of these patients. Demographic characteristics of the participants are summarized in Table 1. The sclerosis in either left or right hippocampus in the two patient groups was corroborated by a volumetric analysis using Freesurfer software (Fischl, 2012; Reuter et al., 2012) that attested an ipsilateral hippocampal volume reduction of an average of 24.51% in the LTLE and 29.71% in the RTLE group compared to healthy controls. Between-group comparisons confirmed the significance of these volume reductions in the atrophic hippocampus (p < 0.05). Volumes and percentage of reduction are summarized in Table 2 (for details on the volumetric analysis, see Data Analysis). The local ethics committee approved this study and informed consent was obtained from each participant.
The material and the scanning protocol used here were previously published by Sammler et al. (2010). The stimulus set consisted of 48 blocks of 6 unfamiliar songs based on a collection of nineteenth century French folk songs (Robine, 1994). Each song within a block was sung by a different singer to avoid adaptation to the singer’s voice (Belin and Zatorre, 2003), had a duration of 2.5 s and was followed by a 0.2 s pause. Repetition of lyrics and/or melodies within blocks was crossed in a 2 × 2 factorial design, forming four conditions. Songs within a block either had the same melodies and same lyrics (SMSL), the same melodies but different lyrics (SMDL), different melodies with same lyrics(DMSL), or different melodies and different lyrics (DMDL). Mode and tempo were balanced across the stimulus set, and each song had an average of 7.65 notes and 5.61 words. Songs in the four conditions did not differ with respect to length and number of word/note, word frequency, interval size, and number of contour reversals. In blocks where lyrics were varied, they did not rhyme, were semantically distant, and differed with respect to syntactic structure avoiding potential adaptation to phonology, semantic content, or syntactic structure (Noppeney and Price, 2004).
Participants were instructed to listen attentively with closed eyes while avoiding moving, humming, or singing along. No behavioral data were collected. Stimuli were presented using E-Prime 1.1 (Psychology Software Tools) and delivered binaurally through air pressure headphones (MR confon). Additionally, participants used earplugs to minimize noise interference. All blocks were presented in one of four pseudorandom orders, with a silent gap between blocks of 10 s (±0.5 s) allowing the hemodynamic response to return to baseline (Belin and Zatorre, 2003). This resulted in a total duration of the experiment of around 30 min. Blocks of the same condition were not presented more than twice in a row. At the end of the experiment, all participants filled in a debriefing questionnaire with several nine-point scales (1 = not at all, 9 = always) in which they rated their attention during listening at 7.63 (Controls), 7.00 (LTLE), 7.57 (RTLE), and the amount of overt and/or covert singing during scanning at 0.00 and 2.89 (Controls), 0.47 and 2.71 (LTLE), and 0.21 and 2.14 (RTLE), showing that they had followed the instructions.
Functional magnetic resonance imaging was performed using a 3-T Siemens TRIO scanner (Siemens, Erlangen, Germany) at the Centre de Neuroimagerie de Recherche at the Institut du Cerveau et de la Moëlle Épinière – ICM (Groupe Hospitalier Pitié-Salpêtrière, Paris, France). Radiofrequency transmission was performed with a body coil and the signal was received with a 12-channel head coil. Before the functional scans, high-resolution T1-weighted images (1 × 1 × 1 mm3 voxel size) were collected for anatomical coregistration using a magnetization-prepared rapid acquisition gradient-echo (MPRAGE) sequence (TR = 2300 ms, TE = 4.18 ms). Subsequently, one series of 595 blood oxygenation level-dependent (BOLD) images was obtained using a single-shot echo-planar gradient-echo (EPI) pulse sequence (TR = 2120 ms, TE = 25 ms, the first six volumes were later discarded to allow for T1 saturation). Forty-four interleaved slices (3 mm × 3 mm × 3 mm voxel size, 10% interslice gap) perpendicular with respect to the hippocampal plane were collected. The field of view was 192 × 192 mm2 with an in-plane resolution of 64 × 64 pixels and a flip angle of 90°. Scanner noise was continuous during the experiment representing a constant auditory background.
The fMRI data were analyzed using SPM8 (Wellcome Trust Centre for Neuroimaging). Preprocessing included spatial realignment and reslicing and coregistration of the anatomical T1 to the mean functional data. The first level analysis was carried out in the native space. Four regressors were built for each experimental condition based on the general linear model (different melodies and different lyrics (DMDL); same melodies and different lyrics (SMDL); different melodies and same lyrics (DMSL) and same melodies and same lyrics (SMSL), and convolved with a hemodynamic response function (HRF). Movement parameters were included as regressors of no interest and serial correlations were modeled with an AR (1) process. A temporal high-pass filter with a cut-off of 200 s was used to eliminate low-frequency drifts. Six one-sample t-tests were computed for each participant: all conditions against silence to establish a “song-sensitive” mask, the main effects of adaptation to lyrics [(DMDL + SMDL) – (DMSL + SMSL)] and to melodies [(DMDL + DMSL) – (SMDL + SMSL)] to identify areas of general adaptation to the repetition of song components, as well as the interaction [(DMSL + SMDL) – (DMDL + SMSL)] to isolate areas of lyrics–melody integration. For the sake of completeness and consistency with the analysis of Sammler et al. (2010), we additionally compared both main effects to identify brain regions that showed an independent processing of either lyrics or melodies (i.e., stronger adaptation for lyrics than for melodies [2 × (SMDL)] and vice versa [2 × (DMSL)]).
Segmentation of the anatomical files was performed with the VBM8 toolbox (Ashburner and Friston, 2005) to form a normalized anatomical image and the DARTEL exported tissue types. A template with eight iterations was created in DARTEL (Ashburner, 2007) including all 43 subjects to improve anatomical accuracy in the normalization of the functional contrast images obtained in the first level. Contrast images were spatially smoothed using a three-dimensional Gaussian kernel with 8 mm full width at half maximum. For the second level, the DARTEL normalized contrast images were normalized to the Montreal Neurological Institute (MNI) space. The automatically generated mask from the first level analysis of each subject was also normalized with this procedure but without smoothing. Statistical analysis was confined to a song-sensitive mask in gray matter to increase signal detection (Friston et al., 1994). To create this mask, a binary mask from the last iteration of the DARTEL template thresholded at 0.3 was overlaid with active voxels in the “all conditions against silence” contrast at p < 0.05 (FWE correction for multiple comparisons), k > 5 for all 43 participants. All voxels that were involved in both were included into the explicit song-sensitive mask for statistics. This mask covered an auditory-motor network, including the temporal gyrus, the PMC, and the cerebellum. For random effects group analyses, the individual contrast images were submitted to one-sample t-tests, separately for healthy controls, LTLE and RTLE patients. Furthermore, two-sample t-tests were computed for all contrasts, comparing each patient group against controls. All SPMs were threshold at p < 0.001 (uncorrected) with a minimum cluster extent of k ≥ 5 voxels. Results will report the peak voxel p value and the number of voxels (k).
To assess the size of the hippocampal sclerosis and surrounding cortex, volumetric measures of hippocampal, entorhinal, and parahippocampal gyrus were obtained for all participants with the Freesurfer image analysis suite (Fischl, 2012; Reuter et al., 2012), which is documented and freely available for downloading online (http://surfer.nmr.mgh.harvard.edu/). Non-parametric tests (Kruskal–Wallis, SPSS 18.0) were used to compare these measures between the patient and controls groups. To control global differences, intracranial volume was included in the analysis as a covariate, which was not found to be significant. The percentage of reduction of each structure was calculated for each patient group in comparison to the control group and is reported in Table 2.
A complete report of the results at threshold p < 0.001 (uncorrected) with a minimum cluster extent of k ≥ 5 voxels can be seen in Table 3. All three groups of participants showed adaptation to lyrics in the left and right STG and STS that was however considerably more extended in Controls (2474 and 2423 voxels) than in LTLE (541 and 388 voxels) and RTLE patients (201 and 165 voxels). Between-group comparisons revealed significantly weaker adaptation effects in the LTLE but not in the RTLE as compared to Controls in the left STS (Figure 1A).
Table 3. Main effects of lyrics and melodies repetition for each group and comparison between Controls and LTLE.
Figure 1. Main effects of Adaptation to Lyrics (A) and Melody (B), and the Interaction (integration contrast) (C). Threshold p < 0.001 k ≥ 5 uncorrected. Results for Control group (red), LTLE (blue), RTLE (green), and Controls vs. LTLE (yellow).
In all three groups, adaptation to melody was found in the left and right STG and STS, again more extended in Controls (2380 and 1830 voxels) than in LTLE (245 and 295 voxels) and RTLE patients (106 and 111 voxels), as well as in the cerebellum. The Control group showed, in addition, adaptation in the left PMC (52 voxels) that was not observed in patients (Figure 1B). However, between-group differences failed to reach significance.
Interaction effects were calculated with the contrast [(DMSL + SMDL) – (DMDL + SMSL)] and were taken to represent an integrated processing of lyrics and melodies in songs. Only the control group showed interaction effects at p < 0.001 k ≥ 5, which were located in the bilateral posterior STG/STS (left: 169 voxels and right: 323 voxels). No such effect was observed in LTLE and RTLE patients. To visualize areas that simply may not have passed our statistical criterion, we inspected the data at a very lenient level of p < 0.05 uncorrected (k > 5). Controls showed an extended region within the left (1936 voxels) and right (2176 voxels) STG/STS (Figure 2A). At this threshold, RTLE patients showed a pattern that was similar to Controls, but considerably less extended (554 and 1501 voxels) (Figure 2B). Interestingly, LTLE patients showed nearly no interaction in the temporal lobe at this very lenient threshold (238 and 35 voxels) (Figure 2C). Indeed, between-group comparisons revealed a significantly weaker interaction effect in the LTLE than the Control group in the right STG (Figure 1C) whereas the difference between the RTLE patients and Controls did not reach significance. Details on interaction effects are shown in Table 4.
Figure 2. “Gradient of integration” for (A) the Control group (B) RTLE and (C) LTLE patients. Specificity for lyrics is shown in red (p < 0.001 k > 5 uncorr.), interaction in dark blue (p < 0.001 k > 5 uncorr.) and weaker interaction in cyan (Interaction at p < 0.05 k > 5 uncorr.).
Greater adaptation to lyrics as compared to melody was found bilaterally in the anterior region of the STG (23 and 196 voxels) in the control group, suggesting an independent processing of lyrics in this region. Greater adaptation to melody as compared to lyrics was obtained bilaterally in the cerebellum in RTLE patients. However, between-group differences failed to reach significance (Figure 2A). Details on independence effects are shown in Table 4.
The aim of the current study was to assess the modulatory effects of a unilateral hippocampal lesion on the incidental emergence of a song memory trace and the integration of lyrics and melodies into a conjunctive representation. To this end, neural adaptation to song repetition – as a proxy for song memory formation – was examined in patients with left or right hippocampal sclerosis in comparison to healthy controls using an fMR-adaptation paradigm. It was hypothesized that damage to the hippocampus may disrupt feedback connections to the lateral temporal lobe and thus preclude the establishment and update of a sensory memory trace. As a consequence, damage to the hippocampus may result in weaker neural adaptation in the STG. In particular, hippocampal lesions could hinder the integration of lyrics and melodies into a unified memory trace (Diana et al., 2007; Staresina and Davachi, 2009; Graham et al., 2010; Shimamura, 2010).
The main findings of this study were indeed that the neural adaptation to lyrics repetition as well as the integration of lyrics and melodies in songs (as reflected by the statistical interaction between adaptation effects for lyrics and melodies) was reduced in patients with left hippocampal sclerosis. More specifically, the direct comparison of these patients with healthy control participants revealed a weaker adaptation to lyrics in the left STS and a weaker integration of lyrics and melodies in the right STG. If one accepts the notion that neural adaptation reflects the emergence of a memory trace (see Introduction), these results are in line with our hypotheses and previous work showing that left hippocampal damage may lead to weaker memory for lyrics (Samson and Zatorre, 1991) and may hinder the integration of lyrics and melodies into a unified memory representation (Samson and Zatorre, 1991; Sammler et al., 2010).
All three groups of participants showed adaptation to the repetition of lyrics or melodies in the bilateral STG and STS, but in both patient groups, these effects were markedly smaller in spatial extent when compared to healthy controls. Notably, patients with left (but not right) hippocampal sclerosis exhibited significantly decreased adaptation to lyrics in the left STS, which is known to play a role in phonemic processing and also known to be crucial for the perception of a sound as speech (Dehaene-Lambertz et al., 2005; Liebenthal, 2005; Möttönen et al., 2006; for a review on STS, see Hein and Knight, 2008). This finding is most likely tied to the role of the left medial temporal lobe in verbal processing (Meyer et al., 2005; Wagner et al., 2008; Greve et al., 2011) and may reflect the perturbed build-up of memory traces for lyrics (and verbal material in general) due to disrupted feedback connections between medial and lateral structures of the left temporal lobe (Eichenbaum, 2000). Such an interpretation could be supported by the verbal memory deficit documented in the LTLE patients of the present study (assessed with the RAVLT) and, although we did not collect behavioral data for this experiment, these results are also in agreement with the behavioral results of Samson and Zatorre (1991). That study showed that the recognition of sung lyrics after listening to unfamiliar songs was impaired in patients with left (but not right) medial temporal lobe lesions.
Although patients with right hippocampal sclerosis showed nominally reduced adaptation and integration effects, these did not significantly differ from those in healthy controls, suggesting rather normal song processing and lyrics–melody integration in these patients. While the latter is in line with previous behavioral data showing spared integration of lyrics and tunes after right anterior temporal lobe resection (Samson and Zatorre, 1991), our hypothesis on reduced adaptation to melodies was not confirmed. This may partly be due to the stimulus material used: even if melodies were repeated to induce adaptation, they differed in octave sung by sopranos, tenors, altos, and bass. Most likely, adaptation effects are not fully robust to transposition of melodies. Furthermore, adaptation to melodies was generally weaker than adaptation to lyrics, as attested by the results in healthy participants, possibly resulting in a floor effect. Our participants may have paid less attention to melodies than to lyrics (as the latter convey the message) leading to weak adaptation, given that a lack of attention reduces adaptation effects (Chee and Tan, 2007). Alternatively, several lines of evidence suggest that melodies may be processed more bilaterally than lyrics (Samson and Zatorre, 1992; Binder et al., 2000; Besson and Schön, 2003; Peretz and Coltheart, 2003; Schön et al., 2005; Patel, 2008; Koelsch, 2012), leading to less severe deficits in processing melodies than in verbal processing after unilateral temporal lobe damage. Further studies will be necessary to clarify this issue.
One novel finding is the main effect of melodies in the cerebellum in all groups (without group differences). Since activity in the cerebellum has been frequently reported in other studies using sung material (Parsons, 2001; Callan et al., 2007; Lebrun-Guillaud et al., 2008; Tillmann et al., 2008; Merrill et al., 2012), these effects may be linked to optimization of the fine sensory acquisition and internalization of input–output characteristics of stimuli, a process related to the creation of internal models of vocal articulation (Parsons, 2001; Callan et al., 2007; Stoodley and Schmahmann, 2009), that may function independently from the hippocampus.
As previously reported (Sammler et al., 2010), healthy participants presented maximum integration of lyrics and melodies in the posterior STS with a continuous decay of the lyrics–melodies integration along the posterior–anterior axis, toward regions of independent processing of lyrics in the anterior STG. These effects were shown bilaterally in the present experiment, expanding the previously reported effect, which was restricted to the left hemisphere. This analysis illustrates a “gradient of integration” from more to less integrated processing. In line with the literature on music and language (Scott et al., 2000; Davis and Johnsrude, 2003; Scott and Johnsrude, 2003; Friederici, 2011; Gow, 2012), this gradient poses an integrative processing of songs at the prelexical and phonemic level in the mid-STS. Consequently, information can be transmitted both along an anterior pathway to the temporal pole for an independent analysis of the linguistic content, and along a posterior pathway to the left PMC for the integrated sensori-motor conversion of the stimuli. In other words, lyrics and melodies might split up in the ventral pathway for semantics and comprehension (Griffiths, 2001; Patterson et al., 2002; Hickok and Poeppel, 2007; Saur et al., 2008; Friederici, 2009, 2011; Hickok et al., 2011) but stay integrated in sensori-motor dorsal pathways (Kiebel et al., 2008; Loui et al., 2009).
Contrary to healthy participants, both patient groups showed very weak levels of lyrics–melody integration in the bilateral mid-STG/STS, and only after lowering the statistical threshold to p < 0.05 (uncorrected). This effect may reside on generally weaker adaptation effects in both patient groups. The spatial extent of this weak lyrics–melody interaction was particularly small in patients with left hippocampal sclerosis who also showed a significantly reduced interaction effect in the right STG as compared to controls. These tendencies suggest a partial (although not complete) disruption of integrated processing in clinical populations and indicate that the conjunctive representation of lyrics and melodies depends on intact medial temporal lobe structures, particularly in the left hemisphere. Overall, this finding is in line with previous studies in patients with anterior temporal lobe resection including parts of the hippocampus (Samson and Zatorre, 1991). These experiments showed a perturbed integration of verbal and melodic song components in patients with left (but not right) temporal lobe resections, i.e., a selective deficit in recognizing lyrics that was independent from recognition memory for melodies. It is worth to mention that in both the present and previous studies, the integration deficit may reside on a more general deficit to process lyrics, as supported by the weaker adaptation for lyrics and reduced performance in neuropsychological tests on verbal memory in our patients with left hippocampal sclerosis.
Taken together, adaptation to lyrics and integration of lyrics and melodies within songs appear to be less efficient in patients with left hippocampal damage as compared to healthy controls. We propose that these lesions may hinder the build-up of a sensory memory trace for lyrics (with rather preserved mechanisms for melodies), which in turn might be at the origin of the reduced integration of lyrics and melody. These combined effects could be attributed to hippocampal malfunction per se or to a more global disconnection of lateral temporal neocortical structures caused by repetitive seizures or epilepsy history (Yasuda et al., 2010; Besson et al., 2012), both of which can disrupt the hippocampal top-down modulatory influence on STG/STS. If this is the case, it is possible that adaptation could also be reduced for stimuli other than lyrics, melodies, or songs, demonstrating a more general adaptation and putative encoding deficit following disruption of cortico-hippocampal processing loops.
Interestingly, an independent analysis of the connectivity profiles in our patients showed asymmetries between the left and right hemispheric lesion groups: LTLE patients exhibited more extended and more strongly left-lateralized disconnections, as opposed to more discrete and bilateral connectivity deficits in RTLE (Besson et al., 2012). Such differences in connectivity profiles provide an additional explanation for the nominally stronger impairments in patients with left hippocampal sclerosis as compared to patients with right hippocampal sclerosis. In sum, the present data indicate that an imbalance in the left hippocampo-cortical system, due to hippocampal sclerosis and/or disrupted connectivity with STG/STS, affects the incidental emergence of a memory trace of verbal song components and precludes the build-up of a conjunctive representation that integrates lyrics and melodies.
To the best of our knowledge, this is the first study to investigate the processing of songs using fMRI in patients with unilateral hippocampal sclerosis. We showed that the adaptation to lyrics and the integration of lyrics and melodies was diminished in lateral temporal lobe regions in patients with left hippocampal sclerosis while a similar but non-significant result pattern was found in patients with right hippocampal sclerosis. These findings suggest the importance of hippocampal top-down modulations on the STG/STS during repetitive exposure to songs. We interpret the observed adaptation patterns to be a result of a disturbed connectivity in a hippocampal–cortical network, weakening the emergence of a memory trace for lyrics and the integrated processing of songs as a unified percept. Overall, these data provide a novel contribution by suggesting that the integration shown in healthy participants is tied to the integrity of the medial temporal lobe and its connections with the lateral temporal cortex.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The authors are grateful to the CENIR team and Diana Omigie for their helpful assistance. Funding: the research leading to these results has received funding from an Early Stage Researcher fellowship to Irene Alonso by the European Community’s Seventh Framework Programme under the Europe, Brain and Music (EBRAMUS) project – grant agreement n°238157 and by a grant from “Agence Nationale pour la Recherche” of the French Ministry of research (project n° ANR-09-BLAN-0310-02) and a grant from the “Institut Universitaire de France” to Séverine Samson.
Axmacher, N., Mormann, F., Fernandez, G., Cohen, M. X., Elger, C. E., and Fell, J. (2007). Sustained neural activity patterns during working memory in the human medial temporal lobe. J. Neurosci. 27, 7807–7816. doi:10.1523/JNEUROSCI.0962-07.2007
Barense, M. D., Gaffan, D., and Graham, K. S. (2007). The human medial temporal lobe processes online representations of complex objects. Neuropsychologia 45, 2963–2974. doi:10.1016/j.neuropsychologia.2007.05.023
Bast, T. (2007). Toward an integrative perspective on hippocampal function: from the rapid encoding of experience to adaptive behavior. Rev. Neurosci. 18, 253–282. doi:10.1515/REVNEURO.2007.18.3-4.253
Baur, B., Uttner, I., Ilmberger, J., Fesl, G., and Mai, N. (2000). Music memory provides access to verbal knowledge in a patient with global amnesia. Neurocase 6, 415–421. doi:10.1080/13554790008402712
Besson, P., Dinkelacker, V., Valabregue, R., Samson, S., Thivard, L., Baulac, M., et al. (2012). “Decreased connectivity of major cortical nodes is more pronounced and lateralized in left versus right mesial temporal sclerosis,” in Poster Presented at 66th American Epilepsy Society Annual Meeting, San Diego.
Bettus, G., Guedj, E., Joyeux, F., Confort-Gouny, S., Soulier, E., Laguitton, V., et al. (2009). Decreased basal fMRI functional connectivity in epileptogenic networks and contralateral compensatory mechanisms. Hum. Brain Mapp. 30, 1580–1591. doi:10.1002/hbm.20625
Binder, J. R., Frost, J. A., Hammeke, T., Bellgowan, P. S., Springer, J., Kaufman, J., et al. (2000). Human temporal lobe activation by speech and nonspeech sounds. Cereb. Cortex 10, 512–528. doi:10.1093/cercor/10.5.512
Blondin, F., and Lepage, M. (2005). Decrease and increase in brain activity during visual perceptual priming: an fMRI study on similar but perceptually different complex visual scenes. Neuropsychologia 43, 1887–1900. doi:10.1016/j.neuropsychologia.2005.03.021
Callan, D. E., Tsytsarev, V., Hanakawa, T., Callan, A. M., Katsuhara, M., Fukuyama, H., et al. (2006). Song and speech: brain regions involved with perception and covert production. Neuroimage 31, 1327–1342. doi:10.1016/j.neuroimage.2006.01.036
Cowell, R. A., Bussey, T. J., and Saksida, L. M. (2006). Why does brain damage impair memory? A connectionist model of object recognition memory in perirhinal cortex. J. Neurosci. 26, 12186–12197. doi:10.1523/JNEUROSCI.2818-06.2006
Cowell, R. A., Bussey, T. J., and Saksida, L. M. (2010). Components of recognition memory: dissociable cognitive processes or just differences in representational complexity? Hippocampus 20, 1245–1262. doi:10.1002/hipo.20865
Crowder, R. G., Serafine, M. L., and Repp, B. (1990). Physical interaction and association by contiguity in memory for the words and melodies of songs. Mem. Cognit. 18, 469–476. doi:10.3758/BF03198480
Dehaene-Lambertz, G., Pallier, C., Serniclaes, W., Sprenger-Charolles, L., Jobert, A., and Dehaene, S. (2005). Neural correlates of switching from auditory to speech perception. Neuroimage 24, 21–33. doi:10.1016/j.neuroimage.2004.09.039
Diana, R. A., Yonelinas, A. P., and Ranganath, C. (2007). Imaging recollection and familiarity in the medial temporal lobe: a three-component model. Trends Cogn. Sci. (Regul. Ed.) 11, 379–386. doi:10.1016/j.tics.2007.08.001
Diehl, B., Tkach, J., Piao, Z., Ruggieri, P., LaPresto, E., Liu, P., et al. (2010). Diffusion tensor imaging in patients with focal epilepsy due to cortical dysplasia in the temporo-occipital region: electro-clinico-pathological correlations. Epilepsy Res. 90, 178–187. doi:10.1016/j.eplepsyres.2010.03.006
Dobbins, I. G., Schnyer, D. M., Verfaellie, M., and Schacter, D. L. (2004). Cortical activity reductions during repetition priming can result from rapid response learning. Nature 428, 316–319. doi:10.1038/nature02400
Focke, N. K., Yogarajah, M., Bonelli, S. B., Bartlett, P. A., Symms, M. R., and Duncan, J. S. (2008). Voxel-based diffusion tensor imaging in patients with mesial temporal lobe epilepsy and hippocampal sclerosis. Neuroimage 40, 728–737. doi:10.1016/j.neuroimage.2007.12.031
Ganel, T., Gonzalez, C. L., Valyear, K. F., Culham, J. C., Goodale, M. A., and Köhler, S. (2006). The relationship between fMRI adaptation and repetition priming. Neuroimage 32, 1432–1440. doi:10.1016/j.neuroimage.2006.05.039
Goh, J. O. S., Chee, M. W. L., Tan, J., Venkatraman, V., Hebrank, A., Leshikar, E., et al. (2007). Age and culture modulate object processing and object-scene binding in the ventral visual areas. Cogn. Affect. Behav. Neurosci. 7, 44–52. doi:10.3758/CABN.7.1.44
Graham, K. S., Barense, M. D., and Lee, A. C. (2010). Going beyond LTM in the MTL: a synthesis of neuropsychological and neuroimaging findings on the role of the medial temporal lobe in memory and perception. Neuropsychologia 48, 831–853. doi:10.1016/j.neuropsychologia.2010.01.001
Greve, A., Evans, C. J., Graham, K. S., and Wilding, E. L. (2011). Functional specialisation in the hippocampus and perirhinal cortex during the encoding of verbal associations. Neuropsychologia 49, 2746–2754. doi:10.1016/j.neuropsychologia.2011.06.002
Lebrun-Guillaud, G., Tillmann, B., and Justus, T. (2008). Perception of tonal and temporal structures in chord sequences by patients with cerebellar damage. Music Percept. 25, 271–283. doi:10.1525/mp.2008.25.4.271
Lee, A. C. (2006). Differentiating the roles of the hippocampus and perirhinal cortex in processes beyond long-term declarative memory: a double dissociation in dementia. J. Neurosci. 26, 5198–5203. doi:10.1523/JNEUROSCI.3157-05.2006
Lee, A. C., Bussey, T. J., Murray, E. A., Saksida, L. M., Epstein, R. A., Kapur, N., et al. (2005). Perceptual deficits in amnesia: challenging the medial temporal lobe ‘mnemonic’ view. Neuropsychologia 43, 1–11. doi:10.1016/j.neuropsychologia.2004.07.017
Lee, A. C., and Rudebeck, S. R. (2010b). Investigating the interaction between spatial perception and working memory in the human medial temporal lobe. J. Cogn. Neurosci. 22, 2823–2835. doi:10.1162/jocn.2009.21396
Lehéricy, S., Cohen, L., Bazin, B., Samson, S., Giacomini, E., Rougetet, R., et al. (2000). Functional MR evaluation of temporal and frontal language dominance compared to the Wada test. Neurology 54, 1625–1633. doi:10.1212/WNL.54.8.1625
Liao, W., Zhang, Z., Pan, Z., Mantini, D., Ding, J., Duan, X., et al. (2011). Default mode network abnormalities in mesial temporal lobe epilepsy: a study combining fMRI and DTI. Hum. Brain Mapp. 32, 883–895. doi:10.1002/hbm.21076
McClelland, J. L., McNaughton, B. L., and O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419.
Merrill, J., Sammler, D., Bangert, M., Goldhahn, D., Lohmann, G., Turner, R., et al. (2012). Perception of words and pitch patterns in song and speech. Front. Psychol. 3:76. doi:10.3389/fpsyg.2012.00076
Meyer, P., Mecklinger, A., Grunwald, T., Fell, J., Elger, C. E., and Friederici, A. D. (2005). Language processing within the human medial temporal lobe. Hippocampus 15, 451–459. doi:10.1002/hipo.20070
Möttönen, R., Calvert, G. A., Jääskeläinen, I. P., Matthews, P. M., Thesen, T., Tuomainen, J., et al. (2006). Perceiving identical sounds as speech or non-speech modulates activity in the left posterior superior temporal sulcus. Neuroimage 30, 563–569. doi:10.1016/j.neuroimage.2005.10.002
Olsen, R. K., Moses, S. N., Riggs, L., and Ryan, J. D. (2012). The hippocampus supports multiple cognitive processes through relational binding and comparison. Front. Hum. Neurosci. 6:146. doi:10.3389/fnhum.2012.00146
O’Reilly, R. C., and Rudy, J. W. (2001). Conjunctive representations in learning and memory: principles of cortical and hippocampal function. Psychol. Rev. 108, 311–345. doi:10.1037/0033-295X.108.2.311
Overath, T., Cusack, R., Kumar, S., Kriegstein, K., von Warren, J. D., Grube, M., et al. (2007). An information theoretic characterisation of auditory encoding. PLoS Biol. 5:e288. doi:10.1371/journal.pbio.0050288
Overath, T., Kumar, S., Kriegstein, K., and von Griffiths, T. D. (2008). Encoding of spectral correlation over time in auditory cortex. J. Neurosci. 28, 13268–13273. doi:10.1523/JNEUROSCI.4596-08.2008
Patterson, R. D., Uppenkamp, S., Johnsrude, I. S., and Griffiths, T. D. (2002). The processing of temporal pitch and melody information in auditory cortex. Neuron 36, 767–776. doi:10.1016/S0896-6273(02)01060-7
Peynircioglu, Z. F., Rabinovitz, B. E., and Thompson, J. L. (2008). Memory and metamemory for songs: the relative effectiveness of titles, lyrics, and melodies as Cues for each other. Psychol. Music 36, 47–61. doi:10.1177/0305735607079722
Reuter, M., Schmansky, N. J., Rosas, H. D., and Fischl, B. (2012). Within-subject template estimation for unbiased longitudinal image analysis. Neuroimage 61, 1402–1418. doi:10.1016/j.neuroimage.2012.02.084
Riley, J. D., Franklin, D. L., Choi, V., Kim, R. C., Binder, D. K., Cramer, S. C., et al. (2010). Altered white matter integrity in temporal lobe epilepsy: association with cognitive and clinical profiles. Epilepsia 51, 536–545. doi:10.1111/j.1528-1167.2009.02508.x
Saito, Y., Ishii, K., Sakuma, N., Kawasaki, K., Oda, K., and Mizusawa, H. (2012). Neural substrates for semantic memory of familiar songs: is there an interface between lyrics and melodies? PLoS ONE 7:e46354. doi:10.1371/journal.pone.0046354
Sammler, D., Baird, A., Valabregue, R., Clement, S., Dupont, S., Belin, P., et al. (2010). The relationship of lyrics and tunes in the processing of unfamiliar songs: a functional magnetic resonance adaptation study. J. Neurosci. 30, 3572–3578. doi:10.1523/JNEUROSCI.2751-09.2010
Samson, S., and Zatorre, R. J. (1991). Recognition memory for text and melody of songs after unilateral temporal lobe lesion: evidence for dual encoding. J. Exp. Psychol. Learn. Mem. Cogn. 17, 793–804.
Saur, D., Kreher, B. W., Schnell, S., Kummerer, D., Kellmeyer, P., Vry, M.-S., et al. (2008). Ventral and dorsal pathways for language. Proc. Natl. Acad. Sci. U.S.A. 105, 18035–18040. doi:10.1073/pnas.0805234105
Schön, D., Gordon, R., Campagne, A., Magne, C., Astésano, C., Anton, J., et al. (2010). Similar cerebral networks in language, music and song perception. Neuroimage 51, 450–461. doi:10.1016/j.neuroimage.2010.02.023
Suarez, R. O., Golby, A., Whalen, S., Sato, S., Theodore, W. H., Kufta, C. V., et al. (2010). Contributions to singing ability by the posterior portion of the superior temporal gyrus of the non-language-dominant hemisphere: first evidence from subdural cortical stimulation, Wada testing, and fMRI. Cortex 46, 343–353. doi:10.1016/j.cortex.2009.04.010
Thivard, L., Hombrouck, J., Tézenas du Montcel, S., Delmaire, C., Cohen, L., Samson, S., et al. (2005). Productive and perceptive language reorganisation in temporal lobe epilepsy. Neuroimage 24, 841–851. doi:10.1016/j.neuroimage.2004.10.001
Tierney, A., Dick, F., Deutsch, D., and Sereno, M. (2012). Speech versus song: multiple pitch-sensitive areas revealed by a naturally occurring musical illusion. Cereb. Cortex 23, 255–263. doi:10.1093/cercor/bhs003
Tillmann, B., Justus, T., and Bigand, E. (2008). Cerebellar patients demonstrate preserved implicit knowledge of association strengths in musical sequences. Brain Cogn. 66, 161–167. doi:10.1016/j.bandc.2007.07.005
Wagner, K., Frings, L., Spreer, J., Buller, A., Everts, R., Halsband, U., et al. (2008). Differential effect of side of temporal lobe epilepsy on lateralization of hippocampal, temporolateral, and inferior frontal activation patterns during a verbal episodic memory task. Epilepsy Behav. 12, 382–387. doi:10.1016/j.yebeh.2007.11.003
Winters, B. D. (2004). Double dissociation between the effects of peri-postrhinal cortex and hippocampal lesions on tests of object recognition and spatial memory: heterogeneity of function within the temporal lobe. J Neurosci. 24, 5901–5908. doi:10.1523/JNEUROSCI.1346-04.2004
Yasuda, C. L., Valise, C., Saúde, A. V., Pereira, A. R., Pereira, F. R., Ferreira Costa, A. L., et al. (2010). Dynamic changes in white and gray matter volume are associated with outcome of surgical treatment in temporal lobe epilepsy. Neuroimage 49, 71–79. doi:10.1016/j.neuroimage.2009.08.014
Keywords: neural adaptation, song, lyrics, hippocampal sclerosis, memory trace, conjunctive representation
Citation: Alonso I, Sammler D, Valabrègue R, Dinkelacker V, Dupont S, Belin P and Samson S (2014) Hippocampal sclerosis affects fMR-adaptation of lyrics and melodies in songs. Front. Hum. Neurosci. 8:111. doi: 10.3389/fnhum.2014.00111
Received: 11 December 2013; Paper pending published: 20 January 2014;
Accepted: 13 February 2014; Published online: 27 February 2014.
Edited by:Eckart Altenmüller, University of Music and Drama Hannover, Germany
Reviewed by:Stephan Schuele, Northwestern University, USA
Kathrin Wagner, University Hospital Freiburg, Germany
Copyright: © 2014 Alonso, Sammler, Valabrègue, Dinkelacker, Dupont, Belin and Samson. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Séverine Samson, Department of Psychology, University of Lille 3, BP 60 149, 59653 Villeneuve d’Ascq Cedex, France e-mail: firstname.lastname@example.org