Abstract
Understanding speech in background noise is a challenging task, especially when the signal is also distorted. In a series of previous studies, we have shown that comprehension can improve if, simultaneously with auditory speech, the person receives speech-extracted low-frequency signals on their fingertips. The effect increases after short audio-tactile speech training. In this study, we used resting-state functional magnetic resonance imaging (rsfMRI) to measure spontaneous low-frequency oscillations in the brain while at rest to assess training-induced changes in functional connectivity. We observed enhanced functional connectivity (FC) within a right-hemisphere cluster corresponding to the middle temporal motion area (MT), the extrastriate body area (EBA), and the lateral occipital cortex (LOC), which, before the training, was found to be more connected to the bilateral dorsal anterior insula. Furthermore, early visual areas demonstrated a switch from increased connectivity with the auditory cortex before training to increased connectivity with a sensory/multisensory association parietal hub, contralateral to the palm receiving vibrotactile inputs, after training. In addition, the right sensorimotor cortex, including finger representations, was more connected internally after the training. The results altogether can be interpreted within two main complementary frameworks. The first, speech-specific, factor relates to the pre-existing brain connectivity for audio–visual speech processing, including early visual, motion, and body regions involved in lip-reading and gesture analysis under difficult acoustic conditions, upon which the new audio-tactile speech network might be built. The other framework refers to spatial/body awareness and audio-tactile integration, both of which are necessary for performing the task, including in the revealed parietal and insular regions. It is possible that an extended training period is necessary to directly strengthen functional connections between the auditory and the sensorimotor brain regions for the utterly novel multisensory task. The results contribute to a better understanding of the largely unknown neuronal mechanisms underlying tactile speech benefits for speech comprehension and may be relevant for rehabilitation in the hearing-impaired population.
Introduction
Resting-state functional magnetic resonance imaging (rsfMRI) is based on the premise that spontaneous low-frequency oscillations in the brain, while, at rest, it can reveal its functional organization and alterations, without the need for the participant to engage in a specific task (Biswal et al., ; Raichle, ). At the same time, the signals obtained from task-MRI and rsfMRI are likely to stem from similar structural connections and neuronal processes (Ngo et al., ). In addition, in resting-state protocols, the oscillations of the blood-oxygenation-level–dependent (BOLD) signal offer a signal-to-noise ratio up to three times higher than the signal increases observed during task-related activities (Fox and Greicius, ). Using a measure of functional connectivity (FC), which represents signal correlations among remote brain regions, rsMRI has been shown to reveal language networks with high sensitivity (e.g., Lemée et al., ; Tie et al., ; Zhu et al., ; Branco et al., ; Sun et al., ; Liu et al., ). Considerable FC changes have been reported as a result of several-minute- or several-week-long cognitive or motor interventions, as well as aging (e.g., Katsuno et al., ; Maruyama et al., ; Cao et al., ; Farrens et al., ; Kawata et al., ; Bamidis et al., ). It has been proposed that, to gain a deeper understanding of changes in brain organization, it is essential to consider functional brain networks.
In the current study, we used the rsfMRI method specifically to investigate changes in functional connectivity following speech comprehension training, combining congruent auditory and vibrotactile inputs on the fingertips.
Audio-vibrotactile interactions are relatively common in our everyday lives, for example, when receiving tactile feedback from a ringing phone, when steering wheel, or while playing computer games. The combination and integration of these two sensory signals seem natural and inherent (Van Békésy, ), which is possibly due to the significant similarities between the two sensory modalities, including their ability to encode the very same oscillatory patterns within an overlapping frequency range using mechanoreceptors. Specifically, for the tactile sense, the Pacinian corpuscles in the skin respond to vibrations of up to 700–1,000 Hz, while the auditory system encodes vibrations in the air through inner hair cells between 20 Hz and 20 kHz (Bolanowski et al., ; Schnupp et al., ).
In several published studies, congruent vibrotactile inputs have been shown to successfully enhance music perception (e.g., Russo et al., ; Sharp et al., ), as well as the localization of sounds in space (Gori et al., ; Occelli et al., ; Snir et al., ). Another specific use of low-frequency vibrations has been to improve speech comprehension. This approach was originally developed to assist the deaf population, with tactile aids designed to convey different features of speech signals via the skin (e.g., Galvin et al., ; Weisenberger and Percy, ). In a series of recent studies by our lab and others, it has been shown that improvement of speech perception is also possible in both typically hearing and hearing-impaired individuals using cochlear implants, with continuous speech-extracted vibrations delivered on the fingertips (Cieśla et al., , ; Fletcher et al., ; Fletcher et al., ; Huang et al., ; Schulte et al., ; Rǎutu et al., ). Most authors implemented some form of manipulation to the speech signal to mimic the acoustic conditions that are most challenging for the hearing-impaired population, such as speech distortions and/or simultaneous background noise.
This rsfMRI study complements our parallel task-fMRI study (manuscript under review; preprint at Cieśla et al., ), with participants performing speech comprehension tasks inside an MRI scanner before and after dedicated audio-tactile speech comprehension training. The speech signals consisted of distorted sentences (vocoded to resemble speech delivered through a cochlear implant system), presented alongside speech background noise. The tactile vibrations corresponded to low frequencies (< 200Hz) extracted directly from the speech inputs using an adjusted STRAIGHT algorithm (Kawahara et al., ). We demonstrated enhanced comprehension following audio-tactile training, which aligns with findings from several other studies that utilized audio–visual speech stimuli, as well as those showing improved intelligibility of distorted speech with practice time (e.g., Casserly and Pisoni, ; Erb et al., ; Guediche et al., ; Bernstein et al., ). The results suggest that the speech-extracted tactile signal can potentially work in a manner similar to the visual signal (such as lip-reading) through temporal synchronization with auditory inputs (O'Sullivan et al., ; Schulte et al., ).
Here, we speculated that training-induced neuronal changes would also manifest in resting-state functional connectivity (FC). Specifically, brain networks were identified using independent component analysis (ICA) and tested for functional connectivity before and after audio-tactile training. Based on our previous behavioral findings indicating successful multisensory integration of speech stimuli (Cieśla et al., , ), we expected to observe enhanced connectivity between the auditory and the somatosensory resting-state networks, as well as within the language network (Price, ; Hertrich et al., ). This finding includes multisensory areas such as the posterior superior temporal sulcus (pSTS), the angular gyrus/supramarginal gyrus (AG/SMG), and the lateral occipital cortex (LOC), which have been shown in the literature for audio-visual and audio-tactile integration (e.g., Renier et al., ; Kassuba et al., ; Landelle et al., ; King et al., ; Nath and Beauchamp, ; Hickok et al., ). The applied outcome measure was the difference in functional connectivity patterns between two resting-state fMRI sessions, before and after audio-tactile speech comprehension training.
Materials and methods
The study took place at the ELSC Neuroimaging Unit in Jerusalem, Israel. All participants gave informed consent and received compensation for their participation. The study procedures complied with the World Medical Association () and received approval from the Ethics Committee of the Hadassah Medical Center in Jerusalem (protocol no. 353-18.1.08).
Participants
The reported data were acquired from 20 adult healthy individuals (8 male/12 female, 26.8+/−4.2), all of whom were right-handed and had no history of neurological or neurodevelopmental impairments.
Experimental procedures
Behavioral experimental procedures
Between the two resting-state fMRI sessions, the participants were tested twice (before and after the training) on their vocoded speech-in-noise comprehension. There were three test conditions: a) auditory sentences (A), b) auditory sentences with congruent tactile vibrations delivered to two fingertips of the right hand (audio-tactile congruent, ATc), and c) auditory sentences accompanied with non-congruent tactile vibrations on the fingertips (audio-tactile non-congruent, ATnc). For each test condition, an individual speech reception threshold (SRT)—i.e., the SNR between the target sentence and the background noise required for 50% understanding—was determined. The used sentences were derived from the Hearing-In-Noise-Test (HINT) database (Nilsson et al., ) and vocoded to resemble stimulation through a cochlear implant (Walkowiak et al., ). In the multisensory test conditions, low-frequency vibrations (< f0 of the spoken sentence, i.e., 200 Hz) were extracted from either the concurrently presented auditory sentence (audio-tactile congruent, ATc) or from another randomly selected sentence from the HINT database (audio-tactile non-congruent, ATnc) and delivered to the index and middle fingers of the right hand through an in-house developed device (Cieśla et al., ). Between the two SRT testing sessions, the participants took part in a 30–45 min training session. They were asked to repeat 148 HINT sentences, one by one, with (group 1) or without (group 2) accompanying congruent tactile vibrations delivered to the fingertips, at individually determined SRTs. The training session continued until all 148 sentences were repeated correctly without feedback. The results of the behavioral study, including pre- and post-training speech comprehension measurements, are published elsewhere (Cieśla et al., ). The resting-state fMRI study was conducted in a subgroup of participants who underwent audio-tactile speech comprehension training.
Resting-state fMRI data acquisition
The resting-state fMRI data were acquired twice using a 3T Skyra MRI scanner by Siemens and a 32-channel head coil, before and after the training, during a 10:11 min session. The parameters were as follows: TR =1.5 s, TE = 32.6 ms, voxel size = 2 × 2 × 2 mm, 72 slices, flip angle = 78, FOV = 192 mm, and bandwidth = 1,736 Hz. The participants had their eyes open and were asked to fixate on a cross presented on the screen. In addition, a high-resolution T1 MR sequence was collected with the following parameters: TR = 2.3 s, TE = 2.98 ms, IT = 900 ms, matrix size = 256 × 256, voxel size = 1 × 1 × 1 mm, 160 slices, and bandwidth = 240Hz/voxel.
Data analysis
Resting-state fMRI data analysis
To evaluate functional connectivity patterns between brain regions, the CONN 22a (Connectivity Toolbox, https://web.conn-toolbox.org/) software was used for data analysis. Preprocessing of the functional data included the following: slice timing correction, motion correction, scrubbing, linear detrending, band-pass filtering (0.008 Hz < f < 0.09 Hz), co-registration to individual T1 structural scans, surface-based spatial normalization (https://surfer.nmr.mgh.harvard.edu/), and spatial smoothing (6 mm Gaussian kernel).
Denoising was applied using default settings, including the regression of potential confounding effects characterized by white matter time series (five CompCor noise components), cerebrospinal fluid (CSF) time series (five CompCor noise components), motion parameters and their first-order derivatives (12 factors), outlier scans (below 25 factors), and linear trends (two factors) within each functional run. This was followed by bandpass frequency filtering of the BOLD time series between 0.008 Hz and 0.09 Hz. The CompCor noise components within the white matter and CSF were estimated by computing the average BOLD signal and the largest principal components orthogonal to the BOLD average, motion parameters, and outlier scans within each participant's eroded segmentation masks. Based on the number of noise terms included in this denoising strategy, the effective degrees of freedom of the BOLD signal after denoising were estimated to range from 59.8 to 125 (average 94.7) across all participants.
Group ICA) was used for statistical analysis. ICA is a data-driven approach that investigates multiple voxel-to-voxel interactions of several networks in the brain simultaneously. Based on the acquired signal, ICA separates noisy components from those representing neuronal activation in the form of resting-state networks. Networks represent sets of regions with voxel-by-voxel correlated BOLD signal fluctuations (Smitha et al., ; Rosazza et al., ). Using the CONN software, we applied a G1 FastICA + GICA3 back-projection procedure with dimensionality reduction set at 64 (64 principal components) and the number of independent components set at 40 (components optimized for orthogonality; 40 is the default value), based on the recorded BOLD signal combined across all participants. Components comprising the language (including auditory cortex), visual, and sensorimotor networks were selected for further analysis (Figure 1, ICAs 1–6, 11, 16). For each of the separately selected eight components, functional connectivity (FC) was estimated with all other voxels in the brain. Next, FC was compared between the PRE and POST training sessions for each component using t-tests. See the subsequent steps of the analysis in Figure 2. Specific outcome brain regions were identified that showed increased FC after the training compared to before the training (Figure 3). The statistical analysis used permutation with the following parameters: voxel-threshold with a p-value of < 0.01 and cluster-mass p-FDR corrected p-value of < 0.05. The regions were labeled using the Harvard-Oxford Atlas.
Figure 1
Figure 2
Figure 3
Results
Resting-state fMRI functional connectivity changes after training in the auditory, sensorimotor, and visual systems
Of the eight comparisons of FC before and after training, four turned out to be statistically significant (Figure 3). Specifically, resting-state functional connectivity increased after, compared to before, the audio-tactile speech comprehension training between bilateral clusters encompassing parts of the lateral occipital cortex (LOC), angular gyrus (AG), posterior medial temporal gyrus (pMTG), and posterior superior temporal sulcus (pSTS; IC2; including functional areas, such as MT +/−44 −68 2 and extrastriate body area (EBA) –/+ 50 −70 5; https://neurosynth.org/), as well as LOC/pMTG/AG on the right side, indicating increased internal connectivity within this network. In addition, increased connectivity was observed between bilateral clusters encompassing parts of the occipital pole (OP), LOC, lingual gyrus (LG), and intracalcarine cortex (ICC; early visual cortex; IC3), as well as a hub representing the left superior parietal lobe (SPL), anterior supramarginal gyrus (aSMG), and postcentral gyrus (PostCG). Increased connectivity was observed in a right-hemisphere cluster encompassing parts of PostCG, precentral gurus (PreCG), and SPL (sensorimotor cortex and operculum, IC6), with an almost overlapping area. At the same time, decreased functional connectivity after training was observed between bilateral clusters encompassing parts of iLOC, AG, pMTG, and pSTS (IC 2) and bilateral anterior insula, as well as between bilateral clusters encompassing parts of the superior temporal gyrus (STG), temporal pole (TP), polar pole (PP), SMG (auditory cortex, IC 4), and OP and LOC on the right side (early visual cortex). The results are depicted in Figures 1–3.
Discussion
As reported in a series of our previous studies using an in-house setup, congruent low-frequency speech-extracted vibrations on the fingertips can improve auditory speech perception in difficult acoustic conditions by an average of 4–6 dB SNR (Cieśla et al., , ). Interestingly, training did not increase this tactile benefit any further, suggesting that only to a certain degree one can use tactile information naturally. In addition, audio-tactile training was found to be more efficient than auditory training, leading to comparable improvement but in more difficult acoustic conditions (Cieśla et al., ). In this study, we present resting-state fMRI findings from a subgroup of participants trained using audio-tactile speech stimulation. The results provide insights into the brain networks likely engaged in the new experience of audio-tactile speech perception and insights that promote training-related improvements in comprehension.
The revealed functional connectivity (FC) results showed several effects when the pre- and post-training resting-state networks were compared, which can be summarized as follows. First, we observed a shift in the functional connectivity of the lateral occipital cortex (LOC) from increased FC with the bilateral dorsal anterior insula to increased internal connectivity (i.e., between the right LOC and itself). Second, there was a change in the functional connectivity of the early visual system, shifting from one with the auditory system before training to one with a hub in the left somatosensory/multisensory association cortex afterward. Third, the sensorimotor system in the right hemisphere showed increased internal FC after training. While the effects involving the auditory and sensorimotor systems were hypothesized following an audio-tactile intervention, the engagement of the early visual system was not. Interestingly, this system was also observed in a parallel task-fMRI study by our lab [under review; see also a preprint at: Cieśla et al., ]. In the previously mentioned study, the participants were asked to repeat distorted sentences inside the MRI scanner, before and after speech comprehension training, with and without tactile vibrations delivered to the fingertips. The effects observed for the trained audio-tactile speech condition partially correspond to those revealed in the current resting-state analysis, including significant involvement of the visual system in both learning and multisensory integration, especially in the early visual regions and LOC. This finding further suggests that the signals obtained from task-fMRI and rsfMRI are likely to stem from similar structural connections and neuronal processes (Ngo et al., ).
We speculate that the revealed “visual” regions (both early V1–V2 and LOC) might represent parts of a network typically reported for audio-visual speech comprehension (e.g., Hertrich et al., ; Beer et al., ; Nath and Beauchamp, ; Hauswald et al., ). The involvement of these regions indicates that the brain functional system engaged in the utterly novel task of processing audio-tactile speech might be first built upon an existing blueprint for connectivity. In the current study, the speech task required comprehension of sentences in a non-native language that were also vocoded (resembling cochlear implant stimulation) and presented against background speech noise, all rendering the task rather challenging. In everyday situations, when exposed to ambiguous speech, either distorted or occurring in a challenging acoustic environment, we naturally search for informative visual cues, such as from lip-reading. These audio-visual associations develop in the early years of life and are practiced on a regular basis throughout lifetime (Hickok et al., ; Dopierała et al., ). Furthermore, the involvement of the early visual cortex has been previously reported for auditory and tactile (e.g., Braille reading) language tasks as well. Several studies have demonstrated this effect in both blind and sighted individuals, indicating that early visual regions have the capacity for amodal language processing (Amedi et al., ; Bedny et al., ; Reich et al., ; Seydell-Greenwald et al., ; Merabet et al., ). In this study, after the audio-tactile speech training, once it became apparent that no informative visual cues were available during task performance, the pre-existing (and potentially automatically engaged in the context of speech perception) connectivity between the auditory system and the early visual system decreased (Panel C in Figure 2).
The other identified “visual” hub in the lateral occipital/temporal lobe encompassed regions such as the middle temporal visual area (MT), EBA, and LOC (Panel D in Figure 2). These brain areas, among other functions, can play a role in non-verbal communication, specifically in encoding lip movements, body gestures, facial expressions, and other language-related spatial cues (Kitada et al., ; Hickok et al., ; Van der Stoep and Alais, ). All these cues can facilitate comprehension in difficult acoustic conditions. Furthermore, the revealed adjacent pSTS is recognized as a key hub for multisensory integration, particularly for audio-tactile and audio-visual inputs. It specifically specializes in processing temporally changing signals, such as speech (King et al., ; Hertrich et al., ; Beer et al., ; Beauchamp et al., ). In a non-speech-specific context, the revealed area of the LOC has also been shown to have multisensory properties (although mainly audio-visual) and has been implicated in activities such as grasping and object recognition (e.g., Kassuba et al., ; Jao et al., ; Brand et al., ), as well as in response to passive vibrotactile stimulation on the hand—potentially related to the mechanism of object manipulation through increased connectivity with SI (Tal et al., ). During the speech comprehension training, the participants both listened to speech stimuli through headphones and had the fingertips of their right hand placed in the tactile device. This specific multisensory setup may also involve body awareness and reaching behavior, which could be reflected in resting-state functional network changes.
The functional connectivity of the LOC after training diminishes with the bilateral anterior insular cortex. This outcome may stem from the insula's role in various cognitive processes, particularly in the context of developing new skills or adapting to new environments (e.g., Gogolla, ; Zühlsdorff et al., ; Ardilla et al., ). Specifically, the revealed dorsal anterior part of the insula, through its multiple connections with the rest of the brain, has been found to participate in language processing and general learning (e.g., Ardilla et al., ; although see Woolnough et al., ), cognitive control, spatial attention tasks, and multimodal integration—as opposed to the social-emotional functions typically attributed to the posterior and ventral-anterior aspects of the insula (Kurth et al., ; Uddin et al., ). In addition, the anterior insula may play a role in bodily ownership, bodily self-awareness, and tactile perception (Wang et al., ). One suggested mechanism is through direct projections from the secondary somatosensory cortex to the insular cortex, which, in turn, innervates regions of the temporal lobe believed to be critical for tactile learning and memory (Kurth et al., ; Augustine, ). The decreased involvement of the insula possibly reflects progress in learning to use the newly available tactile inputs for improved speech comprehension.
Considering the role of the sensorimotor system, we demonstrate two additional effects. First, the applied audio-tactile speech training resulted in increased connectivity from the bilateral early visual cortex to a posterior parietal region on the side opposite to the vibrotactile stimulation. Part of the revealed hub can be considered the somatosensory association cortex, where the experienced low-frequency tactile signal may be analyzed for its temporal and speech features. Other tactile operations attributed to this area include Braille reading, tactile attention, shape recognition, and others (e.g., Kim et al., ; Bodegard et al., ; Li Hegner et al., ). At the same time, the superior parietal lobe—comprising 40% of the revealed cluster—receives inputs not only from the early sensory system, particularly the hand, but also from other modalities, which are further combined to inform behaviors, such as reaching or grasping (Sheth and Young, ; Goodale and Milner, ). Finally, the demonstrated left-hemisphere SMG (Panel A in Figure 2) plays a critical role in the phonological analysis of language inputs, while also having multisensory properties (e.g., Oberhuber et al., ).
The increased functional connectivity after training between the early visual and the sensorimotor system/multisensory is not a straightforward result. It might reflect a mechanism similar to the one reported in some other studies on tactile perception, although in the opposite direction (Tal et al., ; Merabet et al., ). Specifically, Tal et al. showed deactivation of early visual regions (e.g., V1–V2) when participants experienced passive, delicate brushing of various body parts—although a single-case EcoG study by Gaglianese et al. () showed positive involvement of V1 during brushing. In contrast, Merebet et al. found active engagement of V1 in estimating roughness or inter-dot spacing in tactile patterns. This study indicates that the responsiveness of early visual regions during tactile stimulation might be task-specific and could reflect processes such as motion processing, spatial analysis, or tactile-to-visual remapping of inputs. As suggested by some authors, the increased FC with early visual areas after training and its general role in the current experiment might also be mediated by the visualization of the content of the speech signal or the vibrating devices (e.g., Reiner, ).
In addition, after training, we observed increased internal connectivity within the right-hemisphere sensorimotor system (ipsilateral to the tactile finger stimulation). The cluster encompasses the primary (SI), secondary (SII, operculum), and somatosensory association cortices, including areas representing the fingers, as well as the corresponding motor cortex. This finding was unexpected but may be related to several mechanisms. First, although responses to tactile stimuli are found to be mainly contralateral (and especially in SI), both animal and human neuroimaging studies have shown modulation of responses in ipsilateral SII (and to a lesser extent, also in SI), possibly through dense transcallosal connectivity. Moreover, the bilateral sensorimotor association cortex in the posterior parietal cortex is the area where inputs from both hands become integrated (Pala and Stanley, ; Tamè et al., ; Hlushchuk and Hari, ). The sensorimotor cortex is also densely connected with the bilateral motor cortex, specifically the regions representing hands, which enables the successful manipulation of objects. The sensorimotor interactions involve the motor cortex responding to tactile stimulation, as well as the sensory cortex receiving feedback from and regulating motor behavior (Mastria et al., ; Tamè et al., ; Bao et al., ). Further research is needed to understand why the observed effect was ipsilateral to the stimulated hand rather than contralateral.
In summary, the main results of the study highlight the role of the visual network, auditory system, sensorimotor system, and insula in the audio-tactile integration of speech information and learning using our specific multisensory setup. The findings are in partial agreement with our hypotheses— e.g., showing engagement of the early sensory systems, as well as the pSTS and SMG in multisensory integration—although no enhancement of direct connectivity was found between the auditory and the sensorimotor cortex. The lack of such an effect might reflect comparable tactile benefits for speech perception before and after training (Cieśla et al., ). The involvement of the visual system (LOC, early regions), potentially mediating auditory-to-tactile communication and thus the integration of inputs, indicates that functional network changes following perceptual learning might be initially constrained by the existing connections established through experience (Makin and Krakauer, ; Heimler and Amedi, ). Since the audio-tactile speech context is novel—never experienced in a lifetime or in evolution—unlike the well-trained audio-visual speech context, a longer training regime might be necessary to modify pre-existing connectivity patterns. This might, in turn, translate into greater benefits of adding tactile information. In addition, some of the observed results in the visual, sensorimotor, and insular networks can be interpreted within the framework of bodily awareness, grasping, and mental imagery, as the task required successful integration of inputs arriving from different spatial locations. One limitation of the current study is the lack of assessment of functional connectivity changes following auditory-only versus multisensory audio-tactile speech training, especially given our behavioral results showing its significant but less profound effect on learning (Cieśla et al., ).
Statements
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by Ethics Committee of the Hadassah Medical Center in Jerusalem. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
KC: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing. TW: Conceptualization, Methodology, Software, Visualization, Writing – review & editing. AA: Conceptualization, Funding acquisition, Resources, Supervision, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. This research was supported by Polish Ministry of Science and Higher Education MOBILNOSC PLUS V grant (1642/MOB/V/2017/0) to KC, ERC Consolidator Grant (773121 NovelExperiSense) to AA, Horizon GuestXR (101017884) grant to AA; Israeli Science Foundation grant (3709/24) to AA.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
AmediA.RazN.PiankaP.MalachR.ZoharyE. (2003). Early 'visual' cortex activation correlates with superior verbal memory performance in the blind. Nat. Neurosci. 6, 758–766. 10.1038/nn1072
2
ArdillaA.BernalB.RosselliM. (2014). Participation of the insula in language revisited: a meta-analytic connectivity study. J. Neurolinguistics29, 31–41. 10.1016/j.jneuroling.2014.02.001
3
AugustineG. J. (eds.). (2023). Neuroscience (Oxford: Sinauer Associates, Oxford University Press; 7th edition).
4
BamidisP. D.VivasA. B.StyliadisC.FrantzidisC.KladosM.SchleeW.et al. (2014). A review of physical and cognitive interventions in aging. Neurosci. Biobehav. Rev. 44, 206–220. 10.1016/j.neubiorev.2014.03.019
5
BaoS.WangY.EscalanteY. R.LiY.LeiY. (2024). Modulation of motor cortical inhibition and facilitation by touch sensation from the glabrous skin of the human hand. eNeuro. 11:ENEURO.0410-23.2024. 10.1523/ENEURO.0410-23.2024
6
BeauchampM. S.LeeK. E.ArgallB. D.MartinA. (2004). Integration of auditory and visual information about objects in superior temporal sulcus. Neuron41, 809–823. 10.1016/S0896-6273(04)00070-4
7
BednyM.RichardsonH.SaxeR. (2015). “Visual” cortex responds to spoken language in blind children. J. Neurosci. 35, 11674–11681. 10.1523/JNEUROSCI.0634-15.2015
8
BeerA. L.PlankT.MeyerG.GreenleeM. W. (2013). Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory-visual object processing. Front. Integr. Neurosci. 7:5. 10.3389/fnint.2013.00005
9
BernsteinL. E.AuerE. T.JrEberhardtS. P.JiangJ. (2013). Auditory perceptual learning for speech perception can be enhanced by audiovisual training. Front Neurosci. 7:34. 10.3389/fnins.2013.00034
10
BiswalB.YetkinF. Z.HaughtonV. M.HydeJ. S. (1995). Functional connectivity in the motor cortex of resting human brain using echo-planar MRI. Magn. Reson. Med. 34, 537–541. 10.1002/mrm.1910340409
11
BodegardA.GeyerS.GrefkesC.ZillesK.RolandP. E. (2001). Hierarchical processing of tactile shape in the human brain. Neuron31, 317–328. 10.1016/S0896-6273(01)00362-2
12
Bolanowski JrS. J.GescheiderG. A.VerrilloR. T.CheckoskyC. M. (1988). Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am.84, 1680–1694. 10.1121/1.397184
13
BrancoP.SeixasD.DeprezS.KovacsS.PeetersR.CastroS. L.et al. (2016). Resting-state functional magnetic resonance imaging for language preoperative planning. Front. Hum. Neurosci. 10:11. 10.3389/fnhum.2016.00011
14
BrandJ.PiccirelliM.Hepp-ReymondM. C.EngK.MichelsL. (2020). Brain activation during visually guided finger movements. Front. Hum. Neurosci.14:309. 10.3389/fnhum.2020.00309
15
CaoW.LuoC.ZhuB.ZhangD.DongL.GongJ.et al. (2014). Resting-state functional connectivity in anterior cingulate cortex in normal aging. Front. Aging Neurosci. 6:280. 10.3389/fnagi.2014.00280
16
CasserlyE. D.PisoniD. B. (2015). Auditory learning using a portable real-time vocoder: preliminary findings. J. Speech Lang. Hear. Res. 58, 1001–1016. 10.1044/2015_JSLHR-H-13-0216
17
CieślaK.WolakT.AmediA. (2024). Neuronal basis of audio-tactile speech perception (2024). BIORXIV/2024/608369. 10.1101/2024.08.16.608369
18
CieślaK.WolakT.LorensA.HeimlerB.SkarżyńskiH.AmediA.et al. (2019). Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution. Restor. Neurol. Neurosci.37, 155–166. 10.3233/RNN-190898
19
CieślaK.WolakT.LorensA.MentzelM.SkarżyńskiH.AmediA.et al. (2022). Effects of training and using an audio-tactile sensory substitution device on speech-in-noise understanding. Sci. Rep.12, 1–16. 10.1038/s41598-022-06855-8
20
DopierałaA. A. W.PérezD. L.MercureE.PlutaA.Malinowska-KorczakA.EvansS.et al. (2023). The development of cortical responses to the integration of audiovisual speech in infancy. Brain Topogr.36, 459–475. 10.1007/s10548-023-00959-8
21
ErbJ.MollyH.EisnerF.ObleserJ. (2013). The brain dynamics of rapid perceptual adaptation to adverse listening conditions. J. Neurosci.33, 10688–10697. 10.1523/JNEUROSCI.4596-12.2013
22
FarrensA. J.VahdatS.SergiF. (2023). Changes in resting state functional connectivity associated with dynamic adaptation of wrist movements. J. Neurosci.43, 3520–3537. 10.1523/JNEUROSCI.1916-22.2023
23
FletcherM. D.HadeediA.GoehringT.MillsS. R. (2019). Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Sci. Rep.9, 1–8. 10.1038/s41598-019-47718-z
24
FletcherM. D.MillsS. R.GoehringT. (2018). Vibro-tactile enhancement of speech intelligibility in multi-talker noise for simulated cochlear implant listening. Trends Hear.22:2331216518797838. 10.1177/2331216518797838
25
FoxM. D.GreiciusM. (2010). Clinical applications of resting state functional connectivity. Front. Syst. Neurosci. 4:19. 10.3389/fnsys.2010.00019
26
GaglianeseA.BrancoM. P.GroenI.BensonN. C.VansteenselM. J.MurrayM. M.et al. (2020). Electrocorticography evidence of tactile responses in visual cortices. Brain Topogr. 33, 559–570. 10.1007/s10548-020-00783-4
27
GalvinK. L.CowanR. S. C.SarantJ. Z.AlcantaraJ.BlarneyP. J.ClarkG. M.et al. (1991). Use of a multichannel electrotactile speech processor by profoundly hearing-impaired children in a total communication environment. J. Am. Acad. Audiol. 12, 214–225.
28
GogollaN. (2017). The insular cortex. Curr. Biol. 27, R580–R586. 10.1016/j.cub.2017.05.010
29
GoodaleM. A.MilnerA. D. (1992). Separate visual pathways for perception and action. Trends Neurosci. 15, 20–25. 10.1016/0166-2236(92)90344-8
30
GoriM.VercilloT.SandiniG.BurrD. (2014). Tactile feedback improves auditory spatial localization. Front. Psychol. 5:109563. 10.3389/fpsyg.2014.01121
31
GuedicheS.BlumsteinS. E.FiezJ. A.HoltL. L. (2014). Speech perception under adverse conditions: insights from behavioral, computational, and neuroscience research. Front. Syst. Neurosci.7:126. 10.3389/fnsys.2013.00126
32
HauswaldA.LithariC.CollignonO.LeonardelliE.WeiszN. (2018). A visual cortical network for deriving phonological information from intelligible lip movements. Curr. Biol.28, 1453–1459.e3. 10.1016/j.cub.2018.03.044
33
HeimlerB.AmediA. (2020). Are critical periods reversible in the adult brain? Insights on cortical specializations based on sensory deprivation studies. Neurosci. Biobehav. Rev. 116, 494–507. 10.1016/j.neubiorev.2020.06.034
34
HertrichI.DietrichS.AckermannH. (2020). The margins of the language network in the brain. Front. Commun.5:519955. 10.3389/fcomm.2020.519955
35
HickokG.RogalskyC.MatchinW.BasilakosA.CaiJ.PillayS.et al. (2018). Neural networks supporting audiovisual integration for speech: a large-scale lesion study. Cortex. 103, 360–371. 10.1016/j.cortex.2018.03.030
36
HlushchukY.HariR. (2006). Transient suppression of ipsilateral primary somatosensory cortex during tactile finger stimulation. J. Neurosci. 26, 5819–5824. 10.1523/JNEUROSCI.5536-05.2006
37
HuangJ.SheffieldB.LinP.ZengF. G. (2017). Electro-tactile stimulation enhances cochlear implant speech recognition in noise. Sci. Rep.7:2196. 10.1038/s41598-017-02429-1
38
JaoR. J.JamesT. W.JamesK. H. (2015). Crossmodal enhancement in the LOC for visuohaptic object recognition over development. Neuropsychologia. 77, 76–89. 10.1016/j.neuropsychologia.2015.08.008
39
KassubaT.MenzM. M.RöderB.SiebnerH. R. (2013). Multisensory interactions between auditory and haptic object recognition. Cereb. Cortex. 23, 1097–1107. 10.1093/cercor/bhs076
40
KatsunoY.UekiY.ItoK.MurakamiS.AoyamaK.OishiN.et al. (2022). Effects of a new speech support application on intensive speech therapy and changes in functional brain connectivity in patients with post-stroke aphasia. Front. Hum Neurosci. 16:870733. 10.3389/fnhum.2022.870733
41
KawaharaH.Masuda-KatsuseI.de CheveigneA. (1999). Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: possible role of a repetitive structure in sounds. Speech Commun.27, 187–207. 10.1016/S0167-6393(98)00085-5
42
KawataN. Y. S.NouchiR.ObaK.MatsuzakiY.KawashimaR. (2022). Auditory cognitive training improves brain plasticity in healthy older adults: evidence from a randomized controlled trial. Front. Aging Neurosci. 14:826672. 10.3389/fnagi.2022.826672
43
KimJ.MüllerK. R.ChungY. G.ChungS. C.ParkJ. Y.BülthoffH. H.et al. (2015). Distributed functions of detection and discrimination of vibrotactile stimuli in the hierarchical human somatosensory system. Front. Hum. Neurosci. 8:1070. 10.3389/fnhum.2014.01070
44
KingA. J.Hammond-KennyA.NodalF. R. (2019). “Multisensory processing in the auditory cortex,” Multisensory Processes. Springer Handbook of Auditory Research (vol 68), eds. A. Lee, M. Wallace, A. Coffin, A. Popper, R. Fay. (Springer, Cham). 10.1007/978-3-030-10461-0_6
45
KitadaR.YoshiharaK.SasakiA. T.HashiguchiM.KochiyamaT.SadatoN.et al. (2014). The brain network underlying the recognition of hand gestures in the blind: the supramodal role of the extrastriate body area. J. Neurosci. 34, 10096–10108. 10.1523/JNEUROSCI.0500-14.2014
46
KurthF.ZillesK.FoxP. T.LairdA. R.EickhoffS. B. (2010). A link between the systems: functional differentiation and integration within the human insula revealed by meta-analysis. Brain Struct. Funct.214, 519–534. 10.1007/s00429-010-0255-z
47
LandelleC.Caron-GuyonJ.NazarianB.AntonJ. L.SeinJ.PruvostL.et al. (2023). Beyond sense-specific processing: decoding texture in the brain from touch and sonified movement. iScience. 26:107965. 10.1016/j.isci.2023.107965
48
LeméeJ. M.BerroD. H.BernardF.ChinierE.LeiberL. M.MeneiP.et al. (2019). Resting-state functional magnetic resonance imaging versus task-based activity for language mapping and correlation with perioperative cortical mapping. Brain Behav. 9:e01362. 10.1002/brb3.1362
49
Li HegnerY.LeeY.GroddY. W.BraunC. (2010). Comparing tactile pattern and vibrotactile frequency discrimination: a human FMRI study. J. Neurophysiol. 103, 3115–3122. 10.1152/jn.00940.2009
50
LiuC.JiaoL.LiZ.TimmerK.WangR. (2021). Language control network adapts to second language learning: a longitudinal rs-fMRI study. Neuropsychologia150:107688. 10.1016/j.neuropsychologia.2020.107688
51
MakinT. R.KrakauerJ. W. (2023). Against cortical reorganization. eLife 12:e84716. 10.7554/eLife.84716
52
MaruyamaT.TakeuchiH.TakiY.MotokiK.JeongH.KotozakiY.et al. (2018). Effects of time-compressed speech training on multiple functional and structural neural mechanisms involving the left superior temporal Gyrus. Neural. Plast.2018:6574178. 10.1155/2018/6574178
53
MastriaG.ScalitiE.MehringC.BurdetE.BecchioC.SerinoA.et al. (2023). Morphology, connectivity, and encoding features of tactile and motor representations of the fingers in the human precentral and postcentral gyrus. J. Neurosci. 43, 1572–1589. 10.1523/JNEUROSCI.1976-21.2022
54
MerabetL. B.HamiltonR.SchlaugG.SwisherJ. D.KiriakopoulosE. T.PitskelN. B.et al. (2008). Rapid and reversible recruitment of early visual cortex for touch. PLoS ONE 3:e3046. 10.1371/journal.pone.0003046
55
MerabetL. B.SwisherJ. D.McMainsS. A.HalkoM. A.AmediA.Pascual-LeoneA.et al. (2007). Combined activation and deactivation of visual cortex during tactile sensory processing. J. Neurophysiol.97, 1633–1641. 10.1152/jn.00806.2006
56
NathA. R.BeauchampM. S. (2012). A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. Neuroimage59, 781–787. 10.1016/j.neuroimage.2011.07.024
57
NgoG. H.KhoslaM.JamisonK.KuceyeskiA.SabuncuM. R. (2022). Predicting individual task contrasts from resting-state functional connectivity using a surface-based convolutional network. Neuroimage248:118849. 10.1016/j.neuroimage.2021.118849
58
NilssonM.SoliS. D.SullivanJ. A. (1994). Development of the hearing in noise test for the measurement of speech reception thresholds in quiet and in noise. J. Acoust. Soc. Am.95, 1085–1099. 10.1121/1.408469
59
OberhuberM.HopeT. M. H.SeghierM. L.Parker JonesO.PrejawaS.GreenD. W.et al. (2016). Four functionally distinct regions in the left supramarginal gyrus support word processing. Cereb. Cortex. 26, 4212–4226. 10.1093/cercor/bhw251
60
OccelliV.SpenceC.ZampiniM. (2009). Compatibility effects between sound frequency and tactile elevation. Neuroreport20, 793–797. 10.1097/WNR.0b013e32832b8069
61
O'SullivanA. E.CrosseM. J.LibertoG. M. D.de CheveignéA.LalorE. C. (2021). Neurophysiological indices of audiovisual speech processing reveal a hierarchy of multisensory integration effects. J. Neurosci. 41, 4991–5003. 10.1523/JNEUROSCI.0906-20.2021
62
PalaA.StanleyG. B. (2022). Ipsilateral stimulus encoding in primary and secondary somatosensory cortex of awake mice. J. Neurosci. 42, 2701–2715. 10.1523/JNEUROSCI.1417-21.2022
63
PriceC. J. (2012). A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage62, 816–847. 10.1016/j.neuroimage.2012.04.062
64
RaichleM. E. (2009). A paradigm shift in functional brain imaging. J. Neurosci.29, 12729–12734. 10.1523/JNEUROSCI.4366-09.2009
65
RăutuI. S.De TiègeX.JousmäkiV.BourguignonM.BertelsJ. (2023). Speech-derived haptic stimulation enhances speech recognition in a multi-talker background. Sci. Rep.13:16621. 10.1038/s41598-023-43644-3
66
ReichL.SzwedM.CohenL.AmediA. (2011). A ventral visual stream reading center independent of visual experience. Curr. Biol.21, 363–368. 10.1016/j.cub.2011.01.040
67
ReinerM. (2008). “Seeing through touch: the role of haptic information in visualization,” in Visualization: Theory and Practice in Science Education, eds. J. K. Gilbert, M. ReinerM. B.Nakhleh (Springer: Netherlands), 73–84. 10.1007/978-1-4020-5267-5_4
68
RenierL. A.AnurovaI.De VolderA. G.CarlsonS.VanMeterJ.RauscheckerJ. P.et al. (2009). Multisensory integration of sounds and vibrotactile stimuli in processing streams for “what” and “where”. J. Neurosci.29, 10950–10960. 10.1523/JNEUROSCI.0910-09.2009
69
RosazzaC.MinatiL.GhielmettiF.MandelliM. L.BruzzoneM. G. (2012). Functional connectivity during resting-state functional MR imaging: study of the correspondence between independent component analysis and region-of-interest–based methods. Am. J. Neuroradiol.33, 180–18710.3174/ajnr.A2733
70
RussoF. A.AmmiranteP.FelsD. I. (2012). Vibrotactile discrimination of musical timbre. J. Exp. Psychol. Hum. Percept. Perform. 38:822. 10.1037/a0029046
71
SchnuppJ.NelkenI.KingA. (2011). Auditory Neuroscience: Making Sense of Sound. Cambridge, MA: MIT Press. 10.7551/mitpress/7942.001.0001
72
SchulteA.MarozeauJ.RuheA.BüchnerA.KralA.Innes-BrownH.et al. (2023). Improved speech intelligibility in the presence of congruent vibrotactile speech input. Sci. Rep.13:22657. 10.1038/s41598-023-48893-w
73
Seydell-GreenwaldA.WangX.NewportE. L.BiY.Striem-AmitE. (2023). Spoken language processing activates the primary visual cortex. PLoS ONE 18:e0289671. 10.1371/journal.pone.0289671
74
SharpA.HoudeM. S.MaheuM.IbrahimI.ChampouxF. (2019). Improve tactile frequency discrimination in musicians. Exp. Brain Res. 237, 1–6. 10.1007/s00221-019-05532-z
75
ShethV.YoungR. (2014). Ventral and dorsal streams in cortex: focal vs. ambient processing/exploitation vs. exploration. J. Vision.14:51. 10.1167/14.10.51
76
SmithaK. A.Akhil RajaK.ArunK. M.RajeshP. G.ThomasB.KapilamoorthyT. R.et al. (2017). Resting state fMRI: a review on methods in resting state connectivity analysis and resting state networks. Neuroradiol. J. 30, 305–317. 10.1177/1971400917697342
77
SnirA.CieślaK.OzdemirG.VekslarR.AmediA. (2024). Localizing 3D motion through the fingertips: following in the footsteps of elephants. iScience. 27:109820. 10.1016/j.isci.2024.109820
78
SunX.LiL.DingG.WangR.LiP. (2019). Effects of language proficiency on cognitive control: evidence from resting-state functional connectivity. Neuropsychologia129, 263–275. 10.1016/j.neuropsychologia.2019.03.020
79
TalZ.GevaR.AmediA. (2016). The origins of metamodality in visual object area LO: bodily topographical biases and increased functional connectivity to S1. NeuroImage127, 363–375. 10.1016/j.neuroimage.2015.11.058
80
TamèL.BraunC.LingnauA.SchwarzbachJ.DemarchiG.Li HegnerY.et al. (2012). The contribution of primary and secondary somatosensory cortices to the representation of body parts and body sides: an fMRI adaptation study. J. Cogn. Neurosci.24, 2306–2320. 10.1162/jocn_a_00272
81
TamèL.PavaniF.BraunC.SalemmeR.FarnèA.ReillyK. T.et al. (2015). Somatotopy and temporal dynamics of sensorimotor interactions: evidence from double afferent inhibition. Eur. J. Neurosci. 41, 1459–1465. 10.1111/ejn.12890
82
TieY.RigoloL.NortonI. H.HuangR. Y.WuW.OrringerO.et al. (2014). Defining language networks from resting-state fMRI for surgical planning – a feasibility study. Hum. Brain Mapp. 35, 1018–1030. 10.1002/hbm.22231
83
UddinL. Q.NomiJ. S.Hébert-SeropianB.GhaziriJ.BoucherO. (2017). Structure and function of the human insula. J. Clin. Neurophysiol. 34, 300–306. 10.1097/WNP.0000000000000377
84
Van BékésyG. (1959). Similarities between hearing and skin sensations. Psychol. Rev.66, 1–22. 10.1037/h0046967
85
Van der StoepN.AlaisD. (2020). Motion perception: auditory motion encoded in a visual motion area. Curr. Biol. 30, R775–R778. 10.1016/j.cub.2020.05.010
86
WalkowiakA.KostekB.LorensA.ObryckaA.WasowskiA.SkarzynskiH.et al. (2010). Spread of excitation (SoE) - a non-invasive assessment of cochlear implant electrode placement. Cochlear Implants Int.11, 479–481. 10.1179/146701010X12671177204787
87
WangX.WuQ.EganL.GuX.LiuP.GuH.et al. (2019). Anterior insular cortex plays a critical role in interoceptive attention. Elife 8:e42265. 10.7554/eLife.42265.038
88
WeisenbergerJ. M.PercyM. E. (1995). The transmission of phoneme-level information by multichannel tactile speech perception aids. Ear. Hear.16, 392–406. 10.1097/00003446-199508000-00006
89
WoolnoughO.ForsethK. J.RolloP. S.TandonN. (2019). Uncovering the functional anatomy of the human insula during speech. Elife 8:e53086. 10.7554/eLife.53086.sa2
90
World Medical Association (2013). World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA310, 2191–2194. 10.1001/jama.2013.281053
91
ZhuL.FanY.ZouQ.WangJ.GaoJ. H.NiuZ.et al. (2014). Temporal reliability and lateralization of the resting-state language network. PLoS ONE9:e85880. 10.1371/journal.pone.0085880
92
ZühlsdorffK.DalleyJ. W.RobbinsT. W.Morein-ZamirS. (2023). Cognitive flexibility: neurobehavioral correlates of changing one's mind. Cereb. Cortex. 33, 5436–5446. 10.1093/cercor/bhac431
Summary
Keywords
speech comprehension, tactile aid, multisensory training, fMRI, resting-state functional MRI, cochlear implants
Citation
Cieśla K, Wolak T and Amedi A (2025) Resting-state functional connectivity changes following audio-tactile speech training. Front. Neurosci. 19:1482828. doi: 10.3389/fnins.2025.1482828
Received
18 August 2024
Accepted
04 April 2025
Published
29 April 2025
Volume
19 - 2025
Edited by
Corianne Rogalsky, Arizona State University, United States
Reviewed by
Anna Dondzillo, University of Colorado Anschutz Medical Campus, United States
Alina Schulte, Technical University of Denmark, Denmark
Updates
Copyright
© 2025 Cieśla, Wolak and Amedi.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Katarzyna Cieśla kasia.j.ciesla@gmail.com
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.