Impact Factor 3.648 | CiteScore 3.99
More on impact ›

Original Research ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Neurosci. | doi: 10.3389/fnins.2019.01267

Generating Natural, Intelligible Speech from Brain Activity in Motor, Premotor and Inferior Frontal Cortices

  • 1Maastricht University, Netherlands
  • 2University of Bremen, Germany
  • 3Northwestern University, United States
  • 4Virginia Commonwealth University, United States

Neural interfaces that directly produce intelligible speech from brain activity would allow people with severe impairment from neurological disorders to communicate more naturally.
Here, we record neural population activity in motor, premotor and inferior frontal cortices during speech production using electrocorticography (ECoG) and show that ECoG signals alone can be used to generate intelligible speech output that can preserve conversational cues.

To produce speech directly from neural data, we adapted a method from the field of speech synthesis called unit selection, in which units of speech are concatenated to form audible output. In our approach, which we call \emph{Brain-To-Speech}, we chose subsequent units of speech based on the measured ECoG activity to generate audio waveforms directly from the neural recordings.
\emph{Brain-To-Speech} employed the user's own voice to generate speech that sounded very natural and included features such as prosody and accentuation. By investigating the brain areas involved in speech production separately, we found that speech motor cortex provided more information for the reconstruction process than the other cortical areas.

Keywords: Speech, BCI (Brain Computer Interface), ECoG, Synthesis, brain-to-speech

Received: 03 May 2019; Accepted: 07 Nov 2019.

Copyright: © 2019 Herff, Diener, Angrick, Mugler, Tate, Goldrick, Krusienski, Slutzky and Schultz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Dr. Christian Herff, Maastricht University, Maastricht, Netherlands, c.herff@maastrichtuniversity.nl