TUESDAY, January 29, 2019 (HealthDay News). As part of the breakthrough, right from the world of science fiction, the team of researchers used artificial intelligence (AI) to transform brain signals into computer speech.
The feat was accomplished with the help of five epilepsy patients. All were equipped with various types of brain electrodes as part of treating seizures. This allowed researchers to conduct very sensitive brain monitoring, called electrocorticography.
According to the researchers, the end result is a significant leap in the direction of communication between the brain and the computer.
Previous efforts in this direction "were focused on simple computer models that could reproduce sound that sounded about the same as in the original speech, but could not be understood at all," explained the author of the study, Nima Mesgarani. He is an adjunct professor at the Institute of Brain Mind Behavior at Columbia University in New York.
However, the new study used “modern AI” to restore sounds from the brain that were much more comprehensible compared to previous studies, ”said Mesgarani. "This is a huge milestone, and we were not sure that we can achieve it."
Brain activity was monitored while each participant listened to stories and lists of numbers read to them by four different speakers. Brain signal patterns that were recorded while the patients were listening to numbers were blindly entered into a computer algorithm, that is, without any indication of which pattern corresponded to which number.
An artificial intelligence program designed to imitate the nervous structure of the brain, then proceeded to “clear” the sounds produced by the algorithm. The development team noted that this technology is used in Amazon Echo and Apple Siri.
The final product was a sequence of sound tracks with robotic voices, both male and female, that pronounced each number from zero to nine.
During playback, a selected group of 11 listeners found that computer-generated sounds were recognized in about 75 percent of cases, which, according to the team, is a much higher success rate than previously achieved.
“Our algorithm is the first one that generates a sound that is truly understandable to the audience,” said Mesgarani. And this, he added, means that long-term efforts to properly decode the brain are finally being implemented.
“Our voices help to connect us with our friends, family and the world around us, so the loss of voice power due to injury or illness is so destructive. This can happen for a variety of reasons, such as UAS. [amyotrophic lateral sclerosis] or stroke, “resulting in what is known as a“ fixed syndrome, ”he added.
“Our ultimate goal is to develop technologies that can decipher the patient’s inner voice, which cannot speak,” said Mesgarani.
Such innovations also mean better interaction between the brain and the computer, which will open up entirely new platforms for human-machine communication, he added.
In this regard, Mesgarani said that future tests will focus on more complex words and sentence structure. “Ultimately,” he said, “we hope that the system could be part of an implant, similar to those worn by some patients with epilepsy, who translate the imaginary voice of the owner directly into words.”
Dr. Thomas Oxley is director of innovation strategy in the neurosurgery department of the Mount Sinai Hospital Health System in New York. The ability of AI to read a person’s brain “raises important ethical concerns regarding privacy and security that research leaders need to be aware of,” he noted.
However, the breakthrough "is another important step in decoding the brain patterns that underlie thinking," said Oxley.
“This particular job gives hope to people who have difficulty conveying thoughts due to injury or illness,” he said.
The findings were published on January 29th in the journal. Scientific reports,
Even more about the lockdown syndrome at the National Center for the Advancement of Translational Sciences in the USA.
SOURCES: Nima Mesgarani, Ph.D., associate professor, Tsukerman Institute for Brain Behavioral Behavior and Electrical Engineering, Columbia University, New York; Thomas Oxley, MD, MD, clinical instructor and director of innovation strategy, Department of Neurosurgery, Health System, Mount Sinai Hospital, New York; January 29, 2019 Scientific reports