Speech Decoding

Researchers affiliated with Araya Inc. and Sincxpress Education SRL examined a model that predicts spoken words from ultra-high-density EEG to assess whether decoding reflects neural activity rather than electrical activity from speaking muscles. Electrode contributions could not be explained by their mutual information with EMG, and the periods contributing to EEG decoding differed from those supporting EMG-only decoding. Contributing periods partially overlapped between overt and minimally overt speech. Denoising increased contributions from electrodes in speech-related brain areas across all speech conditions. The authors interpret these findings as supporting neural speech decoding with appropriately preprocessed EEG.

Oxford researchers compared six aggregation strategies; training-time logit averaging led in six of seven settings. Aggregating 100 same-phoneme trials yielded 64.1% balanced accuracy in a public-test-split re-evaluation, 17.4 points above the competition champion model. The preprint experiments use MEG recordings.

Researchers proposed BEST, a two-stage method that converts neural activity into text, to examine whether speech brain-computer interfaces need large models. On the Brain-to-Text '24 and '25 benchmarks, BEST achieved word error rates of 6.19% and 1.62%, respectively, using only 3.85% of the parameters required by comparable methods. Word error rate measures errors relative to reference text; lower values indicate fewer errors. The authors argue that larger decoders do not consistently perform better and that alignment between neural signals, speech, and text matters more than model size. These benchmark results do not directly establish everyday communication performance.

Researchers audited three public corpora and found that decoding overt speech may not establish a pipeline's ability to recognize brain-derived speech signals. In ds003626, two lip electrodes achieved an accuracy of 0.7430, compared with 0.4231 for the 128-channel scalp montage; no decodable signal remained in the 8-30 Hz band. By injecting effects of known amplitude into real recordings, the team also measured detection floors and found differences between methods. The authors conclude that muscle signals dominate the overt-speech positive controls in these corpora and recommend reporting band-limited performance, artifact-channel accuracy, and measured detection floors.

Researchers analyzed three public corpora to test whether imagined speech decoders trained on one person's scalp EEG could work for another. Across 482 ordered participant pairs, none remained significant after multiple-comparison correction; mean transfer accuracy was within 0.006 of chance in every corpus. A signal distinguishing the imagery task from pre-stimulus rest instead reflected a post-stimulus state. Only the competition corpus exceeded the within-subject threshold for every participant, but its frequency profile failed an artifact audit. The findings apply to these corpora and the representations examined, not to all imagined speech decoding.

Researchers aligned EEG recorded during imagined speech with text representations to exploit semantic relationships beyond fixed intent labels. Evaluated on five subjects and 39 intent categories in the Chisco dataset, the framework performed comparably to a task-optimized closed-set classifier. Maximum EEG-text similarity identified held-out trial subsets with higher Top-1 and Top-5 accuracy. Mean agreement exceeded the 50% chance reference on all five semantic dimensions. Comparisons with unseen candidate sentences were qualitative demonstrations, not evidence of unrestricted imagined-sentence decoding.

Researchers used natural-language class descriptions to help models learn the correspondence between brain signals and target classes, evaluating the approach in visual, motor, and speech decoding. The study tested seven encoder architectures with independently trained models using scalp and intracranial recordings. Speech-decoding accuracy improved by up to 17.15 percentage points over matched class-label-supervision baselines. The learned representations also supported candidate-set reuse and label-supervised subject and session adaptation. In steady-state visual evoked potential tasks, the method enabled unseen-frequency decoding under specific conditions. These findings do not establish a single model that performs every task or adapts to new subjects without labels. This study is a preprint and has not been peer reviewed.

A contextual prior can raise overall accuracy in MEG speech decoding while making some correctable errors more confident, a preprint reports. Tracking candidate scores across two magnetoencephalography datasets, the authors describe a reversal: when the correct candidate starts near the top of the local ranking, a larger fused margin makes a repair more likely, but when it starts lower, a larger margin makes the error more likely to persist. On MEG-MASC, pooled correctness AUROC is 0.87, while the AUROC separating repairs from residual errors falls from 0.70 to 0.39; errors starting beyond rank 20 account for 46.6% of post-fusion errors. The authors argue confidence should retain local and contextual evidence rather than fused scores alone; reading the two scores separately lifts decoder coverage to 74.5% of windows from 56.7%. The study is a preprint and has not been peer reviewed.

A research team has used a single high-density electrocorticography (ECoG) implant, an electrode array placed on the brain's surface, to decode speech and gestures simultaneously in people with paralysis, ApplySci reported. When participants simply attempted to speak or gesture, a personalized virtual avatar spoke or moved accordingly, either both at once or separately. The researchers expected the brain signals during combined expression to be a simple sum of the two, but the data differed clearly from speech or gestures alone. Participants practiced a limited vocabulary of up to 10 gestures and 10 words or phrases.

Speech brain-computer interfaces aim to restore communication by turning neural activity tied to speech, language or communicative intent into text, synthesized voice or avatar movements. This review argues they are not simple neural-to-text decoders but clinical systems that change over time, in which neural representations, recording hardware, decoding models, language models, feedback and the user's own learning interact. The authors say the next generation should be judged not only on offline accuracy but on stability across sessions, calibration burden, latency, the ability to signal uncertainty, usability and safeguards against decoding words a user did not mean to say. The paper is a preprint and has not been peer reviewed.

Gabriele Cinà reported in an arXiv preprint posted on September 27, 2026, that Jev, a hosted model, can replace the multi-billion-parameter language model that speech neuroprostheses use to rescore candidate sentences, the only GPU-dependent step in the system. Jev returns a probability for each candidate in one call, which is combined with the decoder's own score. On 978 held-out sentences from a participant with ALS, the published decoder alone had an 8.1% word error rate, versus 7.5% with Jev and 7.8% with both OPT-6.7b and Qwen2.5-7B; after re-tuning decoder weights, the figures were 6.9%, 7.2% and 7.4% respectively. Jev needs no GPU, costs $0.07 per thousand sentences, and has 262 ms end-to-end latency over the internet, versus 27 ms for a 7B model on a local GPU. The work is a preprint and has not been peer reviewed.

Researchers have shown that language models do more to help decode imagined speech than heard speech from non-invasive brain recordings. Imagined speech is harder to read than speech a person hears: the brain's response is weaker and noisier, and less data is available. Using a magnetoencephalography (MEG) dataset in which participants both listened to and imagined the same material, the team brought a language model into both training and decoding. Imagined speech gained more from the model than heard speech, and its best results required leaning more heavily on the model. The authors conclude that the weaker the neural evidence, the more useful a language model's contextual prior becomes. The study is a preprint and has not been peer reviewed.

Researchers have proposed SICMD, a method that combines two non-invasive brain signals, functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG), to identify speech a person is hearing from brain activity with a single model that works across different people. fMRI pinpoints where activity occurs and MEG captures when, together giving fuller spatiotemporal information. In cross-subject perceived-speech decoding, the method improved Top-1, Top-10 and Rankacc by more than 10.6%, 10.1% and 1.7% over baseline methods, and cut training costs by 88.8% and 60.5% compared with multi-subject and intra-subject decoding respectively. The work is a preprint and has not been peer reviewed.

Researchers re-examined a multi-class imagined speech EEG dataset, words spoken silently in the mind, under a strictly subject-independent protocol in which a model is never trained on data from the person it is tested on. Spectral bandpower features reached a mean trial-wise accuracy of 49.03% ± 4.18% in coarse-level classification, clearly ahead of the 37.97% ± 3.79% from time-domain statistical features, and a few frequency bands carried most of the discriminative information. The study is a preprint and has not been peer reviewed.

Decoding individual words from scalp EEG and MEG is hard because word-level neural evidence is weak, spread out in time, and entangled with acoustic, lexical and semantic structure. Deep learning can already decode language from intracranial recordings, but extending that to non-invasive recordings remains unsolved. Hierarchical Dynamic Neural Decoding (HDND), proposed by Yueyang Li, Shuran Chen, Wai Ting Siok and Nizhuan Wang, treats word decoding as step-by-step refinement rather than one-shot label retrieval. Across seven EEG and MEG datasets in English, Dutch, Mandarin and Cantonese, covering nine listening, reading and reading-aloud conditions, HDND beat the matched contextual baseline on participant-averaged balanced Top-10 accuracy in all nine and had the highest mean of all compared methods in eight. The work is a preprint and has not been peer reviewed.

This review traces the evolution of P300 brain-computer interfaces from classic spellers to AI agents. The P300 is a positive brain potential that appears about 300 milliseconds after a rare stimulus, and systems use it to determine which character a user wants to select. According to the review, recent systems such as ChatBCI and MindChat, which pair P300 spellers with large language models, cut keystrokes by more than 62% and nearly triple communication speed. The author also proposes an end-to-end architecture: an EEG headset and a real-time CNN detector, topped by an AI-agent layer made up of an LLM planner and IoT control interfaces.

Shanghai-based StairMed has taken its wireless implantable brain-computer interface system into human evaluation for people with speech impairments. The prospective, single-arm trial (NCT07647315) actually began on September 1, 2026, is now recruiting, and plans to enroll 4 participants. Subjects will undergo surgical implantation and brain-control training, followed by an expected 12 months of follow-up recording adverse events, serious adverse events and device deficiencies, along with efficacy data on brain-controlled interaction and communication. Primary completion is expected on August 30, 2028, and the study as a whole on December 30, 2028. The registry record does not disclose the implant site, decoding method or detailed eligibility criteria, and no interim results have been posted, so conclusions on safety and efficacy will have to wait for follow-up data.

P300 BCI spellers are often built by switching on every component that works on its own, on the assumption that more is better. A preprint study tested that assumption with a four-component full-factorial experiment and found that a component's value is conditional, not additive. Subject calibration was the strongest single contributor; Euclidean Alignment made up for the lack of calibration in zero-calibration settings; and stacking components that are each useful on their own could lower performance, an effect the authors call component anti-synergy. Language model support was not universally beneficial either: its effect depended strongly on how strong the underlying EEG pipeline was.

Ten native Mandarin speakers read aloud six single-vowel rhymes while optically pumped magnetometer magnetoencephalography (OPM-MEG) and EEG recorded the same task. In the 150-500 ms window after stimulus onset, OPM-MEG decoded significantly more accurately than EEG with all five classifiers tested. The best-performing combination, common spatial pattern features with a linear support vector machine, reached a mean accuracy of 60.3% for OPM-MEG versus 56.7% for EEG. That suggests speech BCIs that need no implanted electrodes and do not restrict head movement may have a clearer signal path than EEG.

Most speech brain-computer interfaces for people who have lost the ability to speak first decode neural signals into text and then read that text aloud, a two-step chain that adds delay and passes errors from one stage to the next. Brain2Speech-Net skips the text step, mapping long neural recordings directly into the internal representation of a speech synthesis model. The authors report it was the only system in their comparison to produce intelligible speech while generating faster than real time. The work is a preprint and has not been peer reviewed.

A preprint posted to arXiv on September 3, 2026, sets out the 2026 PNPL competition, built on an extended LibriBrain100 dataset that adds 32 subjects at about 40 minutes each and roughly 80 more hours of within-subject data. The competition runs two tracks: a Deep track for within-subject word classification at scale, and a Broad track for cross-subject generalization that steps subject-specific fine-tuning data down from about 40 minutes to 20 and then 10, a clinically feasible range. Winning 2025 submissions reached F1-macro scores of 95.6% on speech detection and 73.6% on phoneme classification.

Researchers have proposed Open-Vocabulary Mutual Information (OVMI), an information-theoretic metric that scores speech brain-computer interfaces on a common scale, and used it to show that accuracy figures computed only over a system's supported vocabulary can overstate how much of a user's intended speech actually gets through. Speech BCIs translate neural activity into language and offer a path to restoring communication for people with paralysis, but systems differ in datasets, recording methods and vocabularies, leaving their reported scores hard to compare. Choosing a vocabulary that maximizes OVMI yielded up to 16.3% relative accuracy improvement across three speech domains; the preprint has not been peer reviewed.

A preprint from Minyi Wang, Zhenqin Wu and Rihui Li reports that aligning EEG signals to an intermediate CLIP layer rather than the final one raised 200-way image retrieval on the THINGS-EEG dataset to 78.1% mean Top-1 accuracy, and to 86.4% with CSLS. Layer-wise contrastive learning selects that layer, which the authors call the Neural Visibility Optimal Layer (NVOL); the same representation then drives generation, with a conditional diffusion prior reconstructing subject-specific NVOL features and mapping them into CLIP space for Stable Diffusion XL, which the authors say beats single-stage final-layer diffusion on semantic and structural metrics. The paper was posted to arXiv on September 2, 2026, and has not been peer reviewed.

LibriBrain100, a magnetoencephalography (MEG) dataset released on August 25, 2026, contains more than 100 hours of high-quality recordings, about 80 of them from a single subject, the deepest within-subject collection of its kind. It is meant as a standardized benchmark for neural speech decoding and ships with open-source tooling and an online competition; a further 32 subjects contribute roughly 40 minutes each to offset thin per-subject data. Using an existing decoding model, the team reported state-of-the-art results on a word-classification benchmark, which it takes as evidence of both data quality and the value of deep within-subject recording.

Thirty-eight typically developing children aged 8 to 12 used a design application to build their own picture-based interface for a P300 brain-computer interface augmentative and alternative communication (P300-BCI-AAC) system, in a study of what children themselves want from such interfaces. They consistently chose preferred colors, animation — zooming most of all — and sound cues: animation aided visual accessibility and target location, background color changes carried preferred colors into the display, and video GIFs and picture overlays added personal relevance. The authors call motion, color, personalization and visual clarity preliminary priorities for pediatric BCI-AAC design, and point to follow-up work with children who use AAC daily and with people who have motor difficulties.

A new study proposes an interpretable brain-computer interface framework that combines affective state recognition with EEG decoding to enable emotion-aware thought-to-speech. Tested on public imagined-speech EEG datasets in subject-independent settings and scored on accuracy, precision, recall, F1-score, inference speed and interpretability, the framework improved decoding reliability over conventional opaque models and produced clinically meaningful explanations, the authors report. They present it as a practical basis for assistive communication tools for people with paralysis, amyotrophic lateral sclerosis, locked-in syndrome and other conditions that disrupt natural speech.