BCIwiki (bciwiki.com) — A method that fuses functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) improved cross-subject decoding of perceived speech, raising Top-1, Top-10 and Rankacc by more than 10.6%, 10.1% and 1.7% over baseline methods, according to a preprint posted to arXiv on September 25, 2026. The paper, by Aoke Zhang and Jing Chen, has been submitted to ICASSP 2027 and has not been peer reviewed.
The authors say perceived speech decoding from non-invasive brain-computer interface (BCI) signals faces two main challenges: extracting neural representations with rich spatiotemporal information, and generalizing across subjects. Earlier studies addressed these problems separately. The proposed Subject-Invariant Cross-Modal Perceived Speech Decoding (SICMD) method is designed to tackle both at once.
The team analyzed fusion method, fusion position, encoder architecture and model inputs. According to the paper, SICMD reduced training costs by 88.8% and 60.5% compared with multi-subject and intra-subject decoding settings, and visualization experiments supported the results.
The method decodes speech that participants hear, not speech they intend to produce. fMRI and MEG systems are large and are currently used mainly in laboratory research.