2026-09-02 00:00 Papers Foundations & Methods Translated from EN

OVMI Metric Puts Speech BCI Results on a Common Scale

Summary Researchers have proposed Open-Vocabulary Mutual Information (OVMI), an information-theoretic metric that scores speech brain-computer interfaces on a common scale, and used it to show that accuracy figures computed only over a system's supported vocabulary can overstate how much of a user's intended speech actually gets through. Speech BCIs translate neural activity into language and offer a path to restoring communication for people with paralysis, but systems differ in datasets, recording methods and vocabularies, leaving their reported scores hard to compare. Choosing a vocabulary that maximizes OVMI yielded up to 16.3% relative accuracy improvement across three speech domains; the preprint has not been peer reviewed.

Why it matters Speech BCI results are reported as percentages over a vocabulary each team chooses for itself, so the numbers different groups publish were never strictly comparable. A metric that prices in what the vocabulary leaves out makes those claims rankable, and the finding that vocabulary selection alone buys up to 16.3% relative accuracy suggests how much reported progress has been design choice rather than better decoding. Whether it matters depends on adoption, since a metric only disciplines a field once the leaderboards use it.

BCIwiki (bciwiki.com) — Speech brain-computer interfaces (BCIs) lack a common measure of progress, as systems differ in vocabularies, datasets, and recording methods. Researchers propose Open-Vocabulary Mutual Information (OVMI), an information-theoretic metric to evaluate communication capabilities on a common scale, in a preprint uploaded to arXiv on September 2, 2026.

The study shows that metrics like accuracy and word error rate computed only over the words a system supports can overstate how much of a user's intended speech the system can convey. Selecting a vocabulary to maximize OVMI yielded up to 16.3% relative improvement in accuracy across three speech domains. The method offers a principled way to compare heterogeneous systems, improve vocabulary design, and measure progress. The work is by Dulhan Jayalath, Benjamin Ballyk, and Oiwi Parker Jones, and has not been peer reviewed.

Compiled by BCIwiki from public sources

Sources · 1

arxiv.org 2026-09-02

Decoding Algorithms EEG Speech Decoding

Read original ↗

Suggest a correction Revisions · none 中 / EN