BCIwiki (bciwiki.com) — A study published in Journal of Neural Engineering on September 25, 2026, reports that motor imagery brain-computer interfaces (BCIs) often struggle to identify movement classes absent from their training data using model uncertainty. The study examines out-of-distribution (OOD) detection: identifying and rejecting inputs from unfamiliar classes instead of forcing a classifier to guess among known categories.
The researchers used a Leave-One-Class-Out setup, training a model on some movement classes and testing whether increased uncertainty could identify the held-out, unfamiliar class. They evaluated seven OOD detection methods and one additional method that has been claimed to improve OOD detection quality.
For many BCI users, uncertainty for familiar, in-distribution classes could exceed uncertainty for OOD classes. The authors attribute this pattern to the high intrinsic variability of electroencephalography (EEG) signals. Consequently, many detection methods that perform well in other machine learning domains were ineffective at identifying unfamiliar motor imagery patterns.
OOD detection performance was also correlated with on-task performance. Deep Ensemble and Monte Carlo-Dropout models achieved an on-task area under the receiver operating characteristic curve (AUC) of 0.9 and OOD detection ability up to 0.7. The authors interpret this as evidence that rejecting unfamiliar cognitive states is feasible for models and participants with high task performance. That condition matters, and the correlation does not establish that improving task performance necessarily improves OOD detection.
The authors present Leave-One-Class-Out evaluation as an experimental paradigm for testing responses to unseen motor imagery actions and describe their work as the first motor imagery BCI benchmark for this type of OOD data. Rejecting unfamiliar inputs is proposed as a way to prevent erroneous actions, but the detection findings do not establish improved safety in actual use.