BCIwiki (bciwiki.com) — Two widely cited EEG foundation models fail to match supervised decoders on motor imagery, and the benchmarks used to evaluate them carry a systematic confound, according to a preprint uploaded to arXiv on September 20, 2026 by Kevin Zhou and Sparsh Roy. Under a validation-locked protocol on the four-class BCI Competition IV-2a dataset, every supervised comparator — including ATCNet and EEG Conformer — outperformed every foundation-model configuration of LaBraM and CBraMod, even after validation-selected fine-tuning.
The study identifies a key confound: foundation models and task-specific decoders are normally evaluated using different input pipelines. When the authors retrained three supervised architectures on the same broadband arrays consumed by the foundation models, accuracy differences flipped sign across architectures — broadband input improved ATCNet by 0.078 accuracy while reducing EEG Conformer accuracy by 0.088. None of the three individual matched-input terms reached significance after multiple-comparison correction at n = 9, so the authors treat the sign variation descriptively. They conclude that a single comparator may not provide an architecture-invariant decomposition of a pretrained-versus-supervised performance gap. The four-class deficit did not reproduce uniformly across datasets: on two-class BNCI2014-004, the separation between fine-tuned CBraMod and supervised comparators was undetectable. Validation-fitted temperature scaling restored foundation-model calibration error to the supervised range despite substantially lower four-class accuracy. The preprint has not been peer reviewed.