BCIwiki (bciwiki.com) — Large language models that correct error-prone text from P300 brain-computer interface spellers can silently swap a user's intended message for a different one, a preprint uploaded to medRxiv on August 27, 2026, found. The researchers, whose affiliations include Rambam Health Care Campus, call this failure mode intent drift. In a simulated benchmark spanning 20 open-weight LLMs and five levels of decoder corruption (0-40% character error rate), drift rose from 2.2% to 60.3%, with an odds ratio of 2.30 per 10-percentage-point increase in error. The 40% error rate represents a stress-test ceiling, not an expected real-world rate, the authors noted. The study has not been peer reviewed.
The benchmark drew on the full ALS message-banking vocabulary and a set of clinically important messages such as medication doses, paired with matched controls. An automated scoring system benchmarked against physician judgments classified each of 4,252,326 model outputs as faithful, degraded, or drifted. Model confidence distinguished correct from incorrect outputs with an AUROC of 0.83 but was poorly calibrated: 28.4% of high-confidence outputs were not faithful to the user's intent. Clinically important messages drifted slightly more than matched controls (odds ratio 1.10). Physician review of a sample showed moderate agreement with automated scoring (kappa 0.41); adjusting for this disagreement lowered drift estimates but did not eliminate the pattern.
A separate subanalysis compared 6 correction strategies across 7 models. No strategy removed drift: cautious approaches reduced it while permissive ones increased it, and even the best strategy still produced a wrong message 18 times per 100 corrections. The authors concluded that evaluations of LLM-assisted BCI communication should report intent drift and confidence calibration alongside speed and accuracy, and that prospective human-in-the-loop testing is needed before deployment. The research was funded by Harvard Catalyst (CTSA UL1TR002541).