The authors note that most existing visual decoding pipelines directly align EEG features with semantic features from pretrained vision models, but EEG signals carry information at multiple levels, and such practice disregards the varying neural visibility of different visual components, leading to cross-modal mismatches. The framework uses layer-wise contrastive learning to select the intermediate CLIP layer that maximizes retrieval performance as NVOL, and builds a hierarchical framework coupling retrieval and generation on it: the retrieval branch fuses multi-NVOL features, aligns them to image embeddings via contrastive learning, and applies CSLS at test time to mitigate hubness; the generation branch reconstructs subject-specific NVOL features from EEG using a conditional diffusion prior, maps them to CLIP space through a lightweight adapter, and drives a pretrained Stable Diffusion XL model. Experiments showed that two-stage NVOL-to-semantic reconstruction outperforms single-stage final-layer diffusion on semantic and structural metrics.
The study was conducted by Minyi Wang, Zhenqin Wu, and Rihui Li.