This paper was accepted at the NeurIPS 2025 Basic Brain and Body Models Workshop.
Self-supervised learning (SSL) provides a promising approach for learning electroencephalogram (EEG) representations from unlabeled data, reducing the need for expensive annotations in clinical applications such as sleep staging and seizure detection. Current EEG SSL techniques mainly use mask reconstruction strategies such as mask autoencoder (MAE) to capture local temporal patterns, but position prediction pre-training remains underexplored despite its potential to learn long-range dependencies of neural signals. We introduce PAirwise Relative Shift or PARS pretraining, a novel pretext task that predicts the relative temporal shift between pairs of randomly sampled EEG windows. Unlike reconstruction-based methods that focus on local pattern recovery, PARS facilitates the encoder to capture the relative temporal organization and long-range dependencies inherent in neural signals. Through a comprehensive evaluation of various EEG decoding tasks, we demonstrate that the PARS pre-trained transducer consistently outperforms existing pre-training strategies in a label-efficient transfer learning setting, establishing a new paradigm for self-supervised EEG representation learning.
**Work done during Apple internship
†Stanford University
‡California Institute of Technology
§University of Amsterdam
