When Pretraining Makes Subject Identity Easier to Read
EEG foundation models are making neural decoding increasingly transferable. Instead of training a separate model for every dataset and task, pretrained backbones can provide reusable representations for downstream problems including motor imagery, emotion recognition, sleep staging, cognitive-state decoding, and speech decoding.
Our analysis, however, revealed a complication: subject identity can become especially easy to recover after foundation-model encoding.
Linear probes identify subjects more effectively from pretrained EEG tokens than from raw EEG features, while the corresponding improvement in task-label probing is much smaller. In other words, pretraining can sharpen information that describes who produced the signal alongside information relevant to what the downstream task is trying to predict.
For cross-subject decoding, this creates a representation-level burden. Standard fine-tuning sends these mixed representations directly to a classifier, which must learn the task decision while suppressing subject-related structure that may not transfer to unseen people.
We developed DEFT — Disentangled EFM Fine-Tuning — around a simple principle: factorize before classifying.
Rather than asking the prediction head to resolve the entire pretrained mixture, DEFT first reorganizes it into a task-oriented content branch and a subject-consistent style branch, then performs downstream prediction from the content representation.
Across the paper’s primary evaluation — six subject-disjoint datasets and four EEG foundation-model backbones — DEFT improves mean Balanced Accuracy by 0.0460 across 24 matched backbone–dataset settings, with a clustered-bootstrap 95% confidence interval of [0.0240, 0.0745].
The result suggests that downstream adaptation deserves attention in its own right: a strong pretrained representation can still contain structure that is poorly aligned with the decision the downstream model ultimately needs to make.
A Mixed Representation Problem
EEG contains substantial subject-specific variation. Anatomy, functional organization, electrode–skin impedance, and recording habits can all contribute stable individual patterns. Foundation-model pretraining does not necessarily remove these factors; in the settings we evaluated, it can make them more explicitly organized in representation space.
We therefore treat the pretrained representation as containing two functional roles.
Content represents task-relevant neural dynamics intended for downstream prediction. Style represents more stable residual structure organized around subject identity.
These terms describe roles induced by the training objectives. The goal is to give task and subject information different representational responsibilities before classification, rather than assuming that the original pretrained tokens already provide the optimal interface for cross-subject decoding.
Factorize Before Classifying
DEFT is inserted after the pretrained EEG backbone and before the downstream prediction head.
At its core is a Recursive Gaussian Disentangler. The module alternates between refining content and style, updating each branch while accounting for the current estimate of the other. Precision-weighted Gaussian updates provide an uncertainty-aware mechanism for this iterative separation.
The resulting branches are shaped by three complementary signals.
A temporal prior encourages content to retain stronger variation across windows while style remains more stable within a trial. A subject-aware contrastive objective encourages style representations from the same person to remain close. A reconstruction objective discourages useful information from being discarded during the split.
Subject identities are used only during training. At inference time, DEFT requires only the EEG input, and prediction is performed from the content branch.
The central idea is therefore architectural rather than task-specific: resolve part of the representation problem before asking the classifier to solve the prediction problem.
What Changes After Disentanglement
The primary evaluation covers CodeBrain, CBraMod, LaBraM, and CSBrain, using the same pretrained checkpoints, subject-disjoint splits, and backbone-update policies for DEFT and vanilla fine-tuning.
Gains appear across different foundation models and EEG tasks
Across 24 matched backbone–dataset combinations, DEFT improves mean Balanced Accuracy by 0.0460, with gains appearing across all four evaluated backbones.
Some settings show substantially larger changes. With CBraMod, Balanced Accuracy increases from 0.4630 to 0.5579 on BCIC-IV-2a motor imagery and from 0.6316 to 0.7292 on Mental Arithmetic.
The broader benchmark spans motor imagery, emotion recognition, sleep staging, workload assessment, speech decoding, and MDD classification. Secondary metrics generally support the same overall trend, although individual decreases remain in some backbone–dataset settings.
Controls with parameter-matched adapters, alternative subject-aware methods, and component ablations further indicate that the gains are not explained solely by additional model capacity or sampling changes.
The content representation becomes more task-oriented
We also examined whether DEFT changes the structure of the learned representation.
After removing subject-level offsets, the content branch improves within-subject class separation on five of six representative datasets. On Mental Arithmetic, the separation score increases from 0.580 to 1.115; on Mumtaz2016, it rises from 2.291 to 6.352.
BCIC-Speech is the exception: its global separation score decreases despite downstream gains across all four backbones, indicating that this representation diagnostic does not capture every form of useful task structure.
Frozen linear probes provide a complementary result. Across all six evaluated settings, the content branch is more predictive of task labels, while the style branch is more predictive of subject identity.
Together, these results suggest that DEFT changes which information becomes most accessible to the downstream classifier.
What the Current Results Establish
The evidence supports functional specialization between the two branches, rather than a proof of complete or uniquely identifiable disentanglement.
Content consistently becomes relatively more useful for task prediction, while style becomes relatively more useful for subject prediction. However, both branches can still retain overlapping information. “Content” and “style” should therefore be interpreted as operational roles induced by the training objectives, rather than pure latent factors that have been completely separated.
A second boundary comes from DEFT’s temporal inductive prior. The method assumes that, for the evaluated cross-subject tasks, task-relevant content tends to exhibit stronger temporal variation while subject-related style remains comparatively stable within a trial. This prior may be less appropriate when stable information itself carries the downstream label.
Finally, the current subject-aware contrastive objective uses subject identities during training, although they are not required at inference time. Learning similar specialization without explicit subject supervision remains an open direction.
These boundaries also define the scope of the main conclusion: DEFT provides empirical evidence that representation-level factorization can improve cross-subject EEG adaptation, while the learned branches should not be interpreted as a definitive decomposition of the underlying EEG-generating factors.
Looking Forward
DEFT raises a broader question for neural foundation models: what should happen between pretraining and prediction?
Large-scale pretraining can make many factors of variation accessible at once. A downstream task may need only a subset of them, while other highly accessible factors can make the final decision problem harder.
This makes adaptation itself a meaningful research problem.
For EEG, one natural next step is to reduce the reliance on explicit subject identities and explore fully unsupervised disentanglement fine-tuning, which the paper identifies as an important future direction.
More broadly, DEFT suggests a simple way to think about downstream interfaces for neural foundation models: understand what pretraining has made easy to read, identify which factors the task should rely on, and shape the representation before the final prediction stage.
