Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval

Qunjie Huang, Weina Zhu

Abstract

Cross-subject EEG-to-image retrieval for visual decoding is challenged by subject shift and hubness in the embedding space, which distort similarity geometry and destabilize top-k rankings, making small-k shortlists unreliable. We introduce SATTC (Structure-Aware Test-Time Calibration), a label-free calibration head that operates directly on the similarity matrix of frozen EEG and image encoders. SATTC combines a geometric expert, subject-adaptive whitening of EEG embeddings with an adaptive variant of Cross-domain Similarity Local Scaling (CSLS), and a structural expert built from mutual nearest neighbors, bidirectional top-k ranks, and class popularity, fused via a simple Product-of-Experts rule. On THINGS-EEG under a strict leave-one-subject-out protocol, standardized inference with cosine similarities, L2-normalized embeddings, and candidate whitening already yields a strong cross-subject baseline over the original ATM retrieval setup. Building on this baseline, SATTC further improves Top-1 and Top-5 accuracy, reduces hubness and per-class imbalance, and produces more reliable small-k shortlists. These gains transfer across multiple EEG encoders, supporting SATTC as an encoder-agnostic, label-free test-time calibration layer for cross-subject neural decoding.

SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval

Abstract

Cross-subject EEG-to-image retrieval for visual decoding is challenged by subject shift and hubness in the embedding space, which distort similarity geometry and destabilize top-k rankings, making small-k shortlists unreliable. We introduce SATTC (Structure-Aware Test-Time Calibration), a label-free calibration head that operates directly on the similarity matrix of frozen EEG and image encoders. SATTC combines a geometric expert, subject-adaptive whitening of EEG embeddings with an adaptive variant of Cross-domain Similarity Local Scaling (CSLS), and a structural expert built from mutual nearest neighbors, bidirectional top-k ranks, and class popularity, fused via a simple Product-of-Experts rule. On THINGS-EEG under a strict leave-one-subject-out protocol, standardized inference with cosine similarities, L2-normalized embeddings, and candidate whitening already yields a strong cross-subject baseline over the original ATM retrieval setup. Building on this baseline, SATTC further improves Top-1 and Top-5 accuracy, reduces hubness and per-class imbalance, and produces more reliable small-k shortlists. These gains transfer across multiple EEG encoders, supporting SATTC as an encoder-agnostic, label-free test-time calibration layer for cross-subject neural decoding.
Paper Structure (26 sections, 28 equations, 2 figures, 2 tables)

This paper contains 26 sections, 28 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. EEG-based Visual Decoding and Cross-Subject Generalization
  4. Label-Free Test-Time Adaptation and Calibration
  5. Hubness, CSLS, and Structure-Aware Retrieval Priors
  6. Method
  7. Label-Free Test-Time Calibration: Problem Setup and Overview
  8. Geometric Normalization via Subject-Adaptive Whitening
  9. Adaptive CSLS Geometric Expert for Hubness Mitigation
  10. Structural Expert from Pre-CSLS Similarity Evidence
  11. Product-of-Experts Fusion and Final Scoring
  12. Experiments
  13. Experimental Setup
  14. Dataset and cross-subject protocol.
  15. Evaluation metrics and baselines.
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Overview of cross-subject EEG-to-image retrieval under leave-one-subject-out (LOSO) evaluation and the proposed SATTC (Structure-Aware Test-Time Calibration) head. Frozen EEG and image encoders produce a similarity matrix whose baseline top-$k$ shortlist is hub-dominated; SATTC performs subject-adaptive whitening, adaptive CSLS, and a structural expert with product-of-experts (PoE) fusion to yield calibrated top-$k$ candidates in a label-free test-time setting.
  • Figure 2: Effect of SAW and SATTC on subject shift, hubness, and shortlist quality. (a) Per-subject Top-5 accuracy under LOSO. (b) Class popularity $N_K(c)$. (c) $\Delta$Recall@K over the Std.+SAW baseline. (d) Distribution of per-class Recall@5 for Std.+SAW and SATTC. SAW improves the standardized baseline, while SATTC further reduces hubness and yields more balanced and reliable small-K shortlists.