Beyond Weight Adaptation: Feature-Space Domain Injection for Cross-Modal Ship Re-Identification

TL;DR

This paper tackles CMS Re-ID under large modality gaps by shifting from weight-space PEFT to feature-space calibration. It introduces Domain Representation Injection (DRI), combining a lightweight Offset Encoder and per-layer Modulators to inject additive domain deviations into a frozen Vision Foundation Model, preserving general knowledge while adapting to cross-modal tasks. Empirical results on HOSS-ReID and CMShipReID show SOTA performance with an order of magnitude fewer trainable parameters, highlighting the efficiency and robustness of feature-space adaptation for cross-modal re-identification. The approach delivers strong, interpretable improvements across optical-SAR and VIS-NIR/TIR settings, with extensive ablations validating architectural choices and injection strategies.

Abstract

Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking, yet it is fundamentally challenged by significant modality discrepancies. Mainstream solutions typically rely on explicit modality alignment strategies; however, this paradigm heavily depends on constructing large-scale paired datasets for pre-training. To address this, grounded in the Platonic Representation Hypothesis, we explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps. Recognizing the suboptimal performance of existing generic Parameter-Efficient Fine-Tuning (PEFT) methods that operate within the weight space, particularly on limited-capacity models, we shift the optimization perspective to the feature space and propose a novel PEFT strategy termed Domain Representation Injection (DRI). Specifically, while keeping the VFM fully frozen to maximize the preservation of general knowledge, we design a lightweight, learnable Offset Encoder to extract domain-specific representations rich in modality and identity attributes from raw inputs. Guided by the contextual information of intermediate features at different layers, a Modulator adaptively transforms these representations. Subsequently, they are injected into the intermediate layers via additive fusion, dynamically reshaping the feature distribution to adapt to the downstream task without altering the VFM's pre-trained weights. Extensive experimental results demonstrate the superiority of our method, achieving State-of-the-Art (SOTA) performance with minimal trainable parameters. For instance, on the HOSS-ReID dataset, we attain 57.9\% and 60.5\% mAP using only 1.54M and 7.05M parameters, respectively. The code is available at https://github.com/TingfengXian/DRI.

Figures (6)

• Figure 1: Schematic illustration of the Cross-Modality Ship Re-Identification task. Take optical-SAR Ship Re-ID as an example, the process is divided into three stages, frames of the same color indicate that the ids of Ships are the same.
• Figure 2: Comparison between Weight-based PEFT and Feature-based PEFT. (a) Reparameterization-based PEFT (e.g., LoRA) reconstructs the original weight matrix $W$ of the VFM by introducing a low-rank incremental matrix $\Delta W$. (b) Additive PEFT (e.g., Adapter) introduces new weight matrices and integrates them into the original network, thereby modifying the model's topology. Both (a) and (b) operate within the weight space. (c) Our proposed Feature-based PEFT introduces external signals to explicitly reshape the feature distribution. This approach maximizes the preservation of general knowledge without altering the weight structure of the VFM.
• Figure 3: Overview of the proposed Domain Representation Injection (DRI) framework. The architecture comprises two primary components: a fully frozen VFM (initialized with DINOv3) and a trainable Domain Representation Injector. Given an input image, the Offset Encoder (OE) extracts the domain-specific representation $f_d$. Subsequently, Modulators (Mod) adaptively transform $f_d$ into specific feature deviations $\Delta x$ based on the context of different layers. These deviations are injected into the Attention and MLP layers of the VFM via additive fusion to reshape the feature distribution. The model is trained using a combination of Triplet Loss and ID Loss.
• Figure 4: Illustration of the proposed Domain Representation Injector. For the $l$-th block of the VFM with input feature $\mathbf{x}_{l-1}$, the domain representation $f_d$ is first derived via the Offset Encoder. Subsequently, $f_d$ is processed by the modulators $\text{Mod}_{\text{attn}}$ and $\text{Mod}_{\text{mlp}}$ to generate the specific feature deviations $\Delta \mathbf{x}^{\ell}_{\text{attn}}$ and $\Delta \mathbf{x}^{\ell}_{\text{mlp}}$, respectively. Finally, $\Delta \mathbf{x}^{\ell}_{\text{attn}}$ is injected into $\mathbf{x}_{l-1}$ via additive fusion, while $\Delta \mathbf{x}^{\ell}_{\text{mlp}}$ is added to the intermediate feature $\mathbf{x}'_{l}$.
• Figure 5: Schematic illustration of different injection positions investigated in the ablation study.Post-Norm (Left): Our proposed strategy where $\Delta \mathbf{x}$ is injected after Layer Normalization, serving as a direct calibration signal for the subsequent modules. Pre-Norm (Middle): Injection is performed before Layer Normalization, where the learned deviation risks being suppressed by the normalization statistics. Residual Only (Right): Injection is applied solely to the residual connections, shifting the output distribution without modulating the internal feature extraction process.