RecycleLoRA: Rank-Revealing QR-Based Dual-LoRA Subspace Adaptation for Domain Generalized Semantic Segmentation

Abstract

Domain Generalized Semantic Segmentation (DGSS) aims to maintain robust performance across unseen target domains. Vision Foundation Models (VFMs) offer rich multi-domain knowledge that can enhance generalization. However, strategies for actively exploiting the rich subspace structures within VFMs remain under-explored, with many existing methods focusing primarily on preserving pre-trained knowledge. Furthermore, their LoRA components often suffer from limited representational diversity and inefficient parameter utilization. We propose RecycleLoRA, which addresses both challenges by employing Rank-Revealing QR Decomposition (RRQR) to systematically exploit VFM's subspace structures and enhance LoRA's representational richness. Our main adapter leverages minor subspace directions identified by RRQR to learn diverse and independent features, achieving competitive performance even when used alone. We further introduce a sub adapter that carefully refines major directions with minimal adjustments, providing complementary improvements to the main adapter's strong baseline performance. This design enables the dual adapters to learn distinct representations without requiring additional regularization losses. Our systematic exploitation of pre-trained subspace structures through RRQR-based initialization leads to superior domain generalization performance. RecycleLoRA achieves state-of-the-art performance on both synthetic-to-real generalization and real-to-real generalization tasks without complex architectures or additional inference latency.

Figures (7)

• Figure 1: Comparison of synthetic-to-real generalization performance (mIoU,%) between our proposed RecycleLoRA and the previous SOTA method, SoMA.
• Figure 2: RecycleLoRA Framework Overview. This figure illustrates the overall workflow of RecycleLoRA. (a) Rank-Revealing QR Decomposition (RRQR) is applied to the pre-trained weight matrix to identify subspace directions ranked by importance. (b) Among the recyclable subspaces selected through RRQR, the minor directions are assigned as initialization values for the main adapter, while the major directions are assigned to the sub adapter. (c) The main adapter's B matrix is initialized with the minor directions, and its A matrix is sparsely initialized by mapping these directions to their corresponding column indices. (d) The sub adapter's B matrix is initialized with the major directions, and its A matrix is sparsely initialized by mapping these directions to their corresponding column indices.
• Figure 3: Cosine similarity heatmaps of LoRA components for (a) RecycleLoRA and (b) SoMA at different ranks (r=16, 32). Left: pairwise similarity among rows of $\mathbf{A}$. Right: pairwise similarity among columns of $\mathbf{B}$. Darker blue colors represent lower similarity.
• Figure 4: Block-wise subspace similarity $\phi$ between main and sub adapters measured using Grassmann Distance on the low-rank matrices. Lower values indicate more orthogonal subspaces.
• Figure 5: Visualization of adapter-induced feature modifications via PCA projection. (a) Input. (b) main adapter PCA Visualization. (c) sub adapter PCA Visualization. The divergent activation patterns reveal complementary feature learning between the dual adapters.