Dual-Domain Representation Alignment: Bridging 2D and 3D Vision via Geometry-Aware Architecture Search

Abstract

Modern computer vision requires balancing predictive accuracy with real-time efficiency, yet the high inference cost of large vision models (LVMs) limits deployment on resource-constrained edge devices. Although Evolutionary Neural Architecture Search (ENAS) is well suited for multi-objective optimization, its practical use is hindered by two issues: expensive candidate evaluation and ranking inconsistency among subnetworks. To address them, we propose EvoNAS, an efficient distributed framework for multi-objective evolutionary architecture search. We build a hybrid supernet that integrates Vision State Space and Vision Transformer (VSS-ViT) modules, and optimize it with a Cross-Architecture Dual-Domain Knowledge Distillation (CA-DDKD) strategy. By coupling the computational efficiency of VSS blocks with the semantic expressiveness of ViT modules, CA-DDKD improves the representational capacity of the shared supernet and enhances ranking consistency, enabling reliable fitness estimation during evolution without extra fine-tuning. To reduce the cost of large-scale validation, we further introduce a Distributed Multi-Model Parallel Evaluation (DMMPE) framework based on GPU resource pooling and asynchronous scheduling. Compared with conventional data-parallel evaluation, DMMPE improves efficiency by over 70% through concurrent multi-GPU, multi-model execution. Experiments on COCO, ADE20K, KITTI, and NYU-Depth v2 show that the searched architectures, termed EvoNets, consistently achieve Pareto-optimal trade-offs between accuracy and efficiency. Compared with representative CNN-, ViT-, and Mamba-based models, EvoNets deliver lower inference latency and higher throughput under strict computational budgets while maintaining strong generalization on downstream tasks such as novel view synthesis. Code is available at https://github.com/EMI-Group/evonas

Figures (11)

• Figure 1: The EvoNAS unified visual architecture, featuring a searchable hybrid VSS-ViT encoder, a Spatial Mamba decoder, and optional task heads for detection, segmentation, and MDE. Searchable components within the Evo-Mamba and Evo-Transformer blocks define the encoder’s supernet search space, enabling flexible and general-purpose architecture optimization across diverse vision tasks.
• Figure 2: Overview of the two-stage supernet optimization with dual-domain knowledge distillation. Stage 1 pretrains the VSS-ViT supernet encoder on ImageNet-1K to establish a stable initialization. Stage 2 adapts the supernet to the target task by aligning student VSS/ViT features with teacher ViT features through spatial- and frequency-domain distillation, while jointly applying prediction-level and ground-truth supervision.
• Figure 3: Overview of the EvoNAS pipeline. The framework integrates distributed multi-model parallel evaluation with multi-objective evolutionary search, enabling efficient validation of candidate architectures across GPUs and progressive approximation of the Pareto frontier in terms of accuracy, latency, and computational complexity.
• Figure 4: Performance–efficiency trade-offs of EvoNAS across four benchmarks: (a) COCO, (b) ADE20K, (c) KITTI, and (d) NYU v2. The x-axis denotes MACs, the y-axis represents task-specific accuracy metrics (mAP, mIoU, or AbsRel), bubble size indicates parameter count, and color encodes inference latency.
• Figure 5: Qualitative comparisons of results on COCO dataset.