UrbanMoE: A Sparse Multi-Modal Mixture-of-Experts Framework for Multi-Task Urban Region Profiling

TL;DR

UrbanMoE addresses the lack of standardized benchmarks for multimodal, multi-task urban region profiling by introducing a sparse multi-modal mixture-of-experts framework and a three-city benchmark. It jointly models carbon emissions, population, and nightlight from multisource inputs—satellite imagery, POI embeddings, and LLM-generated text—via dual-branch SME with task-aware routing, training with a multi-task regression loss $\mathcal{L}$. The key contributions are (i) a standardized multi-modal, multi-task benchmark across three cities, (ii) the UrbanMoE architecture featuring specialized, dual-task, and shared experts with a sparse router and per-task fusion, and (iii) extensive experiments and visualizations demonstrating improved predictive performance, efficiency, and interpretability. The approach yields state-of-the-art results with substantial gains in $R^2$ and reductions in $RMSE$/$MAE$, while maintaining a compact parameter footprint and rapid convergence, enabling practical urban analytics at scale.$

Abstract

Urban region profiling, the task of characterizing geographical areas, is crucial for urban planning and resource allocation. However, existing research in this domain faces two significant limitations. First, most methods are confined to single-task prediction, failing to capture the interconnected, multi-faceted nature of urban environments where numerous indicators are deeply correlated. Second, the field lacks a standardized experimental benchmark, which severely impedes fair comparison and reproducible progress. To address these challenges, we first establish a comprehensive benchmark for multi-task urban region profiling, featuring multi-modal features and a diverse set of strong baselines to ensure a fair and rigorous evaluation environment. Concurrently, we propose UrbanMoE, the first sparse multi-modal, multi-expert framework specifically architected to solve the multi-task challenge. Leveraging a sparse Mixture-of-Experts architecture, it dynamically routes multi-modal features to specialized sub-networks, enabling the simultaneous prediction of diverse urban indicators. We conduct extensive experiments on three real-world datasets within our benchmark, where UrbanMoE consistently demonstrates superior performance over all baselines. Further in-depth analysis validates the efficacy and efficiency of our approach, setting a new state-of-the-art and providing the community with a valuable tool for future research in urban analytics

Figures (6)

• Figure 1: Multimodal multi-task urban region profiling dataset construction.
• Figure 2: The proposed UrbanMoE framework, including (a) overall model architecture and (b) hierarchical expert routing.
• Figure 3: Hyper-parameter analysis results.
• Figure 4: T-SNE visualization of multimodal expert embeddings across tasks and modalities.
• Figure 5: Visualization of sparse gating routing weights.