Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The RoboSense Challenge: Sense Anything, Navigate Anywhere, Adapt Across Platforms

Lingdong Kong, Shaoyuan Xie, Zeying Gong, Ye Li, Meng Chu, Ao Liang, Yuhao Dong, Tianshuai Hu, Ronghe Qiu, Rong Li, Hanjiang Hu, Dongyue Lu, Wei Yin, Wenhao Ding, Linfeng Li, Hang Song, Wenwei Zhang, Yuexin Ma, Junwei Liang, Zhedong Zheng, Lai Xing Ng, Benoit R. Cottereau, Wei Tsang Ooi, Ziwei Liu, Zhanpeng Zhang, Weichao Qiu, Wei Zhang, Ji Ao, Jiangpeng Zheng, Siyu Wang, Guang Yang, Zihao Zhang, Yu Zhong, Enzhu Gao, Xinhan Zheng, Xueting Wang, Shouming Li, Yunkai Gao, Siming Lan, Mingfei Han, Xing Hu, Dusan Malic, Christian Fruhwirth-Reisinger, Alexander Prutsch, Wei Lin, Samuel Schulter, Horst Possegger, Linfeng Li, Jian Zhao, Zepeng Yang, Yuhang Song, Bojun Lin, Tianle Zhang, Yuchen Yuan, Chi Zhang, Xuelong Li, Youngseok Kim, Sihwan Hwang, Hyeonjun Jeong, Aodi Wu, Xubo Luo, Erjia Xiao, Lingfeng Zhang, Yingbo Tang, Hao Cheng, Renjing Xu, Wenbo Ding, Lei Zhou, Long Chen, Hangjun Ye, Xiaoshuai Hao, Shuangzhi Li, Junlong Shen, Xingyu Li, Hao Ruan, Jinliang Lin, Zhiming Luo, Yu Zang, Cheng Wang, Hanshi Wang, Xijie Gong, Yixiang Yang, Qianli Ma, Zhipeng Zhang, Wenxiang Shi, Jingmeng Zhou, Weijun Zeng, Kexin Xu, Yuchen Zhang, Haoxiang Fu, Ruibin Hu, Yanbiao Ma, Xiyan Feng, Wenbo Zhang, Lu Zhang, Yunzhi Zhuge, Huchuan Lu, You He, Seungjun Yu, Junsung Park, Youngsun Lim, Hyunjung Shim, Faduo Liang, Zihang Wang, Yiming Peng, Guanyu Zong, Xu Li, Binghao Wang, Hao Wei, Yongxin Ma, Yunke Shi, Shuaipeng Liu, Dong Kong, Yongchun Lin, Huitong Yang, Liang Lei, Haoang Li, Xinliang Zhang, Zhiyong Wang, Xiaofeng Wang, Yuxia Fu, Yadan Luo, Djamahl Etchegaray, Yang Li, Congfei Li, Yuxiang Sun, Wenkai Zhu, Wang Xu, Linru Li, Longjie Liao, Jun Yan, Benwu Wang, Xueliang Ren, Xiaoyu Yue, Jixian Zheng, Jinfeng Wu, Shurui Qin, Wei Cong, Yao He

TL;DR

RoboSense 2025 presents a comprehensive benchmark to study robustness and generalization in robot sensing across diverse platforms, modalities, and adverse conditions. By unifying five tracks—Driving with Language, Social Navigation, Sensor Placement, Cross-Modal Drone Navigation, and Cross-Platform 3D Object Detection—the paper evaluates perception reliability under domain shifts, sensor failures, and platform discrepancies using standardized datasets, baselines, and evaluation protocols. Across 143 teams and 85 institutions, the competition reveals key trends toward data-centric robustness, geometry-aware representations, and modular, parameter-efficient adaptation, while also highlighting persistent challenges in compound shifts and safe, reliable operation. The findings advance the field toward perception systems that can sense reliably, reason with grounded language, and adapt across heterogeneous robotic platforms in real-world environments.

Abstract

Autonomous systems are increasingly deployed in open and dynamic environments -- from city streets to aerial and indoor spaces -- where perception models must remain reliable under sensor noise, environmental variation, and platform shifts. However, even state-of-the-art methods often degrade under unseen conditions, highlighting the need for robust and generalizable robot sensing. The RoboSense 2025 Challenge is designed to advance robustness and adaptability in robot perception across diverse sensing scenarios. It unifies five complementary research tracks spanning language-grounded decision making, socially compliant navigation, sensor configuration generalization, cross-view and cross-modal correspondence, and cross-platform 3D perception. Together, these tasks form a comprehensive benchmark for evaluating real-world sensing reliability under domain shifts, sensor failures, and platform discrepancies. RoboSense 2025 provides standardized datasets, baseline models, and unified evaluation protocols, enabling large-scale and reproducible comparison of robust perception methods. The challenge attracted 143 teams from 85 institutions across 16 countries, reflecting broad community engagement. By consolidating insights from 23 winning solutions, this report highlights emerging methodological trends, shared design principles, and open challenges across all tracks, marking a step toward building robots that can sense reliably, act robustly, and adapt across platforms in real-world environments.

The RoboSense Challenge: Sense Anything, Navigate Anywhere, Adapt Across Platforms

TL;DR

RoboSense 2025 presents a comprehensive benchmark to study robustness and generalization in robot sensing across diverse platforms, modalities, and adverse conditions. By unifying five tracks—Driving with Language, Social Navigation, Sensor Placement, Cross-Modal Drone Navigation, and Cross-Platform 3D Object Detection—the paper evaluates perception reliability under domain shifts, sensor failures, and platform discrepancies using standardized datasets, baselines, and evaluation protocols. Across 143 teams and 85 institutions, the competition reveals key trends toward data-centric robustness, geometry-aware representations, and modular, parameter-efficient adaptation, while also highlighting persistent challenges in compound shifts and safe, reliable operation. The findings advance the field toward perception systems that can sense reliably, reason with grounded language, and adapt across heterogeneous robotic platforms in real-world environments.

Abstract

Autonomous systems are increasingly deployed in open and dynamic environments -- from city streets to aerial and indoor spaces -- where perception models must remain reliable under sensor noise, environmental variation, and platform shifts. However, even state-of-the-art methods often degrade under unseen conditions, highlighting the need for robust and generalizable robot sensing. The RoboSense 2025 Challenge is designed to advance robustness and adaptability in robot perception across diverse sensing scenarios. It unifies five complementary research tracks spanning language-grounded decision making, socially compliant navigation, sensor configuration generalization, cross-view and cross-modal correspondence, and cross-platform 3D perception. Together, these tasks form a comprehensive benchmark for evaluating real-world sensing reliability under domain shifts, sensor failures, and platform discrepancies. RoboSense 2025 provides standardized datasets, baseline models, and unified evaluation protocols, enabling large-scale and reproducible comparison of robust perception methods. The challenge attracted 143 teams from 85 institutions across 16 countries, reflecting broad community engagement. By consolidating insights from 23 winning solutions, this report highlights emerging methodological trends, shared design principles, and open challenges across all tracks, marking a step toward building robots that can sense reliably, act robustly, and adapt across platforms in real-world environments.
Paper Structure (118 sections, 3 equations, 37 figures, 5 tables)

This paper contains 118 sections, 3 equations, 37 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Driving with Language
  4. Social Navigation
  5. Sensor Placement
  6. Cross-Modal Drone Navigation
  7. Cross-Platform 3D Object Detection
  8. RoboSense Challenge 2025
  9. Track 1
  10. Task Overview
  11. Key Challenges
  12. Information
  13. Evaluation Metrics
  14. Leaderboard
  15. Track 2
  16. ...and 103 more sections

Figures (37)

  • Figure 1: Overview of the RoboSense 2025 Challenge. This competition benchmarks robust robot sensing across platforms (vehicles, drones, mobile robots, and quadrupeds), sensor modalities (RGB, depth, LiDAR, language), task settings (driving with language, social navigation, sensor placement, cross-view matching, and cross-platform 3D detection), and adverse conditions (sensor noise, viewpoint changes, environmental corruptions, and domain shifts). Together, the five tracks evaluate the resilience and adaptability of modern perception systems under real-world variations.
  • Figure 2: Team [TQL]'s dataset expansion and training pipeline. Phase 1 uses pseudo-labels from InternVL3-8B for pre-training, six-view concatenation for multi-view learning, and CoT data from InternVL3-14B-Instruct for reasoning enhancement. Phase 2 applies balanced multi-view SFT with category-wise voting ensemble for final predictions.
  • Figure 3: Team [UCAS-CSU]'s mixture-of-prompts framework. A router classifies test queries and selects task-specific expert prompts, which are combined with multi-view images, visual markers, region crops, and adaptive historical frames before invoking the VLM.
  • Figure 4: Team [CVML]'s Phase-2 metadata-grounded reasoning framework. A rule-based router categorizes questions, then injects nuScenes metadata, zoom-in crops, and object/ego status into task-specific prompts for Qwen 2.5-VL 32B.
  • Figure 5: Team [UQMM]'s fine-tuning pipeline for Senna-VLM jiang2024senna. The model is pretrained on the Senna dataset, then fine-tuned using DriveLM sima2024drivelm for graph-structured VQA and DriveBench for corruption-aware reasoning.
  • ...and 32 more figures