UniOcc: A Unified Benchmark for Occupancy Forecasting and Prediction in Autonomous Driving

TL;DR

UniOcc presents a unified occupancy benchmark and toolkit that combines real-world and simulated data to support occupancy prediction, forecasting, and cooperative forecasting. It introduces per-voxel flow annotations, ground-truth-free evaluation metrics, and a cross-dataset processing pipeline with object identification, tracking, and alignment in voxel space. Empirical results show that voxel-level flows improve forecasting and that diverse data improves cross-domain generalization, while sim data enhances object-shape learning but may affect temporal consistency. The framework enables cross-domain benchmarking, scalable data augmentation, and multi-agent occupancy research, with open-source tooling to accelerate progress in occupancy-centric autonomous driving research.

Abstract

We introduce UniOcc, a comprehensive, unified benchmark and toolkit for occupancy forecasting (i.e., predicting future occupancies based on historical information) and occupancy prediction (i.e., predicting current-frame occupancy from camera images. UniOcc unifies the data from multiple real-world datasets (i.e., nuScenes, Waymo) and high-fidelity driving simulators (i.e., CARLA, OpenCOOD), providing 2D/3D occupancy labels and annotating innovative per-voxel flows. Unlike existing studies that rely on suboptimal pseudo labels for evaluation, UniOcc incorporates novel evaluation metrics that do not depend on ground-truth labels, enabling robust assessment on additional aspects of occupancy quality. Through extensive experiments on state-of-the-art models, we demonstrate that large-scale, diverse training data and explicit flow information significantly enhance occupancy prediction and forecasting performance. Our data and code are available at https://uniocc.github.io/.

Figures (10)

• Figure 1: Our UniOcc framework incorporates various occupancy label generation methods from multiple data sources, provides the training/testing pipeline & toolkit for a variety of occupancy tasks, and supports comprehensive evaluation metrics.
• Figure 2: Comparison of object-level flow and voxel-level flow.
• Figure 3: Visualization of the per-voxel forward flows.
• Figure 4: Illustration of imperfect ground-truth labels. Left: partial car shape from Occ3D tian2024occ3d. Right: a more complete shape predicted by OccWorld zheng2025occworld. Standard IoU may penalize the model for producing a fuller shape, despite it being more realistic.
• Figure 5: An example showing our pipeline for voxel extraction, connected-component segmentation, and bounding-box fitting.