OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity

TL;DR

OccRWKV tackles the efficiency-accuracy trade-off in 3D semantic occupancy prediction by introducing a linear-complexity RWKV-based architecture with three specialized branches (Semantics, Geometry, BEV fusion). By operating in BEV space and employing Sem-RWKV, Geo-RWKV, and BEV-RWKV blocks, the method captures long-range dependencies with reduced computation and memory, achieving 25.1% mIoU on SemanticKITTI while running at 22.2 FPS and requiring modest memory. The approach demonstrates strong performance gains over state-of-the-art baselines, including substantial speedups over Co-Occ, and validates real-world applicability through robot deployment and navigation efficiency improvements. The work highlights the viability of RWKV-based designs for real-time, perception-heavy robotics tasks and provides a practical, deployable solution for obstacle-rich, occluded environments.

Abstract

3D semantic occupancy prediction networks have demonstrated remarkable capabilities in reconstructing the geometric and semantic structure of 3D scenes, providing crucial information for robot navigation and autonomous driving systems. However, due to their large overhead from dense network structure designs, existing networks face challenges balancing accuracy and latency. In this paper, we introduce OccRWKV, an efficient semantic occupancy network inspired by Receptance Weighted Key Value (RWKV). OccRWKV separates semantics, occupancy prediction, and feature fusion into distinct branches, each incorporating Sem-RWKV and Geo-RWKV blocks. These blocks are designed to capture long-range dependencies, enabling the network to learn domain-specific representation (i.e., semantics and geometry), which enhances prediction accuracy. Leveraging the sparse nature of real-world 3D occupancy, we reduce computational overhead by projecting features into the bird's-eye view (BEV) space and propose a BEV-RWKV block for efficient feature enhancement and fusion. This enables real-time inference at 22.2 FPS without compromising performance. Experiments demonstrate that OccRWKV outperforms the state-of-the-art methods on the SemanticKITTI dataset, achieving a mIoU of 25.1 while being 20 times faster than the best baseline, Co-Occ, making it suitable for real-time deployment on robots to enhance autonomous navigation efficiency. Code and video are available on our project page: https://jmwang0117.github.io/OccRWKV/.

Figures (7)

• Figure 1: Comparison of accuracy and efficiency metrics (i.e., FPS and FLOPs) with SoTA methods.
• Figure 2: Overview of the proposed OccRWKV. Semantic and geometry branches learn respective representations, i.e., semantic and geometric, supervised by multi-level auxiliary losses. Finally, features are fused in the BEV fusion branch to generate dense 3D semantic occupancy predictions.
• Figure 3: The overview of the proposed Sem-RWKV and Geo-RWKV blocks is illustrated, please zoom in for details.
• Figure 4: BEV feature fusion branch encoder structure.
• Figure 5: The qualitative comparisons results on the SemanticKITTI validation set.