Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning for Active Perception in Autonomous Navigation

Grzegorz Malczyk, Mihir Kulkarni, Kostas Alexis

TL;DR

This work tackles active perception for autonomous 3D navigation by learning a joint policy that controls both robot motion and an actuated camera. It introduces an ego-centric local occupancy grid, a multi-objective reward that blends goal progress with information gain, and a network that fuses depth, grid, and state information to output 6D actions. Through extensive sim and real-world experiments, including ablations and sim2real demonstrations, the method shows safer, more explorative navigation than fixed-camera baselines, and it demonstrates robust performance in cluttered indoor settings. The approach advances practical autonomy by enabling perception-driven exploration without relying on long-term localization, and it is open-sourced for reproducibility.

Abstract

This paper addresses the challenge of active perception within autonomous navigation in complex, unknown environments. Revisiting the foundational principles of active perception, we introduce an end-to-end reinforcement learning framework in which a robot must not only reach a goal while avoiding obstacles, but also actively control its onboard camera to enhance situational awareness. The policy receives observations comprising the robot state, the current depth frame, and a particularly local geometry representation built from a short history of depth readings. To couple collision-free motion planning with information-driven active camera control, we augment the navigation reward with a voxel-based information metric. This enables an aerial robot to learn a robust policy that balances goal-directed motion with exploratory sensing. Extensive evaluation demonstrates that our strategy achieves safer flight compared to using fixed, non-actuated camera baselines while also inducing intrinsic exploratory behaviors.

Reinforcement Learning for Active Perception in Autonomous Navigation

TL;DR

This work tackles active perception for autonomous 3D navigation by learning a joint policy that controls both robot motion and an actuated camera. It introduces an ego-centric local occupancy grid, a multi-objective reward that blends goal progress with information gain, and a network that fuses depth, grid, and state information to output 6D actions. Through extensive sim and real-world experiments, including ablations and sim2real demonstrations, the method shows safer, more explorative navigation than fixed-camera baselines, and it demonstrates robust performance in cluttered indoor settings. The approach advances practical autonomy by enabling perception-driven exploration without relying on long-term localization, and it is open-sourced for reproducibility.

Abstract

This paper addresses the challenge of active perception within autonomous navigation in complex, unknown environments. Revisiting the foundational principles of active perception, we introduce an end-to-end reinforcement learning framework in which a robot must not only reach a goal while avoiding obstacles, but also actively control its onboard camera to enhance situational awareness. The policy receives observations comprising the robot state, the current depth frame, and a particularly local geometry representation built from a short history of depth readings. To couple collision-free motion planning with information-driven active camera control, we augment the navigation reward with a voxel-based information metric. This enables an aerial robot to learn a robust policy that balances goal-directed motion with exploratory sensing. Extensive evaluation demonstrates that our strategy achieves safer flight compared to using fixed, non-actuated camera baselines while also inducing intrinsic exploratory behaviors.
Paper Structure (20 sections, 11 equations, 7 figures, 1 table)

This paper contains 20 sections, 11 equations, 7 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Contributions
  3. PROBLEM FORMULATION
  4. METHOD
  5. Ego-centric Local Occupancy Grid
  6. Policy Learning
  7. State and Observation Space
  8. Action Space
  9. Reward Design
  10. Implementation
  11. Actuated Camera
  12. Network architecture
  13. Training Environment
  14. EVALUATION STUDIES
  15. Ablation Studies
  16. ...and 5 more sections

Figures (7)

  • Figure 1: The quadrotor platform equipped with an actuated camera system. The local occupancy grid informs the robot about the nearby obstacle while the camera is directed to explore new regions in cluttered environments.
  • Figure 2: The quadrotor platform with its actuated RGB-D camera system.
  • Figure 3: The proposed network architecture for safe navigation with active perception. The network processes depth images as well as the local occupancy grids through dedicated encoder blocks, integrates robot state and camera orientation via MLP and GRU modules, and commands the actions for the robot.
  • Figure 4: Aerial Gym training environment with randomized primitive obstacles ensuring diverse scenarios and sim2real transferability.
  • Figure 5: Gazebo train station navigation experiment. Top-down view comparing trajectories of two methods: one without $n_t$ incorporation and one with $n_t$ The latter demonstrates improved spatial awareness, scaning more of the enviroment while navigating towards the goal.
  • ...and 2 more figures