Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMCO: Adaptive Multimodal Coupling of Vision and Proprioception for Quadruped Robot Navigation in Outdoor Environments

Mohamed Elnoor, Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Tianrui Guan, Vignesh Rajagopal, Dinesh Manocha

TL;DR

AMCO, a novel navigation method for quadruped robots that adaptively combines vision-based and proprioception-based perception capabilities, and a novel planner that selects appropriate gaits and velocities for traversing challenging outdoor environments using the coupled traversability cost map.

Abstract

We present AMCO, a novel navigation method for quadruped robots that adaptively combines vision-based and proprioception-based perception capabilities. Our approach uses three cost maps: general knowledge map; traversability history map; and current proprioception map; which are derived from a robot's vision and proprioception data, and couples them to obtain a coupled traversability cost map for navigation. The general knowledge map encodes terrains semantically segmented from visual sensing, and represents a terrain's typically expected traversability. The traversability history map encodes the robot's recent proprioceptive measurements on a terrain and its semantic segmentation as a cost map. Further, the robot's present proprioceptive measurement is encoded as a cost map in the current proprioception map. As the general knowledge map and traversability history map rely on semantic segmentation, we evaluate the reliability of the visual sensory data by estimating the brightness and motion blur of input RGB images and accordingly combine the three cost maps to obtain the coupled traversability cost map used for navigation. Leveraging this adaptive coupling, the robot can depend on the most reliable input modality available. Finally, we present a novel planner that selects appropriate gaits and velocities for traversing challenging outdoor environments using the coupled traversability cost map. We demonstrate AMCO's navigation performance in different real-world outdoor environments and observe 10.8%-34.9% reduction w.r.t. two stability metrics, and up to 50% improvement in terms of success rate compared to current navigation methods.

AMCO: Adaptive Multimodal Coupling of Vision and Proprioception for Quadruped Robot Navigation in Outdoor Environments

TL;DR

AMCO, a novel navigation method for quadruped robots that adaptively combines vision-based and proprioception-based perception capabilities, and a novel planner that selects appropriate gaits and velocities for traversing challenging outdoor environments using the coupled traversability cost map.

Abstract

We present AMCO, a novel navigation method for quadruped robots that adaptively combines vision-based and proprioception-based perception capabilities. Our approach uses three cost maps: general knowledge map; traversability history map; and current proprioception map; which are derived from a robot's vision and proprioception data, and couples them to obtain a coupled traversability cost map for navigation. The general knowledge map encodes terrains semantically segmented from visual sensing, and represents a terrain's typically expected traversability. The traversability history map encodes the robot's recent proprioceptive measurements on a terrain and its semantic segmentation as a cost map. Further, the robot's present proprioceptive measurement is encoded as a cost map in the current proprioception map. As the general knowledge map and traversability history map rely on semantic segmentation, we evaluate the reliability of the visual sensory data by estimating the brightness and motion blur of input RGB images and accordingly combine the three cost maps to obtain the coupled traversability cost map used for navigation. Leveraging this adaptive coupling, the robot can depend on the most reliable input modality available. Finally, we present a novel planner that selects appropriate gaits and velocities for traversing challenging outdoor environments using the coupled traversability cost map. We demonstrate AMCO's navigation performance in different real-world outdoor environments and observe 10.8%-34.9% reduction w.r.t. two stability metrics, and up to 50% improvement in terms of success rate compared to current navigation methods.
Paper Structure (27 sections, 9 equations, 6 figures, 3 tables)

This paper contains 27 sections, 9 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Traversability Estimation
  4. Exteropercption-based perception
  5. Proprioception-based Perception
  6. Outdoor Navigation
  7. Background
  8. Symbols and Notations
  9. Setup and Conventions
  10. Proprioception-based Terrain Analysis
  11. Vision-based Traversability Types
  12. Our Approach
  13. General Knowledge Map
  14. Recent Traversability History Map
  15. Reliability Estimation of RGB images
  16. ...and 12 more sections

Figures (6)

  • Figure 1: The navigation trajectories generated by our method, AMCO (trot: brown, crawl: red, amble: pink), ProNav elnoor2023pronav (trot: blue, crawl: purple, amble: light blue), GaNav guan2022ga (orange), Spot's Inbuilt (yellow), and RFC kertesz2016rigidity (trot: green, crawl: light green, amble: light beige) in scenario 4, which contains different types of deformable terrains: (A) water puddle, (B) mud, (C) granular. AMCO, ProNav, and RFC can switch between three gaits (trot, crawl, and amble) during navigation as presented in three different colors. Our method AMCO updates its trajectory as traversing the terrain and uses the coupled vision and proprioception to avoid extremely deformable regions such as A and B. It balances between the trajectory length and maintaining stability to reduce potential robot failures on terrains with varying deformability. Conversely, GaNav guan2022ga takes a longer path by avoiding mud and grass altogether. The other methods takes a straight path towards the goal which leads to sinkage and failures.
  • Figure 2: The gaussian ellipses for the PCA components of four different terrains elnoor2023pronav. For each terrain, three types of gait data are shown using different ellipse boundaries. The solid line denotes trot, the dashed line denotes amble, and the dash-dot line is for crawl.
  • Figure 3: The overall system architecture AMCO that couples vision and proprioception perception modules. We use the robot's joint position, forces, and battery current to compute our traversability measure. We also use a semantic segmentation module to classify the terrains. After that, three traversability cost maps are created: general knowledge map (vision-based), history map (vision and proprioception), and proprioception map . All of them are coupled based on the reliability of the input RGB images to create a coupled cost map for the local planner.
  • Figure 4: Reliability Estimation of RGB images: we estimate the reliability of RGB images (top row) by measuring two factors, the image's brightness, and blurriness as both affect semantic segmentation (bottom row). (a) and (b) show two brightness levels for an RGB image. (c) shows a sharp image (with almost no blur). (d) shows how rocky terrains could lead to blurriness in the image.
  • Figure 5: Traversability cost maps of our method. (a) The input RGB image, (b) The segmented image with the color labels as (stable: green, vegetation: red, granular: yellow and poor foothold: orange), (c) The general knowledge map (vision-based), (d) The history map, (e) The proprioception map . (f) The coupled map . Dark blue reflects low traversability cost in (c-d) maps and conversely red indicates high traversability cost. As shown in the figure, concrete shows low cost in the general knowledge map and particularly evident in the proprioception-based cost map as the robots captured the terrain stability using proprioception.
  • ...and 1 more figures