Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Represent Surroundings, Anticipate Motion and Take Informed Actions in Unstructured Environments

Weiming Zhi

TL;DR

The thesis tackles enabling robots to operate in unstructured, dynamic environments by learning representations of surroundings, predicting motion, and selecting informed actions. It develops continuous occupancy representations (Fast-BHM) and continuous spatiotemporal maps to model environmental structure and motion, enabling scalable multi-agent mapping and long-horizon behavior understanding. It then introduces anticipatory navigation through SPAN and OTNet, along with a probabilistic-constraint framework to fuse learning with environment structure, and extends to robot manipulators using Diffeomorphic Templates and Geometric Fabric Command Sequences for globally feasible yet reactive motion. Collectively, the work demonstrates end-to-end learning-and-planning pipelines that exploit environment representations, probabilistic motion predictions, and stable, generalisable control for safe, efficient robot operation in unstructured spaces. These contributions offer scalable, data-driven tools for real-world robotics, with potential impact across autonomous driving, service robots, and collaborative manipulation in crowded or complex environments.

Abstract

Contemporary robots have become exceptionally skilled at achieving specific tasks in structured environments. However, they often fail when faced with the limitless permutations of real-world unstructured environments. This motivates robotics methods which learn from experience, rather than follow a pre-defined set of rules. In this thesis, we present a range of learning-based methods aimed at enabling robots, operating in dynamic and unstructured environments, to better understand their surroundings, anticipate the actions of others, and take informed actions accordingly.

Learning to Represent Surroundings, Anticipate Motion and Take Informed Actions in Unstructured Environments

TL;DR

The thesis tackles enabling robots to operate in unstructured, dynamic environments by learning representations of surroundings, predicting motion, and selecting informed actions. It develops continuous occupancy representations (Fast-BHM) and continuous spatiotemporal maps to model environmental structure and motion, enabling scalable multi-agent mapping and long-horizon behavior understanding. It then introduces anticipatory navigation through SPAN and OTNet, along with a probabilistic-constraint framework to fuse learning with environment structure, and extends to robot manipulators using Diffeomorphic Templates and Geometric Fabric Command Sequences for globally feasible yet reactive motion. Collectively, the work demonstrates end-to-end learning-and-planning pipelines that exploit environment representations, probabilistic motion predictions, and stable, generalisable control for safe, efficient robot operation in unstructured spaces. These contributions offer scalable, data-driven tools for real-world robotics, with potential impact across autonomous driving, service robots, and collaborative manipulation in crowded or complex environments.

Abstract

Contemporary robots have become exceptionally skilled at achieving specific tasks in structured environments. However, they often fail when faced with the limitless permutations of real-world unstructured environments. This motivates robotics methods which learn from experience, rather than follow a pre-defined set of rules. In this thesis, we present a range of learning-based methods aimed at enabling robots, operating in dynamic and unstructured environments, to better understand their surroundings, anticipate the actions of others, and take informed actions accordingly.
Paper Structure (180 sections, 1 theorem, 130 equations, 56 figures, 14 tables, 5 algorithms)

This paper contains 180 sections, 1 theorem, 130 equations, 56 figures, 14 tables, 5 algorithms.

Table of Contents

  1. Declaration
  2. Author Attribution and Publications
  3. Abstract
  4. Acknowledgements
  5. Introduction
  6. Motivations
  7. Contributions
  8. Thesis Structure
  9. Background
  10. Overview
  11. Supervised Regression Problems
  12. Linear Regression Models
  13. Non-linear Features and Kernel Functions: An Example
  14. Regression with Fully-Connected Neural Networks
  15. Optimisation
  16. ...and 165 more sections

Key Result

Theorem 1

Let $\Phi:\mathbb{R}^{d}\rightarrow\mathbb{R}$ be a potential function, and $\psi:\boldsymbol{M}\rightarrow \mathbb{R}^{d}$ be a diffeomorphism mapping between some manifold $\boldsymbol{M}$ to $\mathbb{R}^{d}$ Euclidean space. If the system defined by $\dot{\bm{z}}=-\nabla_{\bm{z}}\Phi(\bm{z}) \in

Figures (56)

  • Figure 1: Contemporary robots deployed in their respective environments. Courtesy of Wikimedia Commons.
  • Figure 2: An example of CMA-ES optimising a cost function for 6 iterations (generations), where the contours are displayed, and lighter colours indicate a lower cost. Samples are given in black, and the ellipse iso-contour of the sampling distribution is in red. We observe that CMA-ES is able to rapidly locate the region with low cost. Figures from cma_figure_cite.
  • Figure 3: An illustration of an example Hilbert Map. (Left) Training data points used for training, blue and brown represent occupied and unoccupied points. (Right) The probability of being occupied is given by a constructed Hilbert Map. Figures from HilbertMaps.
  • Figure 4: The JACO arm used in real-world experiments in \ref{['chap7']} and \ref{['chap8']} has 6 degrees of freedom (not including the gripper), one for each of 6 revolute joints. Figure adapted from jaco_arm_img.
  • Figure 5: The 3 degrees of freedom planar manipulator used in the example. Example and figure from modern_robotics.
  • ...and 51 more figures

Theorems & Definitions (4)

  • Definition 1: Hill:2008
  • Theorem 1: Euclideanising
  • Definition 2: Flows
  • Definition 3: Infinitesimal Generator