Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating saliency scores in point clouds of natural environments by learning surface anomalies

Reuma Arav, Dennis Wittich, Franz Rottensteiner

TL;DR

This work proposes to differentiate objects of interest from the cluttered environment by evaluating how much they stand out from their surroundings by evaluating how much they stand out from their surroundings, i.e., their geometric salience.

Abstract

In recent years, three-dimensional point clouds are used increasingly to document natural environments. Each dataset contains a diverse set of objects, at varying shapes and sizes, distributed throughout the data and intricately intertwined with the topography. Therefore, regions of interest are difficult to find and consequent analyses become a challenge. Inspired from visual perception principles, we propose to differentiate objects of interest from the cluttered environment by evaluating how much they stand out from their surroundings, i.e., their geometric salience. Previous saliency detection approaches suggested mostly handcrafted attributes for the task. However, such methods fail when the data are too noisy or have high levels of texture. Here we propose a learning-based mechanism that accommodates noise and textured surfaces. We assume that within the natural environment any change from the prevalent surface would suggest a salient object. Thus, we first learn the underlying surface and then search for anomalies within it. Initially, a deep neural network is trained to reconstruct the surface. Regions where the reconstructed part deviates significantly from the original point cloud yield a substantial reconstruction error, signifying an anomaly, i.e., saliency. We demonstrate the effectiveness of the proposed approach by searching for salient features in various natural scenarios, which were acquired by different acquisition platforms. We show the strong correlation between the reconstruction error and salient objects.

Evaluating saliency scores in point clouds of natural environments by learning surface anomalies

TL;DR

This work proposes to differentiate objects of interest from the cluttered environment by evaluating how much they stand out from their surroundings by evaluating how much they stand out from their surroundings, i.e., their geometric salience.

Abstract

In recent years, three-dimensional point clouds are used increasingly to document natural environments. Each dataset contains a diverse set of objects, at varying shapes and sizes, distributed throughout the data and intricately intertwined with the topography. Therefore, regions of interest are difficult to find and consequent analyses become a challenge. Inspired from visual perception principles, we propose to differentiate objects of interest from the cluttered environment by evaluating how much they stand out from their surroundings, i.e., their geometric salience. Previous saliency detection approaches suggested mostly handcrafted attributes for the task. However, such methods fail when the data are too noisy or have high levels of texture. Here we propose a learning-based mechanism that accommodates noise and textured surfaces. We assume that within the natural environment any change from the prevalent surface would suggest a salient object. Thus, we first learn the underlying surface and then search for anomalies within it. Initially, a deep neural network is trained to reconstruct the surface. Regions where the reconstructed part deviates significantly from the original point cloud yield a substantial reconstruction error, signifying an anomaly, i.e., saliency. We demonstrate the effectiveness of the proposed approach by searching for salient features in various natural scenarios, which were acquired by different acquisition platforms. We show the strong correlation between the reconstruction error and salient objects.
Paper Structure (25 sections, 9 equations, 10 figures, 7 tables)

This paper contains 25 sections, 9 equations, 10 figures, 7 tables.

Table of Contents

  1. INTRODUCTION
  2. Related work
  3. Handcrafted saliency approaches
  4. Deep learning-based saliency approaches
  5. Methodology
  6. Saliency estimation in 3D point clouds by anomaly detection
  7. Architecture
  8. Loss Function and Training
  9. Inference
  10. Test setup
  11. Experiment setup
  12. Datasets
  13. Validation and test subsets
  14. Evaluation metrics
  15. Baseline approaches
  16. ...and 10 more sections

Figures (10)

  • Figure 1: Overview of the training scheme. We generate random voxel grids from a set of given point-clouds. Next, the shells are generated by setting the inner voxels to zero. The task of the CNN is then to reconstruct the original input. Note that the extracted voxel grid is coloured by the number of points in each cell. This information is used to calculate a weighted reconstruction loss.
  • Figure 2: Illustration of the variational auto-encoder used to reconstruct the surface based on the information in the shell of the voxel grid representation. Blue: Output of downsampling. Red: Output of upsampling. Yellow: Concatenated feature tensors.
  • Figure 3: Datasets analysed in the study. a) Airborne dataset (Dead Sea Coast); b) UAV-borne dataset (Pielach River); c) Terrestrial dataset (Traisenbacher cave). Colours refer to elevation.
  • Figure 8: Dataset #I. Saliency scores estimated for the validation subsets with $n=16$ and different numbers of feature maps in base resolution.
  • Figure 9: Dataset #III. Saliency scores for $f=32$ at different sizes of the voxel grid. It can be seen that as the grid size grows, more regions are marked as salient.
  • ...and 5 more figures