Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Large-scale Spatial Problems with Convolutional Neural Networks

Damian Owerko, Charilaos I. Kanatsoulis, Alejandro Ribeiro

TL;DR

The paper tackles large-scale spatial problems by exploiting CNN shift-equivariance and transfer learning to train on small signal windows while evaluating on much larger inputs. A theoretical bound is derived showing that a CNN trained with windowed data has controlled generalization error when applied to larger signals, with the bound depending on the network’s filter norms, depth, and window sizes. Spatial problems are recast as image-to-image regression by representing point sets with Gaussian mixtures and sampling onto grids for CNN processing. The framework is validated on mobile infrastructure on demand (MID), achieving zero-shot scalability to hundreds of agents and window sizes up to $1600$ meters with linear-time complexity in the area, surpassing previous convex-optimization approaches in scalability. Together, these results establish a principled, efficient route for solving large-scale spatial tasks using CNNs and transfer learning.

Abstract

Over the past decade, deep learning research has been accelerated by increasingly powerful hardware, which facilitated rapid growth in the model complexity and the amount of data ingested. This is becoming unsustainable and therefore refocusing on efficiency is necessary. In this paper, we employ transfer learning to improve training efficiency for large-scale spatial problems. We propose that a convolutional neural network (CNN) can be trained on small windows of signals, but evaluated on arbitrarily large signals with little to no performance degradation, and provide a theoretical bound on the resulting generalization error. Our proof leverages shift-equivariance of CNNs, a property that is underexploited in transfer learning. The theoretical results are experimentally supported in the context of mobile infrastructure on demand (MID). The proposed approach is able to tackle MID at large scales with hundreds of agents, which was computationally intractable prior to this work.

Solving Large-scale Spatial Problems with Convolutional Neural Networks

TL;DR

The paper tackles large-scale spatial problems by exploiting CNN shift-equivariance and transfer learning to train on small signal windows while evaluating on much larger inputs. A theoretical bound is derived showing that a CNN trained with windowed data has controlled generalization error when applied to larger signals, with the bound depending on the network’s filter norms, depth, and window sizes. Spatial problems are recast as image-to-image regression by representing point sets with Gaussian mixtures and sampling onto grids for CNN processing. The framework is validated on mobile infrastructure on demand (MID), achieving zero-shot scalability to hundreds of agents and window sizes up to meters with linear-time complexity in the area, surpassing previous convex-optimization approaches in scalability. Together, these results establish a principled, efficient route for solving large-scale spatial tasks using CNNs and transfer learning.

Abstract

Over the past decade, deep learning research has been accelerated by increasingly powerful hardware, which facilitated rapid growth in the model complexity and the amount of data ingested. This is becoming unsustainable and therefore refocusing on efficiency is necessary. In this paper, we employ transfer learning to improve training efficiency for large-scale spatial problems. We propose that a convolutional neural network (CNN) can be trained on small windows of signals, but evaluated on arbitrarily large signals with little to no performance degradation, and provide a theoretical bound on the resulting generalization error. Our proof leverages shift-equivariance of CNNs, a property that is underexploited in transfer learning. The theoretical results are experimentally supported in the context of mobile infrastructure on demand (MID). The proposed approach is able to tackle MID at large scales with hundreds of agents, which was computationally intractable prior to this work.
Paper Structure (8 sections, 1 theorem, 9 equations, 2 figures, 1 table)

This paper contains 8 sections, 1 theorem, 9 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Learning to process stationary signals
  3. Stationary Stochastic Processes
  4. Convolutional Neural Networks
  5. Training on a window
  6. Representing spatial problems as images
  7. Experimental results
  8. Conclusions

Key Result

Theorem 1

Let $\hat{\calH}$ be a set of filters which achieves a cost of $\calL_\sqcap(\hat{\calH})$ on the windowed problem as defined by eq:mse_window with an input window of width $A$ and an output window width $B$. Then the associated cost $\calL_\infty(\hat{\calH})$ on the original problem as defined by In eq:stationary_bound, $H = \prod_l^L || h_l ||_1$ is the product of all the L1 norms of the CNNs

Figures (2)

  • Figure 1: Example inputs and outputs to the CNN for the MID task at different window widths $A = 320, 640, 960, 1280, 1600$ but with a constant spatial resolution of $\rho = 1.25$ meters per pixel. The top row images represent the positions of the task agents. The bottom row images represent the estimated optimal positions of the communication agents by the CNN. Additionally, the task agent positions are marked in red on the bottom images.
  • Figure 2: The distributions of the minimum transmitter power needed to maintain a normalized communication rate of at least 50% between any two agents. The distributions for each window width are visualized by box plots, with notches representing a 95% confidence interval for the estimate of the median.

Theorems & Definitions (2)

  • Definition 1
  • Theorem 1