Non-submodular Visual Attention for Robot Navigation

TL;DR

The paper tackles task-aware feature selection for Visual-Inertial Navigation under tight computational constraints by formulating a non-submodular MSE objective and solving it with four polynomial-time approximations. It couples a forward-simulation (anticipation) model with IMU and vision models to define feature contributions $\Delta_l$ and the information matrix $\Omega_{\mathsf{S}} = \Omega_{\emptyset} + \sum_{l\in\mathsf{S}} \Delta_l$, targeting minimization of $\mathrm{Trace}(\Omega_{\mathsf{S}}^{-1})$. The authors derive performance guarantees using submodularity ratio $\gamma$ and curvature $\alpha$, and develop a fast low-rank greedy with SMW updates, a randomized greedy, and a linearization-based greedy via a Taylor surrogate, each with concrete computational benefits. Extensive experiments on EuRoC and a control-enabled QCar platform validate the theoretical results, showing near-optimality for greedy variants, real-time feasibility for linearized greedy, and robust performance under practical sensing and control dynamics. The work offers practical guidance for selecting informative visual features in VIN to balance accuracy and on-board computation, with potential extensions to multi-agent settings and broader sensing modalities.

Abstract

This paper presents a task-oriented computational framework to enhance Visual-Inertial Navigation (VIN) in robots, addressing challenges such as limited time and energy resources. The framework strategically selects visual features using a Mean Squared Error (MSE)-based, non-submodular objective function and a simplified dynamic anticipation model. To address the NP-hardness of this problem, we introduce four polynomial-time approximation algorithms: a classic greedy method with constant-factor guarantees; a low-rank greedy variant that significantly reduces computational complexity; a randomized greedy sampler that balances efficiency and solution quality; and a linearization-based selector based on a first-order Taylor expansion for near-constant-time execution. We establish rigorous performance bounds by leveraging submodularity ratios, curvature, and element-wise curvature analyses. Extensive experiments on both standardized benchmarks and a custom control-aware platform validate our theoretical results, demonstrating that these methods achieve strong approximation guarantees while enabling real-time deployment.

Figures (13)

• Figure 1: Detailed illustration of the visual attention mechanism as the front‐end’s filtering block. Image features extracted from each keyframe are combined with motion priors provided by the IMU to form a baseline information estimate. For every candidate feature, the mechanism projects the feature into predicted future poses to simulate its visibility and then computes its expected reduction in estimation error. Features are then ranked by this information‐gain metric, and only the highest‐value features are forwarded to the back‐end optimizer for state estimation. This selective process reduces computational load while maintaining high‐accuracy pose estimates.
• Figure 2: Forward propagation of a feature’s bearing vector from a 3-D object (a stop sign) detected at frame $k$. Left: the robot’s positions and bearing vectors to the stop sign features at time steps $k+1$ and $k+2$. Right: the corresponding camera image planes at those time steps. Colored rectangles around each image indicate whether the feature is visible (green) or not (red). In this example, the predicted bearing at $k+2$ falls outside the camera’s field of view, causing the visibility check to fail. See Section \ref{['sec:experiments:QCar']} for details.
• Figure 3: Performance comparison of different feature selection methods on the MH_01_easy sequence from the EuRoC dataset. Each column corresponds to a single randomly selected video frame. The top plot in each column shows the MSE values versus the number of selected features $\kappa$, while the lower plot presents the computation time on a logarithmic scale to enhance visibility and highlight discrepancies. Methods compared include uniform random selection ("random"), grid-based selection ("grid"), simple greedy ("simple"), fast low-rank greedy ("low-rank"), randomized greedy ("randomized"), and linearization-based greedy ("linearized"). For randomized methods, each experiment was repeated 20 times, and the mean values are reported. The prediction horizon $T$ is set to 13 for the information-aware selection methods, and the hyperparameter $\epsilon$ in the randomized greedy algorithm is set to 0.5. Note that MSE values and feature counts vary across frames, so results are presented for three representative frames without averaging across the sequence.
• Figure 4: Performance comparison of the proposed feature selection methods on the MH_01_easy sequence from the EuRoC dataset. The evaluation considers selecting $\kappa = 1, \ldots, 10$ features from a pool of 10 candidates, allowing comparison against the optimal solution obtained via exhaustive search. The results show that the greedy algorithm achieves identical performance to the optimal method, despite the MSE-based objective not being submodular. The linearized method closely matches this performance with minimal deviation, while the randomized approach exhibits slight fluctuations due to the fixed $\epsilon$, yet follows a similar overall trend.
• Figure 5: (a) Curvature ($\alpha$) and submodularity ratio ($\gamma$) as functions of $\kappa$. (b) Performance bound calculated using $\alpha$ and $\gamma$ values.

Theorems & Definitions (20)

• Remark 1
• Remark 2
• Definition 1
• Definition 2: Curvature, $\alpha$
• Definition 3: Submodularity ratio, $\gamma$
• Proposition 1: Approximate non-submodular maximization bian2017guarantees
• Lemma 1
• proof
• Theorem 1
• proof