N-QR: Natural Quick Response Codes for Multi-Robot Instance Correspondence

TL;DR

The Natural Quick Response codes, or N-QR, is proposed, which enables rapid and reliable correspondence between large-scale teams of heterogeneous robots, using keypoint-based alignment, rapid encoding, and error correction via ensembles of image patches of natural patterns.

Abstract

Image correspondence serves as the backbone for many tasks in robotics, such as visual fusion, localization, and mapping. However, existing correspondence methods do not scale to large multi-robot systems, and they struggle when image features are weak, ambiguous, or evolving. In response, we propose Natural Quick Response codes, or N-QR, which enables rapid and reliable correspondence between large-scale teams of heterogeneous robots. Our method works like a QR code, using keypoint-based alignment, rapid encoding, and error correction via ensembles of image patches of natural patterns. We deploy our algorithm in a production-scale robotic farm, where groups of growing plants must be matched across many robots. We demonstrate superior performance compared to several baselines, obtaining a retrieval accuracy of 88.2%. Our method generalizes to a farm with 100 robots, achieving a 12.5x reduction in bandwidth and a 20.5x speedup. We leverage our method to correspond 700k plants and confirm a link between a robotic seeding policy and germination.

Figures (6)

• Figure 1: N-QR uses naturally-occurring patterns and rapid encoding to help large teams of robots find a unique object. This correspondence allows robotic farmers to track the same plant, from seed to harvest, without extra tagging hardware.
• Figure 2: Robotic farming system and image matching challenge. A seeding robot (left) continuously plants seeds into "rafts" on a moving conveyor belt. After a germination period, these rafts are transferred to a stationary growing robot (right), who supports and monitors growth for the remaining duration of the plant lifecycle. Image-level, raft-level, and pixel-level correspondences are unknown between the seeding and growing robots.
• Figure 3: Matching difficulty. These two normalized raft images ($\mathbf{R}$ and $\mathbf{R^{\prime}}$) constitute a positive match. As an exercise for the reader, we encourage you to find the features that support and discourage this match.
• Figure 4: N-QR iteratively transmits feature embeddings until a minimum distance threshold is met.
• Figure 5: Plant lifecycle with patch-based analysis of growth.