Design and Identification of Keypoint Patches in Unstructured Environments

TL;DR

The paper tackles robust keypoint detection in unstructured environments by designing four distinct, small, rotation- and scale-invariant keypoint patches and a customized Superpoint detector trained on a fully synthesized dataset. The method combines synthetic data generation with random perspective transformations and image deteriorations, plus extra network heads to quantify patch type IDs. Results show high detection and ID matching scores on synthetic validation ($>0.95$) and demonstrate resilience to scale, rotation, and common degradations, with real-world tests confirming practical viability, albeit with degraded performance under extreme noise. The work offers a low-area, robust keypoint design suitable for vision-based autonomy in cluttered or occluded settings, enabling stable localization and planning without relying on fiducial markers.

Abstract

Reliable perception of targets is crucial for the stable operation of autonomous robots. A widely preferred method is keypoint identification in an image, as it allows direct mapping from raw images to 2D coordinates, facilitating integration with other algorithms like localization and path planning. In this study, we closely examine the design and identification of keypoint patches in cluttered environments, where factors such as blur and shadows can hinder detection. We propose four simple yet distinct designs that consider various scale, rotation and camera projection using a limited number of pixels. Additionally, we customize the Superpoint network to ensure robust detection under various types of image degradation. The effectiveness of our approach is demonstrated through real-world video tests, highlighting potential for vision-based autonomous systems.

Figures (8)

• Figure 1: Examples of designs for precise keypoint localization. (a) ArUco3 romero2018speeded. (b) Topotag yu2020topotag. (c) Chromatag chen2021autonomous. (d) Intersense naimark2002circular.
• Figure 2: Training and inference of Superpoint network customized to proposed keypoint patches. (a) 4 types of keypoint patch are deformed with randomness, and drawn on an image from ImageNet deng2009imagenet. (b) The network structure of Superpoint, which predicts 2D keypoint locations and their feature vectors.
• Figure 3: Example of network output on validation dataset with image deterioration. Top. Input images. Upper. Confidence map of keypoints at each pixel. Lower. Associated patch types to which high-confidence keypoints belong. Bottom. The output illustration on the input image highlighting true detections with solid lines, while a dotted line indicating a false positive detection.
• Figure 4: Example of network output on real dataset with various scale.
• Figure 5: Example of network output on real dataset with various pitch angles.