Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViKi-HyCo: A Hybrid-Control approach for complex car-like maneuvers

Edison P. Velasco Sánchez, Miguel Ángel Muñoz-Bañón, Francisco A. Candelas, Santiago T. Puente, Fernando Torres

TL;DR

ViKi-HyCo tackles the problem of precise positioning for non-holonomic car-like robots when the visual target exits the camera FOV by introducing a hybrid control framework that fuses LiDAR-camera depth information with YOLOv5 detections. The method dynamically updates a desired bounding-box-based visual target and blends a visual servoing controller with a metric kinematic controller, switching between them via a controller indicator to maintain smooth motion even under occlusions or detector dropout. A key contribution is the continuous update of the object’s depth and desired features using LiDAR-depth and RGB-D fusion, enabling stable convergence without relying on object trackers. Real-world experiments on the BLUE platform show end-to-end positioning errors of $0.0428 \pm 0.0467$ m (X) and $0.0515 \pm 0.0313$ m (Y), with a runtime under 45 ms per iteration, demonstrating robust, real-time performance for outdoor waste localization tasks and other autonomous positioning applications.

Abstract

While Visual Servoing is deeply studied to perform simple maneuvers, the literature does not commonly address complex cases where the target is far out of the camera's field of view (FOV) during the maneuver. For this reason, in this paper, we present ViKi-HyCo (Visual Servoing and Kinematic Hybrid-Controller). This approach generates the necessary maneuvers for the complex positioning of a non-holonomic mobile robot in outdoor environments. In this method, we use \hbox{LiDAR-camera} fusion to estimate objects bounding boxes using image and metrics modalities. With the multi-modality nature of our representation, we can automatically obtain a target for a visual servoing controller. At the same time, we also have a metric target, which allows us to hybridize with a kinematic controller. Given this hybridization, we can perform complex maneuvers even when the target is far away from the camera's FOV. The proposed approach does not require an object-tracking algorithm and can be applied to any robotic positioning task where its kinematic model is known. ViKi-HyCo has an error of 0.0428 \pm 0.0467 m in the X-axis and 0.0515 \pm 0.0323 m in the Y-axis at the end of a complete positioning task.

ViKi-HyCo: A Hybrid-Control approach for complex car-like maneuvers

TL;DR

ViKi-HyCo tackles the problem of precise positioning for non-holonomic car-like robots when the visual target exits the camera FOV by introducing a hybrid control framework that fuses LiDAR-camera depth information with YOLOv5 detections. The method dynamically updates a desired bounding-box-based visual target and blends a visual servoing controller with a metric kinematic controller, switching between them via a controller indicator to maintain smooth motion even under occlusions or detector dropout. A key contribution is the continuous update of the object’s depth and desired features using LiDAR-depth and RGB-D fusion, enabling stable convergence without relying on object trackers. Real-world experiments on the BLUE platform show end-to-end positioning errors of m (X) and m (Y), with a runtime under 45 ms per iteration, demonstrating robust, real-time performance for outdoor waste localization tasks and other autonomous positioning applications.

Abstract

While Visual Servoing is deeply studied to perform simple maneuvers, the literature does not commonly address complex cases where the target is far out of the camera's field of view (FOV) during the maneuver. For this reason, in this paper, we present ViKi-HyCo (Visual Servoing and Kinematic Hybrid-Controller). This approach generates the necessary maneuvers for the complex positioning of a non-holonomic mobile robot in outdoor environments. In this method, we use \hbox{LiDAR-camera} fusion to estimate objects bounding boxes using image and metrics modalities. With the multi-modality nature of our representation, we can automatically obtain a target for a visual servoing controller. At the same time, we also have a metric target, which allows us to hybridize with a kinematic controller. Given this hybridization, we can perform complex maneuvers even when the target is far away from the camera's FOV. The proposed approach does not require an object-tracking algorithm and can be applied to any robotic positioning task where its kinematic model is known. ViKi-HyCo has an error of 0.0428 \pm 0.0467 m in the X-axis and 0.0515 \pm 0.0323 m in the Y-axis at the end of a complete positioning task.
Paper Structure (17 sections, 26 equations, 19 figures, 1 table)

This paper contains 17 sections, 26 equations, 19 figures, 1 table.

Table of Contents

  1. Introduction
  2. Proposed Controller Architecture
  3. Robot Kinematic Model
  4. Camera Kinematic Model
  5. LiDAR and Frontal Camera Fusion
  6. Object Localization
  7. Desired Features Estimation
  8. ViKi-HyCo Control Law
  9. Evaluation and Results
  10. Experimental Setup
  11. Experiments with the Classic Visual Servoing Controller
  12. Experiments with ViKi-HyCo Method
  13. Comparative Experiments
  14. Robot Placement Experiments
  15. Results
  16. ...and 2 more sections

Figures (19)

  • Figure 1: BLUE: roBot for Localization in Unstructured Environments
  • Figure 2: Sensor transformation systems onboard the BLUE robot. The transformation is determined by the letter $\mathbf{T}$ where the upper index is the origin and the lower index is the destination $_{}^{origin}\mathbf{T}_{destination}$.
  • Figure 3: Kinematic model of BLUE robot.
  • Figure 4: Model of camera system. The point $\mathbf{P}$ corresponds in the image plane to $\mathbf{{f}}$.
  • Figure 5: LiDAR and fusion camera on the point clouds and in the image plane.
  • ...and 14 more figures