Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Modular Object Detection System for Humanoid Robots Using YOLO

Nicolas Pottier, Meng Cheng Lau

TL;DR

This work tackles the challenge of real-time vision for constrained humanoid robots by developing a modular YOLOv9-based perception module integrated with ROS1. It uses CPU-focused optimizations (frame skipping, input resizing, ONNX/OpenVINO conversion, and quantization) and a WebSocket bridge to bridge Python 3-based inference with ROS 1. The authors train task-specific lightweight models for FIRA Hurocup events and evaluate the system against a geometric baseline, demonstrating competitive precision and improved robustness in dynamic, cluttered environments, albeit at lower FPS. The results underscore the viability of deploying modern, data-driven object detectors on resource-limited robots and highlight a path toward hybrid perception pipelines that combine the strengths of both deep learning and traditional geometry for robust, real-time robotic perception.

Abstract

Within the field of robotics, computer vision remains a significant barrier to progress, with many tasks hindered by inefficient vision systems. This research proposes a generalized vision module leveraging YOLOv9, a state-of-the-art framework optimized for computationally constrained environments like robots. The model is trained on a dataset tailored to the FIRA robotics Hurocup. A new vision module is implemented in ROS1 using a virtual environment to enable YOLO compatibility. Performance is evaluated using metrics such as frames per second (FPS) and Mean Average Precision (mAP). Performance is then compared to the existing geometric framework in static and dynamic contexts. The YOLO model achieved comparable precision at a higher computational cost then the geometric model, while providing improved robustness.

A Modular Object Detection System for Humanoid Robots Using YOLO

TL;DR

This work tackles the challenge of real-time vision for constrained humanoid robots by developing a modular YOLOv9-based perception module integrated with ROS1. It uses CPU-focused optimizations (frame skipping, input resizing, ONNX/OpenVINO conversion, and quantization) and a WebSocket bridge to bridge Python 3-based inference with ROS 1. The authors train task-specific lightweight models for FIRA Hurocup events and evaluate the system against a geometric baseline, demonstrating competitive precision and improved robustness in dynamic, cluttered environments, albeit at lower FPS. The results underscore the viability of deploying modern, data-driven object detectors on resource-limited robots and highlight a path toward hybrid perception pipelines that combine the strengths of both deep learning and traditional geometry for robust, real-time robotic perception.

Abstract

Within the field of robotics, computer vision remains a significant barrier to progress, with many tasks hindered by inefficient vision systems. This research proposes a generalized vision module leveraging YOLOv9, a state-of-the-art framework optimized for computationally constrained environments like robots. The model is trained on a dataset tailored to the FIRA robotics Hurocup. A new vision module is implemented in ROS1 using a virtual environment to enable YOLO compatibility. Performance is evaluated using metrics such as frames per second (FPS) and Mean Average Precision (mAP). Performance is then compared to the existing geometric framework in static and dynamic contexts. The YOLO model achieved comparable precision at a higher computational cost then the geometric model, while providing improved robustness.
Paper Structure (26 sections, 7 figures, 5 tables)

This paper contains 26 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Object Detection
  4. YOLO: You Only Look Once
  5. Robotic Applications & Vision Modules
  6. Research Gap
  7. Methodology
  8. Hardware Configuration
  9. YOLOv9 Preparation and Training
  10. ROS Integration
  11. Performance Evaluation
  12. Implementation
  13. Model Deployment
  14. ROS Node Developpement
  15. Pipeline and Behavior Integration
  16. ...and 11 more sections

Figures (7)

  • Figure 1: LIMRL's two humanoid robots. ROBOTIS-OP3, also known as Oscar (pictured left), will be used for this implementation. The vision module will also be installed on Polaris (pictured right).
  • Figure 2: Flowchart showing the interaction between ROS Nodes within the vision model's various environments. The ROS Bridge WebSocket is essential to ensure that both environments can communicate properly.
  • Figure 3: Graph showing the pipeline utilized by Event scripts. The graph shown is for the basketball event.
  • Figure 4: The confusion matrix for YOLOv9-T's marathon model. The matrix suggests the model is frequently confusing left and right arrows due to their similar appearance.
  • Figure 5: All experiments, both static and dynamic, will be conducted on the LIMRL's turf field, similar to the ones utilized at the FIRA competition. This photo shows the setup for the Basketball Event.
  • ...and 2 more figures