Hibikino-Musashi@Home 2025 Team Description Paper

TL;DR

This work addresses the challenge of building capable home-service robots by integrating a synthetic dataset generator for vision, a modular perception stack (YOLOv8, GroundingDINO, NanoSAM), a large-language-model–driven task planner, and a brain-inspired episodic memory system for home adaptation. It combines simulation-based data generation with an open-source development environment to enable rapid development and testing, while reusing established navigation components from RoboCup communities. The key contributions include a dual-object-recognition pipeline, a grasping-pose estimation approach derived from 3D point clouds, a low-power hand-waving action recognizer, and semantic-map–assisted navigation, all orchestrated by an LLM-based planner. The work aims to bridge research and real-world deployment by delivering open-source tools, energy-efficient memory hardware, and continual competition-driven evaluation to refine home-service robotics capabilities.

Abstract

This paper provides an overview of the techniques employed by Hibikino-Musashi@Home, which intends to participate in the domestic standard platform league. The team developed a dataset generator for training a robot vision system and an open-source development environment running on a Human Support Robot simulator. The large-language-model-powered task planner selects appropriate primitive skills to perform the task requested by the user. Moreover, the team has focused on research involving brain-inspired memory models for adaptation to individual home environments. This approach aims to provide intuitive and personalized assistance. Additionally, the team contributed to the reusability of the navigation system developed by Pumas in RoboCup2024. The team aimed to design a home service robot to assist humans in their homes and continuously attend competitions to evaluate and improve the developed system.

Figures (6)

• Figure 1: Example of the prompts for GroundingDino + NanoSAM.
• Figure 2: Overview of the grasping point estimation.
• Figure 3: Overview of the hand waving recognition hand_waving_recognition
• Figure 4: Example of a semantic map
• Figure 5: Overview of the task planning \ref{['fig:task_planning']}