Toward Autonomous Driving by Musculoskeletal Humanoids: A Study of Developed Hardware and Learning-Based Software

TL;DR

This paper tackles autonomous driving with a musculoskeletal humanoid by leveraging human-like body proportions, flexible actuation, and redundant sensing in the Musashi platform.A learning-based software stack with static/inter sensory, dynamic task control, reflex, and recognition modules enables steering and pedal operations under perception and safety constraints, demonstrated via pedestal and steering experiments with recognition.Key contributions include a modular hardware design enabling steering with both arms and a flexible hand/foot sensing system, along with a four-component software architecture that handles static and dynamic motions and safety reflexes.The work highlights practical limitations such as environmental variability, perception under day-time conditions, and slow steering execution, and outlines concrete future directions for online learning, improved sensing, and integration with broader autonomy stacks.

Abstract

This paper summarizes an autonomous driving project by musculoskeletal humanoids. The musculoskeletal humanoid, which mimics the human body in detail, has redundant sensors and a flexible body structure. These characteristics are suitable for motions with complex environmental contact, and the robot is expected to sit down on the car seat, step on the acceleration and brake pedals, and operate the steering wheel by both arms. We reconsider the developed hardware and software of the musculoskeletal humanoid Musashi in the context of autonomous driving. The respective components of autonomous driving are conducted using the benefits of the hardware and software. Finally, Musashi succeeded in the pedal and steering wheel operations with recognition.

Figures (8)

• Figure 1: Autonomous driving by musculoskeletal humanoids
• Figure 2: The overview of the developed musculoskeletal humanoid Musashi. (a) shows the basic musculoskeletal structure. (b) shows the modularized structure of the left arm of Musashi combining various modules, (c) shows the rearrangeable concept of joint modules. (d) shows the detailed structure of muscle module (left figure) and the versatility of muscle arrangement (right figure). (e) shows the nonlinear elastic unit. (f) shows the movable eye unit. (g) shows the flexible musculoskeletal hand with machined springs. (h) shows the foot with six-axis core-shell structural force sensors.
• Figure 3: The realization of respective components of autonomous driving using the characteristics of the developed hardware. (a) shows a steering wheel operation with both arms. (b) shows a variable stiffness control using the nonlinear elastic units. (c) shows the field of view of the movable eye unit. (d) shows the human detection experiment in the side mirror using the high resolution camera. (e) shows an experiment pulling a handbrake. (f) shows an experiment rotating a key. (g) shows an experiment operating a blinker lever by changing the stiffness of fingers. (h) shows a recovering experiment from slipping during brake pedal operation using the developed foot.
• Figure 4: The overview of the developed software with four modules: intersensory network module (static module), dynamic task control network module (dynamic module), reflex module, and recognition module.
• Figure 5: Respective components of autonomous driving using the developed software. (a) shows a steering wheel operation experiment using the online learning of static module. (b) shows a pedal operation experiment using the trained dynamic module. (c) shows a steering wheel operation experiment with and without muscle relaxation control. (d) shows visual recognition of traffic lights and a human, and sound recognition of a car horn.