Shared Autonomy for Proximal Teaching

TL;DR

Z-COACH introduces a theory-driven framework for proximal teaching in motor control by leveraging shared autonomy to both model a learner's Zone of Proximal Development and provide skill-targeted coaching. It combines interpretable skill discovery with language-augmented supervision and a dual-use shared autonomy policy to adjust assistance for learning, evaluated on a high-performance racing task in CARLA with 50 human subjects. Results show improved lap performance, smoother trajectories, and more expert-like steering when using SkillSA coaching, with strong evidence that assistance should be tailored to the learner's current ZPD. The work demonstrates a practical pathway for using semi-autonomous systems not only to assist but to actively teach complex motor skills, with implications for driving, rehabilitation, and other domains requiring structured skill progression.

Abstract

Motor skill learning often requires experienced professionals who can provide personalized instruction. Unfortunately, the availability of high-quality training can be limited for specialized tasks, such as high performance racing. Several recent works have leveraged AI-assistance to improve instruction of tasks ranging from rehabilitation to surgical robot tele-operation. However, these works often make simplifying assumptions on the student learning process, and fail to model how a teacher's assistance interacts with different individuals' abilities when determining optimal teaching strategies. Inspired by the idea of scaffolding from educational psychology, we leverage shared autonomy, a framework for combining user inputs with robot autonomy, to aid with curriculum design. Our key insight is that the way a student's behavior improves in the presence of assistance from an autonomous agent can highlight which sub-skills might be most ``learnable'' for the student, or within their Zone of Proximal Development. We use this to design Z-COACH, a method for using shared autonomy to provide personalized instruction targeting interpretable task sub-skills. In a user study (n=50), where we teach high performance racing in a simulated environment of the Thunderhill Raceway Park with the CARLA Autonomous Driving simulator, we show that Z-COACH helps identify which skills each student should first practice, leading to an overall improvement in driving time, behavior, and smoothness. Our work shows that increasingly available semi-autonomous capabilities (e.g. in vehicles, robots) can not only assist human users, but also help *teach* them.

Figures (6)

• Figure 1: We evaluate Z-COACH on a high performance racing task in a simulated environment of the Thunderhill Raceway Park. Z-COACH identifies which task skills an individual student can only perform with assistance (i.e. within their Zone of Proximal Development), and then provides targeted coaching via skill-focused shared autonomy. Students receiving coaching from Z-COACH generally learned smoother racing lines than students practicing independently for the equivalent amount of time, as shown above by overlaying the trajectories from all participants in our human subject study ($n=50$).
• Figure 2: Overview of Z-COACH, which consists of three stages: (1) an interpretable skill discovery stage to identify task-relevant skills to guide coaching, (2) student modeling, which leverages shared autonomy to identify how a driver's behavior changes with assistance in order to choose skills within their "zone of proximal development", and (3) skill-targeted coaching, which leverages a different form of shared autonomy that forces drivers to control and practice a specific skill. Note that shared autonomy is leveraged twice by Z-COACH: for student modeling and for coaching.
• Figure 3: (Left) Image of our study environment for the High Performance Racing task in simulation. We use the CARLA simulator, with an external Logitech G29 steering wheel and pedals. (Middle) Image of the study interface used during coaching. Participants are provided control input information on a sidebar, given verbal guidance on which skill $z$ they should practice, and drive on the track from ego view. (Right) Participant breakdown of our user study.
• Figure 4: Output segmentations produced by CompILE over an expert driver's trajectory around the Thunderhill West race track. By modifying CompILE to take as input noisy language annotations, the resulting segmentation is more interpretable and aligned with human notions of skills.
• Figure 5: (Left) Comparison of overall performance between Baseline trials, Expert trials, and Evaluation trials from participants either assigned Self-Practice or receiving assistance via Z-COACH, across a variety of quantitative metrics highlights improvement from Z-COACH. (Middle) Fourier transform signal analysis plot of the steering wheel input shows that Z-COACH guides students towards similar steering behavior as an expert HPR driver, including decreasing the amount of large, low frequency turns. (Right) Participant feedback shows that SkillSA, used for coaching, is found challenge participants significantly more than StrongSA, yet comparable in helpfulness. * marks statistical significance ($p < 0.05$ with a paired t-test.)