Table of Contents
Fetching ...

Investigating Remote Hands-On Assistance for Collaborative Development of Embedded Systems

Yan Chen, Jasmine Jones

TL;DR

This study addresses the gap in remote collaboration tools for embedded systems by combining qualitative interviews with a hypothetical teleoperation prototype, Handy. It reveals that developers need richer visual context and safe, controllable physical manipulation to support hardware tasks, beyond what software-centric collaboration tools offer. The authors propose design directions—privacy-aware visual tools, annotation-rich AR guidance, and robotic-assisted hands-on support with controllable autonomy—to enable on-demand, remote collaboration for embedded development. The findings provide an empirical foundation for future remote manipulation systems that can streamline debugging, prototyping, and knowledge sharing in hardware-centric contexts.

Abstract

Developing embedded systems is a complex endeavor that frequently requires collaborative teamwork. With the rise of freelance work and the global shift towards remote work, the need for effective remote collaboration has become crucial for many developers and their clients. However, current communication and coordination tools are predominantly tailored for software development rather than hardware-focused tasks. This study investigates the potential for remote support tools specifically designed for embedded systems development. Through interviews with 12 experienced embedded systems developers, we explored their existing remote work practices, challenges, and requirements. We also conducted a user enactment study featuring a custom-designed remote manipulation agent, Handy, as a theoretical assistant, to identify the kinds of support developers would value in a collaborative setting. Our findings highlight the scenarios and strategies employed in remote work, the specific support needs, and the challenges related to information exchange, coordination, and execution. Additionally, we explore concerns around privacy, control, and trust when using remote physical manipulation tools. This research contributes to the field by integrating the development of embedded systems with the remote, on-demand collaboration and assistance typical of software environments, offering a solid empirical foundation for future research on remote manipulation agents in this area.

Investigating Remote Hands-On Assistance for Collaborative Development of Embedded Systems

TL;DR

This study addresses the gap in remote collaboration tools for embedded systems by combining qualitative interviews with a hypothetical teleoperation prototype, Handy. It reveals that developers need richer visual context and safe, controllable physical manipulation to support hardware tasks, beyond what software-centric collaboration tools offer. The authors propose design directions—privacy-aware visual tools, annotation-rich AR guidance, and robotic-assisted hands-on support with controllable autonomy—to enable on-demand, remote collaboration for embedded development. The findings provide an empirical foundation for future remote manipulation systems that can streamline debugging, prototyping, and knowledge sharing in hardware-centric contexts.

Abstract

Developing embedded systems is a complex endeavor that frequently requires collaborative teamwork. With the rise of freelance work and the global shift towards remote work, the need for effective remote collaboration has become crucial for many developers and their clients. However, current communication and coordination tools are predominantly tailored for software development rather than hardware-focused tasks. This study investigates the potential for remote support tools specifically designed for embedded systems development. Through interviews with 12 experienced embedded systems developers, we explored their existing remote work practices, challenges, and requirements. We also conducted a user enactment study featuring a custom-designed remote manipulation agent, Handy, as a theoretical assistant, to identify the kinds of support developers would value in a collaborative setting. Our findings highlight the scenarios and strategies employed in remote work, the specific support needs, and the challenges related to information exchange, coordination, and execution. Additionally, we explore concerns around privacy, control, and trust when using remote physical manipulation tools. This research contributes to the field by integrating the development of embedded systems with the remote, on-demand collaboration and assistance typical of software environments, offering a solid empirical foundation for future research on remote manipulation agents in this area.
Paper Structure (34 sections, 3 figures, 1 table)

This paper contains 34 sections, 3 figures, 1 table.

Figures (3)

  • Figure 1: A screenshot of a participant verbalizing a request to the hypothetical assistant while working on the hardware aspect of their embedded systems project during the study in Phase 2.
  • Figure 2: Web UI controls. This includes four parts: 1) a natural-language task description, 2) a code editor that includes the task-relevant code, 3) two fixed-position live stream camera views, and 4) a six-slider control panel that corresponds to the six servos of the robot arm.
  • Figure 3: A photo of the robot arm performing a wiring task.