Robot Tasks with Fuzzy Time Requirements from Natural Language Instructions

TL;DR

F fuzzy skills are introduced and investigations reveal that trapezoidal functions best approximate the users' satisfaction, suggesting that users are more lenient if the execution is specified further into the future.

Abstract

Natural language allows robot programming to be accessible to everyone. However, the inherent fuzziness in natural language poses challenges for inflexible, traditional robot systems. We focus on instructions with fuzzy time requirements (e.g., "start in a few minutes"). Building on previous robotics research, we introduce fuzzy skills. These define an execution by the robot with so-called satisfaction functions representing vague execution time requirements. Such functions express a user's satisfaction over potential starting times for skill execution. When the robot handles multiple fuzzy skills, the satisfaction function provides a temporal tolerance window for execution, thus, enabling optimal scheduling based on satisfaction. We generalized such functions based on individual user expectations with a user study. The participants rated their satisfaction with an instruction's execution at various times. Our investigations reveal that trapezoidal functions best approximate the users' satisfaction. Additionally, the results suggest that users are more lenient if the execution is specified further into the future.

Figures (8)

• Figure 1: Natural language instructions are inherently fuzzy, requiring an interpretation within the context of the instruction and instructor. Here, the user's satisfaction varies over time based on the start of the task execution.
• Figure 2: The actor must cope with fuzzy time specifications for each instruction $I_i$ (issued at $0$ s). We consider the user satisfaction ($\psi_i$) when the specific skills ($s_i$) are started, indicated by the boxes $s_1$ to $s_3$. Non-optimized scheduling may lead to low satisfaction ($s_1$, $s_3$) and overlaps.
• Figure 3: The dependency tree encodes grammatical connections between words in a sentence. These dependencies include oblique temporal modifiers (obl:tmod), numbers (nummod), adverbs (advmod), and prepositions (case).
• Figure 4: Users' expectations regarding executing one instruction may vary. Given the instruction "The assignment should start in 30 minutes!", our study participants drew their satisfaction functions. In (a), only ten functions are shown for a better overview. The distribution across all participants is displayed using a histogram (b) or distribution plot (c). In c), the area between the 25- and 75-quantiles is shaded; the minimum and maximum values are dotted.
• Figure 5: Participants of our online user study should draw their satisfaction given instructions with fuzzy time requirements if the task started at the time stamp. Before carrying out the first task, this tutorial page was shown to the participants -- highlighting the core functionalities of the interface.