Inferring geometric constraints in human demonstrations

TL;DR

The paper tackles inferring geometric constraints from human demonstrations by jointly identifying constraint type and parameters using both kinematic data and force/moment measurements. It introduces a generalized rigid-body constraint framework and a library of six geometric constraint models, each with semantic parameters, and fits them per demonstration segment via nonlinear least squares. Ambiguities arising from kinematics are mitigated by incorporating force/torque data and a sample-voting scheme to select the most consistent constraint. Experimental validation with instrumented tools demonstrates robust constraint inference across multiple DOFs and shows that force/moment information improves disambiguation. The approach advances programming-by-demonstration by enabling autonomous extraction of constraint models for hybrid force-position control and richer, semantically meaningful representations of demonstrated tasks.

Abstract

This paper presents an approach for inferring geometric constraints in human demonstrations. In our method, geometric constraint models are built to create representations of kinematic constraints such as fixed point, axial rotation, prismatic motion, planar motion and others across multiple degrees of freedom. Our method infers geometric constraints using both kinematic and force/torque information. The approach first fits all the constraint models using kinematic information and evaluates them individually using position, force and moment criteria. Our approach does not require information about the constraint type or contact geometry; it can determine both simultaneously. We present experimental evaluations using instrumented tongs that show how constraints can be robustly inferred in recordings of human demonstrations.

Figures (5)

• Figure 1: (a) Stylus (Rigid body) – A human demonstrator moves the stylus (using instrumented tongs shown in figure (\ref{['fig:tongs']}) with its tip against a planar surface to estimate the planar surface geometry and tool tip location. (b) shows the trajectory of the rigid body in green. After the demonstration, our algorithm estimates both the location of the plane in the global coordinate frame and the tool tip position relative to the rigid body frame. (c) The estimated plane location compared to ground truth is within the tolerance (sub-mm) of the motion capture system.
• Figure 2: Geometric constraints and their physical counterparts
• Figure 3: (a) Instrumented tongs, (b) Constraint Sabre, (c) Inferred constraint motions. The motions are correctly segmented into free space motion (grey) and constrained motion (colored).
• Figure 4: Left - Shows recorded motion interacting with an axial rotation constraint. Both planar and axial rotation constraint geometry fit the recorded motions. Right - While position errors for both the planar (green) and axial rotation (blue) models are similar, the force errors are significant for the planar one and can be used to determine the axial rotation model as the correct constraint model.
• Figure 5: The point on plane constraint performed with the instrumented tongs. The recorded trajectory is similar to the one shown in figure \ref{['fig:teaser']}. Left - shows the fit error(m) of the point on plane constraint as a function of the number of contiguous samples used during fitting. Projected images of the stylus motion are shown indicating the amount of information over the demonstration. In this particular example, only after reaching 1 second of motion does the fit error drop to an acceptable value. Right - Samples are selected randomly from the entire demonstration and fit. The fit error drops with an increase in the number of samples. Randomly selected samples are usually distinct and provide richer information than an equal number of contiguous samples.