Squidgets: Sketch-based Widget Design and Direct Manipulation of 3D Scene

TL;DR

Squidgets present a stroke-based UI framework for direct, in-situ manipulation of 3D scenes by leveraging user-drawn strokes as handles on curve abstractions. The method frames interaction as an inverse rendering problem that maps a stroke $S$ to attribute changes $A'$ so that the resulting squidget curve $C'=f(A')$ aligns with $S$, using both implicit scene contours and explicitly authored curves. The framework combines implicit NPR-derived curves with user-authored explicit curves, supported by a curve similarity metric to select and update scene attributes, and is implemented in Maya with creation and control workflows. Evaluations with casual users and animation professionals indicate a generally favorable reception, highlighting the approach's intuition and modularity while noting issues like ambiguity and control granularity, which point to opportunities for future refinement and expansion to broader attribute control.

Abstract

Squidgets or 'sketch-widgets' is a novel stroke-based UI framework for direct scene manipulation. Squidgets is motivated by the observation that sketch strokes comprising visual abstractions of scene elements implicitly provide natural handles for the direct manipulation of scene parameters. Configurations of such strokes can further be explicitly drawn by users to author custom widgets associated with scene attributes. Users manipulate a scene by simply drawing strokes: a squidget is selected by partially matching the drawn stroke against both implicit scene contours and explicitly authored curves, and used in-situ to interactively control scene parameters associated with the squidget. We present an implementation of squidgets within the 3D modeling animation system Maya, and report on an evaluation of squidget creation and manipulation, by both casual users and professional artists.

Figures (16)

• Figure 1: Implicit Squidgets comprise a set of perceived curves (2D or 3D) that are already scene curves or computed using an NPR contour rendering of scene objects. In the default control mode, user drawn strokes are matched against all scene squidget curves (explicit and implicit), scene attributes corresponding to the closest matched squidget are then computed and set so the squidget curve best-fits the user stroke, and further be interactively dragged to refine the associated attributes values in real-time.
• Figure 2: Explicit squidgets comprise a set of curves (2D or 3D) that are explicitly authored in ( create mode). During creation, curves are drawn (on planar canvases or directly in 3D) and associated with selected scene object attributes, to form discrete squidget curves, which serve as a bookmark for the configuration of attribute values when created. Multiple discrete squidgets can be connected to define a continuous squidget that interpolates the curves, as well as the union of their bookmarked attribute values. In control mode, user drawn strokes on canvases or 3D objects are matched against explicit and implicit squidgets and the closest matching squidget is selected to manipulate its associated attributes.
• Figure 3: All perceived scene curves are potential squidgets including: real curves (left) that are part of the scene's construction (green), or explicitly authored squidgets (white); and virtual curves (center) imagined by a user (highlighted in red). A user draw stroke (cyan) can manipulate scene attributes (right), such that their associated squidget curve matches the user stroke.
• Figure 4: Explicitly created squidget curves can be manifest on surfaces (a), canvases in space (b-c), or using VR controls (d).
• Figure 5: Explicitly authored curves can be associated with discrete configurations of face attributes to capture a smile (left) and a laugh (right); and further combined into a continuous squidget, where the curves and related face attributes can be interpolated (center).