Squidgets: Sketch-based Widget Design for Scene Manipulation UIST 2025

Joonho Kim, Fanny Chevalier, Karan Singh

University of Toronto

Teaser image for Squidgets project — Humans naturally communicate desired scene changes by over-sketching (a, b). These sketched strokes, when aligned with visual abstraction curves of a scene (c), can effectively express intended changes to scene attributes (d, e). Squidgets or 'sketched widgets' enable interactive manipulation of scene attributes via scene curves (f–h), such as deforming avocado shape attributes to match its silhouette abstraction curve (f). All scene curves can function as squidgets, including predefined rig curves like gaze/jaw controls on a 3D facial rig (g), and user-drawn bookmark curves that capture attribute configurations (e.g., line-of-action curves) to pose an animated character (h).

Abstract

People naturally sketch strokes over graphical scenes to convey scene changes. We propose automatically interpreting these strokes to execute scene changes with squidgets (sketch-widgets), a novel sketch-based UI framework for direct scene manipulation. Squidgets are motivated by the observation that curves resulting from visually abstracting scene elements provide natural handles for the direct manipulation of scene parameters. Additional curves can be defined by users to author custom handles associated with scene attributes. Users manipulate a scene by simply drawing strokes, partially matched against scene curves to select a squidget and interactively control associated parameters. We present an implementation of squidgets within the 3D animation system Maya, showing 2D/3D stroke input to manipulate 2D/3D scenes. We report on a controlled experiment evaluating squidgets on 2D object translation and deformation tasks, and a broader informal study on squidget creation and manipulation.

Downloads

BibTeX


@inproceedings{Kim:Squidgets:2025,
  author    = {Joonho Kim and Fanny Chevalier and Karan Singh},
  title     = {Squidgets: Sketch-based Widget Design for Scene Manipulation},
  booktitle = {Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology},
  series    = {UIST '25},
  year      = {2025},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  location  = {Busan, Republic of Korea},
  doi       = {10.1145/3746059.3747690},
  url       = {https://doi.org/10.1145/3746059.3747690},
  isbn      = {979-8-4007-2037-6/2025/09}
}

Acknowledgements

We would like to thank our user-study participants for feedback; Jenny Oh for help organizing the user study; Damien Masson for data figures; Zhecheng Wang for curve matching; and collaborators credited below. Park Environment and Cartoon Bedroom Set modeled by @anim_matt. Valley Girl characters and Angela © Chris Landreth. Squirrels Rig © Animation Mentor 2022; used with permission (no endorsement). VR environments rendered in © Gravity Sketch. Fruit icons for the user study and in Figure 1 by NicoDigitalStore. This research was supported by NSERC.