PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision

TL;DR

PICASSO tackles the challenge of parameterizing CAD sketches from raster images when parameter-level annotations are scarce. It combines a feed-forward SPN with a differentiable SRN to enable rendering-based, image-level self-supervision for pretraining, enabling zero-shot and few-shot CAD sketch parameterization. The approach achieves strong performance against autoregressive baselines and other non-autoregressive methods, while offering faster inference and robust cross-dataset generalization; it also enables test-time refinement via SRN. Overall, PICASSO reduces reliance on annotated CAD sketches, accelerates parameterization, and opens pathways for integrating hand-drawn sketches into CAD workflows and even 3D modeling.

Abstract

This work introduces PICASSO, a framework for the parameterization of 2D CAD sketches from hand-drawn and precise sketch images. PICASSO converts a given CAD sketch image into parametric primitives that can be seamlessly integrated into CAD software. Our framework leverages rendering self-supervision to enable the pre-training of a CAD sketch parameterization network using sketch renderings only, thereby eliminating the need for corresponding CAD parameterization. Thus, we significantly reduce reliance on parameter-level annotations, which are often unavailable, particularly for hand-drawn sketches. The two primary components of PICASSO are (1) a Sketch Parameterization Network (SPN) that predicts a series of parametric primitives from CAD sketch images, and (2) a Sketch Rendering Network (SRN) that renders parametric CAD sketches in a differentiable manner and facilitates the computation of a rendering (image-level) loss for self-supervision. We demonstrate that the proposed PICASSO can achieve reasonable performance even when finetuned with only a small number of parametric CAD sketches. Extensive evaluation on the widely used SketchGraphs and CAD as Language datasets validates the effectiveness of the proposed approach on zero- and few-shot learning scenarios.

Figures (21)

• Figure 1: PICASSO is a novel self-supervised framework for CAD sketch parameterization. Unlike methods relying on parametric annotations, PICASSO is pretrained via rendering self-supervision on CAD sketch images only, thus drastically reducing the need for parametrically annotated sketches. We demonstrate PICASSO's effectiveness in both few-shot and zero-shot settings.
• Figure 2: Framework overview. PICASSO is composed of two networks, namely the Sketch Parametrization Network (SPN) and Sketch Rendering Network (SRN). Once trained, SRN is kept frozen and used for rendering self-supervision using a multiscale $l2$ loss. This allows for image-level pre-training of the CAD sketch parameterization network SPN.
• Figure 3: Overview of the Sketch Rendering Network (SRN). SRN is modeled by a transformer encoder-decoder that learns a mapping from parametric primitive tokens to the sketch image domain. SRN enables neural differentiable rendering that we leverage for rendering self-supervision of SPN.
• Figure 4: Overview of the Sketch Parameterization Network (SPN). An input raster sketch image is processed by a convolutional backbone and the produced feature map is fed to a transformer encoder-decoder for sketch parameterization. SPN is pre-trained using rendering self-supervision provided by SRN, allowing zero-shot CAD sketch parameterization, and finetuned with parameter-level annotations for few-shot scenario.
• Figure 5: Few-shot setting. Qualitative results of PICASSO learned CAD sketch parameterization from precise and hand-drawn sketches.