Capturing Non-Linear Human Perspective in Line Drawings

TL;DR

This work addresses the gap between artist-drawn line drawings and analytic perspective by learning a 3D, spatially varying perspective deviation $\mathbf{D}(\mathbf{p})$ via an MLP. It aligns sketch contours to 3D shape projections using an HMM-based matching and optimizes $\mathbf{D}(\mathbf{p})$ with a multi-term loss that preserves shape and slope while enforcing smoothness and depth consistency, aided by self-augmentation for generalization. The approach yields human-like, view-consistent projections across unseen shapes and viewpoints, outperforming prior 2D baselines and ablations, and enabling applications in NPR rendering and editing that respect artist-specific perspective biases. These results deepen our understanding of perceptual biases in sketching and offer practical tools for rendering and analyzing human perspective in line drawings.

Abstract

Artist-drawn sketches only loosely conform to analytical models of perspective projection; the deviation of human-drawn perspective from analytical perspective models is persistent and well documented, but has yet to be algorithmically replicated. We encode this deviation between human and analytic perspectives as a continuous function in 3D space and develop a method to learn it. We seek deviation functions that (i)mimic artist deviation on our training data; (ii)generalize to other shapes; (iii)are consistent across different views of the same shape; and (iv)produce outputs that appear human-drawn. The natural data for learning this deviation is pairs of artist sketches of 3D shapes and best-matching analytical camera views of the same shapes. However, a core challenge in learning perspective deviation is the heterogeneity of human drawing choices, combined with relative data paucity (the datasets we rely on have only a few dozen training pairs). We sidestep this challenge by learning perspective deviation from an individual pair of an artist sketch of a 3D shape and the contours of the same shape rendered from a best-matching analytical camera view. We first match contours of the depicted shape to artist strokes, then learn a spatially continuous local perspective deviation function that modifies the camera perspective projecting the contours to their corresponding strokes. This function retains key geometric properties that artists strive to preserve when depicting 3D content, thus satisfying (i) and (iv) above. We generalize our method to alternative shapes and views (ii, iii) via a self-augmentation approach that algorithmically generates training data for nearby views, and enforces spatial smoothness and consistency across all views. We compare our results to potential alternatives, demonstrating the superiority of the proposed approach.

Figures (10)

• Figure 1: Algorithm overview: Given an input sketch (black) and corresponding analytically projected 3D shape contours (orange) in a matching view (a), we match the contours to artist strokes (b). We then use a two-step process to learn the deviation between the contour and sketch projections (c-e): we learn an initial deviation function $\mathbf{D}\xspace({\mathbf{p}}\xspace)$ that balances satisfying the computed matches against adherence to core deviation properties and apply the learned deviation to the input contours (c); we augment our learning data with synthetic sketch/contour pairs and re-learn a deviation that best fits the augmented training set (d,e). The contours projected using our deviation align with the artist's strokes (f). Our deviation consistently generalizes to other views (g).
• Figure 2: We seek for artist-intended matches between contour curves and sketch strokes (a). Using only vertex-to-vertex matching scores whether feature based (DIFT tang2023emergent) (b) or our compatibility based (c) produces locally optimal but globally poor matches; accounting for consistency (d) produces better matches at contour level but can still lead to instances where multiple curves match the same stroke (see inset zoom). Our second matching step resolved these undesirable many-to-one matches (e).
• Figure 3: Self-augmentation impact: (a) input sketch and analytically projected contours; (b) same view inference output using first learned MLP; (c) same view inference output using data augmented MLP, alternative view inference output using first learned MLP (d), alternative view inference output using data augmented MLP (e). The impact of augmentation is more notable for further away camera views.
• Figure 4: Comparison versus DifferSketching xiao2022differsketching. (a) same view/shape sketches in DifferSketching dataset; (b) analytically projected contours of the depicted shapes in a matching view; (c-e) Applying DifferSketching to the contours in (b): (c) DifferSketching pre-trained on full data corpus; (d) trained only on the sketches in (a) and the contours in (b); (e) trained on the blue sketch in (a) and the contour in (b); (f) Our outputs trained the blue sketch in (a) and the contour in (b). In all cases, DifferSketching outputs are distorted, and quality reduces as training set size decreases. Our method capture the artists' deviation and generalizes.
• Figure 5: A gallery of our results. We show (a) overlay of human sketches and shape contours in best matching view (insets show the sketches and contours separately); (b) our learned outputs under the same view as (a); (c) Applying our learned perspectives to contours in different other rotated views. In all examples our outputs match the sketches' perspective deviations.