LassoNet: Deep Lasso-Selection of 3D Point Clouds

TL;DR

LassoNet addresses the challenge of robust, efficient lasso-selection in 3D point clouds by learning a latent mapping f(P,V,L) → P_s from viewpoint and 2D lassos to 3D regions. It introduces a three-stage pipeline—Interaction Encoding, Filtering & Sampling, and a hierarchical PointNet++-style network—to handle data heterogeneity and scale, trained on a dataset of over 30K annotated records across ShapeNet and S3DIS. A formal user study demonstrates that LassoNet significantly improves both efficiency and accuracy over state-of-the-art methods, including CylinderSelection and SpaceCast, across multiple datasets and viewpoints. The work advances interactive visualization for 3D data by enabling accurate multi-region selections in cluttered and occluded scenes, with practical implications for exploratory analysis and visualization pipelines.

Abstract

Selection is a fundamental task in exploratory analysis and visualization of 3D point clouds. Prior researches on selection methods were developed mainly based on heuristics such as local point density, thus limiting their applicability in general data. Specific challenges root in the great variabilities implied by point clouds (e.g., dense vs. sparse), viewpoint (e.g., occluded vs. non-occluded), and lasso (e.g., small vs. large). In this work, we introduce LassoNet, a new deep neural network for lasso selection of 3D point clouds, attempting to learn a latent mapping from viewpoint and lasso to point cloud regions. To achieve this, we couple user-target points with viewpoint and lasso information through 3D coordinate transform and naive selection, and improve the method scalability via an intention filtering and farthest point sampling. A hierarchical network is trained using a dataset with over 30K lasso-selection records on two different point cloud data. We conduct a formal user study to compare LassoNet with two state-of-the-art lasso-selection methods. The evaluations confirm that our approach improves the selection effectiveness and efficiency across different combinations of 3D point clouds, viewpoints, and lasso selections. Project Website: https://lassonet.github.io

Figures (9)

• Figure 1: To select both two wings of the airplane, two sequential lassos from different viewpoints are required, and selection results are combined using either union (top) or subtraction (bottom) Boolean operations.
• Figure 2: LassoNet consists of three stages: In Interaction Encoding stage, we associate point cloud with viewpoint and lasso through 3D coordinate transformation and naive selection; In Filtering and Sampling stage, we reduce the amount of points for network processing through intention filtering and farthest point sampling. Lastly, we build a hierarchical neural network in Network Building stage.
• Figure 3: Overview of network building. The DNN network is built upon (a) PointNet Qi2017a, and we employ a hierarchical structure that generates more local and global features using (b) abstraction and (c) propagation components.
• Figure 4: Exemplar annotation records for point clouds in ShapeNet (left) and S3DIS (right): target and interfering points are colored in yellow and blue respectively, while lassos are in red color.
• Figure 7: Jaccard distances per epoch in training and testing processes for ShapeNet (left) and S3DIS (right) annotations.