Budget-Aware Sequential Brick Assembly with Efficient Constraint Satisfaction

TL;DR

A new method to predict the scores of the next brick position by employing a U-shaped sparse 3D convolutional neural network and a one-initialized brick-sized convolution filter that can readily consider several different brick types by benefiting from modern implementation of convolution operations.

Abstract

We tackle the problem of sequential brick assembly with LEGO bricks to create combinatorial 3D structures. This problem is challenging since this brick assembly task encompasses the characteristics of combinatorial optimization problems. In particular, the number of assemblable structures increases exponentially as the number of bricks used increases. To solve this problem, we propose a new method to predict the scores of the next brick position by employing a U-shaped sparse 3D convolutional neural network. Along with the 3D convolutional network, a one-initialized brick-sized convolution filter is used to efficiently validate assembly constraints between bricks without training itself. By the nature of this one-initialized convolution filter, we can readily consider several different brick types by benefiting from modern implementation of convolution operations. To generate a novel structure, we devise a sampling strategy to determine the next brick position considering the satisfaction of assembly constraints. Moreover, our method is designed for either budget-free or budget-aware scenario where a budget may confine the number of bricks and their types. We demonstrate that our method successfully generates a variety of brick structures and outperforms existing methods with Bayesian optimization, deep graph generative model, and reinforcement learning.

Figures (7)

• Figure 1: Illustration of the efficient constraint satisfaction method with convolution filters for sequential brick assembly. U-Net and $\odot$ denote a U-shaped neural network with sparse 3D convolutional layers and element-wise multiplication, respectively.
• Figure 2: Illustration of how one-initialized brick-size convolution filters work where a red brick is a pivot brick and a green brick has been assembled to the red brick. The grid shows the validity $\mathbf{V}_{t+1}$ of each brick position on the red brick for a brick type $2\!\times\!2$. The pixels in a red rectangle indicate attachable brick positions on the red brick. The green pixels indicate that overlap with the green brick will occur if a new brick is attached to the corresponding position. Therefore, (a) and (c) violate no-overlap and no-isolation, respectively, and (b) satisfies the assembly constraints.
• Figure 3: Qualitative results of structure generation. Best viewed in color.
• Figure 4: Qualitative results of BrECS with $2\!\times\!2$, $2\!\times\!4$, and $2\!\times\!8$ brick types for structure generation. Best viewed in color.
• Figure 5: Overall procedure of our method.