BrickSim: A Physics-Based Simulator for Manipulating Interlocking Brick Assemblies

Abstract

Interlocking brick assemblies provide a standardized yet challenging testbed for contact-rich and long-horizon robotic manipulation, but existing rigid-body simulators do not faithfully capture snap-fit mechanics. We present BrickSim, the first real-time physics-based simulator for interlocking brick assemblies. BrickSim introduces a compact force-based mechanics model for snap-fit connections and solves the resulting internal force distribution using a structured convex quadratic program. Combined with a hybrid architecture that delegates rigid-body dynamics to the underlying physics engine while handling snap-fit mechanics separately, BrickSim enables real-time, high-fidelity simulation of assembly, disassembly, and structural collapse. On 150 real-world assemblies, BrickSim achieves 100% accuracy in static stability prediction with an average solve time of 5 ms. In dynamic drop tests, it also faithfully reproduces real-world structural collapse, precisely mirroring both the occurrence of breakage and the specific breakage locations. Built on Isaac Sim, BrickSim further supports seamless integration with a wide variety of robots and existing pipelines. We demonstrate robotic construction of brick assemblies using BrickSim, highlighting its potential as a foundation for research in dexterous, long-horizon robotic manipulation. BrickSim is open-source, and the code is available at https://github.com/intelligent-control-lab/BrickSim.

Figures (8)

• Figure 1: BrickSim Overview.BrickSim enables high-fidelity simulation of complex brick assembly tasks and seamless integration with diverse robotic platforms. This teaser shows a representative multi-robot workflow for building a brick table alongside a gallery of diverse brick structures. All robotic and structural assets are natively instantiated within the BrickSim environment, arranged here for illustrative purposes.
• Figure 2: Comparison of the temporal behavior of a physically stable brick assembly in MuJoCo todorov2012mujoco and BrickSim. The real assembly remains intact. Starting from the same initial state, the assembly simulated in MuJoCo progressively collapses, while BrickSim preserves the structure, consistent with real-world behavior.
• Figure 2: Comparison of Static Structural Stability Analysis. Stability analysis results on 150 brick assemblies. Ground-truth stability is established by building each structure in reality. Accuracy and average solve time are computed over solvable structures only.
• Figure 3: BrickSim Architecture.BrickSim is built on top of Isaac Sim and augments its simulation stack (blue). The Brick Topology Graph (yellow) maintains a snap-fit connectivity graph between bricks and synchronizes it to the physics engine. The Assembly Monitor (green) detects valid stud-hole engagements and adds new connections, while the Breakage Detector (red) evaluates existing connections under external loads and removes those that fail. BrickSim provides a configurable simulation environment that supports diverse robotic platforms and a rich set of brick assets, including individual bricks and predefined assemblies. Bricks can be manipulated directly from the UI or by a robot controlled via teleoperation or custom policies.
• Figure 4: Brick and Connection Modeling. (a) Brick geometries in the real world and the approximated collision models in BrickSim. (b) Depending on the dimensions of the upper brick, each stud has 3 or 4 contact points with the corresponding hole, represented as red spheres. (c) Bricks are assembled by inserting the studs of the lower brick (green) into the holes of the upper brick (blue). At each contact point $f$, the contact force on the lower brick is decomposed into three components: the effective radial force $F_{r,\mathrm{eff}}^{(f)} = F_r^{(f)} + F_0$, where $F_r^{(f)}$ is the variable radial force and $F_0$ is the preload force due to snap-fit, together with the axial component $F_a^{(f)}$ and the tangential component $F_t^{(f)}$ of the static friction force which resist separation and twisting.