Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DNN-Based Topology Optimisation: Spatial Invariance and Neural Tangent Kernel

Benjamin Dupuis, Arthur Jacot

TL;DR

The paper addresses topology optimization with SIMP when the material density is generated by a neural network that takes spatial coordinates as input. By analyzing the Neural Tangent Kernel (NTK) in the infinite-width limit, it interprets the DNN as inducing a density filter whose characteristics—especially spatial invariance—depend on the embedding of coordinates and network architecture. It identifies translation invariance as a primary source of artifacts and proposes two invariant embeddings (hypertorus and random Fourier features) to enforce spatial invariance, with the filter size controllable via hyperparameters. Empirical results corroborate the theory and show that the learned densities can be upsampled smoothly, suggesting broad applicability to other coordinate-based density generation tasks.

Abstract

We study the Solid Isotropic Material Penalisation (SIMP) method with a density field generated by a fully-connected neural network, taking the coordinates as inputs. In the large width limit, we show that the use of DNNs leads to a filtering effect similar to traditional filtering techniques for SIMP, with a filter described by the Neural Tangent Kernel (NTK). This filter is however not invariant under translation, leading to visual artifacts and non-optimal shapes. We propose two embeddings of the input coordinates, which lead to (approximate) spatial invariance of the NTK and of the filter. We empirically confirm our theoretical observations and study how the filter size is affected by the architecture of the network. Our solution can easily be applied to any other coordinates-based generation method.

DNN-Based Topology Optimisation: Spatial Invariance and Neural Tangent Kernel

TL;DR

The paper addresses topology optimization with SIMP when the material density is generated by a neural network that takes spatial coordinates as input. By analyzing the Neural Tangent Kernel (NTK) in the infinite-width limit, it interprets the DNN as inducing a density filter whose characteristics—especially spatial invariance—depend on the embedding of coordinates and network architecture. It identifies translation invariance as a primary source of artifacts and proposes two invariant embeddings (hypertorus and random Fourier features) to enforce spatial invariance, with the filter size controllable via hyperparameters. Empirical results corroborate the theory and show that the learned densities can be upsampled smoothly, suggesting broad applicability to other coordinate-based density generation tasks.

Abstract

We study the Solid Isotropic Material Penalisation (SIMP) method with a density field generated by a fully-connected neural network, taking the coordinates as inputs. In the large width limit, we show that the use of DNNs leads to a filtering effect similar to traditional filtering techniques for SIMP, with a filter described by the Neural Tangent Kernel (NTK). This filter is however not invariant under translation, leading to visual artifacts and non-optimal shapes. We propose two embeddings of the input coordinates, which lead to (approximate) spatial invariance of the NTK and of the filter. We empirically confirm our theoretical observations and study how the filter size is affected by the architecture of the network. Our solution can easily be applied to any other coordinates-based generation method.

Paper Structure

This paper contains 31 sections, 11 theorems, 83 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Our contribution
  3. Presentation of the method
  4. SIMP method
  5. A modified SIMP approach
  6. Proposed algorithm: SIMP with Neural networks
  7. Theoretical Analysis
  8. Analogy between the Neural Tangent Kernel and filtering techniques
  9. Spatial invariance
  10. Embedding on a hypertorus
  11. Random embeddings for radial kernels
  12. Experimental analysis
  13. Setup
  14. Spectral analysis
  15. Filter radius
  16. ...and 16 more sections

Key Result

Proposition 2.1

Let $\dot{S}$ be the vector with entries $\dot{\sigma}(x_i + \bar{b}(X))$. We have $\nabla_X C = D_X \nabla_Y C$ with: where $\vert . \vert_1$ denotes the $l^1$ norm of a vector. Furthermore $D_X$ is a symmetric positive semi-definite matrix whose null-space is the space of constant vectors and has eigenvalues smaller than $\frac{1}{4}$.

Figures (11)

  • Figure 1: Illustration of our method
  • Figure 2: Example of result of our method with applied forces (red arrow) and a fixed boundary (green). Here we used a Gaussian embedding (see section $4$ for details).
  • Figure 3: Representation of one line of $\Tilde{\Theta}_{\theta}$ on the full torus and of its square root. We used $\beta=0.2$ and $\omega = 3$ (see Section \ref{['experimental_setup']}) here to make the filter visible on the whole torus.
  • Figure 4: Left: empirical NTK of FCNNs with both embedding (a.1, a.2, see Section \ref{['experimental_setup']} for details) or without embedding (a.3 with ReLu, a.4 with tanh). Right: Corresponding shape obtained after training. Note that methods without spatial invariance particularly struggles with this symmetric load case (b.3, b.4) while both "embedded methods" respect the symmetry (b.1, b.2). We also observed that training with non-embedded methods is very unstable
  • Figure 5: Sorted eigenvalues of the empirical NTK with some eigenvectors (reshaped as images). Obtained with a Gaussian embedding.
  • ...and 6 more figures

Theorems & Definitions (13)

  • Proposition 2.1
  • Proposition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 3.4
  • Proposition A.1: Proposition \ref{['implicit_diff']} in the paper
  • Proposition C.1: Proposition \ref{['torus_prop']} in the paper
  • Theorem C.1: Bochner
  • Definition C.1
  • Proposition C.2: Proposition \ref{['embedding']} in the paper
  • ...and 3 more