Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Invariant multiscale neural networks for data-scarce scientific applications

I. Schurov, D. Alforov, M. Katsnelson, A. Bagrov, A. Itin

TL;DR

Data-scarce scientific ML faces challenges in leveraging complex physical structure. The authors propose a compact, symmetry-aware approach that combines translationally invariant CNNs with stacks of dilated convolutions to capture multi-scale physics without downsampling. They validate the method on photonic crystal bandstructure tasks and on learning sign structures in neural quantum states, showing substantive accuracy gains and NAS-guided dilation configurations that further improve performance. The work highlights a practical path to data-efficient scientific modeling with broad applicability and potential extensions to advanced architectures for quantum many-body problems.

Abstract

Success of machine learning (ML) in the modern world is largely determined by abundance of data. However at many industrial and scientific problems, amount of data is limited. Application of ML methods to data-scarce scientific problems can be made more effective via several routes, one of them is equivariant neural networks possessing knowledge of symmetries. Here we suggest that combination of symmetry-aware invariant architectures and stacks of dilated convolutions is a very effective and easy to implement receipt allowing sizable improvements in accuracy over standard approaches. We apply it to representative physical problems from different realms: prediction of bandgaps of photonic crystals, and network approximations of magnetic ground states. The suggested invariant multiscale architectures increase expressibility of networks, which allow them to perform better in all considered cases.

Invariant multiscale neural networks for data-scarce scientific applications

TL;DR

Data-scarce scientific ML faces challenges in leveraging complex physical structure. The authors propose a compact, symmetry-aware approach that combines translationally invariant CNNs with stacks of dilated convolutions to capture multi-scale physics without downsampling. They validate the method on photonic crystal bandstructure tasks and on learning sign structures in neural quantum states, showing substantive accuracy gains and NAS-guided dilation configurations that further improve performance. The work highlights a practical path to data-efficient scientific modeling with broad applicability and potential extensions to advanced architectures for quantum many-body problems.

Abstract

Success of machine learning (ML) in the modern world is largely determined by abundance of data. However at many industrial and scientific problems, amount of data is limited. Application of ML methods to data-scarce scientific problems can be made more effective via several routes, one of them is equivariant neural networks possessing knowledge of symmetries. Here we suggest that combination of symmetry-aware invariant architectures and stacks of dilated convolutions is a very effective and easy to implement receipt allowing sizable improvements in accuracy over standard approaches. We apply it to representative physical problems from different realms: prediction of bandgaps of photonic crystals, and network approximations of magnetic ground states. The suggested invariant multiscale architectures increase expressibility of networks, which allow them to perform better in all considered cases.
Paper Structure (9 sections, 14 equations, 10 figures, 4 tables)

This paper contains 9 sections, 14 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Translationally invariant CNN architectures and multiscale (dilated) convolutions
  3. Translationally invariant CNN architectures
  4. Dilated convolutions and stacks of dilated convolutions
  5. Experiments with photonics datasets
  6. Experiments with neural quantum states
  7. Conclusions
  8. Additional experiments with neural quantum states
  9. Photonic datasets description and additional experiments with them

Figures (10)

  • Figure 1: (a) Top: Typical CNN architecture with only approximate translational invariance (See also Ref. Invariant) (b) Bottom: CNN architecture with exact translational invariance
  • Figure 2: The concept of dilated convolutions. Dark squares designate non-zero parameters of a convolutional kernel. At light squares, parameters are zero. Even though the size of the kernel grows quadratically with the dilation parameter, the number of parameters remains the same. Left: a convolutional kernel of size $3\times 3$ without dilation (or, equivalently, with dilation parameter equal to 1). Right: the same $3\times 3$ convolutional kernel with dilation=2.
  • Figure 3: Dataset of photonic crystals. Top row: an example of a unit cell consisting of two materials with permittivities $\epsilon_{1,2}$ and a set of such unit cells. Center: a unit cell generates a periodic 2D structure (photonic crystal). Bottom row: eigenstates of Maxwell equations are characterised by two numbers (quasimomenta $k_{x,y}$) being located in the first Brillouin zone. A $23 \times 23$ grid was defined, and at each point the first 6 eigenstates were calculated using MPB solver.
  • Figure 4: Examples of $96 \times 96$ unit cells from the third dataset. Complexity factor F=1,2,3,4,5,6 from top left to the bottom right.
  • Figure 5: Triangular 36-spin lattice with periodic boundary conditions. Different couplings $J_1$ and $J_2$ are taken along different directions, with $J_2=0$ corresponding to a bipartite (square) lattice with simple ground state structure, and $J_2/J_1>1.25$ -- to strongly frustrated regime with spin liquid ground state.
  • ...and 5 more figures