Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations

Pradeep Bajracharya, Javier Quetzalcóatl Toledo-Marín, Geoffrey Fox, Shantenu Jha, Linwei Wang

TL;DR

High-performance diffusion simulations incur heavy data-generation costs across large parameter spaces. The authors integrate active learning to train DNN surrogates that selectively query simulations, reducing labeled data needs. In offline emulations of a two-source diffusion problem on a $100\times100$ lattice, TOD-based acquisition with a suitable architecture (notably U-Net) achieved lower weighted MAE with less labeled data than baselines, while the benefits depend strongly on the network design. This work lays a foundation for Smart Surrogates in HPC, enabling on-the-fly data generation steered by active learning to accelerate complex scientific computations.

Abstract

High-performance scientific simulations, important for comprehension of complex systems, encounter computational challenges especially when exploring extensive parameter spaces. There has been an increasing interest in developing deep neural networks (DNNs) as surrogate models capable of accelerating the simulations. However, existing approaches for training these DNN surrogates rely on extensive simulation data which are heuristically selected and generated with expensive computation -- a challenge under-explored in the literature. In this paper, we investigate the potential of incorporating active learning into DNN surrogate training. This allows intelligent and objective selection of training simulations, reducing the need to generate extensive simulation data as well as the dependency of the performance of DNN surrogates on pre-defined training simulations. In the problem context of constructing DNN surrogates for diffusion equations with sources, we examine the efficacy of diversity- and uncertainty-based strategies for selecting training simulations, considering two different DNN architecture. The results set the groundwork for developing the high-performance computing infrastructure for Smart Surrogates that supports on-the-fly generation of simulation data steered by active learning strategies to potentially improve the efficiency of scientific simulations.

Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations

TL;DR

High-performance diffusion simulations incur heavy data-generation costs across large parameter spaces. The authors integrate active learning to train DNN surrogates that selectively query simulations, reducing labeled data needs. In offline emulations of a two-source diffusion problem on a lattice, TOD-based acquisition with a suitable architecture (notably U-Net) achieved lower weighted MAE with less labeled data than baselines, while the benefits depend strongly on the network design. This work lays a foundation for Smart Surrogates in HPC, enabling on-the-fly data generation steered by active learning to accelerate complex scientific computations.

Abstract

High-performance scientific simulations, important for comprehension of complex systems, encounter computational challenges especially when exploring extensive parameter spaces. There has been an increasing interest in developing deep neural networks (DNNs) as surrogate models capable of accelerating the simulations. However, existing approaches for training these DNN surrogates rely on extensive simulation data which are heuristically selected and generated with expensive computation -- a challenge under-explored in the literature. In this paper, we investigate the potential of incorporating active learning into DNN surrogate training. This allows intelligent and objective selection of training simulations, reducing the need to generate extensive simulation data as well as the dependency of the performance of DNN surrogates on pre-defined training simulations. In the problem context of constructing DNN surrogates for diffusion equations with sources, we examine the efficacy of diversity- and uncertainty-based strategies for selecting training simulations, considering two different DNN architecture. The results set the groundwork for developing the high-performance computing infrastructure for Smart Surrogates that supports on-the-fly generation of simulation data steered by active learning strategies to potentially improve the efficiency of scientific simulations.
Paper Structure (24 sections, 4 equations, 7 figures, 1 table)

This paper contains 24 sections, 4 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Background: Diffusion Solver
  4. Data Generation
  5. DNN Surrogates
  6. Loss Definition
  7. Methods: Active Learning of DNN Surrogates for Diffusion Solvers
  8. Active Learning
  9. Entropy:
  10. Temporal Output Discrepancy (TOD):
  11. Parameter Diversity:
  12. Evaluation Metrics
  13. Weighted-MAE
  14. Region-of-Interest MAE
  15. Experiments
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Distribution of parameters used to generate simulation data. (a) Histogram of distance between two sources on the lattice. (b) Histogram of intensity of source 2 which has randomly assigned value between 0 and 1. (c-d): Histograms of the positions of (c) source 1 and (d) source 2 on the lattice.
  • Figure 2: Deep convolutional neural network architecture for the diffusion surrogate. All convolution layers leave the input with the same height and width. Each block is composed of a convolution which increases the channels from $Ch1$ to $Ch2$, followed by a LeakyReLU with slope set at $0.02$.
  • Figure 4: Regions of interest shown with white for a given lattice -- a) Source, b) Field, c) Ring 1, d) Ring 2 and e) Ring 3 for two sources placed randomly in the lattice.
  • Figure 5: Comparison of test weighted-MAE on a) U-Net architecture and b) CNN autoencoder, in log scale, between the ground truth and prediction from the DNN surrogates across different acquisition functions as data acquisition proceeds.
  • Figure 7: Comparison of absolute error maps between DNN predictions and the ground truth using between random (top row), diversity (middle row), and TOD (bottom row) acquisition function at different percentage of labelled data on U-Net architecture for two example lattice (1) and (2)
  • ...and 2 more figures