Information-Theoretic Multi-Model Fusion for Target-Oriented Adaptive Sampling in Materials Design

TL;DR

This work reframes materials design under limited evaluation budgets as an information-theoretic trajectory-discovery problem, treating optimization as entropy reduction of a tripartite system over data, models, and physics. It introduces a four-stage pipeline with dimension-aware capacity control, target-conditioned surrogate shaping, structure-aware candidate generation, and Kalman-inspired multi-model fusion to balance exploitation and exploration. Across $14$ heterogeneous materials tasks (with $N$ in the range $6\times 10^2$ to $4\times 10^6$ and $d$ from $10$ to $10^3$) and complementary $20$-dimensional synthetic benchmarks (Ackley, Rastrigin, Schwefel), the framework achieves robust sample efficiency, often locating top-performing regions within $100$ evaluations and showing strong performance even in ultra-high-dimensional settings like QM9. By prioritizing information gain and trajectory consistency over full surface modeling, the method demonstrates practical potential for autonomous materials discovery under stringent budgets, with a clear pathway for integration with advanced descriptors and generative priors.

Abstract

Target-oriented discovery under limited evaluation budgets requires making reliable progress in high-dimensional, heterogeneous design spaces where each new measurement is costly, whether experimental or high-fidelity simulation. We present an information-theoretic framework for target-oriented adaptive sampling that reframes optimization as trajectory discovery: instead of approximating the full response surface, the method maintains and refines a low-entropy information state that concentrates search on target-relevant directions. The approach couples data, model beliefs, and physics/structure priors through dimension-aware information budgeting, adaptive bootstrapped distillation over a heterogeneous surrogate reservoir, and structure-aware candidate manifold analysis with Kalman-inspired multi-model fusion to balance consensus-driven exploitation and disagreement-driven exploration. Evaluated under a single unified protocol without dataset-specific tuning, the framework improves sample efficiency and reliability across 14 single- and multi-objective materials design tasks spanning candidate pools from $600$ to $4 \times 10^6$ and feature dimensions from $10$ to $10^3$, typically reaching top-performing regions within 100 evaluations. Complementary 20-dimensional synthetic benchmarks (Ackley, Rastrigin, Schwefel) further demonstrate robustness to rugged and multimodal landscapes.

Figures (5)

• Figure 1: Information-theoretic multi-source fusion pipeline for target-oriented adaptive sampling. (a) Problem formalization: We frame data, model beliefs, and physical constraints as a tripartite cognitive system that co-evolves to maximize mutual information with the design target, shifting the goal from exhaustive global modeling to discovering target-relevant trajectories. (b) Information budgeting & capacity alignment: the effective information dimension implied by the current data is estimated and combined with the remaining evaluation budget to match model capacity to problem complexity, enabling hyperparameter adaptation that preserves target-relevant structure while filtering high-dimensional noise. (c) Model-wise information distillation: a heterogeneous reservoir of models is trained under an adaptive bootstrap procedure, producing per-model “cognitive landscapes” that encode complementary inductive biases and uncertainty views. (d) Target-conditioned candidate generation and manifold analysis: when a candidate set is provided, it is directly analyzed for its manifold structure; otherwise, target-relevant candidates are sampled from the collection of cognitive landscapes and then subjected to manifold analysis to characterize the feasible/structured region for search. (e) Model extrapolation scoring: ut-of-bag evaluation during bootstrapping yields model generalization diagnostics, including point-prediction accuracy and distributional predictive quality (f) SNR and globality assessment: combining model outputs with the candidate manifold, we quantify signal-to-noise characteristics and diagnose whether each model’s cognition is predominantly global or local. (g) Information fusion and sampling decision making: multiple fusion modes are used to translate heterogeneous signals into actions. Standard acquisition functions (ACQ) nominate locally promising regions, Kalman-inspired fusion (KF) aggregates multi-model evidence to enable consensus-driven focal jumps, and a uncertainty-weighted variant (rKF) prioritizes consensus probes that most efficiently reduce disagreement and accelerate convergence toward the target region.
• Figure 2: Convergence on 20-dimensional mathematical benchmarks of a) Ackley; b) Rastrigin; c) Schwefel function. Upper panels: best-so-far objective (mean across runs; thin lines show individual runs), with the small panel in the middle is the visualization of the 2D examples for each mathematical function. Lower panels: minimum feature-space distance to the global optimum among proposed candidates (mean across runs; thin lines show individual runs). Initialization uses 10 points; each iteration proposes a batch of 10 candidates; results are averaged over 10 independent runs.
• Figure 3: Composite visualization for Ackley test No. 3, Rastrigin test No. 18 and Schwefel test No. 5 of the multi-model optimization dynamics, data evolution, and performance analysis. a) Dynamics section: Iteration-wise convergence of the objective function and feature-space distance (top and middle), followed by the information bandwidth and quality evolution across clusters (bottom), showing how information flow concentrates and transitions among cluster manifolds during optimization; b) Data analysis section: Internal similarity, novelty–history correlation, and the low-dimensional UMAP projection illustrate structural clustering, exploration–exploitation balance, and trajectory consistency over iterations; c) Score section: Model-level evaluation metrics, including cross-model $R^2–ELPD$ consistency and aggregated score ratios, quantify the ensemble’s predictive stability and calibration through the optimization process.
• Figure 4: Complexity heatmap across all datasets. Each row corresponds to one dataset, sorted in descending order of the normalized total difficulty (hardness $z$-score). Columns are grouped into four categories: Baseline metrics ($d_{\mathrm{int}}$, effective resolution $R_{\mathrm{eff}}=N/d_{\mathrm{int}}$, and objective conflict $C_{\mathrm{obj}}$); Axis A (information-scarcity: start--target graph distance $D_{\mathrm{hop}}$, target--neighborhood sparsity $R_{\rho}$, average mutual information between principal components and aggregation target $\mathrm{MI}_{\mathrm{PC,avg}}$); Axis B (heterogeneity and multimodality: Moran’s $I$, correlation length $\ell_c$, multimodality $K^*$); and Axis C (deceptive and barrier-dominated: required jump $\epsilon^*$, min-cut capacity, bottleneck width $1-B_{\mathrm{mm}}$). Color encodes the standardized hardness ($z$-score), with red indicating more difficult conditions and blue indicating easier ones; white denotes the average level. The right-aligned bar chart shows per-axis composite scores (Baseline, A–C) and overall total hardness, aligned with the heatmap rows.
• Figure 5: Closed-pool optimization behavior on three representative materials datasets. Columns correspond to 3DSC_TC_nosoap (left), alexandria_icams (middle), and QM9 (right). A: Aggregate performance across 10 runs. Success-rate curves (top left), sample-efficiency distributions (top right), best-so-far convergence with property density distribution (bottom left) and proposal-quality migration (bottom right) summarize robustness and sample efficiency under distinct structural regimes. B: Representative single-run analysis. Panels follow the same (a–b–c) decomposition used in the mathematical benchmarks: (a) dynamics and cluster-level bandwidth flow; (b) data-geometry evolution (similarity, novelty, and UMAP trajectories); and (c) model-level score adaptation ($R^2$, $\mathrm{ELPD}$, and deviation-ratio–driven $r$KF weighting). Together these views show how the framework tailors its exploration–exploitation balance to dataset-specific difficulty: repeated cross-basin shifts in heterogeneous 3DSC_TC_nosoap, gradual filtering in high-resolution alexandria_icams, and rapid one-shot localization on the smooth, high-dimensional manifold of QM9.