Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Regression to Classification: Exploring the Benefits of Categorical Representations of Energy in MLIPs

Ahmad Ali

TL;DR

The paper investigates replacing scalar regression in machine-learned interatomic potentials with a histogram-based, categorical energy representation using HL-Gauss. This approach preserves competitive accuracy while providing a principled measure of epistemic uncertainty through distribution entropy. Across experiments on UMA and OMol25, 128-bin HL-Gauss configurations perform comparably to scalar baselines, with entropy offering an informative but imperfect uncertainty signal. The work highlights potential benefits for uncertainty quantification and stability in scaling MLIPs, while noting the need to refine the uncertainty metric further.

Abstract

Density Functional Theory (DFT) is a widely used computational method for estimating the energy and behavior of molecules. Machine Learning Interatomic Potentials (MLIPs) are models trained to approximate DFT-level energies and forces at dramatically lower computational cost. Many modern MLIPs rely on a scalar regression formulation; given information about a molecule, they predict a single energy value and corresponding forces while minimizing absolute error with DFT's calculations. In this work, we explore a multi-class classification formulation that predicts a categorical distribution over energy/force values, providing richer supervision through multiple targets. Most importantly, this approach offers a principled way to quantify model uncertainty. In particular, our method predicts a histogram of the energy/force distribution, converts scalar targets into histograms, and trains the model using cross-entropy loss. Our results demonstrate that this categorical formulation can achieve absolute error performance comparable to regression baselines. Furthermore, this representation enables the quantification of epistemic uncertainty through the entropy of the predicted distribution, offering a measure of model confidence absent in scalar regression approaches.

From Regression to Classification: Exploring the Benefits of Categorical Representations of Energy in MLIPs

TL;DR

The paper investigates replacing scalar regression in machine-learned interatomic potentials with a histogram-based, categorical energy representation using HL-Gauss. This approach preserves competitive accuracy while providing a principled measure of epistemic uncertainty through distribution entropy. Across experiments on UMA and OMol25, 128-bin HL-Gauss configurations perform comparably to scalar baselines, with entropy offering an informative but imperfect uncertainty signal. The work highlights potential benefits for uncertainty quantification and stability in scaling MLIPs, while noting the need to refine the uncertainty metric further.

Abstract

Density Functional Theory (DFT) is a widely used computational method for estimating the energy and behavior of molecules. Machine Learning Interatomic Potentials (MLIPs) are models trained to approximate DFT-level energies and forces at dramatically lower computational cost. Many modern MLIPs rely on a scalar regression formulation; given information about a molecule, they predict a single energy value and corresponding forces while minimizing absolute error with DFT's calculations. In this work, we explore a multi-class classification formulation that predicts a categorical distribution over energy/force values, providing richer supervision through multiple targets. Most importantly, this approach offers a principled way to quantify model uncertainty. In particular, our method predicts a histogram of the energy/force distribution, converts scalar targets into histograms, and trains the model using cross-entropy loss. Our results demonstrate that this categorical formulation can achieve absolute error performance comparable to regression baselines. Furthermore, this representation enables the quantification of epistemic uncertainty through the entropy of the predicted distribution, offering a measure of model confidence absent in scalar regression approaches.

Paper Structure

This paper contains 11 sections, 5 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Background.
  4. Motivation for Classification Formulation
  5. Histogram Prediction and Loss.
  6. Results
  7. Experimental Setup.
  8. Main Results.
  9. Ablations.
  10. Estimating Uncertainty.
  11. Conclusion

Figures (2)

  • Figure 1: Visualizing scalar target as normal distribution, followed by discretizations of normal distribution into a histogram distribution
  • Figure 2: Pearson Correlation Coefficient between Histogram Entropy and the absolute error across training batches. The x-axis is training step, y-axis is the correlation coefficient.