Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do Two AI Scientists Agree?

Xinghong Fu, Ziming Liu, Max Tegmark

TL;DR

This work introduces MASS, a multi-physics AI framework that learns a scalar function $S$ per physical system whose derivatives define dynamics, allowing a single network to capture multiple theories across diverse systems. Through controlled experiments on SHO, pendulum, Kepler, and synthetic relativistic systems, MASS reveals that individual AI scientists can learn diverse explanations yet converge to a common underlying theory as data complexity increases, with a notable shift from Hamiltonian to Lagrangian representations in more complex regimes. The study demonstrates that multiple seeds (AI scientists) generally agree on the learned theory at the level of activations, even when exact weights differ, and shows that the Lagrangian description emerges as the dominant, unifying form in richer theory spaces. Extensions to higher dimensions, including the double pendulum and n-body problems, indicate MASS’s potential for interpretable, scalable AI-driven discovery of physical laws, albeit with computational considerations tied to Hessian inverses and training stability.

Abstract

When two AI models are trained on the same scientific task, do they learn the same theory or two different theories? Throughout history of science, we have witnessed the rise and fall of theories driven by experimental validation or falsification: many theories may co-exist when experimental data is lacking, but the space of survived theories become more constrained with more experimental data becoming available. We show the same story is true for AI scientists. With increasingly more systems provided in training data, AI scientists tend to converge in the theories they learned, although sometimes they form distinct groups corresponding to different theories. To mechanistically interpret what theories AI scientists learn and quantify their agreement, we propose MASS, Hamiltonian-Lagrangian neural networks as AI Scientists, trained on standard problems in physics, aggregating training results across many seeds simulating the different configurations of AI scientists. Our findings suggests for AI scientists switch from learning a Hamiltonian theory in simple setups to a Lagrangian formulation when more complex systems are introduced. We also observe strong seed dependence of the training dynamics and final learned weights, controlling the rise and fall of relevant theories. We finally demonstrate that not only can our neural networks aid interpretability, it can also be applied to higher dimensional problems.

Do Two AI Scientists Agree?

TL;DR

This work introduces MASS, a multi-physics AI framework that learns a scalar function per physical system whose derivatives define dynamics, allowing a single network to capture multiple theories across diverse systems. Through controlled experiments on SHO, pendulum, Kepler, and synthetic relativistic systems, MASS reveals that individual AI scientists can learn diverse explanations yet converge to a common underlying theory as data complexity increases, with a notable shift from Hamiltonian to Lagrangian representations in more complex regimes. The study demonstrates that multiple seeds (AI scientists) generally agree on the learned theory at the level of activations, even when exact weights differ, and shows that the Lagrangian description emerges as the dominant, unifying form in richer theory spaces. Extensions to higher dimensions, including the double pendulum and n-body problems, indicate MASS’s potential for interpretable, scalable AI-driven discovery of physical laws, albeit with computational considerations tied to Hessian inverses and training stability.

Abstract

When two AI models are trained on the same scientific task, do they learn the same theory or two different theories? Throughout history of science, we have witnessed the rise and fall of theories driven by experimental validation or falsification: many theories may co-exist when experimental data is lacking, but the space of survived theories become more constrained with more experimental data becoming available. We show the same story is true for AI scientists. With increasingly more systems provided in training data, AI scientists tend to converge in the theories they learned, although sometimes they form distinct groups corresponding to different theories. To mechanistically interpret what theories AI scientists learn and quantify their agreement, we propose MASS, Hamiltonian-Lagrangian neural networks as AI Scientists, trained on standard problems in physics, aggregating training results across many seeds simulating the different configurations of AI scientists. Our findings suggests for AI scientists switch from learning a Hamiltonian theory in simple setups to a Lagrangian formulation when more complex systems are introduced. We also observe strong seed dependence of the training dynamics and final learned weights, controlling the rise and fall of relevant theories. We finally demonstrate that not only can our neural networks aid interpretability, it can also be applied to higher dimensional problems.

Paper Structure

This paper contains 19 sections, 15 equations, 19 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. MASS: The AI scientist
  4. Method
  5. Experiments
  6. Single scientist: Correlated theories
  7. Multiple systems: Sparsification and diversification
  8. Multiple scientists: Mixture of theories
  9. Exploring the unknown: Lagrangian is all you need
  10. Simple Problems: Hamiltonian is all you need?
  11. Complex Problems: Lagrangian is all you need
  12. Extensions to high dimensions
  13. Discussions
  14. Future Work
  15. Conclusion
  16. ...and 4 more sections

Figures (19)

  • Figure 1: The evolution of AI scientists. Different AI scientists learning from data within the same physical system, even in the simple pendulum, arrives at different results. Theories that fail to support the current data are marked wrong. Surviving AI scientists are exposed to more complex systems, such as the double pendulum. AI scientists modify their theories to model the new data. Ultimately, what will the remaining AI scientists learn?
  • Figure 2: The MASS (Multi-physics AI Scalar Scientist) network.
  • Figure 3: Training results for MASS on the simple harmonic oscillator. (a) MASS (seed 0) trains to an MSE loss of $3 \times 10^{-4}$ over 10000 steps of a batch size of $512$ at each step. The number of significant weights, calculated as the number of weights in the final layer that account the first 99% the total norm, decreases with loss. (b) The recreated motion of a single oscillator accurately captures the frequency and amplitude of the motion.
  • Figure 4: Contour of (a) learnt scalar function $S$, compared with (b) the Hamiltonian $x^2+y^2$. MASS can in general, learn functions that resemble yet differ from conventional physical priors.
  • Figure 5: (a) Weights in final layer (blue) and the mean activation norms (red). The top 5 terms in mean activation magnitude: $S_{yy}^{-1} S_{yy}^{-1} x, \; S_{xy}^{-1} S_{yy} x, \; S_{yy}^{-1} S_{xx} x, \; S_{xx} x, \; S_{yy}^{-1} S_{xy}^{-1} x$. (b) Correlation of significant activations, keeping only indices $i$ contributing to the first cumulative $99\%$ of $\sum_i \mathbb{E}[a_i]$, plotted after hierachial clustering. Most terms are strongly correlated.
  • ...and 14 more figures