Automated Discovery of Integral with Deep Learning

TL;DR

This work investigates whether deep learning can autonomously discover the integral operator from scratch, treating $g(x)=\int_{0}^{x} f(t)\,dt$ as the target derived from a large, randomly generated function corpus. It combines symbolic regression to generate ground-truth integrals with autoregressive (GPT-Neo) and sequence-to-sequence (Flan-T5) models to map $f$ to $g$, and a rule-discovery component to extract analytic integration rules. Results show GPT-Neo achieving higher exact and approximate accuracy than Flan-T5 and symbolic regression across polynomial and non-polynomial functions, and the study derives explicit rules such as $a_k' = \frac{1}{k} a_{k-1}$ for polynomials and analogous relations for exponential and trigonometric cases. The findings demonstrate AI’s potential to autonomously reason about fundamental calculus concepts, laying groundwork for broader automated scientific discovery beyond problem-solving on curated datasets.

Abstract

Recent advancements in the realm of deep learning, particularly in the development of large language models (LLMs), have demonstrated AI's ability to tackle complex mathematical problems or solving programming challenges. However, the capability to solve well-defined problems based on extensive training data differs significantly from the nuanced process of making scientific discoveries. Trained on almost all human knowledge available, today's sophisticated LLMs basically learn to predict sequences of tokens. They generate mathematical derivations and write code in a similar way as writing an essay, and do not have the ability to pioneer scientific discoveries in the manner a human scientist would do. In this study we delve into the potential of using deep learning to rediscover a fundamental mathematical concept: integrals. By defining integrals as area under the curve, we illustrate how AI can deduce the integral of a given function, exemplified by inferring $\int_{0}^{x} t^2 dt = \frac{x^3}{3}$ and $\int_{0}^{x} ae^{bt} dt = \frac{a}{b} e^{bx} - \frac{a}{b}$. Our experiments show that deep learning models can approach the task of inferring integrals either through a sequence-to-sequence model, akin to language translation, or by uncovering the rudimentary principles of integration, such as $\int_{0}^{x} t^n dt = \frac{x^{n+1}}{n+1}$.

Figures (6)

• Figure 1: An illustration of how an autoregressive model like GPT works. It predicts each token based on all previous tokens (including a Begin-Of-Sentence token at the beginning).
• Figure 2: An illustration of a sequence-to-sequence prediction, which translates an English sentence into French.
• Figure 3: The %-Correct and %-Approximately-Correct of Flan-T5 in inferring integrals of polynomial functions.
• Figure 4: The %-Correct and %-Approximately-Correct of Flan-T5 in inferring integrals of non-polynomial functions.
• Figure 5: The %-Correct and %-Approximately-Correct of GPT-Neo in inferring integrals of polynomial functions.