Transforming physics-informed machine learning to convex optimization

TL;DR

The paper addresses optimization bottlenecks in physics-informed machine learning (PIML). It introduces Convex-PIML, which converts PIML losses into a sequence of convex optimization problems using $B$-splines to approximate data and convex surrogates for non-convex terms, enabling reliable application of convex optimization algorithms. An adaptive knot optimization via error estimates is proposed to reduce spectral bias and improve performance. Theoretical guarantees accompany the framework, and empirical tests across problems with different physical priors show effective solutions and broad applicability.

Abstract

Physics-Informed Machine Learning (PIML) offers a powerful paradigm of integrating data with physical laws to address important scientific problems, such as parameter estimation, inferring hidden physics, equation discovery, and state prediction, etc. However, PIML still faces many serious optimization challenges that significantly restrict its applications. In this study, we propose a comprehensive framework that transforms PIML to convex optimization to overcome all these limitations, referred to as Convex-PIML. The linear combination of B-splines is utilized to approximate the data, promoting the convexity of the loss function. By replacing the non-convex components of the loss function with convex approximations, the problem is further converted into a sequence of successively refined approximated convex optimization problems. This conversion allows the use of well-established convex optimization algorithms, obtaining solutions effectively and efficiently. Furthermore, an adaptive knot optimization method based on error estimate is introduced to mitigate the spectral bias issue of PIML, further improving the performance. The proposed theoretically guaranteed framework is tested in scenarios with distinct types of physical prior. The results indicate that optimization problems are effectively solved in these scenarios, highlighting the potential of the framework for broad applications.