Informed Machine Learning with Knowledge Landmarks

Abstract

Informed Machine Learning has emerged as a viable generalization of Machine Learning (ML) by building a unified conceptual and algorithmic setting for constructing models on a unified basis of knowledge and data. Physics-informed ML involving physics equations is one of the developments within Informed Machine Learning. This study proposes a novel direction of Knowledge-Data ML, referred to as KD-ML, where numeric data are integrated with knowledge tidbits expressed in the form of granular knowledge landmarks. We advocate that data and knowledge are complementary in several fundamental ways: data are precise (numeric) and local, usually confined to some region of the input space, while knowledge is global and formulated at a higher level of abstraction. The knowledge can be represented as information granules and organized as a collection of input-output information granules called knowledge landmarks. In virtue of this evident complementarity, we develop a comprehensive design process of the KD-ML model and formulate an original augmented loss function L, which additively embraces the component responsible for optimizing the model based on available numeric data, while the second component, playing the role of a granular regularizer, so that it adheres to the granular constraints (knowledge landmarks). We show the role of the hyperparameter positioned in the loss function, which balances the contribution and guiding role of data and knowledge, and point to some essential tendencies associated with the quality of data (noise level) and the level of granularity of the knowledge landmarks. Experiments on two physics-governed benchmarks demonstrate that the proposed KD model consistently outperforms data-driven ML models.

Figures (14)

• Figure 1: Input space $\Omega$ with a collection of knowledge landmarks $(A_i, B_i)$ and limited space $\Omega^\ast$ for which numeric data are generated by $f(\boldsymbol{x}; \boldsymbol{W})$ for some specific numeric values of $\boldsymbol{W}$, namely $\boldsymbol{w}_0$.
• Figure 2: Matching of an information granule $B$ and a numeric entity $x_0$.
• Figure 3: Spatiotemporal domain and local observation setup for the environmental benchmark. (a) Pollutant concentration field $f(s,t)$ over $\Omega$, computed with baseline parameters. (b) Domain $\Omega$ and local observation window $\Omega_1^\ast$ in the $(s,t)$ space.
• Figure 4: Output space granulation for the environmental model. The histogram shows the distribution of scaled concentration values in $\Omega$, and curves represent five Gaussian membership functions defining the output contexts.
• Figure 5: Input granules $\{A_{ik}\}$ for the environmental model, obtained via conditional FCM for each output context. Each subplot shows eight input granules associated with one output context $B_i$, where the color of each point represents its membership degree.