Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Neural Network for Constraint Acquisition through Tailored Loss Function

Eduardo Vyhmeister, Rocio Paez, Gabriel Gonzalez

TL;DR

This work addresses constraint learning from data by introducing a deep neural network approach based on Symbolic Regression, specifically the Equation Learner (EQL). It pairs EQL with a tailored loss function that combines $z = \frac{\alpha_1}{N} \sum_{i=1}^{N} L_{e}(y,f(x_i)) + \frac{\alpha_2}{N} \sum_{i=1}^{N} L_{P_{\gamma}}(y,f(x_i)) + \alpha_3 L_{anchor}(y,f(x_i))$ to extract linear constraints from datasets, ensuring inequality directions and stability are respected. The method is implemented in Python with TensorFlow 2, using symbolic layers and SymPy, and evaluated on synthetic 2D and 3D geometries (e.g., square, circle, cube) to recover boundary-like inequalities with low constraint-violation rates. The results demonstrate the potential for interpretable, automated constraint discovery and point to future work extending to non-linear primitives and more diverse constraint types, enabling broader applicability and integration with existing frameworks.

Abstract

The significance of learning constraints from data is underscored by its potential applications in real-world problem-solving. While constraints are popular for modeling and solving, the approaches to learning constraints from data remain relatively scarce. Furthermore, the intricate task of modeling demands expertise and is prone to errors, thus constraint acquisition methods offer a solution by automating this process through learnt constraints from examples or behaviours of solutions and non-solutions. This work introduces a novel approach grounded in Deep Neural Network (DNN) based on Symbolic Regression that, by setting suitable loss functions, constraints can be extracted directly from datasets. Using the present approach, direct formulation of constraints was achieved. Furthermore, given the broad pre-developed architectures and functionalities of DNN, connections and extensions with other frameworks could be foreseen.

Deep Neural Network for Constraint Acquisition through Tailored Loss Function

TL;DR

This work addresses constraint learning from data by introducing a deep neural network approach based on Symbolic Regression, specifically the Equation Learner (EQL). It pairs EQL with a tailored loss function that combines to extract linear constraints from datasets, ensuring inequality directions and stability are respected. The method is implemented in Python with TensorFlow 2, using symbolic layers and SymPy, and evaluated on synthetic 2D and 3D geometries (e.g., square, circle, cube) to recover boundary-like inequalities with low constraint-violation rates. The results demonstrate the potential for interpretable, automated constraint discovery and point to future work extending to non-linear primitives and more diverse constraint types, enabling broader applicability and integration with existing frameworks.

Abstract

The significance of learning constraints from data is underscored by its potential applications in real-world problem-solving. While constraints are popular for modeling and solving, the approaches to learning constraints from data remain relatively scarce. Furthermore, the intricate task of modeling demands expertise and is prone to errors, thus constraint acquisition methods offer a solution by automating this process through learnt constraints from examples or behaviours of solutions and non-solutions. This work introduces a novel approach grounded in Deep Neural Network (DNN) based on Symbolic Regression that, by setting suitable loss functions, constraints can be extracted directly from datasets. Using the present approach, direct formulation of constraints was achieved. Furthermore, given the broad pre-developed architectures and functionalities of DNN, connections and extensions with other frameworks could be foreseen.
Paper Structure (10 sections, 6 equations, 2 figures, 2 tables)

This paper contains 10 sections, 6 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Symbolic Regression
  5. Loss Function Definition for Constraint Acquisition
  6. EQL and Loss Function Integration
  7. Experiments definition and setup
  8. Results and Discussion
  9. Conclusions
  10. Acknowledgments.

Figures (2)

  • Figure 1: Schematic representation of the different notions considered for the three terms definition the loss functions: Notion 1 (left panel), Notion 2 (center panel), Notion 3 (right panel). See text for full explanation.
  • Figure 2: Constraints obtained using $y - f(x)$ representation of error within the $L_{e}$ expression: High-granularity Square example (top panel), Circle example (center panel), Low-granularity square example (bottom panel). Red line is equivalent to Result $\# 7$ of Table \ref{['tab2']}, without masking (see text for full explanation).