Model reduction, machine learning based global optimisation for large-scale steady state nonlinear systems

Abstract

Many engineering processes can be accurately modelled using partial differential equations (PDEs), but high dimensionality and non-convexity of the resulting systems pose limitations on their efficient optimisation. In this work, a model reduction, machine-learning methodology combining principal component analysis (PCA) and artificial neural networks (ANNs) is employed to construct a reduced surrogate model, which can then be utilised by advanced deterministic global optimisation algorithms to compute global optimal solutions with theoretical guarantees. However, such optimisation would still be time-consuming due to the high non-convexity of the activation functions inside the reduced ANN structures. To develop a computationally-efficient optimisation framework, we propose two alternative strategies: The first one is a piecewise-affine reformulation of the nonlinear ANN activation functions, while the second one is based on deep rectifier neural networks with ReLU activation function. The performance of the proposed framework is demonstrated through two illustrative case studies.

Figures (9)

• Figure 1: Feed-forward neural network with hyperbolic tangent activation function
• Figure 2: Flow chart of the basic PCA-ANN global optimisation framework
• Figure 3: Adaptive PWA procedure for the hyperbolic activation function
• Figure 4: Deep neural network with rectified linear units
• Figure 5: Solution profiles for dimensionless temperature and concentration (adaptive PWA)