CEopt: A MATLAB Package for Non-convex Optimization with the Cross-Entropy Method

TL;DR

This paper introduces CEopt, a MATLAB tool leveraging the Cross-Entropy method for non-convex optimization, offering robustness and scalability for moderately sized complex problems.

Abstract

This paper introduces CEopt (https://ceopt.org), a MATLAB tool leveraging the Cross-Entropy method for non-convex optimization. Due to the relative simplicity of the algorithm, it provides a kind of transparent ``gray-box'' optimization solver, with intuitive control parameters. Unique in its approach, CEopt effectively handles both equality and inequality constraints using an augmented Lagrangian method, offering robustness and scalability for moderately sized complex problems. Through select case studies, the package's applicability and effectiveness in various optimization scenarios are showcased, marking CEopt as a practical addition to optimization research and application toolsets.

Figures (19)

• Figure 1: Cross-Entropy Method Explained: This method turns a complex optimization problem into a manageable task of estimating a rare event. By using adaptive importance sampling, it iteratively focuses on making the global optimum more likely to find. The mean of the "optimal" distribution estimates the best solution, showcasing how CE simplifies tough optimization challenges into more approachable ones with a probabilistic strategy.
• Figure 2: Cross-Entropy Method Timeline: Beginning with R. Rubinstein's initial proposal in 1997 for rare-event estimation, this timeline tracks the evolution of the method into non-convex optimization and beyond. Highlighting key theoretical advancements and the development of CE software, it illustrates the method's increasing significance in the field of optimization.
• Figure 3: Cross-Entropy Method Flowchart: This diagram shows the CE method's steps, from starting with a sample distribution to finding the best solution. It involves sampling, function evaluation, refining the distribution with the best samples, and repeating these steps until the optimal solution is found.
• Figure 4: CEopt MATLAB Implementation: This diagram outlines the key components of the CEopt package, starting with input checks and setup, through to optimization solvers for various problem types. It highlights the sampling and learning processes used to refine solutions, along with methods for convergence verification and detailed progress reporting, illustrating the systematic approach embedded in CEopt for solving optimization tasks.
• Figure 5: Adaptive Sampling in CE Optimization. The plot showcases the Cross-Entropy method optimizing a one-dimensional Gaussian mixture function (thick blue line). Gaussian distributions, from orange to dark near-black, visualize the adaptive sampling strategy. Early in the search, broader red distributions cover wider areas, while focused near-black distributions towards the end pinpoint the optimum, marked by the black dot. The inset magnifies the final stages, highlighting the precision of the narrowing search distributions.