Neural Simulated Annealing

TL;DR

This work view SA from a reinforcement learning perspective and frame the proposal distribution as a policy, which can be optimised for higher solution quality given a fixed computational budget, and demonstrates that this Neural SA with such a learnt proposal distribution, parametrised by small equivariant neural networks, outperforms SA baselines on a number of problems.

Abstract

Simulated annealing (SA) is a stochastic global optimisation technique applicable to a wide range of discrete and continuous variable problems. Despite its simplicity, the development of an effective SA optimiser for a given problem hinges on a handful of carefully handpicked components; namely, neighbour proposal distribution and temperature annealing schedule. In this work, we view SA from a reinforcement learning perspective and frame the proposal distribution as a policy, which can be optimised for higher solution quality given a fixed computational budget. We demonstrate that this Neural SA with such a learnt proposal distribution, parametrised by small equivariant neural networks, outperforms SA baselines on a number of problems: Rosenbrock's function, the Knapsack problem, the Bin Packing problem, and the Travelling Salesperson problem. We also show that Neural SA scales well to large problems - generalising to significantly larger problems than the ones seen during training - while achieving comparable performance to popular off-the-shelf solvers and other machine learning methods in terms of solution quality and wall-clock time.

Figures (7)

• Figure 1: Neural SA pipeline for the TSP. Starting with a solution (tour) ${\mathbf{x}}_k$, we sample an action ${\mathbf{a}}{=}(i,j)$ from our learnable policy/proposal distribution, defining start $i$ and end $j$ points of a 2-opt move (replacing two old with two new edges). Each pane shows both the linear and graph-based representations for a tour. From ${\mathbf{x}}_k$ and ${\mathbf{a}}$ we form a proposal ${\mathbf{x}}'$ which is either accepted or rejected in a MH step. Accepted moves assign ${\mathbf{x}}_{k+1} {=} {\mathbf{x}}'$; whereas, rejected moves assign ${\mathbf{x}}_{k+1} {=} {\mathbf{x}}_k$.
• Figure 2: (a) Policy network used in all experiments. The same MLP is applied to all inputs pointwise.
• Figure 3: Results on Rosenbrock's function: (a) Example trajectory, moving from red to blue, showing convergence around the minimiser at (1,1) (b) Neural SA has higher acceptance ratio than the baseline, a trend observed in all experiments, (c) Standard deviation of the learned policy as a function of iteration. Large initial steps offer great gains followed by small exploitative steps, (d) A non-adaptive vanilla SA baseline cannot match an adaptive one, no matter the standard deviation.
• Figure 4: Knapsack Policy with logits for $x_i=0$ and $x_i=1$ shown in each pane. Light valuable objects are favoured to insert. Once inserted the policy downweights an object's probably of flipping state again. Interestingly, the ejection probability of heavy, valueless objects is low, perhaps because this only makes sense close to overflowing, although the policy does not receive free capacity as a feature.
• Figure 5: Bin50 primal objective for vanilla, Neural, and Greedy Neural SA with 25th, 50th, and 75th percentiles.