Random is Faster than Systematic in Multi-Objective Local Search

TL;DR

The paper examines the efficiency of neighbourhood exploration in multi-objective local search by comparing systematic (s-PLS) and randomised (r-PLS) Pareto Local Search. Through extensive experiments on four MOCOPs and a theoretical analysis, it shows that r-PLS consistently accelerates progress toward non-dominated solutions, with its advantage persisting in both best-improvement and first-improvement modes. The authors identify a geometric distribution governing the number of good neighbours and provide a formal runtime analysis demonstrating why random sampling yields faster discovery of improvements, particularly in large search spaces. These results offer practical guidance for designing efficient multi-objective local search: favour randomised neighbourhood exploration to achieve faster convergence in typical problem settings.

Abstract

Local search is a fundamental method in operations research and combinatorial optimisation. It has been widely applied to a variety of challenging problems, including multi-objective optimisation where multiple, often conflicting, objectives need to be simultaneously considered. In multi-objective local search algorithms, a common practice is to maintain an archive of all non-dominated solutions found so far, from which the algorithm iteratively samples a solution to explore its neighbourhood. A central issue in this process is how to explore the neighbourhood of a selected solution. In general, there are two main approaches: 1) systematic exploration and 2) random sampling. The former systematically explores the solution's neighbours until a stopping condition is met -- for example, when the neighbourhood is exhausted (i.e., the best improvement strategy) or once a better solution is found (i.e., first improvement). In contrast, the latter randomly selects and evaluates only one neighbour of the solution. One may think systematic exploration may be more efficient, as it prevents from revisiting the same neighbours multiple times. In this paper, however, we show that this may not be the case. We first empirically demonstrate that the random sampling method is consistently faster than the systematic exploration method across a range of multi-objective problems. We then give an intuitive explanation for this phenomenon using toy examples, showing that the superior performance of the random sampling method relies on the distribution of ``good neighbours''. Next, we show that the number of such neighbours follows a certain probability distribution during the search. Lastly, building on this distribution, we provide a theoretical insight for why random sampling is more efficient than systematic exploration, regardless of whether the best improvement or first improvement strategy is used.

Figures (9)

• Figure 1: The hypervolume (HV) trajectory (higher is better) of the s-PLS and r-PLS algorithms across 30 runs on the four MOCOPs with 100 decision variables. The bolded line and shaded area represent the mean and standard deviation of the HV, respectively.
• Figure 2: All non-dominated solutions (i.e., solutions in the archive) obtained by the s-PLS and r-PLS in a typical run on the four MOCOPs, where the Knapsack and NK-Landscape are maximisation problems, and the TSP and QAP are minimisation problems.
• Figure 3: The hypervolume trajectory (higher is better) of the s-PLS, s-PLS$_\nsucc$ (it stops when finding a solution not dominated by the archive), s-PLS$_\prec$ (it stops when finding a solution dominating at least one solution in the archive), and r-PLS across 30 runs on the four MOCOPs with 100 decision variables. The bolded line and shaded area represent the mean and standard deviation of the hypervolume, respectively.
• Figure 4: Two toy examples of the distribution of promising solutions in the archive. Here, a promising solution (filled circle) means all of its neighbours being non-dominated to the archive; an unpromising solution (empty circle) means all of its neighbours being dominated by at least one solution in the archive; and a half-promising solution (half-filled circle) means half of its neighbours being non-dominated to the archive. In Example 1, the archive contains two promising solutions and two unpromising solutions. In Example 2, all the four solutions in the archive are half-promising.
• Figure 5: Goodness-of-fit of the distributions with respect to the number of good neighbours among solutions in the archive during the search process of s-PLS (black) and r-PLS (blue) on the TSP (100 cities). A coloured tick in a row indicates that the corresponding algorithm’s data at that point was not rejected under the model. For example, for the geometric distribution model, the blue ticks virtually cover the whole band. This indicates that the number of good neighbours among solutions in the archive for r-PLS follows a geometric distribution throughout the search process. Note that an algorithm can fit multiple distribution models.

Theorems & Definitions (10)

• Lemma 1
• proof
• Theorem 1: r-PLS is faster than s-PLS in finding the next good solution
• proof
• Proposition 1: r-PLS is faster than s-PLS in Example 1
• proof
• Proposition 2: s-PLS is faster than r-PLS in Example 2
• proof
• Proposition 3: Expected time of r-PLS with Best-from-Multiple-Selections
• proof