When is local search both effective and efficient?

TL;DR

This work analyzes when local search is both effective and efficient on single-peaked fitness landscapes by representing landscapes as binary VCSPs. It introduces directed and oriented VCSPs to capture semismooth landscapes and then defines conditionally-smooth landscapes as a polynomial-time recognizable generalization that remains single-peaked. The paper demonstrates that a broad class of popular local search algorithms (including RandomAscent, Simulated Annealing, various history-based rules, jumping rules, and Kernighan–Lin) are efficient on conditionally-smooth landscapes, while some core methods like Steepest Ascent and RandomFacet can still be intractable on these landscapes. It also proves NP-hardness for recognizing directed/oriented structure, with fixed-parameter tractability by maximum degree, and provides both efficient and hard-instance results to delineate the boundaries of local search efficiency. Overall, the results illuminate when local search is likely to be fast in practice and offer a framework for understanding fitness landscapes in natural and social systems.

Abstract

Combinatorial optimization problems implicitly define fitness landscapes that combine the numeric structure of the 'fitness' function to be maximized with the combinatorial structure of which assignments are 'adjacent'. Local search starts at an assignment in this landscape and successively moves assignments until no further improvement is possible among the adjacent assignments. Classic analyses of local search algorithms have focused more on the question of effectiveness ("did we find a good solution?") and often implicitly assumed that there are no doubts about their efficiency ("did we find it quickly?"). But there are many reasons to doubt the efficiency of local search. Even if we focus on fitness landscapes on the hypercube that are single peaked on every subcube (i.e., semismooth fitness landscapes) where effectiveness is obvious, many local search algorithms are known to be inefficient. Since fitness landscapes are unwieldy exponentially large objects, we focus on their polynomial-sized representations by instances of valued constraint satisfaction problems (VCSP). We define a "direction" for valued constraints such that directed VCSPs generate semismooth fitness landscapes. We call VCSPs oriented if they do not have any pair of variables with arcs in both directions. Since recognizing if a VCSP-instance is directed or oriented is coNP-complete, we generalized oriented VCSPs as conditionally-smooth fitness landscapes that are recognizable in polynomial time for a VCSP-instance. We prove that many popular local search algorithms like random ascent, simulated annealing, history-based rules, jumping rules, and the Kernighan-Lin heuristic are very efficient on conditionally-smooth landscapes. But conditionally-smooth landscapes are still expressive enough so that algorithms like steepest ascent and random facet require a super-polynomial number of steps to find the fitness peak.

Figures (3)

• Figure 4.1: Four instances illustrating the different arc directions of \ref{['def:VCSP_arcs']}. Weights of unary constraints are next to nodes and weights of binary constraints are above the edges.
• Figure 7.1: Haken-Luby gadget with $M_k=\frac{5}{6}(6^k-6)$, $\epsilon_k=n+1-k$, and $K=2n+1$. Constraints of the $k$th of $n$ gadgets are shown: weights of unary constraints are next to their variables and weights of binary constraints are above the edges that specify their scope. Arcs are oriented according to Definition \ref{['alg:VCSP_arcs']}, showing that the instance is oriented. Dotted arcs and vertices illustrate the connection to the neighboring gadgets. For the boundaries: the unary of $(n,1)$ is $(6M_n+24)K>0$, $M_1=0$ and there is no binary constraint $c_{(1,7),(0,1)}$.
• Figure B.1: The instance $\mathcal{C}$ in the reductions from positive SubsetSum of \ref{['prop:rse_NPC']} and \ref{['prop:doublese_NPC']}. The weight of the unary constraints is given near the nodes, whilst the weight of the binary constraints appears on the edges. In both (a) and (b) the variables $0,1,\ldots, n$ are sign-independent. In (a) the SubsetSum instance is encoded in the binary constraints of variables $0,1,\ldots n,c$, and $\ell$ sign-depends on $c$. Only the edge $\{c,l\}$ can be bidirected, and whether or not it is depends on the answer to the SubsetSum instance. In (b) the encoding of SubsetSum is also on the binary constraints, but now $l$ sign-depends on $c$ only when $x_0=x_1=\ldots=x_n=0$, and $c$ sign depends on $l$ if and only if SubsetSum is a yes-instance.

Theorems & Definitions (51)

• Definition 3.1: Kaznatcheev, Cohen and Jeavons repCP
• Definition 3.2: Poelwijk et al.rse2
• Proposition 3.3: Hammer et al.completelyUnimodal, Poelwijk et al.rse2, and Kaznatcheev evoPLS
• Proposition 3.4: Hammer et al.completelyUnimodal, and Kaznatcheev evoPLS
• Definition 4.1
• Definition 4.2
• Definition 4.3
• Definition 5.1
• Proposition 5.2
• Definition 6.1