Iterated Local Search with Linkage Learning

TL;DR

Experiments with NK landscapes, knapsack problem, and feature selection show that local search with linkage learning 2 is able to efficiently build weighted variable interaction graphs, which can provide new insights about the optimization problem and behavior of optimizers.

Abstract

In pseudo-Boolean optimization, a variable interaction graph represents variables as vertices, and interactions between pairs of variables as edges. In black-box optimization, the variable interaction graph may be at least partially discovered by using empirical linkage learning techniques. These methods never report false variable interactions, but they are computationally expensive. The recently proposed local search with linkage learning discovers the partial variable interaction graph as a side-effect of iterated local search. However, information about the strength of the interactions is not learned by the algorithm. We propose local search with linkage learning 2, which builds a weighted variable interaction graph that stores information about the strength of the interaction between variables. The weighted variable interaction graph can provide new insights about the optimization problem and behavior of optimizers. Experiments with NK landscapes, knapsack problem, and feature selection show that local search with linkage learning 2 is able to efficiently build weighted variable interaction graphs. In particular, experiments with feature selection show that the weighted variable interaction graphs can be used for visualizing the feature interactions in machine learning. Additionally, new transformation operators that exploit the interactions between variables can be designed. We illustrate this ability by proposing a new perturbation operator for iterated local search.

Figures (8)

• Figure 1: Empirical VIGw in six steps of LSwLL2. In each step, fitness difference resulted from flipping variable $x_g$ is computed. Flipping the variable is accepted only when fitness improvement is detected, i.e., when $\delta_g(\mathbf{x})>0$. In this case, the variable is indicated in the figure by $x_h$. Initially, the empirical VIGw contains no edges (step 1). Respectively in steps 2 and 3, decision variables $x_4$ and $x_5$ are visited (and stored in list $Q$), resulting in fitness differences equal to -0.5 and -0.6 (stored in list $F$). In step 4, visiting $x_1$ results in fitness improvement. Then, the same sequence of variables (stored in list $Q$) are visited again and the fitness differences are compared to the respective values stored in $F$. In step 6, the fitness difference (-0.8) is not equal to the value stored in $F$ (-0.6), indicating an interaction (edge in the VIGw) between $x_1$ and $x_5$.
• Figure 2: Example of VIGwbP . Decision variable $x_5$ is randomly chosen. Then, $x_5$ and its two neighbors ($x_3$ and $x_6$) in the empirical VIGw (the weights are not showed) are flipped.
• Figure 3: Empirical weighted VIG found by ILS with VIGwbP and LSwLL2 in one run for the NK landscapes ($N=30$, $k=3$) experiment with fixed number of iterations. The widths of the lines are proportional to the weights of the respective edges. The size of the i-th node is proportional to the contribution $f_i$ for the evaluation of the best individual found by ILS. Left: complete graph. Right: only the edges with largest weights are presented. The largest weights are defined according to the procedure to compute the threshold $\beta$ in Section \ref{['sec:VIGwbP']}.
• Figure 4: Empirical weighted VIG found by ILS with VIGwbP and LSwLL2 in one run for the 0-1 knapsack problem ($N=30$) experiment with fixed number of iterations. Here, the size of the i-th node is proportional to the weight of the i-th object of the knapsack problem (heavier objects are represented by larger circles). Left: complete graph. Right: only the edges with largest weights are presented.
• Figure 5: Empirical weighted VIG found by ILS with VIGwbP and LSwLL2 in the first run of the experiments with fixed number of iterations for the feature selection problem with dataset housing. Here, the features selected by ILS are indicated by the blue circles. Left: complete graph. Right: only the edges with largest weights are presented.

Theorems & Definitions (8)

• definition 1
• definition 2
• definition 3
• definition 4
• definition 5
• definition 6
• theorem 1
• corollary 1