Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Solution-Aware Transformers for Efficiently Solving Quadratic Assignment Problem

Zhentao Tan, Yadong Mu

TL;DR

This paper tackles Koopmans-Beckmann's Quadratic Assignment Problem (QAP), a strongly NP-hard COP, by introducing SAWT, a learn-to-improve reinforcement learning method that encodes facilities and locations separately and incorporates solution-aware attention to capture higher-order QAP structure. The model uses independent embeddings for facilities and locations, a Solution Aware Transformer encoder, and a swap-based decoder guided by REINFORCE with a value head, achieving scalable performance without building an association graph. Through extensive experiments on self-generated and QAPLIB benchmarks, SAWT demonstrates strong generalization across problem sizes (up to $n=100$) and favorable inference efficiency compared to exact solvers, with notable gaps reduced as steps increase. The work shows promising practical impact for large-scale QAPs, while acknowledging limitations on certain QAPLIB categories and signaling future directions in meta-learning to further boost generalization.

Abstract

Recently various optimization problems, such as Mixed Integer Linear Programming Problems (MILPs), have undergone comprehensive investigation, leveraging the capabilities of machine learning. This work focuses on learning-based solutions for efficiently solving the Quadratic Assignment Problem (QAPs), which stands as a formidable challenge in combinatorial optimization. While many instances of simpler problems admit fully polynomial-time approximate solution (FPTAS), QAP is shown to be strongly NP-hard. Even finding a FPTAS for QAP is difficult, in the sense that the existence of a FPTAS implies $P = NP$. Current research on QAPs suffer from limited scale and computational inefficiency. To attack the aforementioned issues, we here propose the first solution of its kind for QAP in the learn-to-improve category. This work encodes facility and location nodes separately, instead of forming computationally intensive association graphs prevalent in current approaches. This design choice enables scalability to larger problem sizes. Furthermore, a \textbf{S}olution \textbf{AW}are \textbf{T}ransformer (SAWT) architecture integrates the incumbent solution matrix with the attention score to effectively capture higher-order information of the QAPs. Our model's effectiveness is validated through extensive experiments on self-generated QAP instances of varying sizes and the QAPLIB benchmark.

Learning Solution-Aware Transformers for Efficiently Solving Quadratic Assignment Problem

TL;DR

This paper tackles Koopmans-Beckmann's Quadratic Assignment Problem (QAP), a strongly NP-hard COP, by introducing SAWT, a learn-to-improve reinforcement learning method that encodes facilities and locations separately and incorporates solution-aware attention to capture higher-order QAP structure. The model uses independent embeddings for facilities and locations, a Solution Aware Transformer encoder, and a swap-based decoder guided by REINFORCE with a value head, achieving scalable performance without building an association graph. Through extensive experiments on self-generated and QAPLIB benchmarks, SAWT demonstrates strong generalization across problem sizes (up to ) and favorable inference efficiency compared to exact solvers, with notable gaps reduced as steps increase. The work shows promising practical impact for large-scale QAPs, while acknowledging limitations on certain QAPLIB categories and signaling future directions in meta-learning to further boost generalization.

Abstract

Recently various optimization problems, such as Mixed Integer Linear Programming Problems (MILPs), have undergone comprehensive investigation, leveraging the capabilities of machine learning. This work focuses on learning-based solutions for efficiently solving the Quadratic Assignment Problem (QAPs), which stands as a formidable challenge in combinatorial optimization. While many instances of simpler problems admit fully polynomial-time approximate solution (FPTAS), QAP is shown to be strongly NP-hard. Even finding a FPTAS for QAP is difficult, in the sense that the existence of a FPTAS implies . Current research on QAPs suffer from limited scale and computational inefficiency. To attack the aforementioned issues, we here propose the first solution of its kind for QAP in the learn-to-improve category. This work encodes facility and location nodes separately, instead of forming computationally intensive association graphs prevalent in current approaches. This design choice enables scalability to larger problem sizes. Furthermore, a \textbf{S}olution \textbf{AW}are \textbf{T}ransformer (SAWT) architecture integrates the incumbent solution matrix with the attention score to effectively capture higher-order information of the QAPs. Our model's effectiveness is validated through extensive experiments on self-generated QAP instances of varying sizes and the QAPLIB benchmark.
Paper Structure (30 sections, 21 equations, 4 figures, 13 tables)

This paper contains 30 sections, 21 equations, 4 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Quadratic Assignment Problem
  3. Related Work
  4. Problem Formulation
  5. Solution-Aware Transformers
  6. Embeddings
  7. Location embedding.
  8. Facility embedding.
  9. The Encoder
  10. SAWT-Att.
  11. The Decoder
  12. Reinforcement Learning Training Algorithm
  13. Experiments
  14. Experimental Setup
  15. Instance generation.
  16. ...and 15 more sections

Figures (4)

  • Figure 1: Architecture of our policy network, namely Solution Aware Transformer (SAWT). See main text for more details.
  • Figure 2: Architecture of SAWT module. For a given current solution $\sigma$, we rearrange the rows of the distance matrix to create $\mathbf{D}\sigma = (\mathbf{d}{\sigma(1)}, ..., \mathbf{d}{\sigma(n)})$. Conducting a dot product between the flow matrix $\mathbf{F}$ and the distance matrix $\mathbf{D_\sigma}$ results in the solution-aware matrix $\mathbf{M}\sigma$, which encapsulates the complete information about the QAP's objective. This is because the cost for the current solution $\sigma$ is the sum of $\mathbf{M}\sigma$. By integrating $\mathbf{M}_\sigma$ into the attention module, our goal is to equip the model with incumbent solution information.
  • Figure 3: Structure of MatNet$^{\ast}$.
  • Figure 4: Model behavior on training and testing period.