Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Implementing Recycling Methods for Linear Systems in Python with an Application to Multiple Objective Optimization

Ainara Garcia, Sihong Xie, Arielle Carr

TL;DR

The paper tackles the high computational cost of solving a sequence of linear systems that arise in the predictor step of a Pareto-front computation for multiobjective optimization. It investigates recycling MINRES (RMINRES) and Sparse Approximate Maps (SAMs) to reuse information and map between related coefficient matrices, with Python and PyTorch implementations. Preliminary results from MATLAB benchmarks show RMINRES reducing total iterations from 828 to 796, motivating a Python/PyTorch port and parallel SAMs in future work. The study highlights current PyTorch sparse-tensor limitations and outlines steps toward full integration of recycling methods into the Pareto-front code to enable scalable, cost-efficient multiobjective optimization.

Abstract

Sequences of linear systems arise in the predictor-corrector method when computing the Pareto front for multi-objective optimization. Rather than discarding information generated when solving one system, it may be advantageous to recycle information for subsequent systems. To accomplish this, we seek to reduce the overall cost of computation when solving linear systems using common recycling methods. In this work, we assessed the performance of recycling minimum residual (RMINRES) method along with a map between coefficient matrices. For these methods to be fully integrated into the software used in Enouen et al. (2022), there must be working version of each in both Python and PyTorch. Herein, we discuss the challenges we encountered and solutions undertaken (and some ongoing) when computing efficient Python implementations of these recycling strategies. The goal of this project was to implement RMINRES in Python and PyTorch and add it to the established Pareto front code to reduce computational cost. Additionally, we wanted to implement the sparse approximate maps code in Python and PyTorch, so that it can be parallelized in future work.

Implementing Recycling Methods for Linear Systems in Python with an Application to Multiple Objective Optimization

TL;DR

The paper tackles the high computational cost of solving a sequence of linear systems that arise in the predictor step of a Pareto-front computation for multiobjective optimization. It investigates recycling MINRES (RMINRES) and Sparse Approximate Maps (SAMs) to reuse information and map between related coefficient matrices, with Python and PyTorch implementations. Preliminary results from MATLAB benchmarks show RMINRES reducing total iterations from 828 to 796, motivating a Python/PyTorch port and parallel SAMs in future work. The study highlights current PyTorch sparse-tensor limitations and outlines steps toward full integration of recycling methods into the Pareto-front code to enable scalable, cost-efficient multiobjective optimization.

Abstract

Sequences of linear systems arise in the predictor-corrector method when computing the Pareto front for multi-objective optimization. Rather than discarding information generated when solving one system, it may be advantageous to recycle information for subsequent systems. To accomplish this, we seek to reduce the overall cost of computation when solving linear systems using common recycling methods. In this work, we assessed the performance of recycling minimum residual (RMINRES) method along with a map between coefficient matrices. For these methods to be fully integrated into the software used in Enouen et al. (2022), there must be working version of each in both Python and PyTorch. Herein, we discuss the challenges we encountered and solutions undertaken (and some ongoing) when computing efficient Python implementations of these recycling strategies. The goal of this project was to implement RMINRES in Python and PyTorch and add it to the established Pareto front code to reduce computational cost. Additionally, we wanted to implement the sparse approximate maps code in Python and PyTorch, so that it can be parallelized in future work.
Paper Structure (13 sections, 2 equations, 7 figures, 1 algorithm)

This paper contains 13 sections, 2 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Predictor Corrector Method
  4. Iterative Methods Solving Linear Systems
  5. Recycling Strategies
  6. Implementation
  7. Workflow and Steps
  8. Recycling Minimum Residual (RMINRES)
  9. Sparse Approximate Maps Implementation
  10. Challenges and Proposed Solutions
  11. Discussion
  12. Conclusions and Future Work
  13. Acknowledgments

Figures (7)

  • Figure 1: The predictor step (yellow) and corrector step (green) given an initial point (red).
  • Figure 2: Normwise difference between each pair of matrices in a sequence of 100 systems extracted from b1. Note that the y-axis shows $log\left(\|A_k-A_{k-1}\|_F\right)$.
  • Figure 3: Condition number (y-axis) of matrices from a sequence of 100 systems extracted from b1.
  • Figure 4: Iterations for MINRES and RMINES for a sequence of 100 systems extracted from b1.
  • Figure 5: Code from RMINRES that fails due to underdevelopment of sparse tensors and torch.linalg.solve function.
  • ...and 2 more figures