Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning

Nguyen Anh Minh, Le Dung Muu, Tran Ngoc Thang

TL;DR

The paper tackles nonconvex, multiobjective optimization on unbounded feasible sets by introducing a line-search-free adaptive step-size scheme that yields a monotone multi-gradient descent via a KKT-derived direction $s(x)$ and objective decrease $\Theta(x)=-\tfrac12\|s(x)\|^2$. It provides convergence guarantees under mild assumptions and extends the framework with preference vectors to achieve a more uniform distribution of Pareto points across the front, all while maintaining scalable performance for large-scale multi-task learning problems. Empirical results on synthetic MOPs and real-world MTL tasks (e.g., Multi-MNIST, Drug Review) show faster convergence and improved front coverage compared with PMTL, validating the practical impact of the approach. The proposed line-search-free, adaptive scheme reduces per-iteration cost and offers a flexible, principled method for simultaneously optimizing multiple tasks with diverse trade-offs.

Abstract

We present an adaptive step-size method, which does not include line-search techniques, for solving a wide class of nonconvex multiobjective programming problems on an unbounded constraint set. We also prove convergence of a general approach under modest assumptions. More specifically, the convexity criterion might not be satisfied by the objective function. Unlike descent line-search algorithms, it does not require an initial step-size to be determined by a previously determined Lipschitz constant. The process's primary characteristic is its gradual step-size reduction up until a predetermined condition is met. It can be specifically applied to offer an innovative multi-gradient projection method for unbounded constrained optimization issues. Preliminary findings from a few computational examples confirm the accuracy of the strategy. We apply the proposed technique to some multi-task learning experiments to show its efficacy for large-scale challenges.

Adaptive multi-gradient methods for quasiconvex vector optimization and applications to multi-task learning

TL;DR

The paper tackles nonconvex, multiobjective optimization on unbounded feasible sets by introducing a line-search-free adaptive step-size scheme that yields a monotone multi-gradient descent via a KKT-derived direction and objective decrease . It provides convergence guarantees under mild assumptions and extends the framework with preference vectors to achieve a more uniform distribution of Pareto points across the front, all while maintaining scalable performance for large-scale multi-task learning problems. Empirical results on synthetic MOPs and real-world MTL tasks (e.g., Multi-MNIST, Drug Review) show faster convergence and improved front coverage compared with PMTL, validating the practical impact of the approach. The proposed line-search-free, adaptive scheme reduces per-iteration cost and offers a flexible, principled method for simultaneously optimizing multiple tasks with diverse trade-offs.

Abstract

We present an adaptive step-size method, which does not include line-search techniques, for solving a wide class of nonconvex multiobjective programming problems on an unbounded constraint set. We also prove convergence of a general approach under modest assumptions. More specifically, the convexity criterion might not be satisfied by the objective function. Unlike descent line-search algorithms, it does not require an initial step-size to be determined by a previously determined Lipschitz constant. The process's primary characteristic is its gradual step-size reduction up until a predetermined condition is met. It can be specifically applied to offer an innovative multi-gradient projection method for unbounded constrained optimization issues. Preliminary findings from a few computational examples confirm the accuracy of the strategy. We apply the proposed technique to some multi-task learning experiments to show its efficacy for large-scale challenges.
Paper Structure (13 sections, 11 theorems, 64 equations, 1 figure, 2 algorithms)

This paper contains 13 sections, 11 theorems, 64 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries and Notations
  3. Multi-objective optimization problem
  4. Multi-task Learning Problem
  5. Adaptive multi-gradient for solving Multi-objective optimization problem
  6. Solving subproblem by KKT condition
  7. Compute the adaptive step length
  8. The complete algorithm
  9. Convergence of adaptive multi-gradient methods
  10. Adaptive multi-gradient with preference vectors
  11. The update algorithm
  12. Convergence of adaptive multi-gradient with preference vector
  13. Numerical experiments

Key Result

proposition thmcounterproposition

Dennis The differentiable function $f$ is quasiconvex on $U$ if and only if

Figures (1)

  • Figure 1: Partitioning algorithm with $J=3,\delta=0.2$

Theorems & Definitions (19)

  • definition thmcounterdefinition
  • proposition thmcounterproposition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • ...and 9 more