Improvements to steepest descent method for multi-objective optimization
Wang Chen, Liping Tang, Xinmin Yang
TL;DR
Addressing unconstrained multi-objective optimization, the paper enhances the multi-objective steepest descent (MSD) method by introducing a positive multiplicative step-size modifier $\theta_k$ in the update $x^{k+1}=x^{k}+t_k\theta_k v(x^{k})$. Two concrete schemes are proposed: MSD-I uses a diagonal Hessian approximation via a scalar $\tau_k$ leading to $x^{k+1}=x^{k}+t_k\tau_k^{-1}v(x^{k})$ with a derived update $\tau_{k+1}=\dfrac{2\tau_k\left(\tau_k\sum_{i=1}^m\lambda_i^k(F_i(x^{k+1})-F_i(x^k))+t_k\|v(x^k)\|^2\right)}{t_k^2\|v(x^k)\|^2}$, and is proven to converge to Pareto-critical points; MSD-II uses a second-order model yielding $\theta_k=\bar{\theta}=\dfrac{p_k}{q_k+2t_k\epsilon_k}$ with $p_k=t_k\|v(x^k)\|^2$ and $q_k=t_k v(x^k)^T(\sum_i\lambda_i^k\nabla^2 F_i(x^k)) v(x^k)$, requiring a gradient evaluation at $z^k=x^k+t_k v(x^k)$ and producing $\theta_k\approx p_k/q_k$ after simplifications. Under mild assumptions, MSD-I converges to Pareto-critical points and, for strongly convex MO problems, achieves R-linear convergence to a Pareto optimum; MSD-II exhibits strong empirical performance, though formal convergence theory remains open. Extensive numerical experiments on 32 test problems and large-scale FDS scenarios show the proposed methods outperform the original MSD and the MDSD, with MSD-II delivering the best overall results.
Abstract
In this paper, we propose a simple yet efficient strategy for improving the multi-objective steepest descent method proposed by Fliege and Svaiter (Math Methods Oper Res, 2000, 3: 479--494). The core idea behind this strategy involves incorporating a positive modification parameter into the iterative formulation of the multi-objective steepest descent algorithm in a multiplicative manner. This modification parameter captures certain second-order information associated with the objective functions. We provide two distinct methods for calculating this modification parameter, leading to the development of two improved multi-objective steepest descent algorithms tailored for solving multi-objective optimization problems. Under reasonable assumptions, we demonstrate the convergence of sequences generated by the first algorithm toward a critical point. Moreover, for strongly convex multi-objective optimization problems, we establish the linear convergence to Pareto optimality of the sequence of generated points. The performance of the new algorithms is empirically evaluated through a computational comparison on a set of multi-objective test instances. The numerical results underscore that the proposed algorithms consistently outperform the original multi-objective steepest descent algorithm.