Optimization problems constrained by parameter sums

TL;DR

This paper investigates optimization problems where the objective $f(x)$ is constrained by a simple scale relation $q(x)=c$, with $q(x)$ often being a sum or sum of squares of the parameters. It surveys the challenges of applying general-purpose solvers to sumscale problems, notably scaling indeterminacy and numerical underflow, and proposes practical reformulations including embedding scaling in the objective, log-transformations, leave-one-out schemes, and projection methods. Through illustrative examples such as the Rayleigh Quotient, the extended Rosenbrock function on the unit ball, and an R-help scenario, the work demonstrates that standard optimization tools can be used effectively when these reformulations and projections are carefully employed. The findings highlight that while specialized solvers are fastest for these constrained problems, robust general-purpose approaches—especially projection-based methods like spg with appropriate constraints—offer viable, transferable strategies for a broad user base. Practically, the paper provides actionable guidance on solver selection, parameter handling, and constraint enforcement to solve sumscale problems in environments like R.

Abstract

This article presents a discussion of optimization problems where the objective function f(x) has parameters that are constrained by some scaling, so that q(x) = constant, where this function q() involves a sum of the parameters, their squares, or similar simple function. Our focus is on ways to use standardized optimization programs to solve such problems rather than specialized codes. Examples are presented with R.