Untangling Lariats: Subgradient Following of Variationally Penalized Objectives

TL;DR

This work tackles smoothing of sequences under general variational penalties by framing the problem as $\min_{\mathbf{x}}\sum_i h_i(x_i) + \sum_i g_i(x_i - x_{i+1})$ and introducing the Untangled Lariat, a subgradient-following method founded on Fenchel duality. It develops a lattice-based, convolution-aware solver that handles high-order and non-smooth penalties, recovers classic methods like the fused lasso and isotonic regression as special cases, and extends to multivariate and sparsity-promoting variants with provable convergence properties. The paper also presents empirical demonstrations on financial time series, illustrating the method’s ability to produce piecewise-smooth trajectories with controllable variation, while maintaining competitive runtime and fusion-detection performance. Overall, the Untangled Lariat provides a flexible, unified framework for variational sequence optimization with broad theoretical and practical implications.

Abstract

We describe an apparatus for subgradient-following of the optimum of convex problems with variational penalties. In this setting, we receive a sequence $y_i,\ldots,y_n$ and seek a smooth sequence $x_1,\ldots,x_n$. The smooth sequence needs to attain the minimum Bregman divergence to an input sequence with additive variational penalties in the general form of $\sum_i{}g_i(x_{i+1}-x_i)$. We derive known algorithms such as the fused lasso and isotonic regression as special cases of our approach. Our approach also facilitates new variational penalties such as non-smooth barrier functions. We then derive a novel lattice-based procedure for subgradient following of variational penalties characterized through the output of arbitrary convolutional filters. This paradigm yields efficient solvers for high-order filtering problems of temporal sequences in which sparse discrete derivatives such as acceleration and jerk are desirable. We also introduce and analyze new multivariate problems in which $\mathbf{x}_i,\mathbf{y}_i\in\mathbb{R}^d$ with variational penalties that depend on $\|\mathbf{x}_{i+1}-\mathbf{x}_i\|$. The norms we consider are $\ell_2$ and $\ell_\infty$ which promote group sparsity.

Figures (11)

• Figure 1: Construction of $\mathsf{a}_3(\cdot)$ from $\mathsf{a}_4(\cdot)$ and optimal solution for the fused lasso.
• Figure 2: Construction of $\mathsf{a}_3(\cdot)$ from $\mathsf{a}_4(\cdot)$ and optimal solution for the sparse fused lasso.
• Figure 3: Construction of $\mathsf{a}_3(\cdot)$ from $\mathsf{a}_4(\cdot)$ and optimal solution for isotonic regression.
• Figure 4: Construction of $\mathsf{a}_3(\cdot)$ from $\mathsf{a}_4(\cdot)$ and optimal solution for fused barriers.
• Figure 5: Comparison least-squares sequence approximation with variational penalty of the fused lasso (FL) versus barrier constraints (FB).