Guided Learning: Lubricating End-to-End Modeling for Multi-stage Decision-making

TL;DR

This work proposes Guided Learning, a novel methodological framework designed to enhance end-to-end learning in multi-stage decision-making, and introduces the concept of a ``guide'', a function that induces the training of intermediate neural network layers towards some phased goals, directing gradients away from suboptimal collapse.

Abstract

Multi-stage decision-making is crucial in various real-world artificial intelligence applications, including recommendation systems, autonomous driving, and quantitative investment systems. In quantitative investment, for example, the process typically involves several sequential stages such as factor mining, alpha prediction, portfolio optimization, and sometimes order execution. While state-of-the-art end-to-end modeling aims to unify these stages into a single global framework, it faces significant challenges: (1) training such a unified neural network consisting of multiple stages between initial inputs and final outputs often leads to suboptimal solutions, or even collapse, and (2) many decision-making scenarios are not easily reducible to standard prediction problems. To overcome these challenges, we propose Guided Learning, a novel methodological framework designed to enhance end-to-end learning in multi-stage decision-making. We introduce the concept of a ``guide'', a function that induces the training of intermediate neural network layers towards some phased goals, directing gradients away from suboptimal collapse. For decision scenarios lacking explicit supervisory labels, we incorporate a utility function that quantifies the ``reward'' of the throughout decision. Additionally, we explore the connections between Guided Learning and classic machine learning paradigms such as supervised, unsupervised, semi-supervised, multi-task, and reinforcement learning. Experiments on quantitative investment strategy building demonstrate that guided learning significantly outperforms both traditional stage-wise approaches and existing end-to-end methods.

Figures (5)

• Figure 1: Example pipeline in autonomous driving systems
• Figure 2: Illustration of guided learning in an end-to-end model with an example guide implementation. The model is conceptually separated into $K$ stages, each represented by a function $f_{\theta_k}$ that transforms $h_{k-1}$ to $h_k$. At each stage, a guided head $g_{\phi_k}$ maps $h_k$ to a phased output $c_k$. A guided loss function $L^c_k$ is computed at each stage with different implementations. For example, $L^c_1$ is an unsupervised loss based on feature norms. $L^c_2$ represents a loss defined on partial features using a distance metric $\delta(\cdot, \cdot)$. $L^c_{K-1}$ represents a weighted sum of multiple losses computed with labels in multiple perspectives (e.g. detection loss and segmentation loss), as illustrated in different colors. The pipeline produces a final output $\hat{y}$ and computes a utility-based loss $L_K$ for all samples.
• Figure 3: Illustration of the multi-stage approach and end-to-end approach for quantitative investment.
• Figure 4: Backtest curve
• Figure 5: Parameter sensitivity