Feedback Control for Multi-Objective Graph Self-Supervision

TL;DR

ControlG reimagines multi-objective graph self-supervised learning as a closed-loop scheduling problem. It separates objectives in time, using full-graph sensing to estimate spectral demand $RQ_k$ and interference, planning target budgets via a Pareto-aware log-HV planner, and executing via deficit-tracking PID control to assign discrete single-task blocks. This approach mitigates Disagreement, Drift, and Drought and yields auditable training schedules, with robust improvements across nine graph benchmarks. The method maintains practical compute overhead and offers interpretable insights into which objectives drive learning, enhancing transfer performance for diverse graph datasets.

Abstract

Can multi-task self-supervised learning on graphs be coordinated without the usual tug-of-war between objectives? Graph self-supervised learning (SSL) offers a growing toolbox of pretext objectives: mutual information, reconstruction, contrastive learning; yet combining them reliably remains a challenge due to objective interference and training instability. Most multi-pretext pipelines use per-update mixing, forcing every parameter update to be a compromise, leading to three failure modes: Disagreement (conflict-induced negative transfer), Drift (nonstationary objective utility), and Drought (hidden starvation of underserved objectives). We argue that coordination is fundamentally a temporal allocation problem: deciding when each objective receives optimization budget, not merely how to weigh them. We introduce ControlG, a control-theoretic framework that recasts multi-objective graph SSL as feedback-controlled temporal allocation by estimating per-objective difficulty and pairwise antagonism, planning target budgets via a Pareto-aware log-hypervolume planner, and scheduling with a Proportional-Integral-Derivative (PID) controller. Across 9 datasets, ControlG consistently outperforms state-of-the-art baselines, while producing an auditable schedule that reveals which objectives drove learning.

Figures (6)

• Figure 1: Motivation and overview ofControlG. (a) The key insight: temporal allocation avoids objective conflict by dedicating compute blocks to one objective at a time. (b) The learned schedule is interpretable and auditable---the planner adapts allocation based on interference and demand signals.
• Figure 2: Overview ofControlG. The framework decomposes multi-task graph self-supervised learning into three coupled loops operating at different timescales. (1) SENSE (timescale $u$): A shared encoder $f_\theta$ maps the graph $(G,\mathbf{X})$ to embeddings $\mathbf{Z}$, from which per-task losses $L_k$ are computed. State estimation derives two difficulty signals---Spectral Demand ($\mathrm{RQ}_k$), measuring how high-frequency the learning signal is on the graph, and Interference ($\mathrm{Conf}_k$), measuring gradient conflicts with other Pareto-relevant objectives---which are combined into a composite difficulty state $D_k$. (2) PLAN (timescale $t$): Normalized losses $\tilde{L}_k$ are converted to Log-HV sensitivities$w_k^{\mathrm{HV}} = 1/(r_k - \tilde{L}_k)$, quantifying each objective's marginal contribution to Pareto progress. These are tempered by difficulty to produce a difficulty-adjusted allocation$a_k \propto w_k^{\mathrm{HV}} / D_k$, normalized to an epoch-level target $f(t)\in\Delta^K$. (3) CONTROL (timescale $m$): A PID controller tracks the allocation plan by computing logits $\nu_k$ from deficits $e_k = N_k^{\mathrm{ref}} - N_k$, their integral, and derivative. A stochastic sampler (softmax with $\epsilon$-greedy exploration) selects the task $k_{t,m}$ for each block, updating counts $N_k$ and closing the feedback loop. Dashed arrows indicate cross-module information flow.
• Figure 3: Time per step on Cora. Bar plot comparing wall-clock time (ms) per optimizer step across all methods. ControlG (highlighted in teal) incurs modest overhead compared to simple scheduling baselines (p_par, Random, Round-Robin) but is significantly faster than heavyweight methods like AutoSSL and ParetoGNN.
• Figure 4: Task scheduling timeline.Top: Scatter plot showing which pretext task was selected at each training block (raster view). Bottom: Stacked area chart showing the running proportion of recent blocks allocated to each task. The visualization reveals how ControlG dynamically shifts focus between tasks over training---initially exploring broadly, then concentrating on tasks with higher difficulty or interference signals.
• Figure 5: Allocation deficit trajectories. Deficit $e_k(t)$ for each pretext task over training blocks, showing how far each task is from its target allocation (mean $\pm$ 1 std across seeds). Positive values indicate the task is under-scheduled relative to its target; negative values indicate over-scheduling. The PID controller drives deficits toward zero, ensuring long-run tracking of the planned allocation while allowing short-term deviations for exploration.

Theorems & Definitions (65)

• Proposition 4.1: Spectral demand bounds attainable progress; informal, see Prop. \ref{['prop:rq-progress-bound']} in App. \ref{['app:rq-difficulty']}
• Claim 1.1: Eigenvalue range of the normalized Laplacian
• proof
• Claim 1.2: Edge-domain form of the normalized Laplacian quadratic form
• proof
• Claim 1.3: Matrix-signal extension
• proof
• Claim 1.4: Eigenbasis decomposition for matrix Dirichlet energy
• proof
• Lemma 1.6: First-order progress under Assumption \ref{['assm:lowpass']}