Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Energy-Aware Routing to Large Reasoning Models

Austin R. Ellis-Mohr, Max Hartman, Lav R. Varshney

TL;DR

This work tackles energy-aware routing of tasks to heterogeneous large reasoning models (LRMs) under renewable-energy variability, framing the problem as minimizing auxiliary energy while meeting latency and tolerance constraints. It introduces a stochastic, second-order analysis that links routing decisions to energy dynamics via Brownian-motion-inspired diffusion approximations, identifying drift- and fluctuation-dominated regimes and a critical balance point. The paper derives a policy-agnostic energy lower bound, analyzes myopic baselines, and demonstrates how variance-aware routing can reduce reserve costs; it further grounds dispatch decisions in scaling laws for training-compute and inference-compute to enable practical, energy-efficient policies. Its findings provide principled guidance for variance-aware dispatch and outline extensions toward backpressure-inspired strategies and more complex service graphs, with significant implications for energy-aware AI factories operating under renewable-energy variability.

Abstract

Large reasoning models (LRMs) have heterogeneous inference energy costs based on which model is used and how much it reasons. To reduce energy, it is important to choose the right LRM and operate it in the right way. As a result, the performance of systems that dispatch tasks to different individual LRMs depend on the balance between mean energy provisioning and stochastic fluctuations. The critical regime is the unique operating point at which neither auxiliary energy nor baseline energy is systematically wasted. Increasing baseline supply shifts the system toward persistent over-supply and baseline-energy waste, while reducing supply induces persistent reliance on auxiliary energy. Yet in this regime, performance remains volatility-limited and so a second-order characterization provides further insights that we develop. Here, performance is governed by how variability is absorbed across time, models, and execution choices. This perspective highlights variance-aware routing and dispatch as a principled design axis, and provides a theoretical basis for developing energy-aware model routing policies. Routing behavior is characterized when dispatch policies are based on training-compute and inference-compute scaling laws for LRMs.

Energy-Aware Routing to Large Reasoning Models

TL;DR

This work tackles energy-aware routing of tasks to heterogeneous large reasoning models (LRMs) under renewable-energy variability, framing the problem as minimizing auxiliary energy while meeting latency and tolerance constraints. It introduces a stochastic, second-order analysis that links routing decisions to energy dynamics via Brownian-motion-inspired diffusion approximations, identifying drift- and fluctuation-dominated regimes and a critical balance point. The paper derives a policy-agnostic energy lower bound, analyzes myopic baselines, and demonstrates how variance-aware routing can reduce reserve costs; it further grounds dispatch decisions in scaling laws for training-compute and inference-compute to enable practical, energy-efficient policies. Its findings provide principled guidance for variance-aware dispatch and outline extensions toward backpressure-inspired strategies and more complex service graphs, with significant implications for energy-aware AI factories operating under renewable-energy variability.

Abstract

Large reasoning models (LRMs) have heterogeneous inference energy costs based on which model is used and how much it reasons. To reduce energy, it is important to choose the right LRM and operate it in the right way. As a result, the performance of systems that dispatch tasks to different individual LRMs depend on the balance between mean energy provisioning and stochastic fluctuations. The critical regime is the unique operating point at which neither auxiliary energy nor baseline energy is systematically wasted. Increasing baseline supply shifts the system toward persistent over-supply and baseline-energy waste, while reducing supply induces persistent reliance on auxiliary energy. Yet in this regime, performance remains volatility-limited and so a second-order characterization provides further insights that we develop. Here, performance is governed by how variability is absorbed across time, models, and execution choices. This perspective highlights variance-aware routing and dispatch as a principled design axis, and provides a theoretical basis for developing energy-aware model routing policies. Routing behavior is characterized when dispatch policies are based on training-compute and inference-compute scaling laws for LRMs.
Paper Structure (25 sections, 5 theorems, 82 equations, 4 figures)

This paper contains 25 sections, 5 theorems, 82 equations, 4 figures.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Tasks, requirements, and arrivals
  4. Router actions and task life-cycle
  5. Reliability model and oracle stopping
  6. System energetics
  7. Objective and simplified dynamics
  8. The feasible set
  9. Performance Analysis
  10. Ergodic arrivals and harvesting
  11. Myopic dispatcher scaling analysis
  12. Battery random walk
  13. Diffusion approximation
  14. Routing error as a function of fluctuations
  15. Compute Scaling
  16. ...and 10 more sections

Key Result

Theorem 1

Fix a horizon $T\in\mathbb{N}$, and exogenous sequences $\{R_t\}_{t=0}^{T-1}$ and $\{C_t\}_{t=0}^{T-1}$ in $\mathbb{R}$. Let $t=0,1,\dots,T-1$ and the uncontrolled (possibly negative) battery trajectory $\{B_t\}_{t=0}^T$ be defined by Let the controlled battery trajectory $\{\widetilde{B}_t\}_{t=0}^T$ with nonnegative injections $\{G_t\}_{t=0}^{T-1}$ be defined by where $G_t\ge 0$ for all $t$. C

Figures (4)

  • Figure 1: System diagram
  • Figure 2: Deviation of the normalized expected reserve from drift-only scaling as a function of relative mean--variance ratio $\kappa=\mu T/(\sigma\sqrt{2T/\pi})$. The deviation is largest when drift and fluctuation contributions are of comparable scale and diminishes as drift dominance increases. Note that nonnegative drift denotes the surplus region.
  • Figure 3: Energy consumption and latency per task to generate a response within an error tolerance for a large and small model. The small model is characterized by less-accurate, fast token generation, and the large model is characterized by more-accurate, slow token generation. For simpler tasks, the small model takes less time and energy, however as the task difficulty increases, the small model uses more energy and takes a longer amount of time before generating a correct response leading to a tradeoff before the large model becomes preferred.
  • Figure 4: Expected auxiliary energy consumption $\mathop{\mathrm{\mathbb{E}}}\nolimits[D_T]$ versus time horizon $T$ under varying prediction errors $\mathcal{E}$ for the myopic policy. The zero error policy ($\mathcal{E} = 0$) exhibits square-root scaling throughout (dashed fit), while nonzero error policies transition from fluctuation-dominated (square-root scaling, dashed) to drift-dominated (linear scaling, dotted) regimes at the vertical markers. Shaded regions indicate standard error over 100 trials.

Theorems & Definitions (5)

  • Theorem 1: Cumulative injections equal the maximal deficit of the unconstrained path
  • Theorem 2: Expected deficit scaling across drift regimes
  • Lemma 1: Lumped myopic is pathwise pessimistic
  • Lemma 2: Variance of arrival-feasible consumption
  • Lemma 3: Running minimum of Brownian motion with drift