Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAGE-32B: Agentic Reasoning via Iterative Distillation

Basab Jha, Firoj Paudel, Ujjwal Puri, Ethan Henkel, Zhang Yuting, Mateusz Kowalczyk, Mei Huang, Choi Donghyuk, Wang Junhao

TL;DR

SAGE-32B addresses the reliability gap in autonomous agentic LLMs by engineering a mid-sized model with Iterative Distillation and an Inverse Reasoning (IR) head. The system combines a modified decoder architecture, split embedding spaces, and Landmark Attention to sustain long-horizon planning, while a Distillation and Amplification pipeline transfers capabilities from larger teachers. The IR mechanism, reinforced by Stage 2 Reflective Distillation, Stage 3 DPO for safety, and CodePPO-style RL for function calling, yields strong tool-use reliability with reduced catastrophic error propagation. Across standard reasoning benchmarks and internal tool-use suites, SAGE-32B surpasses open-weight baselines and delivers near GPT-4-class reliability at substantially lower cost, albeit with trade-offs in general-domain knowledge and latency in ultra-long agent loops. The work offers a practical pathway for deploying reliable autonomous systems, with transparent ablations, explicit safety considerations, and a candid discussion of limitations and future directions, including on-device scaling to smaller models and improved API documentation.

Abstract

We demonstrate SAGE-32B, a 32 billion parameter language model that focuses on agentic reasoning and long range planning tasks. Unlike chat models that aim for general conversation fluency, SAGE-32B is designed to operate in an agentic loop, emphasizing task decomposition, tool usage, and error recovery. The model is initialized from the Qwen2.5-32B pretrained model and fine tuned using Iterative Distillation, a two stage training process that improves reasoning performance through rigorously tested feedback loops. SAGE-32B also introduces an inverse reasoning approach, which uses a meta cognition head to forecast potential failures in the planning process before execution. On agentic reasoning benchmarks including MMLU-Pro, AgentBench, and MATH-500, SAGE-32B achieves higher success rates in multi tool usage scenarios compared to similarly sized baseline models, while remaining competitive on standard reasoning evaluations. Model weights are publicly released at https://huggingface.co/sagea-ai/sage-reasoning-32b

SAGE-32B: Agentic Reasoning via Iterative Distillation

TL;DR

SAGE-32B addresses the reliability gap in autonomous agentic LLMs by engineering a mid-sized model with Iterative Distillation and an Inverse Reasoning (IR) head. The system combines a modified decoder architecture, split embedding spaces, and Landmark Attention to sustain long-horizon planning, while a Distillation and Amplification pipeline transfers capabilities from larger teachers. The IR mechanism, reinforced by Stage 2 Reflective Distillation, Stage 3 DPO for safety, and CodePPO-style RL for function calling, yields strong tool-use reliability with reduced catastrophic error propagation. Across standard reasoning benchmarks and internal tool-use suites, SAGE-32B surpasses open-weight baselines and delivers near GPT-4-class reliability at substantially lower cost, albeit with trade-offs in general-domain knowledge and latency in ultra-long agent loops. The work offers a practical pathway for deploying reliable autonomous systems, with transparent ablations, explicit safety considerations, and a candid discussion of limitations and future directions, including on-device scaling to smaller models and improved API documentation.

Abstract

We demonstrate SAGE-32B, a 32 billion parameter language model that focuses on agentic reasoning and long range planning tasks. Unlike chat models that aim for general conversation fluency, SAGE-32B is designed to operate in an agentic loop, emphasizing task decomposition, tool usage, and error recovery. The model is initialized from the Qwen2.5-32B pretrained model and fine tuned using Iterative Distillation, a two stage training process that improves reasoning performance through rigorously tested feedback loops. SAGE-32B also introduces an inverse reasoning approach, which uses a meta cognition head to forecast potential failures in the planning process before execution. On agentic reasoning benchmarks including MMLU-Pro, AgentBench, and MATH-500, SAGE-32B achieves higher success rates in multi tool usage scenarios compared to similarly sized baseline models, while remaining competitive on standard reasoning evaluations. Model weights are publicly released at https://huggingface.co/sagea-ai/sage-reasoning-32b
Paper Structure (67 sections, 1 theorem, 22 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 67 sections, 1 theorem, 22 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Overview
  3. SAGE-32B: The Agentic Specialist
  4. SAGE-32B Architecture Overview
  5. Gated Linear Units (SwiGLU)
  6. Embedding Space Alignments
  7. Landmark Attention for Long Horizons
  8. Model Lineage & Training
  9. Distillation and Amplification Strategy
  10. Stage 2: Reflective Distillation
  11. Stage 3: DPO for Safety
  12. Reinforcement Learning for Function Calling
  13. Agentic Design
  14. Task Decomposition
  15. Inverse Reasoning Architecture
  16. ...and 52 more sections

Key Result

Theorem 1

Let $I(X; Z)$ be the mutual information between the problem statement and the reasoning plan. For any task requiring $\mathcal{K}$ bits of logical depth, an agent without Inverse Reasoning is bounded by: The Inverse Reasoning mechanism explicitly maximizes $I(X;Z)$ by penalizing plans that "forget" constraints, thereby lowering the error floor exponentially with respect to plan length.

Figures (3)

  • Figure 1: SAGE-32B Performance Overview. Outperforming comparable open-weights models (Qwen, Llama) on key reasoning tasks (MATH, MMLU-Pro) while maintaining competitive general capabilities. Note the specific lift in MATH-500 due to the Inverse Reasoning mechanism.
  • Figure 2: Landmark Attention Mechanism allowing 128k context for SAGE-32B
  • Figure 3: Latency-accuracy Pareto frontier. The hybrid system scales performance non-linearly with compute, whereas the baseline flatlines.

Theorems & Definitions (2)

  • proof
  • Theorem 1