Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning

Ibne Farabi Shihab, Sanjeda Akter, Anuj Sharma

TL;DR

The paper addresses enforcing hard constraints during structured NLP inference by reframing parsing and constrained decoding as constraint-propagation problems and solving them with a meta-reinforcement-learning scheduler, MetaJuLS. By representing the inference state as a constraint–variable graph and using a Graph Attention Network trained with MAML, the approach achieves 1.5–2.0× speedups over GPU baselines with negligible accuracy loss (≤0.2%), while enabling rapid cross-language and cross-task adaptation in as little as 5–15 seconds for 5–10 gradient steps. The framework also introduces a safety-aware fallback to preserve accuracy in difficult cases and demonstrates bidirectional transfer to non-linguistic CP benchmarks, indicating broad generalization and potential for green AI via reduced inference work. Together, these results show that learned, state-aware propagation strategies can outperform handcrafted heuristics and enable scalable, robust structured inference across diverse NLP tasks and beyond.

Abstract

Large language models increasingly require structured inference, from JSON schema enforcement to multi-lingual parsing, where outputs must satisfy complex constraints. We introduce MetaJuLS, a meta-reinforcement learning approach that learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining. By formulating structured inference as adaptive constraint propagation and training a Graph Attention Network with meta-learning, MetaJuLS achieves 1.5--2.0$\times$ speedups over GPU-optimized baselines while maintaining within 0.2\% accuracy of state-of-the-art parsers. On Universal Dependencies across 10 languages and LLM-constrained generation (LogicBench, GSM8K-Constrained), MetaJuLS demonstrates rapid cross-domain adaptation: a policy trained on English parsing adapts to new languages and tasks with 5--10 gradient steps (5--15 seconds) rather than requiring hours of task-specific training. Mechanistic analysis reveals the policy discovers human-like parsing strategies (easy-first) and novel non-intuitive heuristics. By reducing propagation steps in LLM deployments, MetaJuLS contributes to Green AI by directly reducing inference carbon footprint.

Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning

TL;DR

The paper addresses enforcing hard constraints during structured NLP inference by reframing parsing and constrained decoding as constraint-propagation problems and solving them with a meta-reinforcement-learning scheduler, MetaJuLS. By representing the inference state as a constraint–variable graph and using a Graph Attention Network trained with MAML, the approach achieves 1.5–2.0× speedups over GPU baselines with negligible accuracy loss (≤0.2%), while enabling rapid cross-language and cross-task adaptation in as little as 5–15 seconds for 5–10 gradient steps. The framework also introduces a safety-aware fallback to preserve accuracy in difficult cases and demonstrates bidirectional transfer to non-linguistic CP benchmarks, indicating broad generalization and potential for green AI via reduced inference work. Together, these results show that learned, state-aware propagation strategies can outperform handcrafted heuristics and enable scalable, robust structured inference across diverse NLP tasks and beyond.

Abstract

Large language models increasingly require structured inference, from JSON schema enforcement to multi-lingual parsing, where outputs must satisfy complex constraints. We introduce MetaJuLS, a meta-reinforcement learning approach that learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining. By formulating structured inference as adaptive constraint propagation and training a Graph Attention Network with meta-learning, MetaJuLS achieves 1.5--2.0 speedups over GPU-optimized baselines while maintaining within 0.2\% accuracy of state-of-the-art parsers. On Universal Dependencies across 10 languages and LLM-constrained generation (LogicBench, GSM8K-Constrained), MetaJuLS demonstrates rapid cross-domain adaptation: a policy trained on English parsing adapts to new languages and tasks with 5--10 gradient steps (5--15 seconds) rather than requiring hours of task-specific training. Mechanistic analysis reveals the policy discovers human-like parsing strategies (easy-first) and novel non-intuitive heuristics. By reducing propagation steps in LLM deployments, MetaJuLS contributes to Green AI by directly reducing inference carbon footprint.
Paper Structure (35 sections, 7 equations, 4 figures, 11 tables)

This paper contains 35 sections, 7 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Background and NLP Problem Formulation
  3. Constituency Parsing as Constraint Propagation
  4. Why RL Is Necessary for Language Inference
  5. Related Work
  6. Method: RL for Adaptive Propagation in NLP Inference
  7. Problem Formulation
  8. Architecture: Graph Attention Network
  9. Meta-Learning for Universal Generalization
  10. Training Procedure
  11. NLP Experiments
  12. Constituency Parsing: Penn Treebank
  13. Dependency Parsing: Universal Dependencies
  14. LLM Constrained Decoding
  15. Safety-Aware Fallback Mechanism
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Constraint--variable graph for parsing "The manager of the company that fired the workers resigned." Rectangles denote span variables, circles denote grammar-rule constraints, and edges indicate their dependencies. The long-range subject--verb agreement between "manager" and "resigned" illustrates complex cross-span interactions.
  • Figure 2: Effect of propagation scheduling on NLP inference. Left: runtime vs. sentence length; right: accuracy vs. time budget. MetaJuLS outperforms static schedulers in both accuracy and runtime.
  • Figure 3: Rapid adaptation across NLP tasks. Left: parsing policy trained on short sentences adapted to longer sentences. Right: policy trained on parsing adapted to constrained decoding. In both cases, MetaJuLS retains most of its improvement over baselines, indicating that the learned scheduling principles transfer within language.
  • Figure 4: Evolution of GAT attention weights during solving of a constraint-based optimization instance. The policy initially focuses on structurally central constraints (red) and gradually shifts to more local constraints (blue) as the solution stabilizes.