Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

You Only Need Your Transformer 25% of the Time: Meaning-First Execution for Eliminating Unnecessary Inference

Ryan Shamim

TL;DR

MFEE introduces Meaning-First Execution, a control-plane gate that decides before inference whether transformer execution is necessary, thereby reducing unnecessary computations while preserving exact outputs when rendering is chosen. The approach achieves $78.1\%$ execution reduction with $100\%$ exact-match equivalence under deterministic decoding on a 1,000-prompt replay, and scales to a 9.24B-parameter model (Gemma 2 9B) with constant gate cost. A formal Theorem (Theorem 1) shows pattern-based routers cannot simultaneously guarantee zero false skips and positive avoidance on feature-collision pairs, motivating semantic gates as essential for correct avoidance. MFEE is designed to be deployable as an upstream, black-box control layer that composes with existing optimizations and safety mechanisms, offering substantial infrastructure savings as models grow in size and capability.

Abstract

Modern AI inference systems treat transformer execution as mandatory, conflating model capability with execution necessity. We reframe inference as a control-plane decision problem: determining when execution is necessary versus when correctness can be preserved through alternative pathways. We introduce Meaning-First Execution (MFEE), a control-plane architecture implementing this framework, selectively invoking transformer inference only when required. MFEE operates as a gating layer above existing stacks without modifying models, weights, or parameters. Across 1,000 diverse prompts under deterministic decoding, MFEE achieves 78.1% execution reduction while maintaining 100% exact-match equivalence for invoked executions. Comparative evaluation reveals pattern-based routers achieve at most 53.3% avoidance with correctness failures, while MFEE reaches 100% avoidance with zero failures through semantic analysis. We prove this limitation via Theorem 1: any router operating solely on finite feature maps cannot simultaneously guarantee zero false skips and positive avoidance on feature-collision pairs. These results establish execution governance as a foundational layer in ML systems infrastructure, orthogonal to model-level optimization techniques.

You Only Need Your Transformer 25% of the Time: Meaning-First Execution for Eliminating Unnecessary Inference

TL;DR

MFEE introduces Meaning-First Execution, a control-plane gate that decides before inference whether transformer execution is necessary, thereby reducing unnecessary computations while preserving exact outputs when rendering is chosen. The approach achieves execution reduction with exact-match equivalence under deterministic decoding on a 1,000-prompt replay, and scales to a 9.24B-parameter model (Gemma 2 9B) with constant gate cost. A formal Theorem (Theorem 1) shows pattern-based routers cannot simultaneously guarantee zero false skips and positive avoidance on feature-collision pairs, motivating semantic gates as essential for correct avoidance. MFEE is designed to be deployable as an upstream, black-box control layer that composes with existing optimizations and safety mechanisms, offering substantial infrastructure savings as models grow in size and capability.

Abstract

Modern AI inference systems treat transformer execution as mandatory, conflating model capability with execution necessity. We reframe inference as a control-plane decision problem: determining when execution is necessary versus when correctness can be preserved through alternative pathways. We introduce Meaning-First Execution (MFEE), a control-plane architecture implementing this framework, selectively invoking transformer inference only when required. MFEE operates as a gating layer above existing stacks without modifying models, weights, or parameters. Across 1,000 diverse prompts under deterministic decoding, MFEE achieves 78.1% execution reduction while maintaining 100% exact-match equivalence for invoked executions. Comparative evaluation reveals pattern-based routers achieve at most 53.3% avoidance with correctness failures, while MFEE reaches 100% avoidance with zero failures through semantic analysis. We prove this limitation via Theorem 1: any router operating solely on finite feature maps cannot simultaneously guarantee zero false skips and positive avoidance on feature-collision pairs. These results establish execution governance as a foundational layer in ML systems infrastructure, orthogonal to model-level optimization techniques.
Paper Structure (93 sections, 11 equations, 1 figure, 6 tables)

This paper contains 93 sections, 11 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. The Unconditional Execution Assumption
  3. Inference as Execution Selection
  4. Contributions
  5. Related Work
  6. Model Compression and Efficiency
  7. LLM Serving Systems and Schedulers
  8. Speculative Execution and Verification
  9. Early Exit and Adaptive Computation
  10. Model Selection and Routing
  11. Caching and Memoization
  12. Mixture-of-Experts and Token Routing
  13. Safety Filters and Content Moderation
  14. Control Planes in Infrastructure Systems
  15. Positioning Summary
  16. ...and 78 more sections

Figures (1)

  • Figure 1: Meaning-First Execution (MFEE) control-plane architecture. Incoming requests are routed through a lightweight semantic gate that conditionally invokes the transformer only when necessary, preserving output equivalence while reducing execution cost.