Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference

Jiangyong Yu, Xiaomeng Han, Xing Hu, Chen Xu, Zhe Jiang, Dawei Yang

TL;DR

The paper addresses the heavy cost of nonlinear operations in LLM inference by introducing Non-uniform Linear Interpolation (NLI), a calibration-free framework that replaces nonlinear evaluations with a FP16 LUT generated via a dynamic-programming search. NLI splits into a software component that computes globally optimal cutpoints with complexity $O(MN^2)$ and a hardware engine that implements two-level address translation and pipelined interpolation for high throughput. The key contributions are the calibration-free DP-based LUT design that yields 259 cutpoints across 10 macro-intervals, the NLI Engine hardware that reduces comparators and power while boosting efficiency, and extensive software and hardware evaluations showing negligible accuracy loss and substantial speedups over state-of-the-art nonlinear units. The approach demonstrates strong generality across LLMs and even vision models, offering a practical path to efficient edge deployment of large models on resource-constrained hardware.

Abstract

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and computational costs. While prior work has achieved significant progress in compressing and accelerating linear layers, nonlinear layers-such as SiLU, RMSNorm, and Softmax-still heavily depend on high-precision floating-point operations. In this paper, we propose a calibration-free, dynamic-programming-optimal, and hardware-friendly framework called Non-uniform Linear Interpolation (NLI). NLI is capable of efficiently approximating a variety of nonlinear functions, enabling seamless integration into LLMs and other deep neural networks with almost no loss in accuracy. NLI ingeniously recasts cutpoint selection as a dynamic-programming problem, achieving the globally minimal interpolation error in O(MxN2) time via Bellman's optimality principle. Based on the NLI algorithm, we also design and implement a plug-and-play universal nonlinear computation unit. Hardware experiments demonstrate that the NLI Engine achieves more than 4x improvement in computational efficiency compared to the state-of-the-art designs.

NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference

TL;DR

The paper addresses the heavy cost of nonlinear operations in LLM inference by introducing Non-uniform Linear Interpolation (NLI), a calibration-free framework that replaces nonlinear evaluations with a FP16 LUT generated via a dynamic-programming search. NLI splits into a software component that computes globally optimal cutpoints with complexity and a hardware engine that implements two-level address translation and pipelined interpolation for high throughput. The key contributions are the calibration-free DP-based LUT design that yields 259 cutpoints across 10 macro-intervals, the NLI Engine hardware that reduces comparators and power while boosting efficiency, and extensive software and hardware evaluations showing negligible accuracy loss and substantial speedups over state-of-the-art nonlinear units. The approach demonstrates strong generality across LLMs and even vision models, offering a practical path to efficient edge deployment of large models on resource-constrained hardware.

Abstract

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and computational costs. While prior work has achieved significant progress in compressing and accelerating linear layers, nonlinear layers-such as SiLU, RMSNorm, and Softmax-still heavily depend on high-precision floating-point operations. In this paper, we propose a calibration-free, dynamic-programming-optimal, and hardware-friendly framework called Non-uniform Linear Interpolation (NLI). NLI is capable of efficiently approximating a variety of nonlinear functions, enabling seamless integration into LLMs and other deep neural networks with almost no loss in accuracy. NLI ingeniously recasts cutpoint selection as a dynamic-programming problem, achieving the globally minimal interpolation error in O(MxN2) time via Bellman's optimality principle. Based on the NLI algorithm, we also design and implement a plug-and-play universal nonlinear computation unit. Hardware experiments demonstrate that the NLI Engine achieves more than 4x improvement in computational efficiency compared to the state-of-the-art designs.
Paper Structure (34 sections, 13 equations, 5 figures, 9 tables, 2 algorithms)

This paper contains 34 sections, 13 equations, 5 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Nonlinear Operations of LLMs
  4. General-Purpose Nonlinear Computation Work
  5. Linear LUT Fitting
  6. Interpolation-Based Approximations and a Unified Error Bound
  7. Piecewise polynomial and spline-base variants.
  8. Linear interpolation
  9. Software Methodology
  10. Proposed Non-Uniform Interpolation LUT via Dynamic Programming
  11. Setup.
  12. Rationale.
  13. Computational cost.
  14. Hardware-Friendly Computation Strategy
  15. Hardware Methodology
  16. ...and 19 more sections

Figures (5)

  • Figure 1: (a) Range of SiLU activations in representative LLMs; values can exceed $\pm100$ (e.g., Qwen2.5-32B, Qwen3-8B). (b) Wikitext-2 perplexity (log-scale) with FP32, NN-LUT, and our NLI. NN-LUT collapses when outliers occur---perplexity skyrockets up to $7.0{\times}10^{4}$---whereas NLI matches FP32 across scales.
  • Figure 2: Approximation quality of the SiLU activation over the range $[-150,150]$, which covers $\ge$99.9% of activations under our measurement protocol (see Figure. \ref{['fig:range_and_compare']} (a) and Appendix. \ref{['app:act-measure']}). Panel (a) and Panel (b) show the result of NN-LUT, and panel (c) shows our NLI framework. For each method we plot (top-left) the function curve, (bottom-left) the absolute error on a logarithmic scale, and (right) a zoom-in over [-5,5]. Red dots denote LUT cutpoints. NN-LUT suffers from pronounced error spikes, whereas NLI preserves near-machine-precision fidelity throughout the LLM-typical range, reducing worst-case error by several orders of magnitude.
  • Figure 3: Comparison between the full-precision reference and our NLI approximation on two representative nonlinear functions—exp and rsqrt. We use 10 macro-intervals: the first and last are not subdivided, and each of the middle eight is uniformly partitioned into 32 bins; accounting for shared endpoints, this yields $2 + 32\times 8 + 1 = 259$ cutpoints. The lower panels plot absolute error (log scale); the worst-case error stays below $1.2\times 10^{-3}$ across the FP16 domain. Additional visualizations and cutpoint layouts are provided in Appendix \ref{['appaddnul']}.
  • Figure 4: The hardware circuit design of a nonlinear computing unit, using two computing circuits sharing a set of Multiply scale factor and function value registers as an example.
  • Figure 5: NLI approximation quality for eight representative nonlinear operators: (a) exp, (b) gelu, (c) rsqrt, (d) reciprocal, (e) hardswish, (f) mish, (g) sigmoid, and (h) tanh. For each operator, the left subplot shows the full-domain fit (top: FP32 reference vs. NLI; bottom: absolute error in log scale), while the right subplot zooms into the high-curvature region. Red dots denote LUT cutpoints generated under an $2{+}8\times32{+}1$ budget. NLI closely overlaps the FP32 reference, keeping the worst-case absolute error within $1.5{\times}10^{-3}$ across the FP16 domain.