Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement

Saba Kublashvili

TL;DR

An adaptive policy system that modulates perturbation magnitude based on activation energy and token-level entropy is described, which incorporates multi-objective verification with iterative refinement for tasks with ground-truth supervision.

Abstract

I introduce Virtual Parameter Sharpening (VPS), an inference-time technique that augments frozen transformer linear layers with dynamic, activation-conditioned low-rank perturbations. Unlike parameter-efficient fine-tuning methods such as LoRA, which learn static low-rank adapters, VPS constructs its perturbation factors on the fly from batch activation statistics and optional gradient signals, enabling test-time adaptation without persistent parameter updates. The perturbation takes the form Delta W = gamma * W^T V U^T W, where selector matrices U and V are constructed via sparse activation-guided selection or Sylvester-coupled regression. We provide a theoretical analysis of the perturbation's spectral properties and describe an adaptive policy system that modulates perturbation magnitude based on activation energy and token-level entropy. This system incorporates multi-objective verification with iterative refinement for tasks with ground-truth supervision. We present the complete algorithmic framework, analyze its mathematical foundations, and discuss the mechanisms by which activation-conditioned computation may enhance reasoning capabilities in large language models. Implementation and experimental code are available at https://github.com/Saba-Kublashvili/vps-virtual-parameter-synthesis .

Virtual Parameter Sharpening: Dynamic Low-Rank Perturbations for Inference-Time Reasoning Enhancement

TL;DR

An adaptive policy system that modulates perturbation magnitude based on activation energy and token-level entropy is described, which incorporates multi-objective verification with iterative refinement for tasks with ground-truth supervision.

Abstract

I introduce Virtual Parameter Sharpening (VPS), an inference-time technique that augments frozen transformer linear layers with dynamic, activation-conditioned low-rank perturbations. Unlike parameter-efficient fine-tuning methods such as LoRA, which learn static low-rank adapters, VPS constructs its perturbation factors on the fly from batch activation statistics and optional gradient signals, enabling test-time adaptation without persistent parameter updates. The perturbation takes the form Delta W = gamma * W^T V U^T W, where selector matrices U and V are constructed via sparse activation-guided selection or Sylvester-coupled regression. We provide a theoretical analysis of the perturbation's spectral properties and describe an adaptive policy system that modulates perturbation magnitude based on activation energy and token-level entropy. This system incorporates multi-objective verification with iterative refinement for tasks with ground-truth supervision. We present the complete algorithmic framework, analyze its mathematical foundations, and discuss the mechanisms by which activation-conditioned computation may enhance reasoning capabilities in large language models. Implementation and experimental code are available at https://github.com/Saba-Kublashvili/vps-virtual-parameter-synthesis .
Paper Structure (47 sections, 3 theorems, 30 equations, 2 tables, 5 algorithms)

This paper contains 47 sections, 3 theorems, 30 equations, 2 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Parameter-Efficient Fine-Tuning
  4. Test-Time Adaptation
  5. Dynamic and Conditional Computation
  6. Self-Consistency and Verification
  7. Method
  8. Problem Setting
  9. VPS Forward Pass
  10. Selector Construction: Builder Algorithms
  11. SK Builder: Sparse Selector via Activation Magnitude
  12. SC Builder: Sylvester-Coupled Refinement
  13. Hybrid Builder
  14. Spectral Norm Control
  15. Adaptive Policy System
  16. ...and 32 more sections

Key Result

Proposition 4.1

Let $\Delta = AB^\top$ where $A \in \mathbb{R}^{d_{in} \times r}$, $B \in \mathbb{R}^{d_{out} \times r}$, and $\text{Clip}_\tau(A, B) = (\tilde{A}, \tilde{B})$ with the clipping operation from Section sec:spectral. Then:

Theorems & Definitions (8)

  • Remark 3.1: Weight-Dependent Structure
  • Definition 3.2: Per-Component Clipping
  • Definition 3.3: Composite Verification Loss
  • Proposition 4.1: Spectral Norm Bound
  • proof
  • Corollary 4.2
  • Proposition 4.3: Selector Rank
  • proof