Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering

TL;DR

The paper tackles diversity collapse in language-model generation by introducing STARS, a training-free, inference-time activation-steering method that diversifies multiple concurrent generation paths through orthogonal directions on the scaled Stiefel manifold. It formalizes a volume-based objective ell(V) = -\log \det((H+V)^T(H+V)) with the constraint V^T V = \alpha I, enabling a Riemannian gradient-descent solution with a convergence guarantee and a fast one-step update for real-time usage. Empirical results on TESTEVAL and LiveIdeaBench demonstrate that STARS yields higher diversity (coverage, originality) while preserving correctness and fluency, outperforming temperature sampling and nucleus sampling, with only modest runtime overhead. Overall, STARS provides a practical, training-free approach to improving LM reasoning and discovery tasks by exploring diverse internal trajectories at inference time.

Abstract

Language models often default to a narrow set of high-probability outputs, leaving their generation paths homogeneous and prone to mode collapse. Sampling-based strategies inject randomness but still struggle to guarantee diversity across multiple concurrent generation runs. We address this limitation by introducing STARS ($\textbf{St}$iefel-based $\textbf{A}$ctivation Steering for Diverse $\textbf{R}$ea$\textbf{S}$oning), a training-free, inference-time intervention method that transforms activation steering into an exploration engine. At each token, STARS collects the hidden activations of concurrent generation runs and optimizes multiple additive steering directions jointly on the Stiefel manifold. STARS maximizes the geometric volume of the steered activations, while the Stiefel manifold induces orthogonality of the steering interventions. This formulation explicitly promotes divergent activation vectors of concurrent generation runs, and implicitly promotes divergent generation trajectories. This manifold optimization formulation can be solved using a Riemannian gradient descent algorithm with convergence guarantees, but this algorithm is too time-consuming for real-time inference. To guarantee low latency, we further design a lightweight one-step update with an aggressive, closed-form stepsize. For test case generation and scientific discovery benchmarks, STARS consistently outperforms standard sampling methods, achieving greater diversity without sacrificing qualitative performance.

Figures (2)

• Figure 1: Schematic overview of our STARS framework. Given an input question, the model generates $N$ candidate sequences in parallel. At each decoding step, hidden states from a designated layer are extracted and used to recompute the steering vectors, which then guide the generation of the next token. This iterative process continues until the end-of-sequence (EOS) token is produced. Paths that emit EOS are dropped from the active set. Once the terminal condition is satisfied, the procedure returns $N$ completed responses.
• Figure 2: Optimality gap comparison of Algorithm \ref{['alg:RGD_line_search']} and Algorithm \ref{['alg:one-step']}. The y-axis is the average relative gap (in percentage) across runs, and the x-axis denotes the iteration number on a logarithmic scale. The standard deviation is very small relative to the average relative gap and is therefore not visually apparent.

Theorems & Definitions (14)

• Proposition 1: Existence of full-rank solution
• Theorem 1: Convergence Rate of Algorithm \ref{['alg:RGD_line_search']}
• Proposition 2: Quadratic approximation of $\mathcal{L}(\eta)$
• Proposition 3: Descent direction
• Proposition 4: Positivesness of $\eta^\star$
• proof : Proof of Proposition \ref{['prop:full_rank']}
• Lemma 1: Second-order boundness of polar retraction
• proof : Proof of Lemma \ref{['lem:second order retraction']}
• Proposition 5: Sufficient descent
• proof : Proof of Proposition \ref{['prop:Sufficient descent']}