Many Minds from One Model: Bayesian Transformers for Population Intelligence

TL;DR

Population Bayesian Transformers (B-Trans) transform a single pre-trained Transformer into a population of diverse yet coherent model instances by treating normalization bias offsets as latent variables drawn from a Gaussian posterior proxy. Temporal consistency is enforced by freezing per-sequence noise, enabling coherent multi-step reasoning while enabling population-level aggregation. The approach yields improved semantic diversity and better task performance in zero-shot generation, RLVR, and label-free Test-Time RL, without retraining the full Bayesian network. The work highlights a practical path to population-level reasoning in large language models by leveraging lightweight parameter-space uncertainty confined to normalization layers.

Abstract

Despite their scale and success, modern transformers are almost universally trained as single-minded systems: optimization produces one deterministic set of parameters, representing a single functional hypothesis about the data. Motivated by the idea that intelligence emerge from many minds, we propose Population Bayesian Transformers (B-Trans), which transform a standard Large Language Model into a Bayesian Transformer model to supports sampling diverse yet coherent model instances from a single set of pre-trained weights. B-Trans introduces a Bayesian-motivated posterior proxy by treating the bias-like offsets in normalization layers as stochastic variables with a Gaussian variational approximation, inducing a distribution over model behavior without the cost of training full Bayesian neural networks. Sampling from this proxy yields a set of model instances with diverse behaviors while maintaining general competence. To preserve coherence within each generation, we freeze the sampled noise at the sequence level, enforcing temporal consistency across tokens. B-Trans allows for population-level decision-making, where aggregating predictions across sampled individuals significantly enhances exploration. Experiments across zero-shot generation, Reinforcement Learning with Verifiable Rewards (RLVR), and RL without explicit labels demonstrate that B-Trans effectively leverage the wisdom of crowds, yielding superior semantic diversity while achieving better task performance compared to deterministic baselines.

Figures (5)

• Figure 1: B-Trans implementation (abstracted). B-Trans operates as a plug-and-play replacement for normalization layers. For each response, it caches a latent offset per sequence and reuse it across autoregressive steps. This approximates drawing a single coherent model instance for each generation pass, enabling diverse yet temporally consistent outputs from one set of weights.
• Figure 2: Pass@$K$ performance on MMLU-Pro across Llama-3.1 and Qwen3 families. Solid lines denote B-Trans, while dashed lines represent deterministic baselines with high-temperature sampling. While performance at $K=1$ is comparable, B-Trans exhibits superior scaling at higher $K$ values. This widening gap indicates that the induced parameter-space diversity yields valid, functional reasoning paths rather than random noise.
• Figure 3: Semantic diversity on INFINITY-CHAT. B-Trans (solid) consistently yields higher embedding distances than baselines (hatched), confirming better output heterogeneity on creativity writing.
• Figure 4: PCA visualization of response embeddings from Qwen3 and Llama-3.1 models. The query is "Output a hard question to humanity (super concise and short), independent of theme." Standard sampling confines models to disjoint semantic regions (their respective "territories"). B-Trans significantly expands this scope, allowing the model to escape its default mode and even explore semantic regions typically associated with the opposing model (highlighted in red box).
• Figure 5: Accuracy curve during training. Reported at every five training steps.