NVIDIA Nemotron 3: Efficient and Open Intelligence

TL;DR

The paper presents Nemotron 3, an open-family of efficient agentic AI models (Nano, Super, Ultra) that combines a hybrid Mamba-Transformer MoE with long-context capability up to $1\,\mathrm{M}$ tokens and multi-environment RL post-training. It introduces LatentMoE to boost accuracy per byte by operating in a reduced latent space and scaling the number of experts and active experts, plus Multi-Token Prediction (MTP) to improve planning and enable speculative decoding. The Super and Ultra models deploy NVFP4 training and LatentMoE, with MTP layers to accelerate long-form generation, while Nano emphasizes cost-efficient inference. The work emphasizes openness, releasing model weights, training software, data, and recipes, and demonstrates tangible gains in throughput, context handling, and multi-task capability across varied agentic tasks in an open framework.

Abstract

We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a Mixture-of-Experts hybrid Mamba-Transformer architecture to provide best-in-class throughput and context lengths of up to 1M tokens. Super and Ultra models are trained with NVFP4 and incorporate LatentMoE, a novel approach that improves model quality. The two larger models also include MTP layers for faster text generation. All Nemotron 3 models are post-trained using multi-environment reinforcement learning enabling reasoning, multi-step tool use, and support granular reasoning budget control. Nano, the smallest model, outperforms comparable models in accuracy while remaining extremely cost-efficient for inference. Super is optimized for collaborative agents and high-volume workloads such as IT ticket automation. Ultra, the largest model, provides state-of-the-art accuracy and reasoning performance. Nano is released together with its technical report and this white paper, while Super and Ultra will follow in the coming months. We will openly release the model weights, pre- and post-training software, recipes, and all data for which we hold redistribution rights.

Figures (8)

• Figure 1: Nemotron 3 models (e.g., Nemotron Nano 3) leverage a hybrid Mamba-Transformer MoE architecture consisting predominantly of interleaved Mamba-2 and MoE layers, with a select few self attention layers.
• Figure 2: The hybrid Mamba-Transformer MoE architecture used by Nemotron 3 models can achieve state-of-the-art accuracy on leading reasoning benchmarks and ultra-long-context tasks while providing throughput improvements over similarly sized Transformer MoEs. For details, please see the Nemotron Nano 3 technical report.
• Figure 3: Standard MoE vs. LatentMoE architectures. In LatentMoE, tokens are projected from the model hidden dimension $d$ into a smaller latent dimension $\ell$ for expert routing and computation, which reduces routed parameter loads and all‑to‑all traffic by a factor of $d / \ell$ (typically about $4\times$). We use this efficiency to increase both the total number of experts and the top-$K$ active experts per token by the same factor $d/\ell$, which improves accuracy per byte while keeping overall inference cost approximately constant.
• Figure 4: Relative difference in train loss (left) and validation loss (right) between models trained with NVFP4 and BF16, shown at two model scales: Nemotron 3 Nano (A3B) and the larger MoE model (A8B). Loss gaps decrease as model size increases (A3B $\to$ A8B). Recipe ablation on Nemotron 3 Nano started from Nemotron 3 NVFP4 checkpoint at 500B tokens, then quantizes sensitive layers (Mamba Output, QKV, and Attention projections) to NVFP4, highlighting the importance of keeping these layers in high precision.
• Figure 5: Downstream task evaluations on 8B active MoE model, trained to 1T tokens. NVFP4 accuracy closely follows BF16 trajectories throughout training. Evaluations are performed in BF16.