Nemotron-CC-v2

Nemotron-Pre-Training-Dataset-v1 Release Data Overview This pretraining dataset, for generative AI model training, preserves high-value math and code while enriching it with diverse multilingual Q&A, fueling the next generation of intelligent, globally-capable models. This dataset supports NVIDIA Nemotron Nano 2, a family of large language models (LLMs) that consists of the NVIDIA-Nemotron-Nano-9B-v2, NVIDIA-Nemotron-Nano-9B-v2-Base, and NVIDIA-Nemotron-Nano-12B-v2-Base models. They are successors of Nemotron-H-8B-Base-8K and Nemotron-H-8B-Reasoning-128K, created with commercial use in mind. The NVIDIA-Nemotron-Nano-9B-v2 model is aligned for human chat preferences and tasks. All of the NVIDIA Nemotron Nano 2 models support a context length of 128K tokens. Our dataset comes in 4 main categories: nvidia/Nemotron-Pretraining-Dataset-sample This dataset includes a small sampled version for inspection and quick experimentation, with 10 representative subsets drawn from different components of the full SFT and pretraining corpora. These include diverse QA data (original and translated), high-quality and synthetic high-quality Common Crawl extractions, math-focused subsets, code metadata, and SFT-style data across code, math, and general domains, as well as synthetic code. nvidia/Nemotron-CC-Math-v1 133B-token high-quality math pretraining dataset from Common Crawl built with a novel Lynx + LLM pipeline that preserves equations and code, standardizes to LaTeX, and removes noise, beating all previous math pretraining datasets on math and improves on code, and reasoning benchmarks. We also regenerated the Nemotron-MIND dataset using Nemotron-cc-math-4plus, our high-quality subset which yielded consistent gains over previous nemotron-MIND. nvidia/Nemotron-CC-v2 Updated English web crawl dataset based on Nemotron-CC with eight additional Common Crawl snapshots (2024–2025), synthetic rephrasing using Qwen3-30B-A3B, filtered for English and globally deduplicated. Includes synth