SmolLM3-3B-ONNX

SmolLM3 Table of Contents 1. Model Summary 2. How to use 3. Evaluation 4. Training 5. Limitations 6. License Model Summary SmolLM3 is a 3B parameter language model designed to push the boundaries of small models. It supports 6 languages, advanced reasoning and long context. SmolLM3 is a fully open model that offers strong performance at the 3B–4B scale. The model is a decoder-only transformer using GQA and NoPE (with 3:1 ratio), it was pretrained on 11.2T tokens with a staged curriculum of web, code, math and reasoning data. Post-training included midtraining on 140B reasoning tokens followed by supervised fine-tuning and alignment via Anchored Preference Optimization (APO). Key features Instruct model optimized for **hybrid reasoning** **Fully open model**: open weights + full training details including public data mixture and training configs **Long context:** Trained on 64k context and suppots up to **128k tokens** using YARN extrapolation **Multilingual**: 6 natively supported (English, French, Spanish, German, Italian, and Portuguese) For more details refer to our blog post: https://hf.co/blog/smollm3 How to use Transformers.js ONNXRuntime Evaluation In this section, we report the evaluation results of SmolLM3 model. All evaluations are zero-shot unless stated otherwise, and we use lighteval to run them. We highlight the best score in bold and underline the second-best score. Instruction Model No Extended Thinking Evaluation results of non reasoning models and reasoning models in no thinking mode. We highlight the best and second-best scores in bold. | Category | Metric | SmoLLM3-3B | Qwen2.5-3B | Llama3.1-3B | Qwen3-1.7B | Qwen3-4B | |---------|--------|------------|------------|-------------|------------|----------| | High school math competition | AIME 2025 | 9.3 | 2.9 | 0.3 | 8.0 | **17.1** | | Math problem-solving | GSM-Plus | 72.8 | 74.1 | 59.2 | 68.3 | **82.1** | | Competitive programming | LiveCodeBench v4 | 15.2 | 10.5 | 3.4 | 15.0 | **24.9** | | Gradu