Stable-DiffCoder 8B Instruct

Stable-DiffCoder-8B-Instruct Introduction We are thrilled to introduce Stable-DiffCoder, which is a strong code diffusion large language model. Built directly on the Seed-Coder architecture, data, and training pipeline, it introduces a block diffusion continual pretraining (CPT) stage with a tailored warmup and block-wise clipped noise schedule. Under identical architecture and data settings, we systematically analyze and design an efficient diffusion training pipeline that is not only stable but also potentially lifts the model’s performance ceiling. With this recipe, Stable-DiffCoder demonstrates overall performance improvements compared to its autoregressive (AR) counterpart across a broad set of code benchmarks, while any-order modeling improves structured code handling for editing and reasoning, and diffusion-based corruption aids learning for low-resource programming languages. Notably, with only CPT followed by supervised fine-tuning, Stable-DiffCoder further surpasses many strong ∼8B AR and diffusion-based code models. These results demonstrate that diffusion-based training can improve code modeling quality beyond what AR training alone can achieve, even under tightly controlled data and architecture constraints. This repo contains the **Stable-DiffCoder-8B-Instruct** model, which has the following features: Type: Mask Diffusion Language Models Training Stage: Pretraining & Post-training Data Source: Public datasets, synthetic data Context Length: 8192 Model Downloads | Model Name | Length | Download | Notes | |---------------------------------------------------------|--------|------------------------------------|-----------------------| | Stable-DiffCoder-8B-Base | 8K | 🤗 Model | Pretrained on our model-centric code data. | | 👉 **Stable-DiffCoder-8B-Instruct** | 8K | 🤗 Model | Instruction-tuned for alignment with user intent. | Requirements Current (v5.3.0) `transformers` is available for inference: Explanation of Inference Parameters `steps`: Number of ste