Cola-DLM

Cola DLM English · 中文 Cola DLM** (`Co`ntinuous `La`tent `D`iffusion `L`anguage `M`odel) is a hierarchical continuous latent-space diffusion language model. It combines a Text VAE with a block-causal Diffusion Transformer (DiT) prior: the VAE maps text into continuous latent sequences and decodes latents back to tokens, while the DiT performs latent prior transport through Flow Matching. This model repository contains the HuggingFace-format checkpoint for the paper **Continuous Latent Diffusion Language Model**. Links **Model repository:** **GitHub repository:** **Paper:** **HuggingFace Daily Paper:** **Project page:** **Blog post:** **Zhihu article:** Model Files The expected repository layout is: The checkpoint consists of two cooperating modules: `ColaDiTModel`: a block-causal 1-D Diffusion Transformer prior over continuous text latents. `ColaTextVAEModel`: a Text VAE encoder and conditional decoder for text-to-latent and latent-to-text mapping. Quickstart Install the Cola DLM code package from the GitHub repository, then install the download helper: Download the model files: Run a minimal Python example: OpenAI-Compatible Serving The companion `openai_adapter/` service in the Cola DLM code release exposes this model through an OpenAI-compatible Chat Completions endpoint: Install the adapter dependencies from the code repository root: Start the service: Then send a request: The adapter currently supports non-streaming completions. Model Details **Architecture:** Text VAE + block-causal DiT latent prior. **Training objective:** two-stage training with Text VAE pretraining followed by joint Text VAE + DiT training using Flow Matching. **Training-compute checkpoint:** the released weights correspond to the 2000 EFLOPs checkpoint reported in the paper's RQ4 scaling curve. **Tokenizer:** OLMo 2 tokenizer with a 100,278-entry vocabulary. **Special token ids:** `pad_token_id=100277`, `eos_token_id=100257`, `im_end_token_id=100265`. **Framework:** PyTorch 2.1+ and Hugging