inkling-GGUF

Read our How to Run Inkling Guide! See Unsloth Dynamic 2.0 GGUFs for our quantization benchmarks. You can now run Inkling in Unsloth Studio with toggles for Thinking. Read our Inkling guide for analysis and instructions. See below for example of 1-bit UD-IQ1_S GGUF running in Unsloth: Inkling BF16 | NVFP4 | Tinker Cookbook | Documentation | Acceptable Use 1. General Information Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers. Languages:** English, with general multilingual capabilities across other languages. 2. Getting Started For accessing Inkling via Tinker: You can get started by referring to the Tinker Cookbook here and associated documentation here. Inkling supports local deployment using the following open-source libraries: SGLang (docs, PR) vLLM (docs, PR) TokenSpeed (docs, PR) Unsloth (docs, PR) Huggingface (docs, PR) API access is also available through third party inference providers. 3. Model Properties Model type Multimodal autoregressive transformer Architecture type A 66-layer decoder-only transformer with a sparse Mixture-of-Experts (MoE) feed-forward backbone: each token is routed to 6 of 256 experts, plus 2 shared experts active on every token. Attention is a hybrid of local and global layers. The model is natively multimodal — images and video are encoded via a hierarchical patch encoder, and audio via discrete token encoding — with all modali