ThinkingCap-Qwen3.6-27B-GGUF

bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF GGUF / llama.cpp quantizations of bottlecapai/ThinkingCap-Qwen3.6-27B — capability of Qwen3.6-27B with **50% less** thinking tokens on average, achieved by finetuning Qwen3.6-27B (Qwen Team, 2026) while preserving the original answer quality and style. ➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.6-27B. About GGUF and quantization GGUF is a single-file model format for running LLMs locally with llama.cpp and compatible runtimes (Ollama, LM Studio, …). The quantized variants below store weights at reduced precision — e.g. ≈4.7 bits per weight for `Q4_K_M` instead of the 16-bit `f16` source — cutting download size and memory severalfold at a small, measured quality cost. Files | File | Quant | Size | |---|---|---| | `ThinkingCap-Qwen3.6-27B-Q4_K_M.gguf` | Q4_K_M | 15.7 GB | | `ThinkingCap-Qwen3.6-27B-Q6_K.gguf` | Q6_K | 20.9 GB | | `ThinkingCap-Qwen3.6-27B-Q8_0.gguf` | Q8_0 | 27.1 GB | | `ThinkingCap-Qwen3.6-27B-f16.gguf` | f16 | 50.9 GB | | `mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf` | mmproj (vision) | 0.9 GB | `f16` is the unquantized source; `Q8_0` is near-lossless; `Q6_K` is a slightly smaller near-lossless option; `Q4_K_M` is the recommended size/quality balance for most local setups. Usage (llama.cpp) Speculative decoding (MTP) llama.cpp can run MTP (multi-token-prediction) self-speculative decoding on these GGUFs for a decode speed-up — no separate draft model needed. Add `--spec-type draft-mtp` when serving: Set the draft length with `--spec-draft-n-max` (e.g. `4`). Requires a recent llama.cpp build with MTP support. Vision (image input) ThinkingCap is a **vision-language model**. Image input needs the multimodal projector `mmproj-ThinkingCap-Qwen3.6-27B-f16.gguf` (in this repo) loaded alongside a text GGUF — the single `f16` mmproj pairs with any of the quants above. **LM Studio / Jan