Qwen2.5-Coder-32B-Instruct-AWQ

Qwen2.5-Coder-32B-Instruct-AWQ Introduction Qwen2.5-Coder is the latest series of Code-Specific Qwen large language models (formerly known as CodeQwen). As of now, Qwen2.5-Coder has covered six mainstream model sizes, 0.5, 1.5, 3, 7, 14, 32 billion parameters, to meet the needs of different developers. Qwen2.5-Coder brings the following improvements upon CodeQwen1.5: Significantly improvements in **code generation**, **code reasoning** and **code fixing**. Base on the strong Qwen2.5, we scale up the training tokens into 5.5 trillion including source code, text-code grounding, Synthetic data, etc. Qwen2.5-Coder-32B has become the current state-of-the-art open-source codeLLM, with its coding abilities matching those of GPT-4o. A more comprehensive foundation for real-world applications such as **Code Agents**. Not only enhancing coding capabilities but also maintaining its strengths in mathematics and general competencies. **Long-context Support** up to 128K tokens. This repo contains the AWQ-quantized 4-bit instruction-tuned 32B Qwen2.5-Coder model**, which has the following features: Type: Causal Language Models Training Stage: Pretraining & Post-training Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias Number of Parameters: 32.5B Number of Paramaters (Non-Embedding): 31.0B Number of Layers: 64 Number of Attention Heads (GQA): 40 for Q and 8 for KV Context Length: Full 131,072 tokens Please refer to this section for detailed instructions on how to deploy Qwen2.5 for handling long texts. Quantization: AWQ 4-bit For more details, please refer to our blog, GitHub, Documentation, Arxiv. Requirements The code of Qwen2.5-Coder has been in the latest Hugging face `transformers` and we advise you to use the latest version of `transformers`. With `transformers<4.37.0`, you will encounter the following error: Also check out our AWQ documentation for more usage guide. Quickstart Here provides a code snippet with `apply_chat_template` to show you ho