DeepSeek-V2.5

Paper Link 👁️ DeepSeek-V2.5 1. Introduction DeepSeek-V2.5 is an upgraded version that combines DeepSeek-V2-Chat and DeepSeek-Coder-V2-Instruct. The new model integrates the general and coding abilities of the two previous versions. For model details, please visit DeepSeek-V2 page for more information. DeepSeek-V2.5 better aligns with human preferences and has been optimized in various aspects, including writing and instruction following: | Metric | DeepSeek-V2-0628 | DeepSeek-Coder-V2-0724 | DeepSeek-V2.5 | |:-----------------------|:-----------------|:-----------------------|:--------------| | AlpacaEval 2.0 | 46.6 | 44.5 | 50.5 | | ArenaHard | 68.3 | 66.3 | 76.2 | | AlignBench | 7.88 | 7.91 | 8.04 | | MT-Bench | 8.85 | 8.91 | 9.02 | | HumanEval python | 84.5 | 87.2 | 89 | | HumanEval Multi | 73.8 | 74.8 | 73.8 | | LiveCodeBench(01-09) | 36.6 | 39.7 | 41.8 | | Aider | 69.9 | 72.9 | 72.2 | | SWE-verified | N/A | 19 | 16.8 | | DS-FIM-Eval | N/A | 73.2 | 78.3 | | DS-Arena-Code | N/A | 49.5 | 63.1 | 2. How to run locally To utilize DeepSeek-V2.5 in BF16 format for inference, 80GB*8 GPUs are required.** Inference with Huggingface's Transformers You can directly employ Huggingface's Transformers for model inference. The complete chat template can be found within `tokenizer_config.json` located in the huggingface model repository. Note: The chat template has been updated compared to the previous DeepSeek-V2-Chat version.** An example of chat template is as belows: You can also add an optional system message: Inference with vLLM (recommended) To utilize vLLM for model inference, please merge this Pull Request into your vLLM codebase: https://github.com/vllm-project/vllm/pull/4650. Function calling Function calling allows the model to call external tools to enhance its capabilities. Here is an example: json { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "s