phi-4

Phi-4 Model Card Phi-4 Technical Report Model Summary | | | |-------------------------|-------------------------------------------------------------------------------| | **Developers** | Microsoft Research | | **Description** | `phi-4` is a state-of-the-art open model built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets. The goal of this approach was to ensure that small capable models were trained with data focused on high quality and advanced reasoning. `phi-4` underwent a rigorous enhancement and alignment process, incorporating both supervised fine-tuning and direct preference optimization to ensure precise instruction adherence and robust safety measures | | **Architecture** | 14B parameters, dense decoder-only Transformer model | | **Inputs** | Text, best suited for prompts in the chat format | | **Context length** | 16K tokens | | **GPUs** | 1920 H100-80G | | **Training time** | 21 days | | **Training data** | 9.8T tokens | | **Outputs** | Generated text in response to input | | **Dates** | October 2024 – November 2024 | | **Status** | Static model trained on an offline dataset with cutoff dates of June 2024 and earlier for publicly available data | | **Release date** | December 12, 2024 | | **License** | MIT | Intended Use | | | |-------------------------------|-------------------------------------------------------------------------| | **Primary Use Cases** | Our model is designed to accelerate research on language models, for use as a building block for generative AI powered features. It provides uses for general purpose AI systems and applications (primarily in English) which require: 1. Memory/compute constrained environments. 2. Latency bound scenarios. 3. Reasoning and logic. | | **Out-of-Scope Use Cases** | Our models is not specifically designed or evaluated for all downstream purposes, thus: 1. Developers should consider common limitations of language models as they select