LLaVA-OneVision-1.5-Instruct-Data

LLaVA-OneVision-1.5 Instruction Data Paper | Code 📌 Introduction This dataset, **LLaVA-OneVision-1.5-Instruct**, was collected and integrated during the development of LLaVA-OneVision-1.5. LLaVA-OneVision-1.5 is a novel family of Large Multimodal Models (LMMs) that achieve state-of-the-art performance with significantly reduced computational and financial costs. This meticulously curated 22M instruction dataset (LLaVA-OneVision-1.5-Instruct) is part of a comprehensive and fully open framework for building high-quality vision-language models entirely from scratch. It has significantly enhanced the performance of Vision-Language Models (VLMs) in structured information processing and knowledge-based question answering tasks. As part of the LLaVA-OneVision-1.5 open-source initiative, we are releasing this dataset to the community in the hope of advancing VLM research and driving further progress in the field. ⚙️ Usage Notes Although the dataset itself is of high quality, we recommend deduplicating and combining it with the FineVision dataset to achieve better training results. 🚀 Sample Usage Below is a quick start guide demonstrating how to use the LLaVA-OneVision-1.5 models with Hugging Face `transformers` for inference. This snippet is directly from the project's GitHub repository. 📊 Data Analysis Distribution of Data Categories Compare and Scaling with FineVision Performance comparison of three datasets (Merge46M, FineVision, and LLaVA-OneVision-1.5-Inst-Data) across 16 benchmarks during the SFT phase, demonstrating the superiority of Merge46M on most benchmarks. 🙏 Acknowledgement We would like to acknowledge the contributions of **FineVision** , whose open dataset served as an important foundation and benchmark for building this SFT dataset. 📜 Cite If you find *LLaVA-OneVision-1.5* useful in your research, please consider to cite the following related papers: