UniDoc-Bench

UNIDOC-BENCH Dataset A unified benchmark for document-centric multimodal retrieval-augmented generation (MM-RAG). Table of Contents Dataset Description Key Features Dataset Structure Domain Descriptions QA Dataset Format Field Descriptions Query Types Data Access Download Options Compressed File Sizes Usage Loading the Dataset Working with PDF Files from Hugging Face Building RAG Database from PDFs Working with PDF Documents Working with QA Pairs Visual Question Answering (VQA) Evaluation Paradigms Benchmark Results Data Quality Citation License Important Usage Notice License Terms Paper Contact Acknowledgments Dataset Description UNIDOC-BENCH is the first large-scale, realistic benchmark for multimodal retrieval-augmented generation (MM-RAG) and Visual Question Answering (VQA) built from 70,000 real-world PDF pages across eight domains. The dataset extracts and links evidence from text, tables, and figures, then generates 1,700+ multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries. Each QA pair includes image paths (`mage_paths`) for the entire pdf for direct VQA evaluation, making it suitable for both MM-RAG and VQA research. Key Features **70,000 real-world PDF pages** across 8 diverse domains **1,700+ multimodal QA pairs** with expert validation **Four evaluation paradigms**: text-only, image-only, multimodal text-image fusion, and multimodal joint retrieval **Visual Question Answering (VQA) support**: Image paths (`longdoc_image_paths` and `gt_image_paths`) for direct VQA evaluation **20% expert validation** ensuring high-quality annotations **Comprehensive query types**: factual retrieval, comparison, summarization, and logical reasoning Dataset Structure The dataset is organized as a single split named `test` containing all 1,742 QA pairs. Each QA pair includes a `domain` field that indicates which of the 8 domains it belongs to. To access domain-specific data, filter the test split by the `domain` field. Da