ParseBench

ParseBench Quick links:** [\[🌐 Website\]](https://parsebench.ai) [\[📜 Paper\]](https://arxiv.org/abs/2604.08538) [\[💻 Code\]](https://github.com/run-llama/ParseBench) ParseBench** is a benchmark for evaluating document parsing systems on real-world enterprise documents, with the following characteristics: **Multi-dimensional evaluation.** The benchmark is stratified into five capability dimensions — tables, charts, content faithfulness, semantic formatting, and visual grounding — each with task-specific metrics designed to capture what agentic workflows depend on. **Real-world enterprise documents.** The evaluation set contains ~2,000 human-verified pages from over 1,200 publicly available documents spanning insurance, finance, government, and other domains, ranging from straightforward to adversarially hard. **Dense test coverage.** Over 169K test rules across the five dimensions, providing fine-grained diagnostic power over precisely where a parser breaks down. **Human-verified annotations.** All annotations are produced through a two-pass pipeline: frontier VLM auto-labeling followed by targeted human correction. **Evaluation code suite.** The benchmark ships with a full evaluation framework supporting end-to-end pipeline evaluation, per-dimension scoring, and cross-pipeline comparison. The evaluation code can be found at ParseBench. Dataset Introduction ParseBench comprises ~2,000 human-verified, annotated pages drawn from publicly available enterprise documents spanning insurance, finance, government, and other domains. The benchmark is stratified into five capability dimensions, each targeting a failure mode that consistently breaks production agentic workflows: **Tables.** Structural fidelity of merged cells and hierarchical headers. A single shifted header or merged-cell error causes an agent to extract values from the wrong column, silently corrupting financial analysis. **Charts.** Exact data point extraction with correct labels from bar, line, pie, and co