LiveResearchBenchFull

Dataset Overview LiveResearchBench provides expert-curated, real-world tasks spanning daily life, enterprise, and academia, each requiring extensive, real-time web search, multi-source reasoning, and cross-domain synthesis. DeepEval offers human-aligned protocols for reliable, systematic evaluation of agentic systems on open-ended deep research tasks. 📌 Quick Links Project Page Paper Codebase Dataset Fields Subsets: question_with_checklist: Full dataset with questions and per-question checklists question_only: Questions without checklists For each entry in the dataset: { 'qid': 'market6VWmPyxptfK47civ', # Unique query identifier 'question': 'What is the size, growth rate...', # Research question 'checklists': [ # List of checklist items for coverage evaluation 'Does the report provide data for the U.S. electric vehicle market...', 'Does the report discuss the size, growth rate...', ... more items ] } Loading the Dataset Default: Static Mode (No Placeholders) The default static mode loads questions and checklists with dates already filled in (e.g., 2025 instead of {{current_year}}): from liveresearchbench.common.io_utils import load_liveresearchbench_dataset Load static version benchmark_data = load_liveresearchbench_dataset(use_realtime=False) Example: Question: "What is the size, growth rate, and segmentation of the U.S. electric vehicle market in 2025?" Realtime Mode For dynamic evaluation with current dates, use realtime mode: Load realtime version (replaces {{current_year}} etc.) benchmark_data = load_liveresearchbench_dataset(use_realtime=True) The following placeholders will be replaced by the current date: {{current_year}} → 2025 (current year) {{last_year}} → 2024 (current year - 1) {{current_date}} → October 29, 2025 (formatted date) Example: Question: "What is the size, growth rate, and segmentation of the U.S. electric vehicle market in 2025?" (automatically updated each year) Accessing Questions and Checklists from liveresearchbench.common.io_utils impor