MMOU

MMOU Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos Project Page · HuggingFace · Videos (Community Hosted) · Paper · Evaluator MMOU evaluates joint audio-visual understanding and reasoning in long and complex real-world videos. Dataset Summary MMOU is a benchmark for evaluating whether multimodal models can jointly reason over video, speech, sound, music, and long-range temporal context in real-world videos. The central challenge is not single-modality recognition in isolation, but tightly coupled audio-visual understanding over extended time horizons where evidence can be sparse, delayed, or distributed across multiple moments. Current models still fail substantially under this setting. Even the strongest systems remain far below human performance when they must integrate both audio and vision across long videos rather than rely on language priors, a single modality, or short-context pattern matching. Ground-truth answers for the main 15,000-question MMOU split are not included in `MMOU.json`. To evaluate models on the main split, users can upload their predictions here. A labeled 5,000-question MMOU Test Mini split is also available for local development, validation, and reproducible benchmarking. Dataset Statistics | Category | Value | | --- | --- | | QA pairs | 15,000 | | Web-collected videos | 9,038 | | Video resolution | 720p | | Average video duration | 711.6 seconds | | Duration range | 7 to 7,255 seconds | | Major domains | 10 | | Fine-grained subcategories | 36 | | Skill categories | 13 | | Average skills per question | roughly 3 simultaneously | | Evaluated models | 20+ | MMOU spans long-form real-world videos collected from the web and was designed to stress temporal grounding, cross-modal reasoning, and robustness to long context rather than short-clip recognition. MMOU Test Mini MMOU Test Mini is a labeled multiple-choice split for fast iteration and reproducible evaluation. It is available as the `