FilmEval

FilmEval FilmEval is a benchmark dataset for evaluating **novel-to-film generation**. It contains source novels grouped by difficulty and generated film/video outputs from multiple inference systems. The dataset is designed for comparing how well different models transform a written story into a complete short film. Dataset Summary FilmEval contains: 15 source novels 3 difficulty levels: `easy`, `medium`, and `hard` 5 source novels per difficulty level 6 model inference result folders 90 generated videos in total, counted from the official `.mp4` files under difficulty folders Chinese and English source texts The current dataset size is approximately **22 GB**. Dataset Structure Data Instances Each data instance is identified by a difficulty level and a novel title. For a source novel: the corresponding generated videos are: Example: Data Fields The dataset is file-based rather than stored as a single JSON/CSV table. Source Novels | Field | Description | |---|---| | `difficulty` | Difficulty split: `easy`, `medium`, or `hard` | | `title` | Novel title, also used as the filename stem | | `text` | Source novel content stored in `.txt` format | Inference Videos | Field | Description | |---|---| | `model` | Inference system name | | `difficulty` | Difficulty split inherited from the source novel | | `title` | Title matched to the source novel filename stem | | `video` | Generated video stored as `.mp4` | Data Splits | Split | Number of Novels | Videos per Model | |---|---:|---:| | `easy` | 5 | 5 | | `medium` | 5 | 5 | | `hard` | 5 | 5 | | **Total** | **15** | **15** | The dataset includes six inference result folders: | Model Folder | Number of Videos | Project Link | |---|---:|---| | `FilmWorld` | 15 | Project | | `MM-StoryAgent` | 15 | GitHub | | `MovieAgent` | 15 | GitHub | | `VideoClaw` | 15 | GitHub | | `VideoGen-of-Thought` | 15 | GitHub | | `ViMax` | 15 | GitHub | | **Total** | **90** | - | Intended Uses FilmEval can be used for: Evaluating novel-to-film generati