shotplan

ShotPlan Training Dataset Multi-shot video training data for **ShotPlan: Cinematic Video Generation with Learnable Planning Token**. 💻 Code: https://github.com/Pensioner-11/ShotPlan 🤖 Models: ShotPlan-Wan2.1-T2V-14B · ShotPlan-Wan2.2-T2V-A14B-HighNoise Contents | Path | Description | |---|---| | `data/train_meta_16fps.json` | 6,404 training samples (metadata + captions) | | `data/videos/V*_16fps.mp4` | 549 source videos, re-encoded to 16 fps | Each sample is an 80-frame (5 s @ 16 fps) window over temporally contiguous shots of a source video, containing 1–3 hard cuts. Metadata format `data/train_meta_16fps.json` is a JSON list. Each record: | Field | Meaning | |---|---| | `file_path` | Video path relative to `data/` | | `start_frame` / `end_frame` | Window boundaries in source-video frame indices (16 fps) | | `cut_at` | Hard-cut positions in frames **relative to `start_frame`** | | `type` | Transition type of this sample (`hardcut`) | | `text` | Hierarchical caption: a global scene description followed by per-shot captions | Curation pipeline Starting from the open-source VidEvent dataset (movie recap videos with naturally dense shot transitions): 1. Standard preprocessing (subtitle and border removal), re-encoding to 16 fps. 2. Shot boundary detection with TransNet V2, segmenting each video into shots. 3. Sliding-window extraction of 5-second windows over contiguous shots (≥2 shots per window, ≥20 frames per shot). 4. Semantic filtering with Gemini 2.5: only windows depicting a consistent event with stable scene context and subject continuity are kept. 5. Hierarchical captioning with Gemini 2.5: one global description (environment, subjects, lighting) plus per-shot captions (action, camera viewpoint, motion). Intended use Training and evaluation of multi-shot / shot-transition-controllable video generation models. This is the dataset used to train the ShotPlan checkpoints listed above. License and provenance The videos are derived from the **VidEvent** dataset (Lia