ST-Evidence-Instruct

ST-Evidence-Instruct Dataset This dataset contains spatiotemporal evidence-based video question answering data for training. This dataset was generated using Gemini and should not be used to develop models that compete with Google. This project also uses the Segment Anything Model 3 (SAM 3) distributed by Meta Platforms, Inc. Use of SAM 3 is subject to the SAM License. This was released for research purposes only, in support of the academic paper *Evidence-Backed Video Question Answering*. Dataset Structure Quick Start 1. Download the dataset 2. Extract compressed files Or use Python: 3. Load the data Dataset Details gen_mask (ST Evidence with Generated Masks) **Source**: CLEVRER videos with masks generated by GroundingDINO **Size**: ~20k samples **Features**: Video frames at 6fps Spatial masks (bbox-based, refined by GroundingDINO) Temporal evidence annotations Question-answer pairs Files:** `st_evidence.csv`: Main annotations with questions, answers, and evidence `st_evidence_meta.pkl`: Additional metadata `video_frames_6fps/`: Video frames sampled at 6fps `masks/`: Generated spatial masks gen_qa/vicas (Generated QA from ViCaS) **Source**: ViCaS videos with QA generated by Gemini + Qwen **Size**: 141k samples **Features**: Multiple-choice questions Temporal segment annotations Spatial mask references Answer candidates Format:** `entry_id`: Unique identifier `video_id`: Video identifier `question`: Question text `answer`: Ground truth answer `candidates`: Multiple choice options (list) `mask_evidence`: Spatial mask references (list) `temporal_evidence`: Temporal segments (list of [start, end] times) Data Format Examples gen_mask CSV columns: `video_id`: Video identifier `question`: Question text `answer`: Answer text `mask_evidence`: Spatial evidence (object references) `temporal_evidence`: Temporal segments (start/end times) gen_qa CSV columns: `entry_id`: Unique sample ID `video_id`: Video file name `video_path`: Relative path to video `question`: Question text `