PLM-Video-Human

Dataset Card for PLM-Video Human PLM-Video-Human is a collection of human-annotated resources for training Vision Language Models, focused on detailed video understanding. Training tasks include: fine-grained open-ended question answering (FGQA), Region-based Video Captioning (RCap), Region-based Dense Video Captioning (RDCap) and Region-based Temporal Localization (RTLoc). [\[📃 Tech Report\]](https://arxiv.org/abs/2504.13180) [\[📂 Github\]](https://github.com/facebookresearch/perception_models/) Dataset Structure Fine-Grained Question Answering (FGQA) A video question answering dataset for fine-grained activity understanding. Contains human-annotated/verified answers to model-generated questions about video clips from open-access video datasets. The questions focus on "what" activities humans perform and "how" they perform these activities. Data fields are: `qa_id`: a `string` feature, unique identifier for the Q&A sample. `segment_id`: a `string` feature, unique identifier for the video segment. `question`: a `string` feature, a model-generated question about the video segment `answer`: a `string` feature, human-annotated or human-verified answer to the question `metadata`: a `dict` of features, representing metadata about the video segment and Q&A pair: `source_video_id`: a `string` feature, video id of untrimmed source video `source_dataset`: a `string` feature, name of the source dataset `source_start_time`: a `float` feature, denoting the start time (seconds) of the video segment in the source video `source_end_time`: a `float` feature, denoting the end time (seconds) of the video segment in the source video `what_description`: a `string` feature, potential activity name shown in video (not verified) `q_type`: a `string` feature, question type `q_subtype`: a `string` feature, question subtype `domain`: a `string` feature, video domain `is_audited`: a `bool` feature, whether the sample has passed a quality audit. A question-answer sample from FGQA looks as foll