action100m-preview

Action100M: A Large-scale Video Action Dataset Paper** | **GitHub** Action100M is a large-scale dataset constructed from 1.2M Internet instructional videos (14.6 years of duration), yielding ~100 million temporally localized segments with open-vocabulary action supervision and rich captions. It serves as a foundation for scalable research in video understanding and world modeling. Load Action100M Annotations Our data can be loaded from the 🤗 huggingface repo at `facebook/action100m-preview` where we released 10% of the full Action100M for preview. For examples of loading from local parquet files (from cloned repo) and visualization, see the GitHub repo. Each `sample` loaded above contains all annotations for one video, and it has three fields: `video_uid` *(string)*: YouTube video id of the source video. `metadata` *(dict)*: video-level metadata (title / description / ASR transcript, etc.) `nodes` *(list[dict])*: annotations for each segments. Each element in `nodes` is a temporally localized segment in the hierachical Tree-of-Captions, it contains: `start`, `end` *(float)*: segment boundaries in seconds within the full video. `node_id` *(string)*: unique id of this segment node. `parent_id` *(string or null)*: id of the parent segment. The root node (corresponding to the entire video) has `parent_id = null`. `level` *(int)*: depth in the hierarchy. Smaller `level` is coarser (longer segments); larger `level` is finer (shorter segments). `plm_caption` *(string or null)*: a caption generated by PLM-3B for this segment. `plm_action` *(string or null)*: a short action label produced by PLM-3B. `llama3_caption` *(string or null)*: middle frame caption produced by LLama-3.2-Vision-11B for leaf nodes. `gpt` *(dict or null)*: main Action100M annotations, available for segments that is not too short: `gpt["summary"]["brief"]`: one-sentence concise caption of the segment. `gpt["summary"]["detailed"]`: longer, detailed summarization of the video segment. `gpt["action"]["brief