molmo-motion-1m

MolmoMotion-1M MolmoMotion-1M is a dataset of **3D point-trajectory annotations** curated across seven video corpora — ego-centric manipulation, real-world robot teleoperation, dynamic real-world scenes, and simulator renders. Each clip ships motion-filtered 3D tracks (and, for most datasets, 2D pixel tracks), a short action caption, per-frame camera, and a train/test split — all frame-aligned to the source video. Datasets We do not re-host the original videos, and for some datasets do not redistribute the upstream coordinates — we ship the annotations/tracks we produced, plus a `reconstruct_*.py` script to regenerate the rest from the original source. **Check each upstream dataset's license before downloading.** | Dataset | Domain | Upstream source | Ships | Reconstruct | |---|---|---|---|---| | egodex | ego-centric manip | apple/EgoDex | tracks + camera | videos | | ytvis | natural-video object tracks | YouTube-VIS 2021 | tracks + camera | videos | | hdepic | ego-centric kitchen | hd-epic.github.io | tracks + camera | videos | | xperience | ego-centric manip (gated) | ropedia-ai/xperience-10m | tracks | videos + camera | | stereo4d | dynamic VR180 scenes | stereo4d.github.io | track index | tracks + camera + videos | | droid | real-world robot teleop | droid-dataset.github.io | tracks + camera | videos | | molmospaces | sim pick-and-place | Ai2 internal sim | everything | — | Each dataset's `README.md` is **authoritative** for its schema and its exact reconstruction procedure. Download Unpack The per-clip directories (`tracks/`, `camera/`, and molmospaces `videos/` + `robot_trajectories/`) ship as ~10 GB tar shards inside those folders. Extract each dataset's shards from its folder: → each shard extracts into its own `tracks/` / `camera/` / `videos/` (stereo4d has no shards). The resulting layout is the tree below. Data format ` /annotations/` holds the JSONs — the **single source of truth**; don't enumerate the `tracks/` directory directly. ` _clips.json` — one e