Cosmos3-DROID

DROID: Distributed Robot Interaction Dataset Dataset Summary DROID (Distributed Robot Interaction Dataset) is a large-scale "in-the-wild" robot manipulation dataset containing 76K teleoperated demonstration trajectories — approximately 350 hours of interaction data — collected across 564 unique scenes, 86 tasks, and 52 buildings over the course of 12 months. The data was collected by 50 data collectors at 18 labs across 13 institutions in North America, Asia, and Europe, on a single shared, open-source robot hardware platform. Every DROID episode contains three synchronized stereo RGB camera streams, camera calibration data, depth information, low-level robot state and control commands, and up to three natural-language task instructions. Dataset Details Dataset Description DROID was designed to enable training of generalizable robot manipulation policies by substantially increasing the diversity of scenes, tasks, objects, viewpoints, and interaction locations over prior real-world robot manipulation datasets. Data was collected on a single uniform robot hardware stack (a Franka Panda 7-DoF arm with a Robotiq 2F-85 gripper) so that demonstrations gathered at any of the 18 collection labs are comparable and reproducible. Each data collection session begins with moving the portable robot platform to a new scene. The data collector chooses third-person camera views that capture interesting behaviors in the scene, performs extrinsic camera calibration using a checkerboard and the OpenCV calibration algorithm, then enters all potential tasks for the scene into the data-collection GUI either by selecting from a list of options or by typing free-form task instructions. During data collection the GUI prompts the data collector with a randomly sampled task from this list for each new episode; this ensures that data collection is not biased toward easier tasks or closer objects. The GUI also periodically prompts the data collector to perform randomly sampled "scene augmentatio