PhysicalAI-Robotics-Locomanipulation-GRAIL

📢 News **[2026-07-15]** Released task-general tracking policy checkpoints trained on the released data. Follow the tracking doc to use them to track our released motion data. [!WARNING] `data/pickup_table` and `data/pickup_ground` were updated on 2026-07-14. If you downloaded it before then, please re-download it. Dataset Overview | Tabletop Pickup | Ground Pickup | |:---:|:---:| | | | | Tabletop Manipulation | Ground Manipulation | |:---:|:---:| | | | | Sitting | Curb | |:---:|:---:| | | | | Slope | Stairs | |:---:|:---:| | | | Description: This dataset contains physics-validated 4D human-object interaction (HOI) trajectories for the **Unitree G1** humanoid robot. It is generated by **GRAIL** (Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors), an end-to-end pipeline that (1) acquires a 3D asset, (2) generates a synthetic character-object interaction video in Blender + Kling AI, (3) reconstructs the 4D HOI (SMPL-X human pose + object 6-DoF) from the video, (4) retargets the human motion to the G1 skeleton, and (5) drives a SONIC tracking policy in Isaac Lab — the released motion data is what the simulated G1 + object realize in simulation. The release is partitioned by HOI category. Each motion ships with: the source synthetic video, the 4D HOI reconstruction (SMPL-X + object pose), the retargeted G1 robot trajectory, the post-RL object trajectory, and the object's USD asset (textures preserved). The repo also ships the **submodule checkpoints** required to re-run the full GRAIL pipeline end-to-end (GEM-SMPL human pose estimation + FoundationPose object 6-DoF tracking + SONIC task general tracking). **Note: the released SONIC checkpoints are intended to help reproduce our pipeline by fine-tuning the policies from these warm-up checkpoints as documented here. Releasing the final task-general tracking policy checkpoints for the released dataset is on our TODO list.** License/Terms of Use: Use of the released dataset is governed by the Apache Licen