MiniCPM-RobotManip

MiniCPM-RobotManip A Smarter and Faster On-Device AI Brain for Robots Github  | Discord  | MiniCPM-RobotManip is a 1.5B vision-language-action model for embodied manipulation with the following highlights: Generalist Manipulation: A unified 1.5B generalist policy that uses one set of weights across all downstream tasks and outperforms larger models such as π₀.₅ and Qwen-VLA across representative evaluations . Streaming Context: Historical observations are continuously incorporated into the model context through streaming inference, reducing per-step compute from 125 TFLOPs to 3.3 TFLOPs while retaining 60 frames of history and supporting up to one minute of visual memory . This moves VLA beyond reactive action generation from single-frame observations toward continuous decision-making grounded in long-horizon visual context. Efficient Inference: Inherits MiniCPM-V 4.6's visual token compression, reducing each frame from 256 to 64 visual tokens for 4× compression. With H100, BF16, and single-frame input, model-forward latency per decision step is 120 ms, compared with 234 ms for π0.5. The measurement excludes task autoregressive decoding. Benchmark Results Inference Example Please Ensure transformers==5.7.0 from __future__ import annotations import argparse import json from pathlib import Path from typing import Sequence import numpy as np import torch from PIL import Image from transformers import AutoModel, AutoProcessor STATE_DIM = 80 IMAGE_SIZE = (448, 448) class MiniCPMVLAInference: """Processor and model wrapper for single-sample VLA inference.""" def __init__( self, checkpoint_path: str | Path = "openbmb/MiniCPM-RobotManip", device: str | torch.device | None = None, ): if device is None: device = "cuda" if torch.cuda.is_available() else "cpu" self.device = torch.device(device) checkpoint = str(checkpoint_path) self.processor = AutoProcessor.from_pretrained(checkpoint, trust_remote_code=True) self.model = AutoModel.from_pretrained(checkpoint, trust_re