Fine-tuning π₀.5 for Pick and Place
Goal: get Pioneer's left arm to pick up a cube and place it in a tray in Isaac Sim, from camera images and a text prompt. The approach: fine-tune the π₀.5 vision-language-action (VLA) model on demonstrations recorded with a leader arm, training only the action expert on top of a frozen vision-language backbone.
The policy after 6,000 weight updates, running in Isaac Sim over several attempts in a row. The two smaller windows are what the policy sees: the left wrist camera and the base camera.