Skip to main content

Fine-tuning π₀.5 for Pick and Place

· 11 min read
Max Yuan

Goal: get Pioneer's left arm to pick up a cube and place it in a tray in Isaac Sim, from camera images and a text prompt. The approach: fine-tune the π₀.5 vision-language-action (VLA) model on demonstrations recorded with a leader arm, training only the action expert on top of a frozen vision-language backbone.

The policy after 6,000 weight updates, running in Isaac Sim over several attempts in a row. The two smaller windows are what the policy sees: the left wrist camera and the base camera.

Critic-Oriented Bimanual Badminton Receive in Simulation

· 14 min read
Sean

Goal: get a stationary bimanual arm to return a badminton serve with either hand, in simulation. The result is one single-arm network that drives both arms, and a small critic model to pick either the left or right arm. On 8192 serves it has never seen, it hits 97.9%, splits the work 49/51 between the arms, and sends 94% of returns over the net.

Compliant Control for the Tendon Hand

· 22 min read
Wilson

Goal: try the impedance controller out in simulation before it touches hardware. Check it actually works, add what it turns out to need (friction feedforward, a torque cap, the wrist angle), and catch problems like the wrist early, so the bench work is just system ID: measuring a few numbers, not debugging the controller.

Each finger holds a target angle and stays soft: it pushes with a force you set and gives way when pushed. It needs only the motor encoders and the wrist angle. All results are from simulation in MuJoCo.

Inverse Kinematics: Arm & Hand

· 5 min read
Wilson

How do you tell a robot arm "put your hand right here," or a robot hand "put your fingertip right here," and have it actually figure out the joint angles to make that happen? That's inverse kinematics, and it's what drives both the bimanual arm setup and the individual hand.