MODULE 2 / LESSON 4 · GROUNDED PERCEPTION
Push-T: turn perception into contact
Drag the blue pusher. Match the T to the outline.
Pose error: 5.0 cm
Three experiments, model and references
- Align. Push the bar near its centre. Reach at least 95% coverage and let the T settle. Reset, then push near an end to compare the rotation.
- Rotate. Use an off-centre contact to turn the T before translating it. Reach 95% coverage without directly moving the T.
- Grounding. Set 8 cm bias and 400 ms delay. Compare the dashed pose with the solid T during a push. Set both to zero and reach 95% physical coverage. This overlay illustrates observation error; it does not control the pusher.
What is measured
Coverage = area(T ∩ goal) / area(goal). The score uses the current physical footprints, not bounding boxes or image similarity. Success requires at least 95% overlap and a settled T.
The T and force-controlled pusher use planar-constrained rigid-body dynamics in PhysX. Mass, contact friction, damping and actuator limits are instructional estimates, not a calibrated robot model. In Grounding, the dashed pose is a biased/delayed copy of solver state: no camera, learned detector or VLM is running. The solid physical T stays visible, and the overlay does not affect control.
Lesson 4 connection: a useful representation must preserve position, orientation, timing and the contact geometry required by the controller.
Task reference: Diffusion Policy / Push-T · Reference implementation (MIT) · Physics and third-party notices