Shaping force profiles via generated contact-sound loudness, Franka robot completes four contact tasks zero-shot
Guanhua Ji, Nadia Figueroa and four other authors self-report: combining generated video and audio, they shape force profiles from the loudness of generated contact sounds, closing the loop on a Franka robot; four tasks including whiteboard wiping and peeling succeed zero-shot where a purely kinematic baseline fails.
ImportanceLocalEvidenceE2 unreplicated
Readers can now know that actions extracted from generated video carry only kinematics and lack force information, making contact tasks prone to failure; Guanhua Ji, Nadia Figueroa and four other authors combine generated video with audio, shaping force profiles from the loudness of generated contact sounds and closing the loop on a Franka robot, filling that gap.
Previously, actions extracted from generated video carried only kinematics and no force information, so contact tasks were prone to failure.
The authors self-report that four tasks, including whiteboard wiping and peeling, succeed zero-shot while a purely kinematic baseline fails; the method can also serve as a data engine for training policies.
Boundary: results are self-reported and not yet independently reproduced; the preprint was submitted to arXiv on September 16 (v2 updated September 17, arXiv:2609.19137), with a project page at dreamingcontactsound.github.io.