Reproduce “You're Pushing My Buttons: Instrumented Learning of Gentle Button Presses” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Raman Talwar, Remko Proesmans, Thomas Lips, Andreas Verleysen, Francis wyffels
Researchers at Ghent University—imec developed an approach leveraging temporary instrumentation during training to improve robotic manipulation. Their method, using a fingertip microphone fine-tuned with instrumented button-state data, reduced the median peak contact force by up to 16% on a Unitree G1 humanoid during button pressing, bringing interaction forces closer to expert demonstrations while maintaining task success.
Key insights
- Temporary training-time instrumentation can effectively guide the learning of deployable representations, such as a fine-tuned audio encoder, without creating dependencies on privileged sensors during inference.
- Audio signals, especially when enhanced by instrumentation-guided fine-tuning, offer critical cues for discerning contact quality that are challenging to obtain solely from vision or kinematics.
- Directly fusing the raw scores (logits) or intermediate feature embeddings from an instrumentation-fine-tuned audio encoder into the policy proves more effective for reducing excessive force than using it as a "soft sensor" or a generic audio encoder.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.