Reproduce “Preferential Bayesian Optimization with Crash Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Johanna Menn, David Stenger, Sebastian Trimpe
The paper presents CrashPBO, an extension to Preferential Bayesian Optimization, that integrates binary crash feedback to safely and efficiently tune parameters for robotic systems using human preferences. The method reduced experimental crashes by up to 63% in synthetic tests and enabled personalized, stable parameter optimization on hardware platforms like quadcopters and robotic pendulums.
Key insights
- Integrating direct binary "crash" feedback via virtual comparisons enables PBO to robustly avoid unsafe regions without requiring explicit, quantified safety constraints.
- The "compare to best" strategy for generating experimental duels proved most effective in CrashPBO, achieving the highest performance and lowest crash rates in synthetic benchmarks.
- CrashPBO allows for the implicit definition of both performance objectives and safety constraints through natural human feedback, broadening PBO's applicability to safety-critical hardware tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.