Reproduce “PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing” on OpenResearch

About this paper

Shengbin Guo, Shaokang He, Chaoyue Meng, Shengpeng Xiao, Xunzhi Xiang, Shaofeng Zhang, Qi Fan

This research introduces PhyEditBench, a real-world, multi-stage benchmark designed to evaluate the physics-based reasoning capabilities of image editing models, alongside PhyWorld, a training-free baseline leveraging video generation for enhanced physical plausibility. The work demonstrates that existing models frequently produce physically implausible edits and that a temporal generation approach can improve physical coherence, with PhyWorld achieving competitive performance using a 5B parameter architecture.

Key insights

• Current state-of-the-art image editing models, both open-source and proprietary, exhibit significant limitations in physics-based reasoning, especially in complex deformation, fluid dynamics, and multi-stage scenarios, often producing physically implausible results.
• Formulating image editing as a temporal generation process, exemplified by PhyWorld, can effectively leverage the inherent temporal causality encoded in video models to achieve more physically plausible and consistent results without extensive fine-tuning.
• The benchmark's Anti-Physics instances revealed that models often rely on memorized statistical priors rather than genuine deductive physical reasoning, highlighting the need for systems capable of strictly adhering to counterfactual physical rules.

Summary by alphaXiv.