Reproduce “ETCHR: Editing To Clarify and Harness Reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Beichen Zhang, Yuhong Liu, Jinsong Li, Yuhang Zang, Jiaqi Wang, Dahua Lin

ETCHR, developed by researchers from The Chinese University of Hong Kong and Shanghai AI Laboratory, enhances multimodal large language models' visual reasoning by training a specialized image-to-image editor to generate and verify visual aids autonomously. The framework consistently improves average Pass@1 accuracy across various MLLMs and tasks, boosting performance by up to 5.47 percentage points.

Key insights

  • Decoupling a specialized image editor from the understanding MLLM overcomes limitations of both tool-based and unified multimodal approaches, offering broader transformation capabilities and higher fidelity.
  • Training the editor to autonomously infer and execute visual transformations from abstract questions (bridging the 'language-side' and 'generation-side' gaps) is critical for reasoning assistance.
  • A dual reward system, combining downstream reasoning utility (guidance) with isolated visual fidelity (correctness), effectively enhances the generative quality and reasoning-awareness of edits in RL.
  • An explicit verification step during inference is crucial for preventing noisy or incorrect edits from misleading the MLLM, though its utility can vary with baseline model accuracy and task complexity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.