Reproduce “FOCUS: Optimal Control for Multi-Entity World Modeling in Text-to-Image Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Eric Tillmann Bill, Enis Simsar, Thomas Hofmann

Researchers at ETH Zürich developed a stochastic optimal control (SOC) framework for text-to-image generation, enabling enhanced multi-subject disentanglement and attribute binding by reformulating flow matching. Their proposed FOCUS objective, applied as either test-time control or a lightweight fine-tuning rule, consistently improved image fidelity and received higher human preference ratings for multi-entity scenes.

Key insights

  • Reformulating text-to-image generation as a stochastic optimal control problem offers a principled, architecture-agnostic framework to unify and improve multi-subject disentanglement.
  • The proposed FOCUS objective effectively quantifies and mitigates attribute leakage and identity entanglement by operating on probabilistic attention maps.
  • Both test-time control and fine-tuning instantiations of the SOC framework consistently improve multi-subject fidelity, with fine-tuning providing better robustness and zero inference-time overhead.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.