Reproduce “Diverse Image Inpainting with Bidirectional and Autoregressive Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yingchen Yu, Fangneng Zhan, Rongliang Wu, Jianxiong Pan, Kaiwen Cui, Shijian Lu, Feiying Ma, Xuansong Xie and 1 more
Researchers from Nanyang Technological University and Alibaba Group developed BAT-Fill, a novel two-stage framework for diverse image inpainting, integrating bidirectional and autoregressive transformers. This system generates multiple realistic and distinct image completions with improved fidelity and diversity over previous methods, achieving the best FID scores across evaluated datasets.
Key insights
- Marrying bidirectional context modeling with autoregressive output dependency in the BAT architecture effectively resolves the trade-off between leveraging full contextual information and ensuring output consistency for diverse generation.
- A two-stage approach, where the BAT handles diverse structure generation at low resolution and a CNN refines high-fidelity textures, is crucial for achieving both high quality and diversity.
- Top-K sampling during the BAT's inference allows for the introduction of diversity by sampling from multiple probable tokens, demonstrating the model's stochastic generation capability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.