Reproduce “Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaofeng Zhang, Jinfa Huang, Qiang Zhou, Zhibin Wang, Fan Wang, Jiebo Luo, Junchi Yan
PQDiff, a diffusion-based model from Shanghai Jiao Tong University and collaborators, introduces a novel framework for image outpainting that generates content at arbitrary continuous multiples in a single step. It achieves new state-of-the-art FID scores (e.g., 21.512 on Scenery at 11.7x expansion) and significantly reduces generation time, operating at 10.2% of previous SOTA methods for large expansions.
Key insights
- The use of Relative Positional Embeddings (RPE) as a query in cross-attention effectively conditions a diffusion model to handle arbitrary and continuous outpainting multiples.
- Training with randomly cropped and resized anchor/target views is crucial for enabling a generative model to generalize to any desired relative position and scale during inference.
- A diffusion-based approach, when combined with appropriate positional conditioning, can perform the entire outpainting process for large expansion factors in a single forward pass, dramatically improving efficiency over multi-step methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.