Reproduce “SE360: Semantic Edit in 360$^\circ$ Panoramas via Hierarchical Data Construction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoyi Zhong, Fang-Lue Zhang, Andrew Chalmers, Taehyun Rhee
A framework called SE360 enables semantic object editing in 360° panoramas by employing a Transformer-based diffusion model and a two-phase data generation pipeline. It produces geometrically consistent and semantically accurate edits, achieving a CS_pers of 29.354 and LPIPS of 0.076 for object addition, and demonstrating superior performance in user studies compared to existing methods.
Key insights
- Explicitly encoding 3D spherical geometry and employing a symmetric circular padding strategy (Pad-and-Unpad) is critical for achieving geometrically consistent and seamless edits in 360° panoramas.
- An autonomous, coarse-to-fine data generation pipeline, leveraging Physical Affiliation Analysis, can construct large-scale, semantically rich training data for complex 360° scenes.
- A two-stage training approach, involving initial training on a larger, potentially noisier dataset followed by fine-tuning on a smaller, high-fidelity dataset, effectively balances generative capability with artifact reduction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.