Reproduce “Prompting Diffusion Representations for Cross-Domain Semantic Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rui Gong, Martin Danelljan, Han Sun, Julio Delgado Mangas, Luc Van Gool
This paper presents a method for cross-domain semantic segmentation that leverages frozen, pretrained diffusion models and prompt engineering. The approach enhances generalization by using prompt randomization during training and enables efficient test-time adaptation through prompt tuning, achieving improved performance on several challenging benchmarks.
Key insights
- Diffusion models, pre-trained on diverse datasets, naturally learn more disentangled and domain-invariant visual representations suitable for generalization.
- Text prompts can effectively condition diffusion model backbones to extract features that separate semantic content from domain-specific attributes.
- Prompt randomization during training and prompt tuning during inference offer parameter-efficient ways to enhance domain generalization and adaptation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.