Reproduce “Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, Wangmeng Zuo
DiffTPT, a method developed by researchers from A*STAR, MBZUAI, Australian National University, and Harbin Institute of Technology, improves Test-time Prompt Tuning by using pre-trained diffusion models for diverse data augmentation. This approach achieves a 5.13% average increase in zero-shot accuracy over prior TPT methods on out-of-distribution datasets and maintains robustness across various cross-dataset generalization tasks.
Key insights
- Pre-trained diffusion models can effectively generate visually rich and diverse augmented data that significantly enhances the robustness of prompts for zero-shot generalization.
- Maintaining semantic consistency between original and augmented data via cosine similarity-based filtration is crucial to prevent spurious augmentations from degrading model performance.
- Combining both traditional and diffusion-based augmentation strategies offers synergistic benefits, creating a more comprehensive augmented data pool for test-time adaptation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.