Reproduce “LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yabin Zhang, Wenjie Zhu, Chenhang He, Lei Zhang
LAPT learns prompts for vision-language-model out-of-distribution detection to reduce sensitivity to class-prompt wording. It trains separate contexts for in-distribution and mined negative labels using automatically collected generated and retrieved examples, while keeping CLIP encoders frozen. On OpenOOD with ImageNet-1k, it raises near-OOD AUROC from 76.92 to 82.63 versus NegLabel, while lowering FPR95 from 68.18 to 58.94; results are benchmark-specific.
Key insights
- Distribution-aware prompts that learn specific token sequences for in-distribution and negative classes perform significantly better than unified or class-specific prompt designs for OOD detection.
- A combination of image retrieval from web-scale datasets and synthetic image generation using text-to-image models effectively provides sufficient and diverse training data for automated prompt tuning.
- Ablation studies reveal that optimal performance for learnable prompts is achieved with 4-8 tokens, and placing these tokens at the end of the prompt yields the best results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.