Reproduce “Open-World Dynamic Prompt and Continual Visual Representation Learning” on OpenResearch

About this paper

Youngeun Kim, Jun Fang, Qin Zhang, Zhaowei Cai, Yantao Shen, Rahul Duggal, Dripta S. Raychaudhuri, Zhuowen Tu and 2 more

Researchers from Yale University and AWS AI Labs introduce Dynamic Prompt and Representation Learner (DPaRL), a method addressing a new Open-World Continual Representation Learning setting. DPaRL significantly improves Recall@1 by an average of 4.7% over previous state-of-the-art methods in 10 continual learning stages and 9.1% in 100 stages on open-world image retrieval benchmarks by dynamically generating prompts and jointly learning representations.

Key insights

• Dynamically generated prompts that integrate both image-specific and stage-specific information are crucial for enabling prompt-based models to generalize effectively to truly unseen categories in open-world continual learning.
• Jointly updating the backbone's parameters via PEFT along with the prompt generation mechanism is more effective for open-world generalization than relying solely on a frozen backbone, as it allows for better integration of diverse semantics.
• Learnable 'stage tokens,' managed with a scalable mechanism like a FIFO queue, provide an efficient way to integrate historical knowledge from various continual learning stages without an unbounded increase in parameters.

Summary by alphaXiv.