Reproduce “Understanding Visual Concepts Across Models” on OpenResearch

About this paper

Brandon Trabucco, Max Gurinas, Kyle Doherty, Ruslan Salakhutdinov

Researchers at Carnegie Mellon University found that soft prompt-tuned word embeddings for visual concepts are highly model-specific and generally non-transferable across different large multimodal models (LMMs) and tasks, such as generation, detection, and classification. The study revealed a "fracturing" of the embedding space, indicating that performant solutions can be found as arbitrary small perturbations around unrelated word embeddings, primarily impacting the final layers of text encoders.

Key insights

• The vector embedding space exhibits "fracturing," where performant solutions for a visual concept can be found as small perturbations around *any* prior embedding, even for semantically unrelated words.
• Soft prompts primarily target and modify activations in the *final layers* of text encoders; early layers often retain the semantic content of the initial anchor word.
• New embeddings for visual concepts do not appear to operate through simple algebraic combinations of existing word embeddings but rather as specialized, model-specific perturbations.

Summary by alphaXiv.