Reproduce “VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision-Language Models” on OpenResearch

About this paper

Christos Ziakas, Alessandra Russo

Researchers at Imperial College London developed VITA, a method for zero-shot value function learning that improves Vision-Language Models' generalization and temporal reasoning via test-time adaptation. VITA effectively provides reward shaping for offline reinforcement learning, achieving an interquartile mean return of 0.815 on Meta-World MT10, which exceeds the simulation's native fuzzy-logic dense rewards.

Key insights

• Test-time adaptation of a small, meta-learned module can effectively imbue frozen VLM representations with both generalization capabilities and robust temporal reasoning without extensive retraining.
• Implicitly retaining temporal context within the adapter's parameters through sequential, per-timestep updates is more effective for value function estimation and less prone to overfitting than explicit recurrent models or memoryless adaptation.
• Mitigating shortcut learning through dissimilarity-based sampling during meta-training is crucial for developing value functions that rely on meaningful semantic and temporal understanding rather than superficial chronological cues.

Summary by alphaXiv.