Reproduce “TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeremy Andrew Irvin, Emily Ruoyu Liu, Joyce Chuyi Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, Stefano Ermon
Researchers at Stanford University developed TEOCHAT, a Vision-Language Model capable of conversational interaction and reasoning over temporal sequences of Earth Observation data. It sets new benchmarks by outperforming existing generalist VLMs on temporal tasks and demonstrating competitive performance against specialist models and proprietary foundation models.
Key insights
- Jointly training TEOCHAT on both single-image and temporal instruction-following tasks significantly enhanced its temporal reasoning capabilities.
- Explicitly embedding image references (e.g., "Image 1") within prompts proved crucial for the LLM to effectively differentiate and reason about distinct temporal steps.
- Leveraging Video-LLaVA initialization and fine-tuning the vision-language projector layer were critical architectural choices that contributed to superior performance compared to alternative configurations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.