Reproduce “RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yelin Wang, Zijia Song, Shuo Ye, Chuanguang Yang, Miaoyu Wang, Yong Xu, Zhulin An, Yongjun Xu and 1 more
RSICCLLM is introduced as a 7-billion parameter multimodal large language model, along with a novel post-training framework and large-scale instruction (RSICI) and preference (RSICP) datasets for remote sensing image change captioning. This model achieves leading performance in describing fine-grained changes between bi-temporal satellite images and generalizes effectively across diverse remote sensing data.
Key insights
- Explicitly modeling geometric and textural changes via specialized convolutions and Hough transforms significantly improves a VLM's ability to perceive and understand temporal differences in bi-temporal remote sensing images.
- Large-scale, synthetically generated instruction and preference datasets are crucial for effectively adapting general-purpose VLMs to specialized remote sensing tasks with limited real-world annotated data.
- Dual-Negative Preference Optimization, using systematically constructed positive-negative pairs, can substantially enhance a model's discriminative power and descriptive accuracy beyond supervised fine-tuning alone, with a stabilization term critical for training stability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.