Reproduce “CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiming Wang, Mingze Wang, Sheng Xu, Yanjing Li, Baochang Zhang
CCExpert, developed by Beihang University, enhances Remote Sensing Image Change Captioning (RSICC) by integrating a Difference-aware Integration Module and leveraging a new 200,000-image-pair CC-Foundation dataset. The framework achieved a new state-of-the-art S*m score of 81.80 on the LEVIR-CC benchmark, improving caption accuracy and detail.
Key insights
- Explicitly capturing and integrating multi-scale differential features significantly boosts an MLLM's ability to perceive and describe subtle changes in remote sensing imagery.
- Domain-specific data-driven continued pretraining, combined with the preservation of the MLLM's foundational knowledge, is crucial for effectively adapting powerful general models to specialized tasks like RSICC.
- A three-stage progressive training process ensures deep and effective integration of novel modules with pretrained MLLMs, optimizing performance for RSICC tasks without undermining existing knowledge.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.