Reproduce “A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Swadhin Das, Divyansh Mundra, Priyanshu Dayal, Raksha Sharma
Researchers introduced a lightweight transformer model with edge-aware fusion (KD-EBF) for remote sensing image captioning, combining a distilled ConvNeXt encoder and DistilGPT2 decoder with fused edge-detected image inputs. The KD-EBF model achieved higher performance across various metrics (e.g., BLEU-4, CIDEr) compared to state-of-the-art baselines on SYDNEY, UCM, and RSICD datasets, while also reducing computational cost.
Key insights
- Lightweight transformer architectures, coupled with knowledge distillation, can significantly reduce computational cost while maintaining or improving performance in RSIC.
- Explicitly incorporating fine-grained structural features via edge-aware fusion at the input level enhances the model's ability to generate more accurate and descriptive captions.
- A unified input pipeline for original and edge-enhanced images effectively leverages multi-faceted image representations without increasing model complexity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.