Reproduce “LUT-Fuse: Towards Extremely Fast Infrared and Visible Image Fusion via Distillation to Learnable Look-Up Tables” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xunpeng Yi, Yibing Zhang, Xinyu Xiang, Qinglong Yan, Han Xu, Jiayi Ma
LUT-Fuse introduces an infrared and visible image fusion framework that leverages learnable look-up tables and an efficient distillation strategy to achieve real-time processing speeds, even on low-power edge devices. The method fuses images at 4.7-23.2 ms per frame on PC and 18.23 ms for 480P resolution on a mobile platform, while preserving competitive fusion quality and excelling in downstream tasks like object detection.
Key insights
- Knowledge distillation can effectively transfer intricate fusion logic from a complex deep network to a compact, learnable look-up table (LUT) structure.
- A learnable LUT architecture can integrate both low-order image feature approximations and high-level contextual scene embeddings to efficiently represent multi-modal fusion relationships.
- Regularization for smoothness and monotonicity is vital to ensure stability, prevent artifacts, and enhance the generalization of the learned LUTs, particularly when direct ground truth is unavailable.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.