Reproduce “Transferring Physical Priors into Remote Sensing Segmentation via Large Language Models” on OpenResearch

About this paper

Yuxi Lu, Kunqi Li, Zhidong Li, Xiaohan Su, Biao Wu, Chenya Huang, Bin Liang

PriorSeg transfers category-level physical ranges into a trainable refiner attached to a frozen foundation model (FM), using language-model-generated priors to create simulated training maps and correct initial segmentations. On WHU-OPT-SAR, SkySense-O with PriorSeg reaches 42.98% mIoU, versus 39.83% alone, with real physical inputs. This segmentation gain does not verify the priors’ physical accuracy.

Key insights

• Large Language Models can efficiently extract and structure domain-specific physical knowledge into a knowledge graph, automating the definition of plausible physical ranges for numerous semantic categories.
• Training with simulated physical variables, generated based on LLM-derived knowledge, is sufficient to internalize physics-aware reasoning in segmentation models, enabling visual-only inference with improved physical plausibility.
• A modular, residual refinement approach allows existing, frozen foundation models to incorporate physics-informed corrections, enhancing segmentation accuracy and physical consistency without expensive full retraining or architectural changes.

Summary by alphaXiv.