Reproduce “Parameter-Efficient Fine-Tuning of Vision Foundation Model for Forest Floor Segmentation from UAV Imagery” on OpenResearch

About this paper

Mohammad Wasil, Ahmad Drak, Brennan Penfold, Ludovico Scarton, Maximilian Johenneken, Alexander Asteroth, Sebastian Houben

Researchers at Bonn-Rhein-Sieg University of Applied Sciences adapted the Segment Anything Model (SAM) using Parameter-Efficient Fine-Tuning (PEFT) to perform automated, category-specific semantic segmentation of forest floor objects from UAV imagery. The adapted models, particularly those using AdapterH and LoRA, significantly outperformed zero-shot SAM, achieving mIoU values up to 0.416 with millions of fewer trainable parameters than full fine-tuning.

Key insights

• PEFT methods are effective for adapting large vision foundation models to niche remote sensing datasets, enabling knowledge transfer with a limited number of trainable parameters.
• Modifying SAM's mask decoder to directly predict semantic categories is feasible and crucial for enabling automated, prompt-free segmentation in practical environmental monitoring applications.
• The placement and architectural specifics of PEFT modules (e.g., AdapterH vs. AdapterL) significantly influence performance, with AdapterH and LoRA demonstrating superior efficacy for this task.

Summary by alphaXiv.