Reproduce “ETHcavation: A Dataset and Pipeline for Panoptic Scene Understanding and Object Tracking in Dynamic Construction Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lorenzo Terenzi, Julian Nubert, Pol Eyschen, Pascal Roth, Simin Fei, Edo Jelavic, Marco Hutter
Researchers at ETH Zurich developed "ETHcavation," a dataset and perception pipeline enabling autonomous excavators to perform panoptic scene understanding and object tracking in dynamic construction environments. The system integrates fine-tuned 2D panoptic segmentation with 3D LiDAR mapping and an online motion planner, demonstrating robust, reactive path planning in complex, real-world construction scenarios.
Key insights
- Fine-tuning large pre-trained panoptic segmentation models on a relatively small, domain-specific dataset, when combined with a general dataset like COCO, effectively addresses data scarcity and prevents catastrophic forgetting, yielding robust performance in specialized environments.
- Mask2Former with a Swin-Tiny backbone provides an optimal balance between high panoptic quality (PQ 0.68) and real-time inference speed (8 images/s), making it suitable for deployment on autonomous construction machinery.
- The integration of 2D panoptic segmentation, 3D LiDAR, and Kalman filter tracking enables explicit differentiation between static and dynamic objects and their tracking over time, which is critical for reactive and safe autonomous navigation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.