Reproduce “YOLOE-26: Integrating YOLO26 with YOLOE for Real-Time Open-Vocabulary Instance Segmentation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ranjan Sapkota, Manoj Karkee

YOLOE-26 supports instance segmentation with targets specified by text, visual examples, or a built-in vocabulary. It represents detected objects and prompts as embeddings, matching them to produce categories and masks. For YOLOE-26x-seg-pf, at 640-pixel resolution on minival, prompt-free evaluation reports mAP50–95 of 29.9 and mAP50 of 38.7. The paper reports no YOLOE-26 latency or direct comparison with YOLOE.

Key insights

  • It is possible to achieve real-time open-vocabulary instance segmentation on edge devices by leveraging efficient CNN-based YOLO architectures and re-parameterization techniques to avoid the computational overhead of transformer models.
  • Unified object embedding spaces, combined with specific prompt alignment (RepRTA for text, SAVPE for visual) and lazy matching strategies (LRPC for prompt-free), allow for flexible and efficient open-world perception.
  • Eliminating NMS and implementing end-to-end inference, inherited from YOLOv26, is crucial for maintaining real-time performance and determinism when extending to open-vocabulary tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.