Reproduce “RaCo: Ranking and Covariance for Practical Learned Keypoints” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Abhiram Shenoi, Philipp Lindenberger, Paul-Edouard Sarlin, Marc Pollefeys
RaCo combines a repeatable-point detector, a ranker for cross-view matches, and a covariance head estimating uncertainty in pixels. It trains with synthetic homographies and image augmentations, without covisible training pairs or rotation-equivariant layers. On HPatches, rotation augmentation yields 78.3% two-pixel rotation repeatability AUC, versus 24.9% without it; this is a detector result, not evidence of improved reconstruction or system speed.
Key insights
- Extensive data augmentation encompassing full 360-degree rotations can effectively impart state-of-the-art rotational robustness to keypoint detectors, eliminating the need for computationally intensive equivariant network designs.
- Explicitly training a differentiable ranker to optimize match counts across varying keypoint budgets provides substantial performance improvements, particularly when subsampling for resource-constrained applications.
- Learning metric, anisotropic spatial covariances for keypoints is achievable through self-supervision and provides physically interpretable uncertainty measures that significantly enhance the quality of downstream 3D processing tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.