Reproduce “Puzzle Similarity: A Perceptually-guided Cross-Reference Metric for Artifact Detection in 3D Scene Reconstructions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nicolai Hermann, Jorge Condor, Piotr Didyk

A new cross-reference metric, "Puzzle Similarity," detects and localizes visual artifacts in novel views generated from 3D scene reconstructions by comparing patches in deep feature space against reference images. The metric provides high-resolution artifact maps, demonstrating superior correlation with human perception compared to existing methods, and supports applications like automatic image restoration.

Key insights

  • Comparing image patches in the latent space of a pre-trained CNN to a pool of reference patches effectively models human perception of reconstruction artifacts.
  • Integrating feature similarities from multiple CNN layers, weighted empirically, enhances the robustness and accuracy of artifact localization across diverse distortions.
  • The data-driven deep feature similarity approach can outperform traditional human visual system models (like VDPs) for detecting complex generative artifacts in 3D scene reconstructions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.