Reproduce “Unraveling the geometry of visual relational reasoning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiaqi Shang, Gabriel Kreiman, Haim Sompolinsky

The authors test whether visual reasoning models transfer abstract rules to rules withheld from training, using a row-matching task and four models. They analyze rule-manifold geometry, including separation, dimensionality, and alignment, to explain generalization. SCL had the lowest error across 15 held-out-rule splits; an SNR-based objective beat Prototypical loss, but not significantly cross-entropy, limiting evidence of improvement over that baseline.

Key insights

  • A geometric theory based on 'rule manifolds' accurately predicts generalization error in neural networks across diverse architectures on abstract visual reasoning tasks.
  • Generalization ability is governed by an interplay of geometric properties, particularly the signal (separation of rule manifolds), their effective dimensionality, and signal-noise overlap.
  • A consistent trade-off exists between increasing the generalization signal and maintaining high dimensionality, driven by a structured compression mechanism that aligns representations into a low-dimensional subspace.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.