Reproduce “Latent Bottlenecked Attentive Neural Processes” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Leo Feng, Hossein Hajimirsadeghi, Yoshua Bengio, Mohamed Osama Ahmed

Latent Bottlenecked Attentive Neural Processes (LBANPs) introduce a scalable Neural Process architecture that resolves the quadratic computational cost of attention-based variants while maintaining strong predictive performance. The model achieves linear complexity for context encoding and constant complexity per target query, allowing it to scale effectively to large datasets and resource-constrained environments.

Key insights

  • A compact, fixed-size set of latent vectors can effectively represent information from a variable and potentially large context dataset through iterative attention.
  • Decomposing the attention mechanism into a linear-cost context encoding step and a constant-cost (per target) querying step significantly improves scalability without sacrificing predictive power.
  • The number of latent vectors offers a direct control knob to balance the model's computational requirements against its predictive performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.