Reproduce “Compositional meta-learning through probabilistic task inference” on OpenResearch

About this paper

Jacob J. W. Bakermans, Pablo Tano, Reidar Riveland, Charles Findling, Alexandre Pouget

The authors present a model that learns reusable transformations and a controller for choosing their sequence, then infers tasks by testing candidate sequences against observed outputs without updating its weights. In a constructed rule-learning task, module and gating accuracy correlations plateau at 1 across five training seeds; it solves a sparse-feedback test task four times longer than any training task.

Key insights

• Meta-learning can be effectively reframed as compositional inference over fixed computational building blocks, rather than relying on parameter adaptation for new task acquisition.
• An explicit modular architecture with a learned gating mechanism provides a strong inductive bias for compositionality, allowing the model to learn and reuse both individual computations and their sequencing rules.
• Integrating probabilistic inference with expressive neural networks enables rapid task acquisition from minimal, even sparse, feedback by constraining the search space of possible solutions with a learned 'task grammar'.

Summary by alphaXiv.