Reproduce “Manifold Bandits: Bayesian Curriculum Learning over the Latent Geometry of Large Language Models” on OpenResearch

About this paper

Darrien McKenzie, Nicklas Hansen, Xiaolong Wang

Researchers at UC San Diego introduced a Bayesian curriculum learning framework, Manifold Bandits, which optimizes large language model training by leveraging the latent geometric structure of task problems. The method improves training efficiency and generalization by adaptively selecting prompts based on a combination of their expected learning signal, diversity across the task manifold, and relevance to target evaluations.

Key insights

• The learning signal during LLM training is correlated with the model's latent task geometry, allowing a hierarchical structure to efficiently propagate evidence and guide problem selection across related tasks.
• Endogenous non-stationarity of learning signals can be effectively managed through Bayesian filtering, which adaptively reduces reliance on stale evidence for surprising observations.
• Effective LLM training requires balancing immediate learning signal (productivity) with broad coverage of problem types (diversity) and explicit consideration of relevance to target evaluations (utility) for improved generalization and targeted capability development.

Summary by alphaXiv.