Reproduce “Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Garrett Baker, Vinayak Pathak, Daniel Murfet, Susan Wei
Posterior Prefix Tuning (PPT) optimizes a continuous prompt distribution in the latent posterior space of Bayes-filtered transformers (BFTs), enabling efficient elicitation of specific model behaviors without requiring backpropagation through the transformer. The method demonstrates competitive or superior performance compared to backpropagation-based approaches on synthetic tasks, significantly reducing the computational cost for iterating on different utility functions.
Key insights
- The behavior of Bayes-filtered transformers can be controlled by optimizing their latent posterior distributions, which factorizes the complex discrete prompt search into a more tractable continuous optimization problem.
- Decoupling the optimization process from transformer forward/backward passes, through latent-space gradient estimation, allows for 'utility amortization,' significantly reducing marginal computational cost when optimizing for multiple objectives.
- Rao–Blackwellization effectively reduces variance in gradient estimates, leading to improved optimization performance for certain utility functions within the PPT framework.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.