Reproduce “Adaptive Multilevel Twisted Sequential Monte Carlo for Rare Events Estimation in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixuan Liu, Fangzheng Wu, Brian Summa, Zizhan Zheng
Researchers at Tulane University developed Adaptive Multilevel Twisted Sequential Monte Carlo (Adaptive Multilevel Twisted SMC) to enhance rare-event probability estimation in large language models by overcoming the positive-sample bottleneck. The method employs a sequence of progressively rarer intermediate events, achieving superior hit rates and lower estimation error in discovering extremely rare unsafe behaviors like toxic content generation and jailbreak prompts compared to existing Twisted SMC variants.
Key insights
- Learning twist functions via a sequence of progressively rarer intermediate events effectively mitigates the positive-sample bottleneck, making rare event estimation more tractable.
- An adaptive threshold selection mechanism, based on weighted quantiles, allows for automatic, statistically consistent progression through rarity levels.
- The multilevel construction fundamentally improves sample complexity from an inverse dependence on rare-event probability to a polylogarithmic dependence, even with imperfect proposals.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.