Reproduce “Flaming-hot Initiation with Regular Execution Sampling for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weizhe Chen, Zhicheng Zhang, Guanlin Liu, Renjie Zheng, Wenlei Shi, Chen Dun, Zheng Wu, Xing Jin and 1 more
Researchers at the University of Southern California, Carnegie Mellon University, and ByteDance Inc. developed Flaming-hot Initiation with Regular Execution (FIRE) sampling, which applies high-temperature sampling to initial tokens to improve large language models' success rates and response diversity in reasoning tasks like math and code generation during both inference and reinforcement learning.
Key insights
- Diversifying the initial token of an LLM generation, leveraging its role as an 'attention sink,' effectively steers the model towards more varied and potentially correct reasoning paths.
- Increased diversity in generated samples directly correlates with improved pass rates for complex reasoning tasks when multiple attempts are allowed, validating the method's approach.
- FIRE sampling can be seamlessly integrated into reinforcement learning frameworks like Proximal Policy Optimization (PPO), enhancing the efficiency of LLM alignment by providing more diverse and successful training samples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.