Reproduce “NgramRL: Accelerating RL Post-Training for Structured Generation via Rollout-Aware N-gram Speculative Decoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nilesh Prasad Pandey, Jangseon Park, Jason Kong, Onat Gungor, Tajana Rosing
NgramRL accelerates reinforcement-learning rollouts by drafting from a bounded, per-prompt cache of recent completions, which the target policy verifies with speculative correction. Tokens committed per target forward pass equals one plus mean accepted draft length. Across Qwen3-8B and DeepSeek-7B on three tasks, it reaches 3.60–7.77 in the warm-cache regime; this measures target forwards, not runtime or total computation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.