Reproduce “Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kate McCurdy, Jacob Hoover Vigly, Thomas Hikaru Clark, Karolina Stańczak, Yahya Emara, Joshua B. Tenenbaum, Alexander K. Lew, Ryan Cotterell and 1 more
The authors guide small language models toward specified grammar constructions by pairing programmatic checks, called potentials, with sequential Monte Carlo (SMC), which extends and reweights candidate responses at inference time. In five A2 tasks, Qwen 2 0.5B with a template prompt reaches 98% target-grammar accuracy with eight particles, versus 58% with one; this is generated-turn coverage, not evidence of learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.