Reproduce “BitMTP: When Multi-Token Prediction Meets Low-Bit Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ning Zhang, Shihao Wang, Jinrui Zhang, Chaodong Xiao, Lei Zhang
BitMTP trains a model for ordinary next-token generation and parallel block proposals, then commits confident prefixes or verifies proposals against its autoregressive path. Its runtime aligns accepted text, stored attention keys and values, and subsequent execution. On NVIDIA H20, Bonsai-1.7B at D=8 hybrid decoding reaches 1,110.65 tokens/s versus 469.26, with comparable mean quality: 47.91% versus 47.36%. Results cover tested models and hardware, not larger models or end-to-end latency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.