Reproduce “Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jianxing Qin, Alexander Du, Danfeng Zhang, Matthew Lentz, Danyang Zhuo

Vosti makes inference outputs reproducible across runtime scheduling and cache variations by fixing computation choices for a model and deployment and checking request-history handling and GPU kernels. It matched all 5,488 tested pairs of complete raw next-token logit vectors across seven Llama and Gemma models and four execution variations. This guarantee applies only within a fixed qualified deployment configuration.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.