Reproduce “No Single Tokenizer Feature Reliably Predicts Downstream Language Model Performance” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Antonio Lopardo, Luis Frentzen Salim, Shikhar Srivastava, Kevin Jiang, Mridul Sharma, Catherine Arnett

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.