Reproduce “Together Compiler: Continual Compilation for AI Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Reyna Abhyankar, Dylan Lim, Drew Wadsworth, Venkata Hasith Vattikuti, YuTong Julia Zhang, Jung Hwan Heo, Taeksang Kim, Kesavan Ramakrishnan and 4 more
TKCC enables agents to optimize inference deployments as workloads change by hot-reloading candidate forward passes in a persistent engine, screening them before representative measurements. Its cost model selects GPU-parallelism configurations. On B200 GLM 5.3, two-tier measurement evaluated 1,354 candidates versus 168 for a single slow tier and found a 7.2% faster solution; the result is specific to that comparison.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.