Reproduce “HELIX: HEterogeneous LLM Inference with FleXible and Fine-Grained Partitioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Victoria Clerico, William Simon, Corey Liam Lammie, Orhun Görkem, Athanasios Vasilopoulos, Hsinyu Tsai, Jeronimo Castrillon, Abu Sebastian and 1 more
HELIX searches per-operation placements across accelerators for each model and workload, rather than fixing attention and feed-forward layers to devices. It uses profiled costs and memory budgets to compile partitions for a unified vLLM engine. In the modeled H100, 3DCIM, and DRAM-PIM setup, Llama 8B single-step latency is 1.8–3.5× faster than fixed baselines below 64 concurrent queries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.