Reproduce “Characterize LSM-tree Compaction Performance via On-Device LLM Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiabiao Ding, Yina Lv, Qiao Li, Zhirong Shen, Chun Jason Xue
An empirical study characterizes the trade-offs in using Large Language Models (LLMs) for real-time Log-Structured Merge-tree (LSM-tree) compaction optimization, revealing that large cloud-hosted LLMs incur prohibitive latency for edge deployment, while smaller on-device LLMs face reasoning challenges but can be effective in simplified tasks. The research provides foundational insights into the viability and limitations of LLM-driven autonomous system tuning.
Key insights
- Large-scale cloud-hosted LLMs incur high inference latencies (hundreds of seconds per iteration) due to network overhead and queuing, making them unsuitable for real-time, latency-sensitive system tuning.
- Small-scale on-device LLMs exhibit significant reasoning limitations, including failures to adhere to instruction formats, incomplete outputs, repetitive generation, and unstable temperature-based exploration.
- There is a direct correlation between model scale and reasoning quality, but an inverse correlation with inference speed; small LLMs can demonstrate competent tuning in constrained parameter subspaces, suggesting potential for focused applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.