Reproduce “LP-Spec: Leveraging LPDDR PIM for Efficient LLM Mobile Speculative Inference with Architecture-Dataflow Co-Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyuan He, Zhantong Zhu, Yandong He, Tianyu Jia
Peking University researchers developed LP-Spec, an architecture-dataflow co-optimization framework for efficient Large Language Model speculative inference on mobile devices. It integrates a GEMM-enhanced LPDDR5-PIM with NPU-PIM parallel execution and dynamic scheduling to optimize performance and energy, achieving 4.59x higher throughput and 7.56x better energy efficiency compared to mobile NPU speculative inference baselines.
Key insights
- Adapting PIM architectures from GEMV to GEMM operations is critical for efficiently accelerating speculative inference workloads, which are compute-intensive, on resource-constrained mobile devices.
- Co-designing hardware (e.g., Near-Data Memory Controller for concurrent operations and data reallocation) with intelligent software (dynamic scheduling and token pruning) is essential for maximizing heterogeneous resource utilization and energy efficiency under mobile constraints.
- Dynamically optimizing the token tree structure based on prediction accuracy and hardware costs significantly reduces redundant computation and improves overall energy efficiency without sacrificing inference accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.