Reproduce “PALUTE: Processing-In-Memory Acceleration via Lookup Table for Edge LLM Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Runyang Tian, Yanru Chen, Weihong Xu, Tajana Šimunić Rosing
PALUTE introduces a Processing-In-Memory (PIM) accelerator leveraging Lookup Tables (LUTs) and Monolithic 3D (M3D) DRAM to enable efficient Large Language Model inference on edge devices. It achieves 7,738 Tokens Per Second per Watt (TPS/W) for Qwen3-4B with W4A4 quantization, representing a 12.8x energy efficiency improvement over CHIME.
Key insights
- The vertical architecture of VBL M3D DRAM enables massive in-DRAM parallelism for LUT lookups, significantly enhancing throughput while minimizing area overhead.
- Replacing complex arithmetic operations, including dequantization and nonlinearities, with in-memory LUT queries directly addresses the memory wall and associated energy overheads.
- Implementing efficient, near-memory LUT generation on the logic die ensures that LUT preparation does not become a new bottleneck, supporting a complete end-to-end inference pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.