Reproduce “CHIME: Chiplet-based Heterogeneous Near-Memory Acceleration for Edge Multimodal LLM Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanru Chen, Runyang Tian, Yue Pan, Zheyu Li, Weihong Xu, Tajana Rosing
UC San Diego researchers developed CHIME, a chiplet-based heterogeneous near-memory acceleration system that combines Monolithic 3D (M3D) DRAM and M3D RRAM to optimize multimodal Large Language Model (MLLM) inference on edge devices. The system achieves up to 54x speedup and 246x energy efficiency compared to edge GPUs by mitigating memory bottlenecks and tailoring memory technologies to MLLM demands.
Key insights
- Integrating complementary memory technologies (DRAM for speed, RRAM for density/capacity) is essential for simultaneously meeting the diverse bandwidth, capacity, and energy efficiency demands of MLLMs on edge devices.
- A careful hardware-software co-design, including specialized data partitioning and kernel fusion tailored to each chiplet's capabilities, is crucial for minimizing cross-chiplet data transfers and maximizing computational locality.
- Tiered KV cache management effectively leverages intrinsic memory characteristics, allowing for efficient handling of long sequence lengths while mitigating RRAM endurance concerns by writing cold blocks infrequently.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.