Reproduce “MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture” on OpenResearch

About this paper

Kunming Shao, Ming Zeng, Xin Yuan, Binbin Liao, Yangming Zhang, Wei Wang, Tim Kwang-Ting Cheng, Chi-Ying Tsui

ReXpert improves interactive mixture-of-experts decoding by keeping expert weights resident in ReRAM and sharing a hot expert’s read-out among nearby idle processing groups. On Qwen3.5-35B-A3B, its modeled FFN-pool latency is 1.93 μs/token versus 18.26 for H20 at batch 8 and equal peak compute. Weight-movement energy is 20× lower at batch 64; results are modeled, not silicon measurements.

Key insights

• Efficient MoE FFN serving under strict latency Service-Level Objectives (SLOs) prioritizes high 'bandwidth density' and bounded locality-preserving sharing over simply increasing batch sizes.
• Core-local multicast pooling significantly recovers processing element occupancy by enabling idle units to 'borrow' hot expert weights, thereby mitigating performance bottlenecks from routing skew.
• Optimal hardware design for local pooling exists, balancing occupancy gains with communication overhead, as demonstrated by the 'side-4' core configuration achieving peak efficiency.

Summary by alphaXiv.