Reproduce “A Mess of Memory System Benchmarking, Simulation and Application Profiling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pouya Esmaili-Dokht, Francesco Sgherzi, Valeria Soldera Girelli, Isaac Boixaderas, Mariana Carmin, Alireza Monemi, Adria Armejach, Estanislao Mercadal and 10 more

The Mess framework unifies memory system benchmarking, simulation, and application profiling, addressing inaccuracies and fragmentation in existing tools. It provides a more comprehensive view of memory behavior and demonstrates significantly improved simulation accuracy and speed (e.g., 1.3% average IPC error, 13x-15x faster than prior methods) by leveraging detailed hardware characterization.

Key insights

  • Memory write operations generally degrade performance (higher latency, lower saturated bandwidth) compared to reads across most memory systems, with exceptions like CXL memory expanders optimizing for balanced traffic.
  • A "wave form" effect was identified where increasing memory access rate can lead to a decrease in measured bandwidth, particularly on some Intel and AMD platforms, correlating with increased row-buffer misses.
  • Commonly used CPU simulators (ZSim, gem5) and external cycle-accurate DRAM models (DRAMsim3, Ramulator) exhibit widespread inaccuracies, including unrealistically low latencies and overestimated bandwidths, often failing to replicate real system trends.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.