Reproduce “GPT-OSS-20B: A Comprehensive Deployment-Centric Analysis of OpenAI's Open-Weight Mixture of Experts Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Deepak Kumar, Divakar Yadav, Yash Patel

Researchers performed a deployment-centric analysis of OpenAI's projected GPT-OSS-20B, a 20.9B parameter Mixture-of-Experts model, against dense baselines on a single H100 GPU. The evaluation demonstrated that the MoE model achieves higher decode throughput, uses substantially less peak VRAM, and is more energy-efficient, particularly when considering performance per active parameter.

Key insights

  • Mixture-of-Experts (MoE) architectures, as exemplified by GPT-OSS-20B, offer superior decode throughput, significantly lower peak VRAM usage, and improved energy efficiency compared to larger dense models for single-GPU inference, despite incurring higher Time-To-First-Token (TTFT).
  • Normalizing performance by active parameters through the Active Parameter Efficiency (APE) metric clearly highlights the substantial efficiency gains and better resource utilization of MoE models per active computation.
  • While increasing context lengths generally degrades throughput and increases energy consumption for all evaluated models, the MoE architecture maintains its absolute leadership in decode throughput even at longer context lengths.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.