Reproduce “One Model, Multiple Compute Budgets: Multi-Ratio ConceptMoE for Flexible Compute Across Turns” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiyang Fu, Zihao Huang, Dongchen Han, Shukai Gong, Zhen Dong, Qiyang Min, Daquan Zhou
Multi-Ratio ConceptMoE lets one shared model vary conversation-compression settings across turns, avoiding backbone switches. It uses ratio-specific boundary routers and shared modules to process compressed concepts at CR2, CR4, or CR8. On CodeChat it reaches 43.78% Combined Joint accuracy at 1.0999 PFLOPs, versus 37.91% at 1.8662 PFLOPs for Large MoE; these workload-based algorithmic FLOPs do not measure latency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.