Reproduce “MoETok: Sparse Expert Routing for 1D Visual Tokenizers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahao Guo, Bao Tang, Haoxiang Cao, Jingfeng Yao, Bo Li, Wenyu Liu, Kun Gai, Changqian Yu and 1 more
MoETok uses sparse expert routing to let each token follow a different processing path in an image representation. A learned router selects experts for each token, while reconstruction and visual-feature objectives train the tokenizer without role labels. On matched ImageNet 1D Patch + Query models, it improves linear-probe accuracy, zero-shot Top-1, reconstruction rFID, and unguided generation gFID versus Dense; these results do not establish runtime savings or prove role specialization causes the gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.