Reproduce “TokBench: Evaluating Your Visual Tokenizer before Visual Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Junfeng Wu, Dongliang Luo, Weizhi Zhao, Zhihao Xie, Yuanhao Wang, Junyi Li, Xudong Xie, Yuliang Liu and 1 more

Researchers at Huazhong University of Science and Technology developed TokBench, a benchmark to evaluate the fine-grained visual feature preservation of visual tokenizers, particularly for text and human faces. The benchmark demonstrates that current tokenizers frequently struggle with these details and that conventional reconstruction metrics often fail to reflect the quality of these perceptually critical elements.

Key insights

  • Traditional metrics like rFID and LPIPS often fail to accurately reflect the reconstruction quality of fine-grained text and facial features, demonstrating inconsistencies with human judgment.
  • Most modern visual tokenizers struggle significantly to preserve fine-grained features, especially at smaller object scales, highlighting a fundamental limitation in current architectures.
  • Continuous VAEs consistently outperform discrete tokenizers in fine-grained feature preservation, particularly for faces and at higher reconstruction resolutions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.