Reproduce “Pre-training Vision Transformers with Very Limited Synthesized Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryo Nakamura, Hirokatsu Kataoka, Sora Takashima, Edgar Josafat Martinez Noriega, Rio Yokota, Nakamasa Inoue
The authors introduce OFDB, a formula-driven supervised learning approach that stores one representative fractal image per category and creates variation through training-time augmentation. They evaluate 2D and 3D datasets on transfer tasks, including ImageNet-1k fine-tuning. With ViT-Base, 3D-OFDB-21k reaches 82.7, versus 81.8 for ImageNet-21k, despite using 21,000 rather than 14 million images; ViT-T scores trail ImageNet-21k.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.