Reproduce “Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Heming Zou, Yunliang Zang, Wutong Xu, Xiangyang Ji

Fly-CL adapts image classifiers to new classes while keeping a pretrained image encoder frozen. It expands image features with a sparse random projection, applies top-k inhibition, and trains an L2-regularized classifier with adaptive regularization. On CUB-200-2011 with ViT-B/16, it reports 4.43 seconds total training per task and 93.84% overall accuracy, a benchmark result rather than a deployment guarantee.

Key insights

  • Neurobiological principles, specifically the efficient pattern separation mechanisms observed in the fly olfactory system, can be effectively translated into artificial intelligence frameworks to address decorrelation challenges.
  • The KC MBON pathway in the fly olfactory circuit, beyond the PN KC transformation, theoretically contributes to decorrelation through a learning rule equivalent to ridge classification.
  • Optimized operations like weight-sparse random projection, adaptive GCV for regularization, and Cholesky factorization enable substantial computational speedups in continual learning while maintaining high accuracy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.