Reproduce “ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy” on OpenResearch

About this paper

Kian Kenyon-Dean, Zitong Jerry Wang, John Urbanik, Konstantin Donhauser, Jason Hartford, Saber Saberian, Nil Sahin, Ihab Bendidi and 5 more

Researchers at Recursion developed a framework for biological representation learning in cell microscopy, combining a curated dataset (Phenoprints-16M) with a 1.9 billion-parameter Vision Transformer and a method to select optimal intermediate layers. This approach achieved state-of-the-art performance, including a 60% improvement in genetic perturbation separability and a 42% relative improvement in compound-gene activity prediction.

Key insights

• Data quality and curation, even when reducing dataset size, can lead to superior biological representations compared to simply using larger, uncurated datasets.
• Intermediate layers of large Vision Transformers often contain more biologically relevant and linearly separable features than their final encoder layers.
• Scaling laws observed in natural language processing and computer vision extend to complex biological microscopy data, suggesting further performance gains with increased model scale and computation.

Summary by alphaXiv.