Reproduce “At First Sight: Zero-Shot Classification of Astronomical Images with Large Multimodal Models” on OpenResearch

About this paper

Dimitrios Tanoglidis, Bhuvnesh Jain

General-purpose Vision-Language Models effectively perform zero-shot classification of astronomical images using natural language prompts, bypassing the need for specific training on domain data. A leading model achieved over 80% accuracy in distinguishing low-surface-brightness galaxies from artifacts and 67% for multi-class galaxy morphology, demonstrating a viable approach to mitigate data scarcity in astronomy.

Key insights

• Large Multimodal Models can effectively perform zero-shot classification of astronomical images, leveraging their general visual and linguistic understanding to interpret domain-specific prompts.
• Natural language descriptions alone can serve as sufficient guidance for generalist AI models to categorize complex astronomical phenomena, addressing challenges associated with labeled data scarcity.
• This approach suggests a paradigm shift in astronomical data analysis, enabling rapid initial classification and exploration without extensive task-specific model development or data labeling efforts.

Summary by alphaXiv.