Reproduce “Solar-VLM: Multimodal Vision-Language Models for Augmented Solar Power Forecasting” on OpenResearch

About this paper

Hang Fan, Haoran Pei, Runze Liang, Weican Liu, Long Cheng, Wei Wei

Solar-VLM introduces a framework for multi-site photovoltaic (PV) power forecasting, integrating temporal observations, satellite imagery, and textual weather information through large vision-language models and spatiotemporal graph attention. The model consistently outperformed various baselines, achieving an 8.5% reduction in MSE and an 18.3% reduction in MAE for a 48-step forecasting horizon compared to the next best model, while exhibiting improved prediction stability.

Key insights

• Jointly integrating heterogeneous modalities (temporal, visual, textual) with explicit spatiotemporal modeling significantly enhances multi-site PV power forecasting accuracy, particularly for longer prediction horizons.
• Leveraging pretrained large vision-language models for feature extraction from visual and textual data provides rich semantic context, reducing the need for extensive domain-specific pretraining for these modalities.
• The combination of structured graph learning for spatial dependencies and adaptive cross-site attention for multimodal fusion is crucial for effectively capturing complex inter-site relationships in distributed PV systems.

Summary by alphaXiv.