Reproduce “Beyond the Camera: A Tri-Modal Driving World Model for Future RGB, Thermal, and LiDAR Forecasting in Adverse Weather” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Faysal Mehrab Chowdhury, Md. Mahfuzur Rahman, Ahmed Rafi Hasan, Kishor Datta Gupta, Mohd Ariful Haque, Savannah Shannon

The authors adapt Cosmos-Predict2.5 to forecast synchronized RGB, thermal, and LiDAR futures by placing the sensor streams in one tiled video, with LiDAR depth densified for video encoding. Across DSERT-RoLL validation, tri-sensor LiDAR mean absolute error at two seconds is 4.68 meters versus 6.81 meters for single-sensor forecasting. Camera metrics incur small losses, and persistence remains better at near range.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.