Reproduce “MORPH: PDE Foundation Models with Arbitrary Data Modality” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mahindra Singh Rautela, Alexander Most, Siddharth Mansingh, Bradley C. Love, Alexander Scheinker, Diane Oyen, Nathan Debardeleben, Earl Lawrence and 1 more

MORPH learns from heterogeneous PDE data with a shared representation and component-wise processing, field fusion, and axial attention. Pretrained only on two-dimensional incompressible flow, it was evaluated with frozen weights on six fluid-related targets across one to three dimensions and had positive Gap-Closure Ratio on all six. This means improvement over an untrained baseline, not matching target-trained models, with weaker transfer on turbulence and multiphysics targets.

Key insights

  • A unified data format combined with a modular, shape-agnostic transformer architecture enables effective learning across highly heterogeneous scientific datasets, facilitating universal PDE surrogacy.
  • Factorized axial attention provides a computationally efficient mechanism for capturing global spatiotemporal dependencies in high-dimensional data, outperforming both full and sparse self-attention in accuracy-compute trade-offs.
  • Parameter-efficient fine-tuning methods like LoRA can effectively adapt large PDE foundation models to specific downstream tasks, significantly reducing the computational and data burden while recovering most performance gains.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.