Reproduce “Physics-Guided Transformer (PGT): Physics-Aware Attention Mechanism for PINNs” on OpenResearch

About this paper

Ehsan Zeraatkar, Rodion Podorozhny, Jelena Tešić

Researchers at Texas State University developed the Physics-Guided Transformer (PGT), which embeds physical principles derived from Green's functions directly into its attention mechanism and uses a FiLM-modulated SIREN decoder. This architecture demonstrates improved accuracy in reconstructing continuous physical fields from sparse observations, achieving a 38-fold reduction in L2 error compared to PINN on a 1D heat equation and balancing high data fidelity with strong physical consistency for 2D Navier-Stokes equations.

Key insights

• Directly embedding physical structure (via Green's functions) into the self-attention mechanism provides a strong inductive bias, significantly enhancing reconstruction accuracy by shaping latent representations towards physically coherent solutions.
• Physics-guided attention and PDE residual loss are complementary; the former improves representation and stability, while the latter ensures pointwise compliance, enabling the model to achieve both high data fidelity and strong physical consistency.
• The FiLM-modulated SIREN decoder adaptively controls its spectral response based on global context, which is critical for accurately reconstructing high-frequency details prevalent in complex physical fields.

Summary by alphaXiv.