Reproduce “Algebraformer: A Neural Approach to Linear Systems” on OpenResearch

About this paper

Pietro Sittoni, Francesco Tudisco

A neural architecture named Algebraformer processes linear systems, including ill-conditioned ones, by encoding matrix columns and right-hand side entries as tokens for a Transformer. The model achieves robust solutions under noisy conditions, demonstrates efficient transfer learning for different boundary value problems, and accelerates iterative numerical methods such as Newton's method.

Key insights

• Transformers can effectively process matrix-based numerical problems like linear systems by employing a structured, column-wise input encoding, achieving practical scalability.
• An end-to-end neural solver provides a general and robust alternative to classical methods, reducing the need for specialized numerical expertise and manual tuning.
• Deep learning models can implicitly learn to handle ill-conditioning and noisy inputs, yielding solutions that are more robust than some traditional exact methods.

Summary by alphaXiv.