Reproduce “Efficient Training of Deep Neural Operator Networks via Randomized Sampling” on OpenResearch

About this paper

Sharmila Karumuri, Lori Graham-Brady, Somdatta Goswami

Researchers at Johns Hopkins University introduce a randomized sampling technique for training Deep Operator Networks (DeepONets) that significantly reduces training time and improves model generalization. This method achieves comparable or superior accuracy while cutting computational costs by sampling a subset of output evaluation points during each training iteration.

Key insights

• Randomized sampling for trunk network inputs in DeepONet training leads to significantly reduced training time and often improves model generalization.
• Smaller effective batch sizes, achieved through sampling, help the model converge to 'flatter minima' in the loss landscape, resulting in improved performance on unseen data.
• The increased variance in loss with randomized sampling is hypothesized to aid in more effective exploration of the parameter space, contributing to lower overall test loss.

Summary by alphaXiv.