Reproduce “PAON: A New Neuron Model using Padé Approximants” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Onur Keleş, A. Murat Tekalp
Researchers developed PAON (Padé Approximant-based Operational Neuron), a new neuron model that leverages Padé approximants to integrate linear and nonlinear operations within a single unit. Applied to single-image super-resolution, PadéNet, incorporating PAONs, demonstrated improved PSNR and SSIM metrics and superior visual quality compared to ResNet baselines and other operational neuron models.
Key insights
- Padé approximants, as rational functions, offer improved approximation capabilities over wider domains compared to polynomial or Taylor series-based methods, enhancing neuron expressiveness.
- Integrating linear combination and nonlinear activation directly within a single Padé-based neuron creates a more expressive computational unit than traditional separated approaches.
- An adaptive "Shifter" module for input transformation can further optimize the effectiveness of Padé approximations by mapping inputs to more favorable representation regions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.