Reproduce “HOSC: A Periodic Activation with Saturation Control for High-Fidelity Implicit Neural Representations” on OpenResearch

About this paper

Michal Jan Wlodarczyk, Danzel Serrano, Przemyslaw Musialski

This research introduces HOSC (Hyperbolic Oscillator with Saturation Control), a novel periodic activation function for Implicit Neural Representations that explicitly decouples frequency selection from gradient scaling. HOSC enables stable training and high-fidelity reconstructions by providing an explicit parameter to control gradient magnitude, yielding superior results in audio and competitive performance across images, video, NeRFs, and SDFs.

Key insights

• HOSC successfully decouples the parameter controlling spectral support (ω₀) from the parameter controlling gradient magnitude (β), offering an interpretable mechanism for tuning training dynamics.
• The parameter β enables HOSC to operate in distinct regimes: a 'smooth regime' (low β) for high-dimensional signals like video, and a 'sharp regime' (high β) for tasks requiring localized, high-magnitude gradients, such as audio.
• Analysis of the empirical Neural Tangent Kernel (NTK) reveals that β controls the kernel scale and interaction strength, increasing diagonal dominance and thus shifting learning towards more self-dominated features.

Summary by alphaXiv.