Reproduce “WARP Logic Neural Networks” on OpenResearch

About this paper

Lino Gerlach, Thore Gerlach, Liv Våge, Elliott Kauffman, Isobel Ojalvo

WARP Logic Neural Networks introduce a framework utilizing Walsh-Hadamard transform for compact and differentiable Boolean function parametrization, combined with stochastic smoothing and residual initialization, to enable scalable and stable training of deep logic networks. This approach, developed by researchers from Princeton University and the ESA Advanced Concepts Team, achieves superior performance and hardware efficiency by effectively addressing the limitations of prior logic neural networks, yielding higher accuracy with fewer binarization bits.

Key insights

• The Walsh-Hadamard transform offers the most parameter-efficient and fully expressive way to represent any Boolean function, encompassing prior LNNs as restricted cases.
• Stochastic smoothing and residual initialization are effective in mitigating the 'discretization gap' and improving training stability for deep logic networks.
• Dynamically learnable input binarization thresholds significantly enhance accuracy with fewer bits, improving hardware efficiency and adaptability to diverse datasets.

Summary by alphaXiv.