Reproduce “Neural Scaling Laws for Boosted Jet Tagging” on OpenResearch

About this paper

Matthias Vigl, Nicole Hartman, Michael Kagan, Lukas Heinrich

This research investigates neural scaling laws for boosted jet classification in High Energy Physics, demonstrating that cross-entropy loss scales as a power law with model capacity and dataset size, similar to general machine learning models. The study identifies an irreducible loss of 0.32 and quantifies the diminishing returns of data repetition compared to training on unique, larger datasets.

Key insights

• Neural scaling laws, characterized by power-law dependencies of loss on model capacity and dataset size, are applicable to boosted jet classification in HEP, aligning with principles observed in other machine learning domains.
• While data repetition can improve performance, it is less compute-efficient than training on unique, larger datasets, requiring significantly more compute for equivalent performance due to diminishing returns.
• The choice of input features (e.g., full features vs. kinematics) and particle multiplicity fundamentally determines the irreducible loss ($L_\infty$) and thus the ultimate performance ceiling, while the data scaling exponent remains relatively stable.

Summary by alphaXiv.