Reproduce “Slowing Learning by Erasing Simple Features” on OpenResearch

About this paper

Lucia Quirke, Nora Belrose

Researchers at EleutherAI explored how neural networks learn when low-order statistical features are removed from image classification datasets, introducing novel quadratic concept erasure methods. They observed that linear feature removal consistently slowed learning, whereas quadratic erasure yielded complex outcomes, including a "backfiring" effect where advanced models initially struggled but then leveraged inadvertently injected higher-order information to achieve lower final losses.

Key insights

• The consistent increase in learning difficulty (prequential MDL) and final loss from linear erasure (LEACE) across architectures highlights the fundamental importance of linear features for neural network efficiency.
• Quadratic concept erasure methods exhibit complex, method-dependent effects, with QLEACE demonstrating a "backfiring" phenomenon where advanced models exploit inadvertently injected higher-order information after an initial period of high loss.
• ALF-QLEACE, by ablating directions of highest variance differences, sometimes results in a smaller learning difficulty increase than linear erasure, suggesting it could inadvertently simplify data or act as a form of data augmentation.

Summary by alphaXiv.