Reproduce “Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruixiang Tang, Dehan Kong, Longtao Huang, Hui Xue
Large Language Models demonstrate a propensity for "shortcut learning" during in-context learning, prioritizing spurious correlations within prompts over robust semantic understanding, with larger models exhibiting increased susceptibility to these shortcuts. For instance, GPT2-large experienced a 41.45% performance drop on the MR dataset when a common word shortcut was introduced.
Key insights
- LLMs act as "lazy learners" during ICL, readily exploiting shortcuts in prompts, often leading to significant performance degradation on anti-shortcut examples.
- A "reverse scaling phenomenon" was discovered, where larger LLMs exhibited a greater susceptibility to shortcut learning, even while showing improved performance on clean tasks, challenging assumptions about model scaling and generalization.
- Shortcut learning is robust across various trigger types and prompt formats, and can be detected using interpretability techniques like LIME, which showed models prioritizing shortcut tokens.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.