Reproduce “ReAGent: A Model-agnostic Feature Attribution Method for Generative Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhixue Zhao, Boxuan Shan
ReAGent, a model-agnostic feature attribution method for generative language models, recursively computes token importance by observing changes in predictive likelihoods after contextually informed input perturbations. This approach consistently demonstrates superior faithfulness compared to baselines across various decoder-only LMs and generation tasks, proving suitable for black-box models while significantly reducing computational overhead.
Key insights
- Contextually informed input perturbations, rather than simple masking or random replacements, are crucial for efficient and faithful assessment of token importance in generative LMs.
- Model-agnostic methods relying solely on forward passes can achieve high faithfulness for black-box generative LMs, overcoming the limitations of internal-access or gradient-based approaches.
- Evaluating FA faithfulness for generative models requires comparing full predictive distributions over the vocabulary (e.g., using Hellinger distance) instead of relying on single token likelihoods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.