Reproduce “Toward Explaining Large Language Models in Software Engineering Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Antonio Vitale, Khai-Nguyen Nguyen, Denys Poshyvanyk, Rocco Oliveto, Simone Scalabrino, Antonio Mastropaolo
FeatureSHAP is a model-agnostic, black-box explainability framework that attributes Large Language Model outputs in software engineering tasks to high-level semantic features rather than individual tokens. Evaluated across code generation and summarization, it reliably assigned negligible importance to irrelevant input and received high ratings for usefulness and faithfulness from software practitioners.
Key insights
- Attributing LLM outputs to semantically meaningful, high-level features is more interpretable and useful for software engineers than traditional token-level explanations.
- Model-agnostic and black-box compatible XAI is crucial for real-world adoption, as many powerful LLMs are proprietary and only accessible via APIs.
- The modular design of FeatureSHAP (splitter, modifier, comparator) allows for customization and extensibility to various software engineering tasks, enhancing its versatility and applicability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.