Reproduce “Explainable Mapper: Charting LLM Embedding Spaces Using Perturbation-Based Explanation and Verification Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyuan Yan, Rita Sevastjanova, Sinie van der Ben, Mennatallah El-Assady, Bei Wang
Explainable Mapper pairs an interactive mapper-graph workspace with LLM agents that propose interpretations of embedding clusters and paths, then tests their stability by perturbing examples and comparing regenerated explanations. In a BERT case study, “since” progressed from mixed uses to a middle-layer path and three later components; a “my” explanation scored 0.83, measuring repeatability, not truth.
Key insights
- The "divergence-convergence" paradigm, utilizing LLM agents for both hypothesis generation and explanation verification, offers an effective method to explore and validate complex data structures in a scalable manner.
- Combining topological data analysis with LLM-based agents provides a robust approach to extract and validate linguistic insights from high-dimensional embedding spaces, helping to mitigate LLM hallucinations.
- A defined taxonomy of explorable mapper elements (nodes, edges, paths, components, trajectories) enables a structured investigation of linguistic information at different granularities within LLM embeddings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.