Reproduce “When Trees Are Not Enough: Learning Mixed-Topology Feature Graphs with Adaptive Graph Sparse Autoencoders” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaozuo Shen, Yifei Cai, Tian Tan, Rui Ning, Chunsheng Xin, Hongyi Wu

AG-SAE learns a graph of sparse autoencoder features by testing complete parent sets against null, subset, and alternative explanations, then using accepted relations as training feedback and repeatedly revising the dictionary and graph. On Gemma-2-2B MiniPile activations, 69.56% of sampled relations passed predictive, reconstruction, and blinded semantic tests, versus 40.63% for full-space Vanilla SAE post-hoc; this is not ground-truth proof.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.