Reproduce “Output-Aware Block Influence: Jacobian-Lens Weighting for Depth Pruning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kyle Lemoi, Jonathan Yong, Shubhankar Tripathy, Priestley Fernandes, Anirban Majumder, Zarreen Reza
The authors introduce J-BI, an output-aware block-pruning score that transports each block’s input and output toward the final residual stream before measuring similarity. It retains ShortGPT’s angle-based criterion while accounting for downstream effects. With eight blocks removed from Gemma-3-12B-IT, J-BI Mag reaches 50.6% filtered accuracy versus BI’s 31.3%, but remains below dense performance; throughput was not measured.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.