Reproduce “Identified-Set Geometry of Distributional Model Extraction under Top-$K$ Censored API Access” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenhua Nie, ZiCheng Zhu, Jianan Wu, Binhan Luo, Haoran Zheng, Jyh-Shing Roger Jang
The authors characterize uncertainty in teacher next-token distributions exposed through top-K logits and separately measure capability transfer. They derive an exact per-position total-variation ambiguity bound and compare distillation access modes on GSM8K. Top-K, full-logit, and teacher-generated reasoning recover 11.8%, 56.5%, and 96.1% of private capability, respectively; generation is stronger access, and the bound does not cover autoregressive access.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.