Reproduce “SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang
SHINE converts a passage into a reusable LoRA adapter, allowing a language model to answer questions without the passage in its prompt. It generates adapter weights from layerwise memory states using a hypernetwork, without test-time gradient optimization. On MS MARCO MQA, SHINE scores 55.6 answer F1 versus 33.0 for SFT and 69.4 for full-context prompting; results are setup-specific.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.