Reproduce “Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Minki Kang, Sung Ju Hwang, Gibbeum Lee, Jaewoong Cho
LaPael moves paraphrasing into a model’s early layers by learning input-dependent perturbations from paraphrases and reusing them during fine-tuning on knowledge updates. The method improves question-answering F1, reaching 84.75 on ArchivalQA-syn versus 76.16 for fine-tuning alone. Its evaluations cover 7B models and knowledge-injection QA; in a direct comparison, RAG scores higher, and LaPael shows a knowledge-retention trade-off.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.