Reproduce “CaLLM: Continual Adaptation of LLMs via Non-Parametric Adapter Routing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bilge Celik, Anna Vettoruzzo, Joaquin Vanschoren
CaLLM routes data to adapters during continual fine-tuning, aiming to limit forgetting while allowing tasks to share or isolate updates. It embeds batches with a frozen model, compares them with stored prototypes, and reuses, creates, or repurposes an adapter without a learned gate. On batched TRACE-8, it scores 0.41 versus TreeLoRA’s 0.39, but results depend on benchmark and setting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.