Reproduce “Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Vladimir Araujo, Marie-Francine Moens, Tinne Tuytelaars

Researchers at KU Leuven developed L2R (Learning to Route), a framework for continual learning with language models that utilizes isolated adapter training and a memory-based router employing a Gumbel-sigmoid distribution. This approach enabled more effective dynamic composition of parameter-efficient fine-tuning (PEFT) modules, achieving superior state-of-the-art performance across various continual learning benchmarks, including matching or exceeding upper-bound performance in some task-incremental learning scenarios.

Key insights

  • Training PEFT modules in isolation significantly improves their task-specific specialization by eliminating cross-talk from previously learned modules.
  • A learned routing function, trained on a small memory of past examples, can intelligently compose multiple adapters to achieve superior generalization compared to simple similarity-based selection.
  • Utilizing a Gumbel-sigmoid distribution for router output is crucial for enabling 'task-level modularity,' allowing multiple relevant adapters to co-activate and contribute, which outperforms competitive Softmax-based routing.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.