Reproduce “Adapting Auxiliary Losses Using Gradient Similarity” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar, Mehrdad Farajtabar, Razvan Pascanu, Balaji Lakshminarayanan
The authors introduce a gradient-similarity gate that admits auxiliary updates to shared parameters only when their gradients align with the main-task gradient. In a gridworld where a teacher’s +10 rewards are removed, the method reaches baseline return in about one-third the training steps and averages 3 points after 10,000 steps. With a perfect same-task teacher, it underperforms simple addition.
Key insights
- The proposed update rule is theoretically guaranteed to converge to a local minimum of the main loss, preventing divergence even if the auxiliary task is detrimental.
- Gradient cosine similarity empirically correlates with intuitive notions of task relatedness, providing a dynamic measure of an auxiliary task's current utility.
- The unweighted (binary) variant is often more robust to noisy gradient estimates, relying only on the sign of the cosine similarity rather than its precise magnitude.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.