Reproduce “AdaTT: Adaptive Task-to-Task Fusion Network for Multitask Learning in Recommendations” on OpenResearch

About this paper

Danwei Li, Zhengyu Zhang, Siyang Yuan, Mingze Gao, Weilin Zhang, Chaofei Yang, Xi Liu, Jiyan Yang

AdaTT, an Adaptive Task-to-Task Fusion Network, introduces a multi-level architecture with adaptive gating and residual connections to explicitly model task relationships and balance shared/specific knowledge in multitask learning for recommendations. The model consistently outperforms state-of-the-art baselines on large-scale industrial datasets, demonstrating faster convergence and superior performance across diverse task groups.

Key insights

• Adaptive, direct task-to-task knowledge fusion, driven by input-conditioned gating, is more effective than static or indirect sharing methods for improving performance and adaptability in MTL.
• The explicit residual design, which integrates distinct modules for task-specific and shared knowledge, is crucial for mitigating negative transfer and enhancing robustness across diverse tasks.
• Multi-level fusion allows for deeper knowledge abstraction and more flexible sharing patterns, with lower levels discerning broad task group relationships and higher levels enabling fine-grained, dynamic knowledge transfer.

Summary by alphaXiv.