Reproduce “AutoTask: Task Aware Multi-Faceted Single Model for Multi-Task Ads Relevance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shouchang Guo, Sonam Damani, Keng-hao Chang
Microsoft AI researchers developed AutoTask, a multi-faceted attention model that re-frames multi-task ad relevance as a language modeling problem, achieving strong generalization to unseen ad scenarios and competitive performance with task-specific models. The model consistently outperformed a generalized baseline DNN, notably increasing ROC AUC for unseen tasks like "Clinics" from 54.381% to 75.127%.
Key insights
- Re-framing multi-task feature combination as a "language modeling" problem with token and position embeddings can improve convergence and model performance in non-NLP domains.
- Introducing a specific "task ID encoding" directly into the feature representation is crucial for effectively instilling task awareness and significantly boosting generalization capabilities, especially to unseen tasks.
- A multi-faceted architecture, combining dedicated task-aware feature processing and cross-task interaction modeling via auto-regressive attention, effectively balances shared and distinct characteristics across diverse tasks within a single model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.