Reproduce “Query as Anchor: Scenario-Adaptive User Representation via Large Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahao Yuan, Yike Xu, Jinyong Wen, Baokun Wang, Ziyi Gao, Xiaotong Lin, Yun Liu, Xing Fu and 4 more
The Ant Group developed the "Query as Anchor" (Q-Anchor) framework, which leverages a large language model to generate scenario-adaptive user representations by conditioning embeddings on explicit queries. This approach achieved state-of-the-art performance across 10 real-world Alipay business tasks and yielded significant improvements in online A/B tests, demonstrating efficient multi-scenario deployment.
Key insights
- Conditioning user embeddings on explicit queries enables a single LLM-based model to dynamically adapt to diverse downstream scenarios, providing a 'one-model-for-many' solution.
- The combination of an industrial-scale pre-training dataset (`UserU`) and a hierarchical encoder successfully integrates language-centric LLMs with sparse, multimodal user behavioral data.
- A 'Scaling Paradox' was observed for discriminative user embedding tasks, where larger LLM backbones did not monotonically improve performance, suggesting different optimal scaling laws compared to generative tasks.
- Soft prompt tuning effectively specializes the embedding space for specific tasks and provides interpretability by dynamically shifting the model's attention to scenario-relevant modalities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.