Reproduce “STEC: See-Through Transformer-based Encoder for CTR Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Serdarcan Dilbaz, Hasan Saribas
STEC, a transformer-based encoder, unifies multiple feature interaction learning strategies and extracts intrinsic bilinear interactions from scaled dot-product attention for click-through rate prediction. The model achieves state-of-the-art accuracy on public datasets and demonstrated a 0.72% increase in CTR during an industrial online A/B test, while maintaining high computational efficiency.
Key insights
- Bilinear interactions are inherently present within the commonly used scaled dot-product attention mechanism and can be explicitly extracted.
- Directly fusing bilinear interactions from multiple architectural levels, rather than solely relying on the final layer, enhances prediction performance by allowing lower-order interactions to contribute.
- A model can achieve state-of-the-art CTR prediction accuracy and maintain high computational efficiency by strategically leveraging and unifying different feature interaction learning strategies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.