Reproduce “OptEmbed: Learning Optimal Embedding Table for Click-through Rate Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fuyuan Lyu, Xing Tang, Hong Zhu, Huifeng Guo, Yingxue Zhang, Ruiming Tang, Xue Liu
OptEmbed, from McGill University and Huawei Noah's Ark Lab, presents a three-phase framework for learning optimal embedding tables for click-through rate prediction models, simultaneously pruning redundant embeddings, optimizing feature-wise dimensions, and ensuring hardware compatibility. The system achieved statistically significant AUC increases of up to 0.15% on KDD12 and Avazu datasets with approximately 50% parameter reduction, and substantial memory savings on Criteo while maintaining performance.
Key insights
- An 'optimal' embedding table requires simultaneously addressing redundant embeddings, flexible dimensionality, and hardware compatibility.
- A unified framework for pruning and dimension search can achieve better trade-offs between performance and memory efficiency than isolated approaches.
- The L1-norm of embeddings can serve as an effective, trainable importance indicator for pruning, enabling dynamic sparsity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.