Reproduce “Controllable and Diverse Text Generation in E-commerce” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huajie Shao, Jun Wang, Haohong Lin, Xuezhou Zhang, Aston Zhang, Heng Ji, Tarek Abdelzaher
Researchers from the University of Illinois at Urbana-Champaign and Alibaba Group developed Apex, a framework integrating Conditional Variational Autoencoders, a Proportional-Integral (PI) controller, and Transformer architecture for controllable and diverse text generation. This approach precisely manages the diversity-accuracy trade-off and keyword ordering, resulting in a 13.17% improvement in Click-Through Rate for product descriptions on Taobao.
Key insights
- Closed-loop feedback control, specifically a PI controller, can robustly solve the KL-vanishing problem in CVAEs and dynamically tune the diversity-accuracy trade-off, moving beyond heuristic annealing schedules.
- Integrating explicit keyword and keyword order embeddings into a Transformer-based CVAE allows for highly accurate control over the sequential presentation of information in generated text.
- A well-tuned KL-divergence, rather than simply maximizing it, is crucial for achieving an optimal balance between perceived fluency/relevance and the desired level of text diversity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.