Reproduce “Towards Unifying Feature Interaction Models for Click-Through Rate Prediction” on OpenResearch

About this paper

Yu Kang, Junwei Pan, Jipeng Jin, Shudong Huang, Xiaofeng Gao, Lei Xiao

Researchers from Shanghai Jiao Tong University and Tencent developed the IPA framework to systematically unify and analyze feature interaction models for Click-Through Rate prediction. Their analysis led to the design of PFL, a novel model that demonstrated superior performance on public benchmarks and delivered 0.9% to 3.7% GMV lifts during online A/B tests in Tencent’s advertising platform.

Key insights

• More complex Interaction Functions, particularly the Projected Product, consistently yield higher performance and effectively mitigate the "embedding collapse" phenomenon by enabling embeddings to span a larger, more robust space.
• Field-wise Layer Pooling consistently outperforms Global-wise Layer Pooling, suggesting that structured, field-specific interactions are more effective than global aggregations.
• Layer Aggregators, especially Layer-wise Aggregation, demonstrate an ability to adaptively learn the inherent interaction order of data, assigning lower weights to redundant higher-order layers when the model's maximum order exceeds the true data order.

Summary by alphaXiv.