Reproduce “CoLES: Contrastive Learning for Event Sequences with Self-Supervision” on OpenResearch

About this paper

Dmitrii Babaev, Ivan Kireev, Nikita Ovsov, Mariya Ivanova, Gleb Gusev, Ivan Nazarov, Alexander Tuzhilin

CoLES introduces a self-supervised contrastive learning method for generating high-quality embeddings from discrete event sequences, leveraging a "Random Slices" augmentation strategy. This approach enhances downstream machine learning tasks, yielding performance improvements and generating hundreds of millions of dollars yearly in financial gains for a major financial services company.

Key insights

• Real-world discrete event sequences exhibit "periodicity and repeatability," allowing for the creation of effective self-supervision signals through random sub-sequence extraction.
• The "Random Slices" augmentation strategy effectively captures these inherent patterns, outperforming other sampling methods for event sequence representation learning.
• Pre-trained CoLES embeddings, especially when fine-tuned, consistently improve performance on downstream tasks, particularly in scenarios with limited labeled data.
• CoLES significantly reduces the need for extensive manual feature engineering, providing high-quality, ready-to-use representations for complex event data.

Summary by alphaXiv.