Reproduce “Label Attention Network for Temporal Sets Prediction: You Were Looking at a Wrong Self-Attention” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elizaveta Kovtun, Galina Boeva, Andrey Shulga, Alexey Zaytsev
Researchers at Skolkovo Institute of Science and Technology and Sber AI Lab developed LANET, a Transformer-based model that predicts future label sets by effectively aggregating historical information into time- and set structure-aware views. This approach consistently outperformed existing models across four datasets, demonstrating up to a 179% improvement in Weighted F1 score on the Dunnhumby-Carbo dataset.
Key insights
- The way historical information is presented to the self-attention mechanism is critical for performance, highlighting the importance of input preparation.
- Label linkages and inter-set label relationships are highly influential for temporal set prediction, potentially more so than purely temporal ordering.
- Explicit temporal information contributes to model accuracy, as shown by performance drops in ablation studies when time representations were removed.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.