Reproduce “Deep Representation Learning for Prediction of Temporal Event Sets in the Continuous Time Domain” on OpenResearch

About this paper

Parag Dutta, Kawin Mayilvaghanan, Pratyaksha Sinha, Ambedkar Dukkipati

The Indian Institute of Science developed TESET, a framework grounded in Temporal Point Processes, which predicts the continuous-time occurrence and composition of event sets rather than individual events. The model achieved absolute improvements of 0.12 in Dice Score and 0.34 in Mean Absolute Error for event set and time prediction, respectively, on the Synthea dataset compared to prior methods.

Key insights

• Learning contextual embeddings through self-supervised contrastive learning effectively captures the interdependencies between events within a set, which is crucial for accurate set prediction.
• Custom SpatioTemporal Encodings, integrating positional and temporal information, allow a Transformer architecture to model sequences of event sets in continuous time while accounting for both intra-set (unordered events) and inter-set (non-uniform time intervals) relationships.
• Utilizing a Probabilistic Bayesian Transformer with distributional prediction heads provides stable training and enables the model to output continuous intensity distributions for event sets and their times, offering more comprehensive predictions than point estimates.

Summary by alphaXiv.