Reproduce “SERT: A Transfomer Based Model for Spatio-Temporal Sensor Data with Missing Values for Environmental Monitoring” on OpenResearch

About this paper

Amin Shoari Nejad, Rocío Alaiz-Rodríguez, Gerard D. McCarthy, Brian Kelleher, Anthony Grey, Andrew Parnell

SERT (Spatio-temporal Encoder Representations from Transformers) and SST-ANN (Sparse Spatio-Temporal Artificial Neural Network) are deep learning models introduced for forecasting environmental sensor data while inherently handling missing values. SERT demonstrates superior spatio-temporal forecasting performance and computational efficiency, with SST-ANN offering interpretability by quantifying variable importance.

Key insights

• Transformer architectures can be effectively adapted to intrinsically handle sparse, irregularly sampled spatio-temporal environmental sensor data, overcoming limitations of traditional imputation-reliant methods.
• Simplified deep learning models, like SST-ANN, can provide crucial interpretability for complex spatio-temporal forecasting tasks, offering insights into variable importance and influence direction.
• Computational optimization and a separate location encoding strategy result in a highly efficient transformer-based model (SERT) suitable for real-time environmental monitoring.

Summary by alphaXiv.