Reproduce “Learning from Points and Patches: A Pre-trained Hybrid Attention Transformer for Time Series Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peiwang Tang, Cheng HE, Yupeng Ye, Weitai Zhang, Qinghua Sun, Defu Lian
TimeHAT trains a time-series model with point- and patch-level masking for contextual reconstruction and future-tail prediction. Its encoder combines bidirectional exchange among historical tokens with directional attention in deeper layers, protecting targets from future information. In examples across five datasets, point masking preserves local detail more than block masking. On ETTh2, pre-training yields MSE 0.300 versus 0.321 for causal-extrapolation-only, not a cross-benchmark average.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.