Reproduce “Statistical comparisons of time-series feature sets on classification tasks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Trent Henderson, Ben D. Fulcher

Henderson and Fulcher systematically evaluated the classification performance of six open-source univariate time-series feature sets, along with simple baselines, across 124 real-world problems. Their magnitude-sensitive benchmarking methodology revealed broad similarities in performance among complex feature sets but identified specific problem contexts where certain sets offered distinct advantages or disadvantages, with tsfresh generally performing strongest.

Key insights

  • Despite significant differences in size and composition, open-source time-series feature sets exhibit broad similarities in overall classification performance, with statistical ties dominating pairwise comparisons.
  • Problem-specific strengths and weaknesses exist for individual feature sets, where unique feature compositions can lead to substantial performance advantages or disadvantages on particular datasets.
  • Simple, interpretable baseline feature sets (e.g., based on quantiles or Fourier coefficients) can be highly competitive and even outperform more complex, dedicated feature sets on specific time-series classification tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.