Reproduce “ConTSG-Bench: A Unified Benchmark for Conditional Time Series Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shaocheng Lan, Shuqi Gu, Zhangzhi Xiong, Kan Ren

ConTSG-Bench introduces a unified framework for evaluating conditional time series generation models, featuring aligned multimodal datasets and a comprehensive metric suite to assess generation fidelity, condition adherence, fine-grained control, compositional generalization, and downstream utility. The benchmark identifies that current models universally struggle with precise fine-grained control and compositional generalization, revealing critical areas for future research.

Key insights

  • A significant discrepancy exists between generation fidelity and condition adherence, indicating that models producing realistic data do not inherently guarantee faithful adherence to specified conditions.
  • The difficulty of handling morphological versus conceptual conditions is dataset-dependent, with models showing varying sensitivity to semantic abstraction levels across different domains.
  • A trade-off often exists between strong condition adherence and compositional generalization, where models with better adherence tend to show larger performance degradation on novel attribute combinations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.