Reproduce “Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting” on OpenResearch

About this paper

Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu

TempoWave, a multi-wavelet number embedding approach from the University of Southern California, enables Large Language Models (LLMs) to accurately process and forecast continuous-valued time series data. This method consistently improved forecasting performance, achieving state-of-the-art on 7 out of 10 metrics across five benchmarks with an average of 7.0% relative MAE improvement.

Key insights

• Encoding digits through structured multi-scale wavelet patterns creates numerically coherent and stable embeddings that maintain structural differences under LLM internal operations like normalization.
• A dedicated, numerically grounded embedding interface allows LLMs to capture latent numerical semantics and smooth temporal continuity more effectively, enhancing their internal numerical reasoning.
• Integrating rich contextual information (statistical features, situational context) with a faithful numerical embedding is critical for LLMs to achieve superior and robust time series forecasting performance.

Summary by alphaXiv.