Reproduce “VNIbCReg: VICReg with Neighboring-Invariance and better-Covariance Evaluated on Non-stationary Seismic Signal Time Series” on OpenResearch

About this paper

Daesoo Lee, Erlend Aune, Nadège Langet, Jo Eidsvik

VNIbCReg, a self-supervised learning method, effectively learns representations from non-stationary seismic signal time series by integrating VICReg's feature decorrelation with Temporal Neighborhood Coding's temporal context awareness. On the Åknes rock slope dataset, this approach achieved a linear evaluation accuracy of 0.716 and robust fine-tuning performance, reaching 0.717 with only 5% of labeled data.

Key insights

• Combining robust feature decorrelation and variance regularization with explicit temporal neighborhood awareness is crucial for effective self-supervised learning on non-stationary time series.
• Employing temporally consistent 'Neighboring Crop' data augmentation significantly improves representation learning compared to random cropping for non-stationary data.
• Refining the TNC loss by explicitly encouraging correlation between neighboring samples and decorrelation between non-neighboring samples enhances the linear separability and quality of learned features.

Summary by alphaXiv.