Reproduce “An LLM-based Two-Stage Transformer Framework for Cross-Domain Bearing Fault Diagnosis with Limited Data” on OpenResearch

About this paper

Jinghan Wang, Feng Cheng, Wentao Wu, Hang Li, Gaoliang Peng, Tianchen Liu

Researchers from the Harbin Institute of Technology developed a two-stage Transformer framework for bearing fault diagnosis, effectively addressing cross-domain data shifts and limited labeled target data through explicit knowledge transfer. The framework achieved an average accuracy of 92.61% using only 10% of labeled target data, demonstrating a 17.24 percentage point improvement over existing state-of-the-art methods.

Key insights

• Explicit knowledge transfer mechanisms, such as initializing encoder parameters and modulating feature embeddings with fault prototypes, significantly enhance diagnostic accuracy in limited-data, cross-domain scenarios.
• A two-stage framework that first learns universal fault patterns from multiple source domains and then explicitly guides adaptation to a new target domain can effectively overcome "dual-shift" challenges.
• Combining parameter-efficient fine-tuning (LoRA) with targeted data augmentation strategies effectively preserves prior knowledge while adapting to new domains with minimal labeled data.

Summary by alphaXiv.