Reproduce “Japanese SimCSE Technical Report” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda
Nagoya University researchers developed and extensively evaluated Japanese SimCSE models, identifying optimal configurations and releasing state-of-the-art sentence embeddings that surpass previous public and some proprietary models on Japanese Semantic Textual Similarity (STS) tasks, achieving an average of 81.91 Spearman correlation.
Key insights
- Supervised SimCSE, particularly when trained with JSNLI, generally yields superior performance on STS tasks compared to unsupervised methods, though well-tuned unsupervised models can achieve competitive results.
- Waseda University's RoBERTa (nlp-waseda/roberta-large-japanese) emerged as the most robust base model across both supervised and unsupervised settings.
- For unsupervised training, Wikipedia-based corpora (Wiki40B, Wikipedia) proved more effective than web corpora like BCCWJ and CC100.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.