Reproduce “Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gregor Wiedemann, Eugen Ruppert, Raghav Jindal, Chris Biemann
This research enhances offensive language detection for German Twitter data by fine-tuning a BiLSTM-CNN with transfer learning and user cluster information. It demonstrates that unsupervised pre-training using LDA topics on a large corpus, combined with a 'Single Bottom-up Unfreezing' strategy, substantially improves performance, especially when integrating thematic user cluster features.
Key insights
- Unsupervised topic transfer using LDA on a large, unlabeled in-domain corpus yielded the most effective pre-training for offensive language detection in German.
- The 'Single Bottom-up Unfreezing' strategy proved most effective for fine-tuning, suggesting that training lower, general layers before higher, specific ones is beneficial.
- Incorporating thematic user cluster information significantly boosted model generalization, improving F1-scores by over 10 percentage points for binary classification on unseen user accounts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.