Reproduce “Random Text Perturbations Work, but not Always” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhengxiang Wang

This study critically evaluates random text perturbations for data augmentation in Natural Language Processing, demonstrating that their effectiveness is not universal but highly dependent on the amount of original training data and the task's semantic sensitivity. It shows these perturbations can be detrimental with limited data and only provide gains, primarily in precision, with sufficiently large datasets.

Key insights

  • Random text perturbations are not a universally effective data augmentation approach for NLP; their utility is highly task-specific and critically dependent on the quantity of original training data.
  • Augmenting small datasets with these perturbation techniques can be detrimental, leading to a consistent decrease in model performance.
  • When augmentation does yield benefits, performance improvements are primarily driven by increased precision scores, suggesting models make fewer false positive predictions with sufficient data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.