Reproduce “The Woman Worked as a Babysitter: On Biases in Language Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng

Researchers from ISI at USC and UCLA systematically analyzed biases in open-domain language generation, introducing 'regard' as a more precise bias metric than sentiment. Their analysis revealed that large language models like GPT-2 exhibit pronounced gender, racial, and sexual orientation biases in generated text, with 'regard' often highlighting more negative associations for certain demographics than sentiment analysis alone.

Key insights

  • "Regard" is a distinct and more appropriate metric for bias than general sentiment, especially in contexts like occupations where sentiment analysis may fail to capture nuanced social connotations.
  • Transfer learning with BERT models enables effective automatic classification of "regard" in text, achieving over 80% accuracy even with a relatively small annotated dataset.
  • Relying solely on sentiment analysis can underestimate the true magnitude of biases present in natural language generation outputs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.