Reproduce “A Distributional Approach to Controlled Text Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Muhammad Khalifa, Hady Elsahar, Marc Dymetman

The GDC framework unifies pointwise and distributional control for text generation by defining an optimal energy-based model and using an adaptive policy gradient to approximate it. This approach effectively mitigates social biases in language models and enhances generation diversity and quality compared to existing methods.

Key insights

  • A unique, optimal target distribution (EBM) for controlled generation can be formally derived using Information Geometry, satisfying constraints with minimal deviation from the original LM.
  • Decoupling the definition of the target distribution from the process of sampling provides a more transparent and diagnosable approach to controlled text generation.
  • An adaptive policy gradient method can effectively train an autoregressive model to approximate complex, theoretically optimal EBMs, preserving generation quality and diversity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.