Reproduce “Learning with Fenchel-Young Losses” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mathieu Blondel, André F. T. Martins, Vlad Niculae

This work introduces Fenchel-Young losses, a generic and principled framework for constructing convex loss functions from "regularized prediction functions" via Fenchel duality. It unifies a wide array of existing losses, from regression to structured prediction, and establishes a theoretical equivalence between output sparsity and the existence of separation margins for losses derived from generalized entropies.

Key insights

  • Fenchel-Young losses are inherently convex with respect to the prediction scores, enabling efficient optimization regardless of the complexity of the output regularization.
  • The framework provides a systematic paradigm for loss design, where desired output properties (e.g., sparsity, geometric domain constraints) guide the automatic generation of a corresponding convex loss.
  • A fundamental theoretical link exists between the sparsity of a predictive distribution and the presence of a "separation margin" for the corresponding Fenchel-Young loss, explaining why certain losses exhibit sparse outputs and margins while others do not.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.