Reproduce “Stolen Probability: A Structural Weakness of Neural Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
David Demeter, Gregory Kimmel, Doug Downey
Research identifies a "stolen probability effect" in neural language models using dot-product softmax, demonstrating that words with embeddings interior to the convex hull of the vocabulary space are structurally limited in the maximum probability they can be assigned. This geometric constraint affects the expressiveness of NNLMs, particularly for less frequent words.
Key insights
- The dot-product softmax fundamentally limits the maximum probability assignable to words whose embeddings lie in the interior of the convex hull of the embedding space.
- This "stolen probability effect" represents a structural weakness rooted in the geometric arrangement of word embeddings, distinct from other NNLM bottlenecks.
- The expressive limitations for interior words persist across different embedding dimensions and are not fully overcome by word-specific bias terms or increased model capacity alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.