Reproduce “Training Autoregressive Language Models on Rounded, Bounded, and Censored Numerical Labels via Exact Likelihood” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingquan Yan, Yuwei Miao, Yang Xie, Junzhou Huang
The authors introduce boundary marginalization, training language models on probabilities of all numerical values consistent with a report, using digit tokens rather than guessed point labels. In GPU-kernel latency tests, it achieved CRPS of 8.262 ± 0.299 × 10⁻³ ms, versus 8.990 ± 0.780 × 10⁻³ ms for an event-aware histogram head under the same supervision across three paired seeds.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.