Reproduce “SHIFT-JEV: Certified Abstention and Gain-Gated Continual Adaptation for LLM-Based Decision Models under Distribution Shift” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Muhammad Atif Saeed, Azka Atiq, Adarsh Kumar, Akhtar Jamil, Uzair Iqbal, Roland Aydin
SHIFT-JEV maps fixed-choice language-model decisions to probabilities, then uses learned risk scores, abstention, monitoring, and cautious updates. On Qwen3-0.6B, its uncertainty score with pooled certification exceeded a 10% target risk in 5 of 27 tests, versus 15 for max-probability. Adaptation averaged 70.48% accuracy on 0.6B, but gains did not generalize to 4B; guarantees require exchangeable calibration and deployment data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.