Reproduce “On the Robustness of Answer Formats in Medical Reasoning Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pittawat Taveekitworachai, Natpatchara Pongjirapat, Krittaphas Chaisutyakorn, Piyalitt Ittichaiwong, Tossaporn Saengja, Kunat Pipatanakul
The authors test medical models across multiple-choice, open-ended, and ranked-list formats. They define answer-format robustness as structural compliance, separate from correctness, although unparsable outputs also count as incorrect. With a shared Qwen2.5-7B-Instruct backbone, QA-trained RFT has 9.79% List compliance versus 59.73% for SFT; judge-based MRR raises List-RFT QA compliance from 40.19% to 65.92%. They do not establish clinical readiness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.