Reproduce “Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng and 4 more
AoR (Aggregation of Reasoning) changes how language models select among sampled reasoning chains. It groups chains by answer, scores reasoning quality locally and across answers, then samples more chains when leading scores are close. On AQuA with GPT-3.5-Turbo-0301 chain-of-thought prompting, AoR(20,40) achieves 75.9% accuracy versus 68.7% for Self-Consistency(40), but this comparison does not isolate the effect of scoring.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.