Reproduce “LLMartini: Seamless and Interactive Leveraging of Multiple LLMs through Comparison and Composition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yingtian Shi, Jinda Yang, Yuhan Wang, Yiwen Yin, Haoyu Li, Kunyu Gao, Chun Yu
Researchers from multiple institutions developed LLMartini, an interactive system designed to facilitate the comparison and composition of outputs from multiple large language models. The system significantly reduces task completion time and cognitive load while improving usability for users engaging with diverse LLM responses.
Key insights
- A foundational user study confirmed that 93% of LLM users concurrently compare and often manually integrate outputs, following a consistent 'Browse-Compare-Integrate' pattern.
- User motivations for multi-LLM use include seeking information breadth, depth, different output styles, and encountering a 'surprise' factor in responses.
- Task purpose influences comparison behavior, with subjective tasks leading to more extensive multi-model exploration than tasks with fixed answers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.