Reproduce “Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Akriti Jain, Anish Mulay, Divyansh Verma, Aishani Pandey, Pritika Ramu, Aparna Garimella
DECISIVE is an interactive framework that guides users through complex decisions by objectively scoring options from unstructured documents and actively learning subjective preferences using a Bayesian model. The framework achieves up to 90% Top-1 accuracy, outperforming LLM-based approaches and other decision frameworks, while efficiently converging on recommendations with an average of 5-6 questions.
Key insights
- Separating objective factual extraction from unstructured documents from the subjective modeling of user preferences allows for both robust grounding and personalization in decision support.
- A Bayesian, particle-based approach effectively models and refines latent user preferences through a series of interactive pairwise tradeoff questions.
- Optimizing question selection by maximizing Expected Information Gain on the final decision distribution, rather than general preference uncertainty, leads to more efficient and targeted user interaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.