Reproduce “Digital Pantheon: Simulating and Auditing Coalition Formation with LLM Agents” on OpenResearch

About this paper

Dylan Van Mulders, Matthias Bogaert, Dirk Van den Poel

This work introduces "Digital Pantheon," a framework using fine-tuned Large Language Model (LLM) agents to simulate political coalition formation, grounding their negotiation behavior in official party manifestos. It demonstrates that combining Supervised Fine-Tuning, Direct Preference Optimization, and Retrieval-Augmented Generation enables agents to maintain partisan consistency, and its explainability tools track policy lineage and quantify party influence, showing how simulated agreements align with real-world outcomes.

Key insights

• The SFT+DPO+RAG architecture effectively enabled LLM agents to maintain consistent partisan behavior and ground policy proposals in specific manifestos throughout extended multi-party negotiations.
• The MILT framework revealed that a significant majority (71.1%) of simulated policy provisions had verifiable lineage to originating party manifestos, demonstrating the effectiveness of the grounding mechanism.
• Simulated policy provisions with a traceable lineage to manifestos were significantly more likely to align with a real-world coalition agreement (G = 0.32-0.37) compared to ungrounded or hallucinated content (G = 0.164).
• The Coalition Influence Score (CIS) consistently identified the largest party as the 'winner' in simulated negotiations, with their influence largely stemming from direct retention of manifesto positions, replicating canonical coalition dynamics.

Summary by alphaXiv.