Reproduce “Inferring Latent Intentions: Attributional Natural Language Inference in LLM Agents” on OpenResearch

About this paper

Xin Quan, Jiafeng Xiong, Marco Valentino, André Freitas

This research introduces Attributional Natural Language Inference (Att-NLI) to enable large language model (LLM) agents to infer latent intentions in multi-agent environments. It presents Undercover-V, a verifiable social deduction game, and the Attributional Score metric to evaluate Att-NLI, demonstrating that neuro-symbolic agents achieve an average spy win rate of 17.08%, a 24.22% improvement over standard Att-NLI agents.

Key insights

• Formalizing intention inference as Attributional NLI and integrating principles from social psychology provides a robust framework for LLMs to achieve deeper, human-like social intelligence in multi-agent settings.
• The neuro-symbolic approach, by combining LLMs with external theorem provers like Isabelle/HOL, significantly enhances the logical soundness and robustness of intention inference, leading to more rational agent behavior.
• Verifiable game environments that constrain agent behavior (e.g., by prohibiting lying) are essential for rigorously evaluating and comparing complex, intention-aware reasoning capabilities in multi-agent systems.

Summary by alphaXiv.