Reproduce “Neuro Symbolic Knowledge Reasoning for Procedural Video Question Answering” on OpenResearch

About this paper

Basura Fernando, Thanh-Son Nguyen, Hong Yang, Tzeh Yuan Neoh, Hao Zhang, Ee Yeo Keat

This research introduces Knowledge Module Learning (KML), a neuro-symbolic framework, for robust and interpretable procedural video question answering. KML learns neural knowledge modules for relations within a Procedural Knowledge Graph (PKG) and composes them into executable reasoning programs, achieving up to 78.1% accuracy on the new PKR-QA benchmark and outperforming existing neuro-symbolic and VLM baselines.

Key insights

• Learning neural knowledge modules for explicit relations within a knowledge graph enables scalable, interpretable, and uncertainty-aware multi-hop reasoning, effectively bridging neural generalization with symbolic structure.
• Decoupling program synthesis (by LLMs) from program execution (by learned neural modules) allows for flexible reasoning logic while maintaining adherence to domain-specific knowledge and providing traceable intermediate states.
• The theoretical analysis demonstrates that KML's contrastive loss ensures effective separation of valid entities and that Lipschitz continuity of modules controls error propagation in multi-step reasoning.

Summary by alphaXiv.