Reproduce “CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao and 2 more
This research introduces CityEQA, the first embodied question answering benchmark dataset for complex 3D urban environments, featuring open-vocabulary questions that require active exploration. The proposed hierarchical LLM agent, PMA, outperforms other automated baselines in question answering accuracy and exploration efficiency, achieving 60.73% of human performance.
Key insights
- Hierarchical LLM-driven planning and cognitive mapping are effective for long-horizon embodied question answering tasks in complex urban environments.
- An object-centric cognitive map is crucial for active perception and distinguishing similar landmarks in ambiguous city scenes.
- Fine-grained observation adjustments by a dedicated collector module significantly improve answer accuracy by refining the agent's view of target objects.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.