Reproduce “COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs” on OpenResearch

About this paper

Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, Yejin Choi

Researchers at the Allen Institute for AI developed ATOMIC2020, a new commonsense knowledge graph, and used it to fine-tune neural knowledge models (COMET). This approach significantly improved the generation of accurate, implicit commonsense knowledge for unseen entities, with COMET models trained on ATOMIC2020 outperforming few-shot GPT-3 despite having substantially fewer parameters.

Key insights

• High-quality, declarative symbolic knowledge graphs are critical for significantly enhancing language models' ability to precisely express specific types of implicit commonsense knowledge, even for entities not seen during training.
• ATOMIC2020 successfully captures knowledge types that are particularly challenging for large language models to acquire implicitly from raw text, making it a valuable resource for guiding LLM fine-tuning.
• The accuracy and quality of a training knowledge graph are more crucial for effective generalization and knowledge generation by neural models than mere scale or volume of tuples.

Summary by alphaXiv.