Reproduce “Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tianqing Fang, Zeming Chen, Yangqiu Song, Antoine Bosselut
Researchers from Hong Kong University of Science and Technology and EPFL introduce COM2, a large-scale dataset for complex, multi-hop commonsense reasoning derived from existing knowledge graphs. Fine-tuning models on the synthetically generated COM2 dataset significantly improves their zero-shot performance on diverse commonsense tasks, with fine-tuned models achieving over 6% higher accuracy than GPT-4 on the COM2 benchmark itself.
Key insights
- Large-scale synthetically generated training data, derived from knowledge graphs, can effectively train language models for complex, multi-hop commonsense reasoning, often outperforming much larger models lacking such specialized training.
- Pre-processing steps like node densification and plausibility filtering are crucial for improving the quality and utility of distantly supervised data derived from existing knowledge graphs.
- Verbalizing structured queries into coherent, rich natural language contexts (e.g., using LLMs like ChatGPT) is important for effective model training and generalization to diverse downstream tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.