Reproduce “LRG at SemEval-2021 Task 4: Improving Reading Comprehension with Abstract Words using Augmentation, Linguistic Features and Voting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Abheesht Sharma, Harshit Pandey, Gunjan Chhablani, Yash Bhartia, Tirtharaj Dash
The LRG team at BITS Pilani, Goa Campus, addressed the challenge of reading comprehension with abstract words for SemEval-2021 Task 4. Their approach combined fine-tuned BERT-based models with context-aware voting mechanisms and specific linguistic features, achieving 78.98% accuracy on the Non-Specificity subtask and demonstrating model interpretability.
Key insights
- Context-handling strategies like "Voting" significantly improve performance by allowing Transformer models to process longer articles, overcoming their input length limitations.
- Targeted integration of linguistic knowledge, such as the "Hyponyms Options Method" for non-specificity, can effectively enhance model accuracy for abstract semantic tasks.
- The use of interpretability techniques like Integrated Gradients reveals that models implicitly learn to attend to relevant abstract and concrete words, providing insight into their reasoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.