Reproduce “Finding Common Ground: Annotating and Predicting Common Ground in Spoken Conversations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Magdalena Markowska, Mohammad Taghizadeh, Adil Soubki, Seyed Abolghasem Mirroshandel, Owen Rambow
Researchers from Stony Brook University and the University of Guilan introduced the first corpus explicitly annotated for common ground in naturalistic spoken conversations, encompassing event extraction, individual beliefs, and three types of common ground updates. Initial experiments using FLAN-T5 achieved a Fleiss' kappa of 0.70 for common ground annotation and demonstrated that models could predict common ground updates with gold belief information, improving performance with increased context.
Key insights
- The annotation scheme effectively captured the dynamic negotiation of common ground, with a Fleiss' kappa of 0.70 indicating substantial inter-annotator agreement for this complex task.
- While challenging, especially for minority belief classes and implicit event generation, machine learning models can begin to track individual beliefs and common ground updates, with data augmentation proving beneficial for class imbalance.
- Access to individual belief states (Bel(A) and Bel(B)) is crucial for accurate common ground classification, as models perform significantly better when provided with this information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.