Reproduce “Reevaluating Data Partitioning for Emotion Detection in EmoWOZ” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Moeen Mostafavi, Michael D. Porter
This research from the University of Virginia identifies and corrects a critical dataset shift in EmoWOZ, a large-scale corpus for emotion detection in task-oriented dialogues. It introduces a novel stratified sampling method for conversational data, demonstrating that the new partitioning leads to more consistent model performance and reliable evaluation compared to the original, flawed splits.
Key insights
- Dataset shift is a pervasive problem, particularly when adapting existing datasets (like MultiWOZ/EmoWOZ) for new machine learning tasks.
- Meticulous data partitioning is crucial for robust model evaluation and comparison, especially for complex sequential data like conversational dialogues with unevenly distributed labels.
- Stratified sampling based on emotion sequences, segregating frequent from non-frequent patterns, effectively mitigates distributional inconsistencies in emotion-annotated conversational data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.