Reproduce “An Empirical Study of Benchmarking Chinese Aspect Sentiment Quad Prediction” on OpenResearch

About this paper

Junxian Zhou, Haiqin Yang, Ye Junpeng, Yuxuan He, Hao Mou

Researchers from DataStory and IDEA constructed two large-scale, high-density Chinese Aspect Sentiment Quad Prediction (ASQP) datasets, Car-ASQP and Digital-ASQP, derived from real-world online platforms. They empirically evaluated state-of-the-art baselines and GPT series models on these and existing English datasets, finding that while the new datasets pose significant challenges for current models, GPTs primarily struggle with precise boundary distinction of aspect and opinion terms rather than overall sentiment.

Key insights

• The newly constructed Chinese ASQP datasets (Car-ASQP, Digital-ASQP) are significantly larger, denser, and more challenging than existing English datasets, reflecting real-world linguistic complexity.
• Large language models (GPT series) perform poorly on strict ASQP evaluation metrics primarily due to imprecise boundary distinction of aspect and opinion terms, rather than a failure to identify core entities or sentiment.
• Specialized generation-based ASQP models generally outperform pipeline methods but still show lower F1 scores on the more challenging new Chinese datasets.

Summary by alphaXiv.