Reproduce “Modeling Human Perspectives with Socio-Demographic Representations” on OpenResearch

About this paper

Leixin Zhang, Cagri Coltekin

Socio-Contrastive Learning (SCL) was introduced to model diverse human perspectives in subjective NLP tasks like hate speech and toxicity detection by integrating rich socio-demographic features. This method, developed by researchers at the University of Tübingen, achieved F1 scores of 0.725 on hate speech and 0.645 on toxicity datasets, outperforming baseline approaches by leveraging comprehensive socio-demographic information.

Key insights

• Comprehensive integration of socio-demographic features, particularly through sophisticated fusion and contrastive representation learning, significantly improves the prediction of individual human perspectives in subjective NLP tasks.
• The contrastive learning component within SCL is crucial for refining socio-demographic representations, making them more discriminative and strengthening their association with systematic annotation patterns.
• Specific socio-demographic attributes such as education, political ideology, income, and age groups show strong correlations with annotator perspective divergence, providing insights into the sources of disagreement.

Summary by alphaXiv.