Reproduce “High Risk of Political Bias in Black Box Emotion Inference Models” on OpenResearch

About this paper

Hubert Plisiecki, Paweł Lenartowicz, Maria Flakus, Artur Pokropek

Researchers at the Polish Academy of Sciences empirically demonstrated pervasive political bias in a Polish sentiment analysis model, showing that human annotator biases during training systematically skew emotion inference results. This bias significantly affects predictions for political figures and is primarily rooted in the annotation process, not public trust or inherent linguistic portrayals.

Key insights

• Systematic political bias in emotion inference models is substantial and pervasive, primarily stemming from human annotators' subjective political leanings propagating into the training data.
• This bias cannot be fully explained by public trust in politicians or the inherent linguistic content of the training data texts themselves, but rather how those texts were perceived and labeled by annotators.
• Mitigating such biases is complex, as pruning training data reduces but does not completely eliminate the bias, suggesting more indirect or deeply embedded sources may exist.

Summary by alphaXiv.