Reproduce “From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection” on OpenResearch

About this paper

Yuan Cao, Feixiang Liu, Xinyue Wang, Yihan Zhu, Hui Xu, Zheng Wang, Qiang Qiu

Researchers at the Institute of Computing Technology, Chinese Academy of Sciences, developed PerDet-R1, a two-stage LLM framework that redefines MBTI personality detection as a multi-class ranking task. This approach achieved state-of-the-art results on the Kaggle and PANDORA datasets, showing up to an 8.79% improvement in multiclass F1-score by capturing nuanced trait interactions more effectively than classification methods.

Key insights

• Reconceptualizing personality detection as a ranking task rather than classification more accurately reflects psychometric realities and better captures nuanced trait preferences and interdependencies.
• A two-stage training approach combining SFT for initial reasoning and GRPO for fine-tuning via reinforcement learning is crucial for robust and generalizable LLM performance in subjective assessments.
• The designed rule-based reward function, which integrates NDCG and dimension similarity, is critical for stable learning, preventing reward hacking, and enabling autonomous discovery of complex personality patterns.

Summary by alphaXiv.