Reproduce “Conf-Profile: A Confidence-Driven Reasoning Paradigm for Label-Free User Profiling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yingxin Li, Jianbo Zhao, Xueyu Ren, Jie Tang, Wangjie You, Xu Chen, Kan Zhou, Chao Feng and 3 more
CONF-PROFILE presents a confidence-driven, label-free framework for user profiling, supported by ProfileBench, a new real-world benchmark derived from a major video platform. The framework achieved an average F1 score of 73.03 across six profiling dimensions, outperforming existing state-of-the-art methods by explicitly integrating confidence signals throughout its two-stage training pipeline.
Key insights
- Confidence is a crucial, inherent signal in LLMs that can be effectively leveraged across the entire training pipeline to enhance reliability and manage uncertainty in label-free tasks.
- A two-stage approach combining supervised fine-tuning with pseudo-labels and unsupervised reinforcement learning effectively distills complex reasoning into smaller models without human annotations.
- Real-world, multi-dimensional benchmarks that include explicit 'Unknown' options are essential for evaluating and developing robust, practical user profiling systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.