Reproduce “Do We Fully Understand Students' Knowledge States? Identifying and Mitigating Answer Bias in Knowledge Tracing” on OpenResearch

About this paper

Chaoran Cui, Hebo Ma, Chen Zhang, Chunyun Zhang, Yumo Yao, Meng Chen, Yuling Ma

Research from Shandong University of Finance and Economics introduces an unbiased evaluation framework and the CORE (COunterfactual REasoning) framework to address "answer bias" in Knowledge Tracing (KT) models. The CORE method, which applies a causality-driven approach, consistently improves the performance of various KT models on debiased test sets by 2-5% in accuracy and AUC, enabling more accurate assessments of student knowledge states.

Key insights

• Answer bias is a pervasive and overlooked issue in Knowledge Tracing, leading existing models to rely on superficial statistical correlations rather than true knowledge assessment.
• A causality perspective, specifically by disentangling direct and indirect causal effects, provides a robust method to identify and mitigate answer bias in KT models.
• Explicitly modeling and removing the component of prediction attributable solely to question-specific bias significantly enhances the generalizability and reliability of KT models for assessing student knowledge.

Summary by alphaXiv.