Reproduce “CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experiments” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xuan Liu, Haoyang Shang, Haojian Jin

CoBRA measures and adjusts language-model agents toward specified response tendencies using classic social-science tasks. Its Cognitive Bias Index summarizes observable choices, and a regulation loop can tune prompts, activations, or model parameters. Across four paradigm pairs, calibration reduced cross-model response variance from 0.187 to 0.043, a 77.0% reduction. The index describes task-specific behavior, not an agent’s internal cognition.

Key insights

  • Cognitive biases in LLM agents can be explicitly and quantitatively controlled, moving beyond qualitative prompt engineering to systematic behavioral programming.
  • Classic social science experiments provide a robust, validated framework for operationalizing, measuring, and regulating specific behavioral biases in AI.
  • Interventions in an LLM's input, activation, and parameter spaces can achieve fine-grained, monotonic, and generalizable control over agent biases with minimal disruption to general capabilities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.