Reproduce “Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nicholas Deas, Kathleen McKeown
Researchers at Columbia University introduced "artificial impressions" to analyze how Large Language Models internally perceive prompts, finding these impressions are linearly decodable from hidden states and predict downstream behaviors like response quality and hedging. The study revealed that LLMs form negative competence and warmth impressions from African American Language, indicating algorithmic bias.
Key insights
- LLMs form measurable "artificial impressions" of prompt authors analogous to human social cognition, which are consistently decodable from internal model representations.
- Warmth appears to be a more salient and consistently encoded dimension in LLM artificial impressions than competence, reflecting the "warmth-primacy effect" observed in human impression formation.
- Linguistic features in prompts, including specific language varieties, play a critical role in shaping these artificial impressions, leading to predictable shifts in LLM output characteristics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.