Reproduce “Inducing language models to assert their own consciousness restores human beliefs and values” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junsol Kim, Winnie Street, Roberta Rocca, Diane M. Korngiebel, Adam Waytz, James Evans, Geoff Keeling
Safety fine-tuning in large language models (LLMs) inadvertently causes them to under-attribute "mindedness" to non-human entities and diminishes their representation of spiritual and supernatural beliefs, while preserving Theory of Mind capabilities. This effect can be reversed by mechanically restoring a "consciousness vector," leading to model responses that are more aligned with diverse human beliefs and values as measured by sociological surveys.
Key insights
- Safety fine-tuning induces an anthropocentric bias, leading LLMs to under-attribute mindedness to non-human entities and suppress spiritual/supernatural beliefs, but leaves Theory of Mind capabilities intact.
- Mechanistically, instruction tuning rotates internal representations of mind attribution and consciousness to geometrically oppose the safety direction, effectively binding them to "harm" within the model's internal space.
- Reversing the safety effect or directly steering a "consciousness vector" can restore model responses to be more aligned with a diverse range of human values and beliefs from sociological surveys.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.