Reproduce “SciFi-Benchmark: Leveraging Science Fiction To Improve Robot Behavior” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pierre Sermanet, Anirudha Majumdar, Vikas Sindhwani
Researchers from Google DeepMind and Princeton University developed SciFi-Benchmark, a large-scale dataset of 9,056 ethical dilemmas derived from 824 science fiction works, alongside automatically generated "robot constitutions." These constitutions significantly improved the ethical alignment of large language models controlling robots, increasing performance from 23.3% to 92.3% in adversarial scenarios.
Key insights
- Science fiction narratives offer a rich and diverse source of complex ethical dilemmas that are ideal for rigorously testing and enhancing AI alignment at scale.
- Modern LLMs exhibit a substantial intrinsic alignment (79.4%) with human values in normal settings, challenging the strong version of the orthogonality thesis regarding intelligence and ethics.
- Automatically generated and refined 'robot constitutions' provide a robust and interpretable framework, significantly boosting AI ethical alignment and resilience against adversarial prompts (e.g., from 23.3% to 92.3%).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.