Reproduce “Assessing Human Rights Risks in AI: A Framework for Model Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vyoma Raman, Camille Chabot, Betsy Popken
A framework derived from the UN Guiding Principles on Business and Human Rights enables the computational evaluation of generative AI models for human rights risks, moving beyond policy-level discussions to specific model outputs. Applied to LLMs in political news journalism, the framework reveals varied risks to access to information and freedom of thought based on model behavior in handling misinformation and framing.
Key insights
- Human rights risks in AI are emergent properties of sociotechnical systems, emphasizing the necessity of context-specific and ecologically valid evaluations tailored to the model's deployment conditions.
- Operationalizing the UNGPs provides a legally grounded and internationally recognized basis for computational auditing of AI models, offering a more robust approach to accountability than purely ethical guidelines.
- Effective AI risk assessment must be multi-dimensional, analyzing not only factual accuracy but also subtle framing biases, ideological amplification, and representational distortion, especially concerning vulnerable groups.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.