Reproduce “Beyond accuracy: what AI assurance needs to establish in public services” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Louise Yorke
The authors examine whether GPT-4o topic classifications provide dependable evidence for public-sector decisions and how sampled UK and EU documents address assurance. They compare eight configurations on UK parliamentary questions against human-coded labels, finding accuracy of 68.2% to 70.3%, yet different labels for 24.3% of complete cases. This is one task and model family, not evidence of effects on decisions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.