Reproduce “Circuits of LLM Classifiers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hadi Khalaf, Sajani Vithana, Kunal Talreja, Haewon Jeong, Viveck R. Cadambe, Flavio Calmon
LMCircuit adaptively selects among prompted language model judges and decides whether to stop or query again for binary classification. It learns label and judge-response probabilities from labeled data, then uses dynamic programming to select queries and predictions. Across seven tasks, it outperforms individual judges and routing and cascade baselines at nearly all tested budgets. It does not establish real-world optimality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.