Reproduce “VCBench: Benchmarking LLMs in Venture Capital” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rick Chen, Joseph Ternasky, Afriyie Samuel Kwesi, Ben Griffin, Aaron Ontoyin Yin, Zakari Salifu, Kelvin Amoaba, Xianling Mu and 2 more

VCBench introduces the first anonymized benchmark for predicting founder success in venture capital, utilizing a rigorously validated multistage anonymization pipeline for 9,000 founder profiles. This work demonstrates that state-of-the-art Large Language Models, such as GPT-4o, can achieve higher predictive precision (29.1%) compared to human expert baselines in early-stage investment forecasting.

Key insights

  • The multistage anonymization pipeline effectively reduced re-identification risk by over 80% for online adversaries and 90% for offline adversaries while maintaining predictive utility.
  • An unexpected finding was that including unbucketed QS university rankings further reduced re-identification rates, as LLMs frequently misused current ranking lists, enhancing anonymity.
  • Large Language Models can achieve predictive performance in early-stage venture forecasting that surpasses established human expert baselines, even when operating on anonymized data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.