Reproduce “A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rachel M. Harrison
This preliminary study compares ChatGPT-3.5's performance to human performance on a random number generation task. It finds that the LLM generates sequences that are generally more 'random' than human-generated ones, particularly in its extreme avoidance of repeating digits, although it still deviates from perfectly uniform randomness.
Key insights
- Large Language Models (LLMs) like ChatGPT-3.5 occupy a unique intermediate position, generating sequences that are generally more 'random' than human outputs but still deviate from perfect theoretical randomness.
- ChatGPT-3.5 exhibits an extreme 'algorithmic bias' through a near-total aversion to repeating digits, a pattern humans also avoid but to a lesser degree than the LLM.
- The model's generation process is not merely a reflection of its human-generated training data, but also shaped by underlying computational mechanisms that minimize redundancy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.