Reproduce “A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marcelo A. Montemurro, Mirko Degli Esposti
Researchers at The Open University and University of Bologna developed a method to generate symbolic sequences that simultaneously maintain empirical symbol frequencies, like Zipf's law, and long-range correlations, quantified by the Detrended Fluctuation Analysis exponent. This approach, validated on language and genomic DNA, produces surrogates with exact frequency matches and identical DFA scaling behavior to original texts, addressing a long-standing gap in surrogate data methodologies.
Key insights
- The method enables the precise disentanglement of first-order symbol frequencies and second-order long-range correlations from higher-order structural elements in complex symbolic systems.
- A significant portion of the observed long-range structure in written language and genomic DNA can be explained by the interplay of empirical lexical frequencies and linear long-memory processes, even when higher-order structures are randomized.
- The proposed framework provides a more sophisticated and realistic null model, facilitating refined hypothesis testing on the generative and cognitive mechanisms underlying complex symbolic sequences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.