Reproduce “ONTO: A Token-Efficient Columnar Notation for LLM Input Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Harshavardhanan Deekeswar
This research introduces ONTO, a token-efficient columnar notation designed for optimizing structured data input to large language models. The format achieves a 46% to 51% reduction in token count compared to JSON and YAML, leading to a 9% to 10% decrease in inference latency while maintaining LLM comprehension.
Key insights
- Serialization for LLM input represents a distinct problem from general document interchange, requiring a purpose-built format optimized for token efficiency under context window constraints.
- The primary source of token redundancy in structured data for LLMs is the per-record repetition of field names, which significantly outweighs the overhead from verbose punctuation.
- Adopting a 'schema-once, data-many' principle with a columnar structure can drastically reduce token counts for structured data while preserving LLM's ability to comprehend and process the information effectively, even with modest indentation overhead for nesting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.