Reproduce “Hillview: A trillion-cell spreadsheet for big data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mihai Budiu, Parikshit Gopalan, Lalith Suresh, Udi Wieder, Han Kruiger, Marcos K. Aguilera
Hillview is a distributed spreadsheet system that enables interactive visualization of datasets containing up to 1.4 trillion cells. It achieves this by combining a distributed architecture with "vizketches," a novel approach based on mergeable summaries, demonstrating faster response times and significantly reduced data transfer compared to general-purpose analytics engines for visualization tasks.
Key insights
- Mergeable summaries, conceptualized as "vizketches," provide an efficient foundation for distributed interactive visualization by reducing data transfer and enabling parallel computation.
- Specialized engines designed specifically for visualization tasks can significantly outperform general-purpose analytics engines for ad-hoc data exploration on massive datasets.
- By computing only what is visually discernible and allowing efficient merging of partial results, the system can provide progressive visualizations with initial results displayed quickly.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.