Reproduce “Approximate Stream Analytics in Apache Flink and Apache Spark Streaming” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Do Le Quoc, Ruichuan Chen, Pramod Bhatotia, Christof Fetze, Volker Hilt, Thorsten Strufe

StreamApprox is a system that introduces approximate computing principles to stream analytics, leveraging an Online Adaptive Stratified Reservoir Sampling (OASRS) algorithm. The system achieved throughput improvements of 1.15x to 3x on Spark Streaming while maintaining accuracy loss typically under 1% for aggregate queries with 10% sampling.

Key insights

  • Online Adaptive Stratified Reservoir Sampling (OASRS) successfully combines reservoir and stratified sampling, enabling real-time approximate analytics on heterogeneous data streams without requiring complete dataset knowledge.
  • Applying approximate computing to stream processing can yield substantial throughput gains (e.g., up to 3x) while maintaining high accuracy, typically under 1% loss for aggregate queries.
  • The system's ability to adapt sampling parameters based on a cost function allows for flexible trade-offs between accuracy, latency, and resource utilization in dynamic stream environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.