Reproduce “FlowLog: Efficient and Extensible Datalog via Incrementality” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hangdong Zhao, Zhenghong Yu, Srinag Rao, Simon Frisk, Zhiwei Fan, Paraschos Koutris
FlowLog, developed by researchers from Microsoft Gray Systems Lab, University of Wisconsin-Madison, and Meta, introduces an efficient and extensible Datalog engine that leverages Differential Dataflow while incorporating Datalog-specific optimizations. The system achieves superior parallel scalability and robust performance across complex recursive workloads, reducing memory consumption by an average of 3.5x compared to its DDlog baseline.
Key insights
- Decoupling the Datalog rule logic from the underlying execution framework (Differential Dataflow) enables systematic, Datalog-aware optimization while retaining the benefits of an incremental backend.
- A "robustness-first" approach to query optimization, using structural cost models and avoiding worst-case scenarios, is more effective for dynamic recursive workloads than traditional cost-based methods.
- Memory efficiency, achieved through optimizations like logic fusion and subplan sharing, is critical for the practical scalability and adoption of incremental Datalog systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.