Reproduce “Large-Scale Graphs Community Detection using Spark GraphFrames” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Elena-Simona Apostol, Adrian-Cosmin Cojocaru, Ciprian-Octavian Truică
Researchers from the National University of Science and Technology Politehnica Bucharest developed a distributed framework for community detection in large-scale graphs using Apache Spark's GraphFrames API. This approach successfully implemented and evaluated Louvain, Fast Greedy, and K-Cliques algorithms on real-world datasets, demonstrating effective scalability and addressing the computational challenges of massive graph analysis.
Key insights
- Apache Spark GraphFrames provides a viable platform for implementing and executing community detection algorithms at scale.
- The performance of distributed community detection algorithms depends on factors like algorithm design, graph structure, and resource allocation.
- There is an optimal balance point for resource allocation (workers, cores, memory); exceeding this may not improve performance due to overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.