Reproduce “HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maoliang Li, Jiayu Chen, Zihao Zheng, Ziqian Li, Xinhao Sun, Guojie Luo, Chenchen Liu, Xiang Chen
HeRo, an adaptive orchestration system developed by researchers from Peking University and collaborators, enables efficient on-device deployment of agentic Retrieval-Augmented Generation (RAG) workflows on heterogeneous mobile SoCs. The system achieves up to 10.94x lower end-to-end latency compared to single-accelerator baselines and up to 1.5x speedup over static heterogeneous mapping by dynamically managing sub-stage execution across CPU, GPU, and NPU.
Key insights
- Optimal performance for agentic RAG on heterogeneous mobile SoCs requires dynamic, cross-layer scheduling that simultaneously considers the RAG graph's evolving topology and underlying hardware characteristics like PU affinity and memory bandwidth.
- Aggressive parallelism without considering shared-memory bandwidth contention can lead to significant slowdowns on critical-path tasks, negating the benefits of concurrent execution.
- A combination of fine-grained workload partitioning, dynamic critical-path guidance, and contention-aware concurrency management is essential for efficiently orchestrating complex and dynamic AI workflows on edge devices.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.