Reproduce “Edge Large AI Models: Collaborative Deployment and IoT Applications” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixin Wang, Yuanming Shi, Khaled. B. Letaief
Researchers from The Hong Kong University of Science and Technology and ShanghaiTech University developed frameworks for collaboratively deploying Large AI Models (LAMs) on edge devices, addressing their high resource demands. Their approach utilizes federated fine-tuning with parameter-efficient techniques and microservice-based inference to enable real-time, privacy-preserving, and personalized intelligent IoT applications, demonstrating substantial reductions in memory and latency for Chain-of-Thought reasoning.
Key insights
- Parameter-Efficient Fine-Tuning (PEFT) like LoRA is essential for making federated fine-tuning of Large AI Models (LAMs) viable on resource-limited edge devices by significantly reducing communication and computation.
- Decomposing LAMs into functional microservices allows for dynamic and optimized orchestration across heterogeneous edge devices, effectively reducing inference latency and improving resource utilization.
- An orthogonal projection mechanism within federated unlearning can effectively remove the influence of specific client data while preserving overall model utility and addressing privacy concerns.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.