Reproduce “Pruning-Aware Multi-Cluster Co-Inference for Large AI Models in AI-RANs” on OpenResearch

About this paper

Xiaowen Cao, Zhonghao Lyu, Shicheng Chu, Zezhong Zhang, Dingzhu Wen, Guangxu Zhu, Kaibin Huang, Shuguang Cui and 1 more

Researchers from Shenzhen University and The Chinese University of Hong Kong (Shenzhen) developed a pruning-aware multi-cluster co-inference framework for large AI models in AI-RANs. The framework jointly optimizes model pruning, resource allocation, and task scheduling, achieving improved inference quality and resource efficiency in multi-user, multi-cluster edge environments for tasks like image classification and video-to-text captioning.

Key insights

• Modeling heterogeneous device contributions through importance coefficients (quantified by Shapley Value) is crucial for guiding pruning and resource allocation decisions to optimize overall inference quality.
• A rate-distortion-based analysis effectively characterizes the trade-off between model pruning levels and inference performance, providing a theoretical foundation for pruning-aware co-inference.
• The coupling between semantic importance, model pruning, and network resource allocation (e.g., bandwidth) is dynamic; its influence varies significantly depending on the pruning regime, suggesting uniform pruning is often suboptimal.

Summary by alphaXiv.