Reproduce “Hetero-SplitEE: Split Learning of Neural Networks with Early Exits for Heterogeneous IoT Devices” on OpenResearch

About this paper

Yuki Oda, Yuta Ono, Hiroshi Nakamura, Hideki Takase

Hetero-SplitEE introduces a distributed training framework for deep neural networks on heterogeneous IoT devices, allowing clients to select different split points and integrating early exits for adaptive inference. The framework demonstrates improved accuracy over non-collaborative methods for complex tasks like CIFAR-100, achieving up to 39.37% server-side accuracy with a Layer-3 split compared to 25.81% for the distributed baseline.

Key insights

• Allowing heterogeneous split points combined with early exits effectively addresses the challenge of device diversity in collaborative deep learning for IoT, enabling broader participation of resource-constrained devices.
• Server-side aggregation in split learning is particularly beneficial for complex tasks, where it substantially boosts accuracy compared to non-collaborative methods.
• The proposed framework offers flexibility through distinct training strategies (Sequential vs. Averaging) to accommodate varying server resources and desired levels of parallelism.

Summary by alphaXiv.