Reproduce “Non-Federated Multi-Task Split Learning for Heterogeneous Sources” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yilin Zheng, Atilla Eryilmaz

Multi-Task Split Learning (MTSL) is a framework that leverages data heterogeneity in distributed machine learning by combining Split Learning and Multi-Task Learning without a federation process. The system shows superior accuracy and communication efficiency compared to federated approaches across various datasets, particularly under increasing data heterogeneity and noise.

Key insights

  • Multi-Task Split Learning (MTSL) embraces data heterogeneity by allowing clients to have unique learning objectives and data distributions, leveraging differences rather than treating them as obstacles.
  • The framework enables independent learning rate tuning for each client and the central server, which theoretical analysis and experiments show can accelerate convergence.
  • By eliminating the federation process (parameter averaging), MTSL avoids performance degradation caused by conflicting gradient information when data distributions or tasks differ significantly across clients.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.