Reproduce “SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yimeng Shan, Zhaorui Zhang, Sheng Di, Yu Liu, Xiaoyi Lu, Benben Liu
SplitFT divides LLM fine-tuning between clients and a server, allowing client cut layers to vary and assigning a lower-rank LoRA update at the boundary. Clients send intermediate activations, so communication remains part of the method. On GPT2-small and WikiText2-v1, cutlayer 2 reported 810.4379 seconds mean elapsed time versus 6897.3059 for No Cut; the configurations used different ranks, on an RTX 3090 testbed with five clients.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.