Reproduce “TIFeD: a Tiny Integer-based Federated learning algorithm with Direct feedback alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Luca Colombo, Alessandro Falcetta, Manuel Roveri
Politecnico di Milano researchers introduced TIFeD, an integer-based federated learning algorithm featuring Direct Feedback Alignment, designed to enable on-device training of deep learning models on extremely resource-constrained IoT devices. This approach demonstrated the capability to train feed-forward networks achieving 0.94 accuracy on MNIST and convolutional neural network classifiers reaching 0.76 accuracy on CIFAR10.
Key insights
- Direct Feedback Alignment (DFA) is computationally lighter and more amenable to integer-only arithmetic than Backpropagation, making it suitable for tiny device training.
- Implementing the entire learning process with integer-only arithmetic significantly reduces memory consumption and broadens compatibility to devices lacking Floating-Point Units.
- Distributing training by assigning individual layers to specific client groups (single-layer modality) can drastically lower local computational and memory requirements, though it may slightly impact final accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.