Reproduce “Efficient-Adam: Communication-Efficient Distributed Adam” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Congliang Chen, Li Shen, Wei Liu, Zhi-Quan Luo

Efficient-Adam distributes training by quantizing updates sent from workers to a server and back, while retaining error-feedback residuals at both ends. This reduces communicated bits, not elapsed time. On IMDB binary sentiment classification, Example 2 achieved the highest test accuracy among compared methods after 500 MB of communication; this result is specific to that setting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.