Reproduce “Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rongyu Zhang, Xiaowei Chi, Guiliang Liu, Wenyi Zhang, Yuan Du, Fangxin Wang
The authors introduce HA-Fedformer for unimodal training with multimodal prediction in federated learning. Clients train on one modality, while the server uses posterior-based encoder aggregation and cross-modal decoder aggregation to combine their models. On CMU-MOSEI sentiment analysis, HA-Fedformer++ achieves 48.6% seven-class accuracy, versus 43.8% for FedMSplit. This result concerns sentiment benchmarks and assumes multimodal input at test time.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.