Deep model fusion: A survey

W Li, Y Peng, M Zhang, L Ding, H Hu… - arxiv preprint arxiv …, 2023 - arxiv.org
Deep model fusion/merging is an emerging technique that merges the parameters or
predictions of multiple deep learning models into a single one. It combines the abilities of …

[HTML][HTML] Understanding global aggregation and optimization of federated learning

SI Nanayakkara, SR Pokhrel, G Li - Future Generation Computer Systems, 2024 - Elsevier
We investigate the hypothesis that exploring Federated Learning (FL) aggregation methods
can enhance training processes within FL frameworks, particularly in resource-constrained …

Convergence analysis of sequential federated learning on heterogeneous data

Y Li, X Lyu - Advances in Neural Information Processing …, 2023 - proceedings.neurips.cc
There are two categories of methods in Federated Learning (FL) for joint training across
multiple clients: i) parallel FL (PFL), where clients train models in a parallel manner; and ii) …

The limits and potentials of local sgd for distributed heterogeneous learning with intermittent communication

KK Patel, M Glasgow, A Zindari… - The Thirty Seventh …, 2024 - proceedings.mlr.press
Local SGD is a popular optimization method in distributed learning, often outperforming mini-
batch SGD. Despite this practical success, proving the efficiency of local SGD has been …

Tackling noisy clients in federated learning with end-to-end label correction

X Jiang, S Sun, J Li, J Xue, R Li, Z Wu, G Xu… - Proceedings of the 33rd …, 2024 - dl.acm.org
Recently, federated learning (FL) has achieved wide successes for diverse privacy-sensitive
applications without sacrificing the sensitive private information of clients. However, the data …

The power of extrapolation in federated learning

H Li, K Acharya, P Richtárik - arxiv preprint arxiv:2405.13766, 2024 - arxiv.org
We propose and study several server-extrapolation strategies for enhancing the theoretical
and empirical convergence properties of the popular federated learning optimizer FedProx …

Understanding and mitigating dimensional collapse in federated learning

Y Shi, J Liang, W Zhang, C Xue… - IEEE Transactions on …, 2023 - ieeexplore.ieee.org
Federated learning aims to train models collaboratively across different clients without
sharing data for privacy considerations. However, one major challenge for this learning …

Fednlr: Federated learning with neuron-wise learning rates

H Wang, P Zheng, X Han, W Xu, R Li… - Proceedings of the 30th …, 2024 - dl.acm.org
Federated Learning (FL) suffers from severe performance degradation due to the data
heterogeneity among clients. Some existing work suggests that the fundamental reason is …

Federated learning with manifold regularization and normalized update reaggregation

X An, L Shen, H Hu, Y Luo - Advances in Neural …, 2023 - proceedings.neurips.cc
Federated Learning (FL) is an emerging collaborative machine learning framework where
multiple clients train the global model without sharing their own datasets. In FL, the model …

Federated learning while providing model as a service: Joint training and inference optimization

P Han, S Wang, Y Jiao, J Huang - IEEE INFOCOM 2024-IEEE …, 2024 - ieeexplore.ieee.org
While providing machine learning model as a service to process users' inference requests,
online applications can periodically upgrade the model utilizing newly collected data …