Poster
2

On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing

Avatar
Voices Powered byElevenlabs logo

Avatar
user

What is the main idea of this paper?

AI: The main idea of this paper is to propose and analyze decentralized federated learning algorithms for optimizing non-convex loss functions in the presence of communication noise and heterogeneity across clients, and to compare the performance of FedNDL1, FedNDL2, and FedNDL3 through both theoretical analysis and experiments. The paper also investigates the impact of noisy communication channels on the convergence of decentralized federated learning algorithms.

Authors

Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi, Stanislaw H /. Zak

Abstract

Decentralized learning and optimization is a central problem in control that encompasses several existing and emerging applications, such as federated learning. While there exists a vast literature on this topic and most methods centered around the celebrated average-consensus paradigm, less attention has been devoted to scenarios where the communication between the agents may be imperfect. To this end, this paper presents three different algorithms of Decentralized Federated Learning (DFL) in the presence of imperfect information sharing modeled as noisy communication channels. The first algorithm, Federated Noisy Decentralized Learning (FedNDL1), comes from the literature, where the noise is added to their parameters to simulate the scenario of the presence of noisy communication channels. This algorithm shares parameters to form a consensus with the clients based on a communication graph topology through a noisy communication channel. The proposed second algorithm (FedNDL2) is similar to the first algorithm but with added noise to the parameters, and it performs the gossip averaging before the gradient optimization. The proposed third algorithm (FedNDL3), on the other hand, shares the gradients through noisy communication channels instead of the parameters. Theoretical and experimental results demonstrate that under imperfect information sharing, the third scheme that mixes gradients is more robust in the presence of a noisy channel compared with the algorithms from the literature that mix the parameters.

Follow Us on

0 comments

Add comment
Recommended SciCasts