Summary: A Federated Learning Algorithm that Optimizes the Communication Overhead among Multiple Users
What
Federated learning is a form of collaborative machine learning, which allows to maintain user’s privacy while using user’s data. The proposed technology is a time-triggered algorithm for exchanging and aggregating data between local users and a global server in federated learning. The method allows to improve convergence time and accuracy in federated learning, if compared to traditional models.
Why
The emerging federated learning framework offers a new way to train machine learning models in a privacy-preserving manner. However, traditional federated learning models, such as synchronous federated learning, asynchronous federated learning, or federated learning systems with asynchronous tiers (FedAT), all suffer from stragglers and communication overhead issues.
Benefits
The proposed technology allows to overcome the known problems of federated learning models, as it provides a novel, time-triggered algorithm that improves converged test accuracy, under highly imbalanced and non-independent and identically distributed data, and that substantially reduces the communication overhead.
The technology allows to implement together different advantageous features available from other known federated learning models, such as intra-tier aggregation and synchronous updates.
Finally, the model is highly adaptable, and its parameters can be tuned to implement not only this newly proposed time-triggered federated learning, but also classical synchronous federated learning, classical asynchronous federated learning, or Fed-AT.
Opportunity
The technology is protected by a GB priority patent application and is available for licensing
The Science
Figure: Federated learning usually comprises the following rounds:
- a global aggregation round, wherein a plurality of local models are sent by different local users to a global server, and a corresponding global model is trained based on the received local models, and
- a local updating round, wherein the updated global model is distributed to the local users and updated with local data to create a corresponding updated local model.
The proposed technology implements a time-triggered intra-tier aggregation round. In particular each global aggregation round is triggered after a predetermined time by receiving updated local models from at least one tier of local users. Furthermore, the global aggregation round has been designed to save storage space on the server and to generate a robust global model, which takes into account all the users contribution.
Patent Status
Pending GB Priority patent application
Further Information
Zhou, X. et al., (2022), "Time-triggered Federated Learning over Wireless Network", arXiv: 2204.12426, doi:10.48550/arXiv.2204.12426.