Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning / Sun, Y.; Zhou, S.; Niu, Z.; Gunduz, D.. - 2022-:(2022), pp. 3388-3393. (Intervento presentato al convegno 2022 IEEE International Conference on Communications, ICC 2022 tenutosi a COEX, kor nel 2022) [10.1109/ICC45855.2022.9839279].

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

Zhou S.;Gunduz D.
2022

Abstract

Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.
2022
2022 IEEE International Conference on Communications, ICC 2022
COEX, kor
2022
2022-
3388
3393
Sun, Y.; Zhou, S.; Niu, Z.; Gunduz, D.
Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning / Sun, Y.; Zhou, S.; Niu, Z.; Gunduz, D.. - 2022-:(2022), pp. 3388-3393. (Intervento presentato al convegno 2022 IEEE International Conference on Communications, ICC 2022 tenutosi a COEX, kor nel 2022) [10.1109/ICC45855.2022.9839279].
File in questo prodotto:
File Dimensione Formato  
Time-Correlated_Sparsification_for_Efficient_Over-the-Air_Model_Aggregation_in_Wireless_Federated_Learning.pdf

Accesso riservato

Tipologia: Versione pubblicata dall'editore
Dimensione 1.13 MB
Formato Adobe PDF
1.13 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
2202.08420.pdf

Open access

Tipologia: Versione dell'autore revisionata e accettata per la pubblicazione
Dimensione 405.45 kB
Formato Adobe PDF
405.45 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1286891
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 8
  • ???jsp.display-item.citation.isi??? 3
social impact