The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variable, quantified by the mutual information. The hidden variables and the source can be arbitrarily correlated. The optimal trade-off between rates of relevance and compression (or complexity) is obtained through a singleletter characterization, referred to as the rate-relevance region. Conditions of successive refinabilty are given. Binary source with BSC hidden variables and binary source with BSC/BEC mixed hidden variables are both proved to be successively refinable. We further extend our result to Guassian models. A counterexample of successive refinability is also provided.

The multi-layer information bottleneck problem / Yang, Q.; Piantanida, P.; Gunduz, D.. - 2018-:(2018), pp. 404-408. (Intervento presentato al convegno 2017 IEEE Information Theory Workshop, ITW 2017 tenutosi a Kaohsiung Exhibition Center, twn nel 2017) [10.1109/ITW.2017.8278006].

The multi-layer information bottleneck problem

D. Gunduz
2018

Abstract

The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variable, quantified by the mutual information. The hidden variables and the source can be arbitrarily correlated. The optimal trade-off between rates of relevance and compression (or complexity) is obtained through a singleletter characterization, referred to as the rate-relevance region. Conditions of successive refinabilty are given. Binary source with BSC hidden variables and binary source with BSC/BEC mixed hidden variables are both proved to be successively refinable. We further extend our result to Guassian models. A counterexample of successive refinability is also provided.
2018
2017
2017 IEEE Information Theory Workshop, ITW 2017
Kaohsiung Exhibition Center, twn
2017
2018-
404
408
Yang, Q.; Piantanida, P.; Gunduz, D.
The multi-layer information bottleneck problem / Yang, Q.; Piantanida, P.; Gunduz, D.. - 2018-:(2018), pp. 404-408. (Intervento presentato al convegno 2017 IEEE Information Theory Workshop, ITW 2017 tenutosi a Kaohsiung Exhibition Center, twn nel 2017) [10.1109/ITW.2017.8278006].
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1202772
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 7
  • ???jsp.display-item.citation.isi??? 5
social impact