In-memory computing architectures based on Resistive random access memory technologies (RRAM) are a promising candidate for the development of ultra-low power hardware accelerators that could enable the deployment of deep neural networks inference algorithms on energy constrained devices at the edge of the communication network. However, the study of the reliability of such circuits is non-trivial due to the intrinsic RRAM devices nonlinearity and stochasticity. For instance, RRAM devices are subject not only to device-to-device and cycle-to-cycle resistance variations but also to Random Telegraph Noise which introduces additional time dependent resistance fluctuations that could result in reduced circuit performance. Previous studies exploited simplified statistical models to show that such device nonidealities may reduce the classification accuracy even when binarized neural networks are employed. However, a circuit reliability analysis based on full circuit-level simulations is still missing. In this work, we develop and train a low-bit precision neural network which employs binary weights and 4-bits activations. We further analyze the impact of RRAM nonidealities (e.g., variability and Random Telegraph Noise) on the classification accuracy by means of full circuit-level simulations enabled by a physics-based RRAM compact model, calibrated on experimental data from the literature. Results show that combining binary weights with low-precision activations allows retaining software-level accuracy even in the presence of Random Telegraph Noise and weight variability.

Low-Bit Precision Neural Network Architecture with High Immunity to Variability and Random Telegraph Noise based on Resistive Memories / Zanotti, Tommaso; Puglisi, Francesco Maria; Pavan, Paolo. - (2021), pp. 1-6. ((Intervento presentato al convegno 2021 IEEE International Reliability Physics Symposium (IRPS) tenutosi a Monterey, CA, USA nel 2021 [10.1109/IRPS46558.2021.9405103].

Low-Bit Precision Neural Network Architecture with High Immunity to Variability and Random Telegraph Noise based on Resistive Memories

Tommaso Zanotti
;
Francesco Maria Puglisi;Paolo Pavan
2021

Abstract

In-memory computing architectures based on Resistive random access memory technologies (RRAM) are a promising candidate for the development of ultra-low power hardware accelerators that could enable the deployment of deep neural networks inference algorithms on energy constrained devices at the edge of the communication network. However, the study of the reliability of such circuits is non-trivial due to the intrinsic RRAM devices nonlinearity and stochasticity. For instance, RRAM devices are subject not only to device-to-device and cycle-to-cycle resistance variations but also to Random Telegraph Noise which introduces additional time dependent resistance fluctuations that could result in reduced circuit performance. Previous studies exploited simplified statistical models to show that such device nonidealities may reduce the classification accuracy even when binarized neural networks are employed. However, a circuit reliability analysis based on full circuit-level simulations is still missing. In this work, we develop and train a low-bit precision neural network which employs binary weights and 4-bits activations. We further analyze the impact of RRAM nonidealities (e.g., variability and Random Telegraph Noise) on the classification accuracy by means of full circuit-level simulations enabled by a physics-based RRAM compact model, calibrated on experimental data from the literature. Results show that combining binary weights with low-precision activations allows retaining software-level accuracy even in the presence of Random Telegraph Noise and weight variability.
2021
2021 IEEE International Reliability Physics Symposium (IRPS)
Monterey, CA, USA
2021
1
6
Zanotti, Tommaso; Puglisi, Francesco Maria; Pavan, Paolo
Low-Bit Precision Neural Network Architecture with High Immunity to Variability and Random Telegraph Noise based on Resistive Memories / Zanotti, Tommaso; Puglisi, Francesco Maria; Pavan, Paolo. - (2021), pp. 1-6. ((Intervento presentato al convegno 2021 IEEE International Reliability Physics Symposium (IRPS) tenutosi a Monterey, CA, USA nel 2021 [10.1109/IRPS46558.2021.9405103].
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11380/1245902
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 3
social impact