The Cherenkov Telescope Array (CTA) is an initiative that is currently building the largest gamma-ray ground Observatory that ever existed. A Science Alert Generation (SAG) system, part of the Array Control and Data Acquisition (ACADA) system of the CTA Observatory, analyses online the telescope data - arriving at an event rate of tens of kHz - to detect transient gamma-ray events. The SAG system also performs an online data quality analysis to assess the instruments' health during the data acquisition: this analysis is crucial to confirm good detections. A Python and a C++ software library to perform the online data quality analysis of CTA data, called rta-dq-lib, has been proposed for CTA. The Python version is dedicated to the rapid prototyping of data quality use cases. The C++ version is optimized for maximum performance. The library allows the user to define, through XML configuration files, the format of the input data and, for each data field, which quality checks must be performed and which types of aggregations and transformations must be applied. It internally translates the XML configuration into a direct acyclic computational graph that encodes the dependencies of the computational tasks to be performed. This model allows the library to easily take advantage of parallelization at the thread level and the overall flexibility allow us to develop generic data quality analysis pipelines that could also be reused in other applications....

rta-dq-lib: a software library to perform online data quality analysis of scientific data / Baroncelli, Leonardo; Bulgarelli, Andrea; Parmiggiani, Nicolo; Fioretti, Valentina; Addis, Antonio; De Cesare, Giovanni; Di Piano, Ambra; Conforti, Vito; Gianotti, Fulvio; Russo, Federico; Maurin, Gilles; Vuillaume, Thomas; Aubert, Pierre; Garcia, Emilio; Zoccoli, Antonio. - (2021). ((Intervento presentato al convegno Astronomical Data Analysis Software and Systems XXX. ASP Conference Series tenutosi a virtual conference nel 8-12 November 2020.

rta-dq-lib: a software library to perform online data quality analysis of scientific data

Di Piano, Ambra;
2021

Abstract

The Cherenkov Telescope Array (CTA) is an initiative that is currently building the largest gamma-ray ground Observatory that ever existed. A Science Alert Generation (SAG) system, part of the Array Control and Data Acquisition (ACADA) system of the CTA Observatory, analyses online the telescope data - arriving at an event rate of tens of kHz - to detect transient gamma-ray events. The SAG system also performs an online data quality analysis to assess the instruments' health during the data acquisition: this analysis is crucial to confirm good detections. A Python and a C++ software library to perform the online data quality analysis of CTA data, called rta-dq-lib, has been proposed for CTA. The Python version is dedicated to the rapid prototyping of data quality use cases. The C++ version is optimized for maximum performance. The library allows the user to define, through XML configuration files, the format of the input data and, for each data field, which quality checks must be performed and which types of aggregations and transformations must be applied. It internally translates the XML configuration into a direct acyclic computational graph that encodes the dependencies of the computational tasks to be performed. This model allows the library to easily take advantage of parallelization at the thread level and the overall flexibility allow us to develop generic data quality analysis pipelines that could also be reused in other applications....
Astronomical Data Analysis Software and Systems XXX. ASP Conference Series
virtual conference
8-12 November 2020
Baroncelli, Leonardo; Bulgarelli, Andrea; Parmiggiani, Nicolo; Fioretti, Valentina; Addis, Antonio; De Cesare, Giovanni; Di Piano, Ambra; Conforti, Vito; Gianotti, Fulvio; Russo, Federico; Maurin, Gilles; Vuillaume, Thomas; Aubert, Pierre; Garcia, Emilio; Zoccoli, Antonio
rta-dq-lib: a software library to perform online data quality analysis of scientific data / Baroncelli, Leonardo; Bulgarelli, Andrea; Parmiggiani, Nicolo; Fioretti, Valentina; Addis, Antonio; De Cesare, Giovanni; Di Piano, Ambra; Conforti, Vito; Gianotti, Fulvio; Russo, Federico; Maurin, Gilles; Vuillaume, Thomas; Aubert, Pierre; Garcia, Emilio; Zoccoli, Antonio. - (2021). ((Intervento presentato al convegno Astronomical Data Analysis Software and Systems XXX. ASP Conference Series tenutosi a virtual conference nel 8-12 November 2020.
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1289237
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact