With the democratization of data science libraries and frame- works, most data scientists manage and generate their data analytics pipelines using a collection of scripts (e.g., Python, R). This marks a shift from traditional applications that communicate back and forth with a DBMS that stores and manages the application data. While code debuggers have reached impressive maturity over the past decades, they fall short in assisting users to explore data-driven what-if sce- narios (e.g., split the training set into two and build two ML models). Those scenarios, while doable programmati- cally, are a substantial burden for users to manage them- selves. Dagger (Data Debugger) is an end-to-end data de- bugger that abstracts key data-centric primitives to enable users to quickly identify and mitigate data-related problems in a given pipeline. Dagger was motivated by a series of interviews we conducted with data scientists across several organizations. A preliminary version of Dagger has been in- corporated into Data Civilizer 2.0 to help physicians at the Massachusetts General Hospital process complex pipelines.
Dagger: A Data (not code) Debugger / Kindi Rezig, El; Cao, Lei; Simonini, Giovanni; Schoemans, Maxime; Madden, Samuel; Tang, Nan; Ouzzani, Mourad; Stonebraker:, Michael. - (2020). (Intervento presentato al convegno 10th Conference on Innovative Data Systems Research tenutosi a Amsterdam, The Netherlands nel January 12-15, 2020).