Introduction: The introduction of robot-assisted surgical devices requires the application of objective performance metrics to verify performance levels. Objective: To develop and validate (face, content, response process, and construct) the performance metrics for a robotic dissection task using a chicken model. Methods: In a procedure characterization, we developed the performance metrics (i.e., procedure steps, errors, and critical errors) for a robotic dissection task, using a chicken model. In a modified Delphi panel, 14 experts from four European Union countries agreed on the steps, errors, and critical errors (CEs) of the task. Six experienced surgeons and eight novice urology surgeons performed the robotic dissection task twice on the chicken model. In the Delphi meeting, 100% consensus was reached on five procedure steps, 15 errors and two CEs. Novice surgeons took 20 min to complete the task on trial 1 and 14 min during trial two, whereas experts took 8.2 min and 6.5 min. On average, the Expert Group completed the task 56% faster than the Novice Group and made 46% fewer performance errors. Results: Sensitivity and specificity for procedure errors and time were excellent to good (i.e., 1.0-0.91) but poor (i.e., 0.5) for step metrics. The mean interrater reliability for the assessments by two robotic surgeons was 0.91 (Expert Group inter-rater reliability = 0.92 and Novice Group = 0.9). Conclusions: We report evidence which supports the demonstration of face, content, and construct validity for a standard and replicable basic robotic dissection task on the chicken model.

Development and Validation of the Metric-Based Assessment of a Robotic Dissection Task on an Avian Model / Puliatti, S.; Amato, M.; Mazzone, E.; Rosiello, G.; De Groote, R.; Berquin, C.; Piazza, P.; Farinha, R.; Mottrie, A.; Gallagher, A. G.. - In: JOURNAL OF SURGICAL RESEARCH. - ISSN 0022-4804. - 277:(2022), pp. 224-234. [10.1016/j.jss.2022.02.056]

Development and Validation of the Metric-Based Assessment of a Robotic Dissection Task on an Avian Model

Puliatti S.;
2022

Abstract

Introduction: The introduction of robot-assisted surgical devices requires the application of objective performance metrics to verify performance levels. Objective: To develop and validate (face, content, response process, and construct) the performance metrics for a robotic dissection task using a chicken model. Methods: In a procedure characterization, we developed the performance metrics (i.e., procedure steps, errors, and critical errors) for a robotic dissection task, using a chicken model. In a modified Delphi panel, 14 experts from four European Union countries agreed on the steps, errors, and critical errors (CEs) of the task. Six experienced surgeons and eight novice urology surgeons performed the robotic dissection task twice on the chicken model. In the Delphi meeting, 100% consensus was reached on five procedure steps, 15 errors and two CEs. Novice surgeons took 20 min to complete the task on trial 1 and 14 min during trial two, whereas experts took 8.2 min and 6.5 min. On average, the Expert Group completed the task 56% faster than the Novice Group and made 46% fewer performance errors. Results: Sensitivity and specificity for procedure errors and time were excellent to good (i.e., 1.0-0.91) but poor (i.e., 0.5) for step metrics. The mean interrater reliability for the assessments by two robotic surgeons was 0.91 (Expert Group inter-rater reliability = 0.92 and Novice Group = 0.9). Conclusions: We report evidence which supports the demonstration of face, content, and construct validity for a standard and replicable basic robotic dissection task on the chicken model.
277
224
234
Development and Validation of the Metric-Based Assessment of a Robotic Dissection Task on an Avian Model / Puliatti, S.; Amato, M.; Mazzone, E.; Rosiello, G.; De Groote, R.; Berquin, C.; Piazza, P.; Farinha, R.; Mottrie, A.; Gallagher, A. G.. - In: JOURNAL OF SURGICAL RESEARCH. - ISSN 0022-4804. - 277:(2022), pp. 224-234. [10.1016/j.jss.2022.02.056]
Puliatti, S.; Amato, M.; Mazzone, E.; Rosiello, G.; De Groote, R.; Berquin, C.; Piazza, P.; Farinha, R.; Mottrie, A.; Gallagher, A. G.
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S002248042200138X-main.pdf

non disponibili

Tipologia: Versione dell'editore (versione pubblicata)
Dimensione 2.8 MB
Formato Adobe PDF
2.8 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1281311
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact