Current approaches for movie description lack the ability to name characters with their proper names, and can only indicate people with a generic "someone" tag. In this paper we present two contributions towards the development of video description architectures with naming capabilities: firstly, we collect and release an extension of the popular Montreal Video Annotation Dataset in which the visual appearance of each character is linked both through time and to textual mentions in captions. We annotate, in a semi-automatic manner, a total of 53k face tracks and 29k textual mentions on 92 movies. Moreover, to underline and quantify the challenges of the task of generating captions with names, we present different multi-modal approaches to solve the problem on already generated captions.

Towards Video Captioning with Naming: a Novel Dataset and a Multi-Modal Approach / Pini, Stefano; Cornia, Marcella; Baraldi, Lorenzo; Cucchiara, Rita. - 10485(2017), pp. 384-395. ((Intervento presentato al convegno 19th International Conference on Image Analysis and Processing, ICIAP 2017 tenutosi a Catania, Italy nel 11-15 September 2017 [10.1007/978-3-319-68548-9_36].

I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Powered by IRIS-about IRIS-Utilizzo dei cookie

 Copyright © 2022