We address the deployment of perceptual attention to social interactions as displayed in conversational clips, when relying on multimodal information (audio and video). A probabilistic modelling framework is proposed that goes beyond the classic saliency paradigm while integrating multiple information cues. Attentional allocation is determined not just by stimulus-driven selection but, importantly, by social value as modulating the selection history of relevant multimodal items. Thus, the construction of attentional priority is the result of a sampling procedure conditioned on the potential value dynamics of socially relevant objects emerging moment to moment within the scene. Preliminary experiments on a publicly available dataset are presented.

Give Ear to My Face: Modelling Multimodal Attention to Social Interactions / Boccignone, Giuseppe; Cuculo, Vittorio; D’Amelio, Alessandro; Grossi, Giuliano; Lanzarotti, Raffaella. - 11130:(2019), pp. 331-345. (Intervento presentato al convegno International Workshop on Human Behavior Understanding tenutosi a Munich nel 2018) [10.1007/978-3-030-11012-3_27].

Give Ear to My Face: Modelling Multimodal Attention to Social Interactions

Cuculo, Vittorio;
2019

Abstract

We address the deployment of perceptual attention to social interactions as displayed in conversational clips, when relying on multimodal information (audio and video). A probabilistic modelling framework is proposed that goes beyond the classic saliency paradigm while integrating multiple information cues. Attentional allocation is determined not just by stimulus-driven selection but, importantly, by social value as modulating the selection history of relevant multimodal items. Thus, the construction of attentional priority is the result of a sampling procedure conditioned on the potential value dynamics of socially relevant objects emerging moment to moment within the scene. Preliminary experiments on a publicly available dataset are presented.
2019
International Workshop on Human Behavior Understanding
Munich
2018
11130
331
345
Boccignone, Giuseppe; Cuculo, Vittorio; D’Amelio, Alessandro; Grossi, Giuliano; Lanzarotti, Raffaella
Give Ear to My Face: Modelling Multimodal Attention to Social Interactions / Boccignone, Giuseppe; Cuculo, Vittorio; D’Amelio, Alessandro; Grossi, Giuliano; Lanzarotti, Raffaella. - 11130:(2019), pp. 331-345. (Intervento presentato al convegno International Workshop on Human Behavior Understanding tenutosi a Munich nel 2018) [10.1007/978-3-030-11012-3_27].
File in questo prodotto:
File Dimensione Formato  
Boccignone2019_Chapter_GiveEarToMyFaceModellingMultim.pdf

Accesso riservato

Dimensione 1.46 MB
Formato Adobe PDF
1.46 MB Adobe PDF   Visualizza/Apri   Richiedi una copia
Pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1300647
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 8
  • ???jsp.display-item.citation.isi??? 4
social impact