Here we introduce an approximated differentiable renderer to refine a 6-DoF pose prediction using only 2D alignment information. To this end, a two-branched convolutional encoder network is employed to jointly estimate the object class and its 6-DoF pose in the scene. We then propose a new formulation of an approximated differentiable renderer to re-project the 3D object on the image according to its predicted pose; in this way the alignment error between the observed and the re-projected object silhouette can be measured. Since the renderer is differentiable, it is possible to back-propagate through it to correct the estimated pose at test time in an online learning fashion. Eventually we show how to leverage the classification branch to profitably re-project a representative model of the predicted class (i.e. a medoid) instead. Each object in the scene is processed independently and novel viewpoints in which both objects arrangement and mutual pose are preserved can be rendered. Differentiable renderer code is available at:https://github.com/ndrplz/tensorflow-mesh-renderer.

End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization / Palazzi, Andrea; Bergamini, Luca; Calderara, Simone; Cucchiara, Rita. - (2019), pp. 702-715. (Intervento presentato al convegno Second Workshop on 3D Reconstruction Meets Semantics (3DRMS) tenutosi a Munich, Germany nel 8 - 14 September 2018) [10.1007/978-3-030-11015-4_53].

End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization

Andrea Palazzi
;
Luca Bergamini
;
Simone Calderara;Rita Cucchiara
2019

Abstract

Here we introduce an approximated differentiable renderer to refine a 6-DoF pose prediction using only 2D alignment information. To this end, a two-branched convolutional encoder network is employed to jointly estimate the object class and its 6-DoF pose in the scene. We then propose a new formulation of an approximated differentiable renderer to re-project the 3D object on the image according to its predicted pose; in this way the alignment error between the observed and the re-projected object silhouette can be measured. Since the renderer is differentiable, it is possible to back-propagate through it to correct the estimated pose at test time in an online learning fashion. Eventually we show how to leverage the classification branch to profitably re-project a representative model of the predicted class (i.e. a medoid) instead. Each object in the scene is processed independently and novel viewpoints in which both objects arrangement and mutual pose are preserved can be rendered. Differentiable renderer code is available at:https://github.com/ndrplz/tensorflow-mesh-renderer.
2019
23-gen-2019
Second Workshop on 3D Reconstruction Meets Semantics (3DRMS)
Munich, Germany
8 - 14 September 2018
702
715
Palazzi, Andrea; Bergamini, Luca; Calderara, Simone; Cucchiara, Rita
End-to-end 6-DoF Object Pose Estimation through Differentiable Rasterization / Palazzi, Andrea; Bergamini, Luca; Calderara, Simone; Cucchiara, Rita. - (2019), pp. 702-715. (Intervento presentato al convegno Second Workshop on 3D Reconstruction Meets Semantics (3DRMS) tenutosi a Munich, Germany nel 8 - 14 September 2018) [10.1007/978-3-030-11015-4_53].
File in questo prodotto:
File Dimensione Formato  
palazzi_eccvw.pdf

Open access

Tipologia: Versione dell'autore revisionata e accettata per la pubblicazione
Dimensione 1.34 MB
Formato Adobe PDF
1.34 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/1167726
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 1
social impact