In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage.

Are loss functions all the same? / Rosasco, L; DE VITO, Ernesto; Caponnetto, A; Piana, M; Verri, A.. - In: NEURAL COMPUTATION. - ISSN 0899-7667. - STAMPA. - 16:5(2004), pp. 1063-1076. [10.1162/089976604773135104]

Are loss functions all the same?

DE VITO, Ernesto;
2004

Abstract

In this letter, we investigate the impact of choosing different loss functions from the viewpoint of statistical learning theory. We introduce a convexity assumption, which is met by all loss functions commonly used in the literature, and study how the bound on the estimation error changes with the loss. We also derive a general result on the minimizer of the expected risk for a convex loss function in the case of classification. The main outcome of our analysis is that for classification, the hinge loss appears to be the loss of choice. Other things being equal, the hinge loss leads to a convergence rate practically indistinguishable from the logistic loss rate and much better than the square loss rate. Furthermore, if the hypothesis space is sufficiently rich, the bounds obtained for the hinge loss are not loosened by the thresholding stage.
2004
16
5
1063
1076
Are loss functions all the same? / Rosasco, L; DE VITO, Ernesto; Caponnetto, A; Piana, M; Verri, A.. - In: NEURAL COMPUTATION. - ISSN 0899-7667. - STAMPA. - 16:5(2004), pp. 1063-1076. [10.1162/089976604773135104]
Rosasco, L; DE VITO, Ernesto; Caponnetto, A; Piana, M; Verri, A.
File in questo prodotto:
Non ci sono file associati a questo prodotto.
Pubblicazioni consigliate

Licenza Creative Commons
I metadati presenti in IRIS UNIMORE sono rilasciati con licenza Creative Commons CC0 1.0 Universal, mentre i file delle pubblicazioni sono rilasciati con licenza Attribuzione 4.0 Internazionale (CC BY 4.0), salvo diversa indicazione.
In caso di violazione di copyright, contattare Supporto Iris

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11380/4590
Citazioni
  • ???jsp.display-item.citation.pmc??? 19
  • Scopus 339
  • ???jsp.display-item.citation.isi??? 262
social impact