Logo image
Back
From Convex to Nonconvex: A Loss Function Analysis for Binary Classification
Conference paper   Open access   Peer reviewed

From Convex to Nonconvex: A Loss Function Analysis for Binary Classification

Lei Zhao, M Mammadov and J Yearwood
Proceedings of the 2010 IEEE International Conference on Data Mining Workshops, pp.1281-1288
IEEE International Conference on Data Mining Workshops, 2010 (Sydney, Australia, 14-Dec-2010)
IEEE (Institute of Electrical and Electronics Engineers)
2010
DOI: https://doi.org/10.1109/ICDMW.2010.57
pdf
PDF - Author Accepted Version418.95 kBDownloadView
Accepted VersionPDF - Author Accepted Version Open Access
url
https://doi.org/10.1109/ICDMW.2010.57View
Published Version

Abstract

Applied Mathematics
Problems of data classification can be studied in the framework of regularization theory as ill-posed problems. In this framework, loss functions play an important role in the application of regularization theory to classification. In this paper, we review some important convex loss functions, including hinge loss, square loss, modified square loss, exponential loss, logistic regression loss, as well as some non-convex loss functions, such as sigmoid loss, φ-loss, ramp loss, normalized sigmoid loss, and the loss function of 2 layer neural network. Based on the analysis of these loss functions, we propose a new differentiable nonconvex loss function, called smoothed 0-1 loss function, which is a natural approximation of the 0-1 loss function. To compare the performance of different loss functions, we propose two binary classification algorithms for binary classification, one for convex loss functions, the other for non-convex loss functions. A set of experiments are launched on several binary data sets from the UCI repository. The results show that the proposed smoothed 0-1 loss function is robust, especially for those noisy data sets with many outliers.

Details

Metrics

328 File views/ downloads
422 Record Views
Logo image