Maximization of AUC and Buffered AUC in binary classification
Loading...
Authors
Norton, Matthew
Uryasev, Stan
Subjects
Buffered Probability of Exceedance
Conditional-Value-at-Risk
AUC
ROC curve
Buffered AUC
Support Vector Machine
Convex programming
Classification performance metric
Generalization
Conditional-Value-at-Risk
AUC
ROC curve
Buffered AUC
Support Vector Machine
Convex programming
Classification performance metric
Generalization
Advisors
Date of Issue
2018-07-05
Date
July 05, 2018
Publisher
SpringerLink
Language
Abstract
In binary classification, performance metrics that are defined as the probability that some error exceeds a threshold are numerically difficult to optimize directly and also hide potentially important information about the magnitude of errors larger than the threshold. Defining similar metrics, instead, using Buffered Probability of Exceedance (bPOE) generates counterpart metrics that resolve both of these issues. We apply this approach to the case of AUC, the Area Under the ROC curve, and define Buffered AUC (bAUC). We show that bAUC can provide insights into classifier performance not revealed by AUC, while being closely related as the tightest concave lower bound and representable as the area under a modified ROC curve. Additionally, while AUC is numerically difficult to optimize directly, we show that bAUC optimization often reduces to convex or linear programming. Extending these results, we show that AUC and bAUC are special cases of Generalized bAUC and that popular Support Vector Machine (SVM) formulations for approximately maximizing AUC are equivalent to direct maximization of Generalized bAUC. We also prove bAUC generalization bounds for these SVM’s. As a central component to these results, we provide an important, novel formula for calculating bPOE, the inverse of Conditional Value-at-Risk. Using this formula, we show that particular bPOE minimization problems reduce to convex and linear programming.
Type
Article
Description
The article of record as published may be found at http://dx.doi.org/10.1007/s10107-018-1312-2
Series/Report No
Department
Operations Research (OR)
Organization
Naval Postgraduate School (U.S.)
Identifiers
NPS Report Number
Sponsors
USA Air Force Office of Scientific Research
DARPA
DARPA
Funder
FA9550-12-1-0427
FA9550-11-1-0258
FA9550-11-1-0258
Format
38 p.
Citation
Norton, Matthew, and Stan Uryasev. "Maximization of auc and buffered auc in binary classification." Mathematical Programming (2016): 1-38.
Distribution Statement
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.