NEURAL NETWORKS FOR CONSTRAINED OPTIMIZATION

Abstract

A major challenge associated with training a classifier is controlling the ratio of false positive and false negative error rates (FPR/FNR), particularly when one error type dominates the overall error rate (ER) or is connected to costly real-world mistakes and needs to be minimized relative to the others. For many classification tasks, such as computer vision, this problem is exacerbated by the need to use neural network (NN) classification algorithms. In a ship detection problem, using an NN to classify satellite images as containing a ship or not, we demonstrate that tuning the ratio of these ERs is extremely challenging. We show how standard class-weighting (CW) procedures are unstable and ineffective for this task, but are additionally affected in unpredictable or undesirable ways by NN architectural choices and choice of optimization algorithm. However, we find that the simple and computationally inexpensive Threshold-Adjustment (TA) technique (applied to a standard NN architecture and objective) is as effective and more stable than the CW procedure and, for our ship detection problem, even outperforms much more advanced algorithms based on alternating stochastic optimization techniques. To effectively apply the TA technique, we introduce a simple adjustment to the NN output probabilities. We show predictable effects on the ratio of FPR/FNR, which can be used to tune NN classifiers when different ratios of FPR/FNR are desired, depending upon the application.