ANOMALY DETECTION USING A VARIATIONAL AUTOENCODER NEURAL NETWORK WITH A NOVEL OBJECTIVE FUNCTION AND GAUSSIAN MIXTURE MODEL SELECTION TECHNIQUE
Alt, Jonathan K.
MetadataShow full item record
Anomalies in data often convey critical information that can be leveraged in a variety of applications. For the military engaged in combat, this can amount to identifying threats early and preserving a lethal edge over an adversary. In other more benign cases it can corrupt data integrity and lead to ineffective application of other data analysis techniques. To tackle the problem of anomaly detection, there are several common methods provided in statistics and machine learning literature, including variational autoencoders (VAEs). Using a VAE, we develop a novel objective function to improve its performance detecting anomalies. Additionally, we introduce a modeling pipeline that works in the fully unsupervised context, where one does not know the true proportion of anomalies present in the data. To construct this pipeline, we fit reconstruction errors using a Gaussian mixture model (GMM) and select the model whose characteristics best match our performance metrics. Using our approach, we observe an increase in anomalies detected against a standard objective function, and we measure an average improvement of 0.4021 in F1 scores. We show our findings using four labeled benchmark data sets and apply our conclusions on an open-source, unlabeled data set taken from USASpending.gov.
Approved for public release. distribution is unlimited
Showing items related by title, author, creator and subject.
Reeder, Ben (Monterey, California. Naval Postgraduate School, 2016);Very large subaqueous sand dunes were discovered on the upper continental slope of the northern South China Sea (SCS) in the spring of 2007 in water depths of 160 m to 600 m. While subaqueous dunes are found in many locations ...
ENSO forced variations of the sea surface temperature and adjusted sea level along the west coast of the United States Skillman, John B. (Monterey, California. Naval Postgraduate School, 1993-12);Daily coastal surface temperature and adjusted sea level data for the period 1955-1988 were used to characterize the surface temperature and adjusted sea level anomalies, and the propagation of features along the west coast ...
Harr, P.A.; Elsberry, R.L. (American Meteorological Society, 1995-05-01);The basic structure of the variability of the large-scale circulations over the tropical western Pacific is investigated with respect to its influence on tropical cyclone characteristics. A vector empirical orthogonal ...