Machine Learning Techniques for Identifying Anomalous Network Traffic

Abstract

Cyber investigations often involve analysis of large volumes of log files, including network flow data. Machine learning (ML) techniques allow analysts and examiners to more quickly identify traffic flows relevant to the investigation. The research will focus on the analysis of network flow data generated by the Audit Record Generation and Utilization System (ARGUS). Examples of anomalous traffic patterns of interest (not an exhaustive list) include traffic spikes, malware beaconing, command and control (C2) activity, data exfiltration, and scanning. The objective of the proposed study is to analyze network flow data with ML and heuristics algorithms to optimize time spent by analysts and investigators during cyber network forensic investigations (including, but not limited to, cyber incident handling and incident response investigations). We are analyzing ARGUS, and other network flow application data, with ML algorithms, with a focus on targeting and optimizing indicators-of-compromise (IOCs). ML is being leveraged to mine network flows to optimize the determination and identification of an ongoing compromise, or historical evidence of compromise (mining C2 channel data, beaconing, data exfiltration, unexpected encrypted traffic, or other anomalous network traffic). After an extensive review of various market solutions, we found that there is a general paucity of specific products addressing forensic analysis of anomalous network traffic a number of vendor products are headed in the direction of using ML algorithms that can be considered as a solution in analyzing network traffic flow. An analysis of the 10+ possibilities have been produced.