A BENCHMARK FRAMEWORK AND SUPPORT FOR AT-SCALE BINARY VULNERABILITY ANALYSIS

Abstract

Today, software is integrated into nearly every aspect of our lives and so are its vulnerabilities. Exploited software vulnerabilities can have detrimental financial, social, and economic effects. Researchers rely on Vulnerability Analysis Tools and Techniques (VATT) to amplify the vulnerability analysis process. There are hundreds of VATTs on the market, but there is no way to compare their relative efficacy. We developed a framework for the Benchmark for Vulnerability Analysis Tools and Techniques (BVATT). In addition to providing key metrics for quantifying the performance of a particular VATT, the proposed framework ensures that BVATT will facilitate the comparison of different VATTs in a manner that is repeatable, reproducible, fair, verifiable, and relevant. Additionally, in the past decade, there has been a noteworthy increase of VATTs that leverage machine-learning and data-mining techniques to identify vulnerabilities. Yet, there is no open-source tool to synthesize the extraction, cleaning, and transformation of common features from binary files to be compatible with these techniques. We develop such a tool, and call it BiSECT (Binary Synthesized Extraction, Cleaning, and Transformation). BiSECT reduces the barrier to entry and makes binary vulnerability analysis using data mining and machine learning more accessible to researchers.