Security Measurement—Establishing Confidence That System and Software Security Is Sufficient

Abstract

Evaluating the software assurance of a product as it functions within a specific system context involves assembling carefully chosen metrics that demonstrate a range of behaviors to establish confidence that the product functions as intended and is free of vulnerabilities. The first challenge is to establish that the requirements define the appropriate security behavior and the design addresses these security concerns. The second challenge is to establish that the completed product, as built, fully satisfies the specifications. Measures to provide assurance must, therefore, address requirements, design, construction, and test. We know that software is never defect free. According to Jones and Bonsignour (2012), the average defect level in the United States is 0.75 defects per function point or 6,000 per million lines of code (MLOC) for a high-level language. Thus, software, on average, cannot always function perfectly as intended. Additionally, we cannot establish that software is completely free from vulnerabilities based on our research which indicates that 5% of defects should be categorized as vulnerabilities. So how can we establish reasonable confidence in software security? To answer this question, the Software Engineering Institute (SEI) is researching how measurement can be used to establish confidence in software security. This paper will share our progress to date.