CERTS: a comparative evaluation method for risk management methodologies and tools

Abstract

This thesis develops a comparative evaluation method for computer security risk management methodologies and tools. The subjective biases inherent to current comparison practices are reduced by measuring unique characteristics of computer security risk management methodologies. Standardized criteria are established and described by attributes which in turn are defined by metrics that measure the characteristics. The suitability of a method or tool to a particular organizational situation can then be analyzed objectively. Additionally, our evaluation method facilitates the comparison of methodologies and tools to each other. As a demonstration of its effectiveness, our method is applied to four distinct risk management methodologies and four risk management tools. Alternative models for utilizing the evaluation method are presented as well as possible directions for their application. Without an adequate means of comparing and evaluating risk management decision-making methodologies, the metadecision (the selection of a risk management method or tool) becomes arbitrary and capricious, thereby making an inappropriate selection more likely. Selection of an inappropriate method or tool could lead to excessive costs, misdirected efforts, and the loss of assets. The systematic and standard comparison method developed in this thesis resolves that problem.