Issues and Methods for Assessing COTS Reliability, Maintainability, and Availability
Abstract
Many vendors produce products that are not domain specific (e.g., network server) and have limited functionality (e.g., mobile phone). In contrast, many customers of COTS develop systems that am domain specific (e.g., target tracking system) and have great variability in functionality (e.g., corporate information system). This discussion takes the viewpoint of how the customer can ensure the quality of COTS components. In evaluating the benefits and costs of using COTS, we must consider the environment in which COTS will operate. Thus we must distinguish between using a non-mission critical application like a spreadsheet program to produce a budget and a mission critical application like military strategic and tactical operations. Whereas customers will tolerate an occasional bug in the former, zero tolerance is the rule in the latter. We emphasize the latter because this is the arena where there are major unresolved problems in the application of COTS. Furthermore, COTS components may be embedded in the larger customer system. We refer to these as embedded systems. These components must be reliable, maintainable, and available, and must be with the larger system in order for the customer to benefit from the advertised advantages of lower development and maintenance costs. Interestingly, when the claims of COTS advantages are closely examined, one finds that to a great extent these COTS components consist of hardware and office products, not mission critical software [1]. Obviously, COTS components are different from custom components with respect to one or more of the following attributes: source, development paradigm, safety, reliability, maintainability, availability, security, and other attributes. However, the important question is whether they should be treated differently when deciding to deploy them for operational use; we suggest the answer is no. We use reliability as an example to justify our answer. In order to demonstrate its reliability, a COTS component must pass the same reliability evaluations as the custom components, otherwise the COTS components will be the weakest link in the chain of components and will be the determinant of software system reliability. The challenge is that there will be less information available for evaluating COTS components than for custom components but this does not mean we should despair and do nothing. Actually, there is a lot we can do even in the absence of documentation on COTS components because the customer will have information about how COTS components are to be used in the larger system. To illustrate our approach, we will consider the reliability, maintainability, and availability (RMA) of COTS components as used in larger systems. Finally, COTS suppliers might consider increasing visibility into their products to assist customers in determining the components' fitness for use in a particular application. We offer ideas of information that would be useful to customers, and what vendors might do to provide it.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.Collections
Related items
Showing items related by title, author, creator and subject.
-
The implementation of CRM at FISC Norfolk Detachment Philadelphia
Tierney, Joseph A. (Monterey, California. Naval Postgraduate School, 2003-06);Customer Relations Management (CRM) can be defined in many ways. In this thesis, it is defined as the business process an organization uses to provide exceptional acquisition services to existing customers to retain their ... -
Measuring Air Force Contracting customer satisfaction
Davis, Jamie (Monterey, California: Naval Postgraduate School, 2015-12);This research gathers background information to identify which customer satisfaction elements should be included in a standardized tool that measures the level of customer satisfaction for AF Contracting’s external and ... -
Multi-phase-mission reliability of maintained systems
Bell, Merlin Gene (Monterey, California. Naval Postgraduate School, 1975-12);In a phased mission the functional organization of the system changes at selected times which mark the boundaries of the phases of the mission. Existing methods for analysis of phased missions are modified and extended ...