The Correct Use of Subject Matter Experts in Cost Risk Analysis

dc.contributor.authorBraxton, Peter J.
dc.contributor.authorDruker, Eric R.
dc.contributor.authorColeman, Richard L.
dc.contributor.corporateAcquisition Management (AM)
dc.contributor.corporateAcquisition Research Program (ARP)
dc.contributor.departmentAcquisition Management
dc.contributor.departmentOther Research Faculty
dc.date30-Apr-10
dc.date.accessioned2013-05-08T21:19:43Z
dc.date.available2013-05-08T21:19:43Z
dc.date.issued2010-04-30
dc.descriptionProceedings Paper (for Acquisition Research Program)en_US
dc.description.abstractSubject Matter Experts (SMEs) are commonly used in cost risk analysis (and in other fields as well) for values that either are not available in historical data or for which no appropriate analogy can be found. Problems commonly arise in two areas in particular: (1) when multiple experts give opinions on a single effect or entity and the inputs are not identical in distribution (which is almost inevitable), and (2) when a single expert provides distributional information that is intractable or suspiciously unlikely in its form (which is common).This paper will put forward correct solutions in case (1), in which the authors'' experience shows that practitioners (and even experts) use incorrect solutions. It is important to note that the commonly exercised incorrect solution underestimates the dispersion, and thus the 80th percentile, in some cases by a large margin. The authors believe that their solution is rare and, further, are unaware of any use of the solution, and will recommend tenets to guide the practitioner. In preparation for the solutions laid out above, the authors will first describe the method of expert-based risk analysis, with the erroneous method for combining SME testimony, and then show the correction. An analytical treatment will quantify the impacts of the erroneous approach. The paper will also explain why the new method of conflating expert assessments is to be preferred to the common Delphi technique, which may fall prey to both anchoring and domination by a vocal minority. The paper will also briefly address case (2) by presenting common examples of problematic formulations and proposed resolutions. These include intractable specification of a triangular distribution, specification of a discrete categorical distribution when triangular was intended, and specification of a triangular with low and high values that are not the true extremes as well as errors committed by the risk analyst. In any situation, correct treatment of risk is important. In the current era, with 80th percentiles required for all weapon systems cost estimates by the Weapon Systems Acquisition Reform Act of 2009, and budgeting to the 80th percentile as the default practice, the correct determination of the distribution is more important than ever before.en_US
dc.description.distributionstatementApproved for public release; distribution is unlimited.
dc.description.sponsorshipNaval Postgraduate School Acquisition Research Programen_US
dc.identifier.npsreportNPS-AM-10-031
dc.identifier.urihttps://hdl.handle.net/10945/33464
dc.publisherMonterey, California. Naval Postgraduate Schoolen_US
dc.relation.ispartofseriesAcquisition Research Symposium
dc.rightsThis publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.en_US
dc.subject.authorCost Risk Analysisen_US
dc.titleThe Correct Use of Subject Matter Experts in Cost Risk Analysisen_US
dc.typeTechnical Reporten_US
dspace.entity.typePublication
relation.isOrgUnitOfPublication394ae3f1-b0b4-47ac-ba4b-b6ea3bc4dab9
relation.isOrgUnitOfPublicationbc7988c8-57ff-485c-9348-877f4ef91f4e
relation.isOrgUnitOfPublication.latestForDiscoverybc7988c8-57ff-485c-9348-877f4ef91f4e
relation.isSeriesOfPublicatione652aa3b-f22b-4814-8e9d-6391f82531e0
relation.isSeriesOfPublication.latestForDiscoverye652aa3b-f22b-4814-8e9d-6391f82531e0
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
NPS-AM-10-031.pdf
Size:
397.52 KB
Format:
Adobe Portable Document Format