Improving Defense Acquisition Processes with Evidence-based Analysis: An Illustrative Case Using the DoD SBIR Program
Abstract
This paper proposes and demonstrates that experimental and quasi-experimental program evaluation methods can be applied to some parts of the defense acquisition system to provide evidence of program effectiveness. The specific example presented is a quasi-experimental evaluation of the Department of Defense Small Business Innovation Research (DoD SBIR) program. Quasi-experimental methods are a set of program evaluation techniques that allow researchers to approximate the results of an experimental study, such as a randomized controlled trial, without performing the experiment. The paper performs a quasi-experimental evaluation of the DoD SBIR program, which provides evidence that the program is effective at transitioning SBIR-funded technologies into other DoD programs. This demonstration that quasi-experimental methods can be used to evaluate certain aspects of the DoD acquisition system provides policy analysts with new tools to meet Congressional requirements for acquisition system evaluation. The paper recommends that more quasi-experimental studies be conducted and actual experimental studies be executed. These methods can help the DoD overcome the well-documented deficiency in evaluating the effectiveness of its acquisition systems. The Office of Management and Budget, the Government Accountability Office and the House Armed Services Committee unanimously agree that the DoD does not objectively measure the performance of its acquisition system.
Description
Proceedings Paper (for Acquisition Research Program)
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.NPS Report Number
NPS-AM-10-056Related items
Showing items related by title, author, creator and subject.
-
Small Business Research in a World of Skewed Returns
Edison, Toby (2012-04-30); NPS-AM-12-C9P05R02-052This paper observes that outcomes from small business support programs are likely to have a small fraction of observations with extremely positive outcomes that may impact program evaluations and conclusions. Current ... -
COMPARISON OF NAVAL ACQUISITION PROCESSES BETWEEN THE UNITED STATES AND TAIWAN
Liu, Chih-chieh (Monterey, CA; Naval Postgraduate School, 2021-06);This research uses a case study approach to analyze the respective naval acquisition processes of the United States and Taiwan. The methodology enables a comparison of the acquisition systems used by the U.S. Navy (USN) ... -
Using "clickers" in the classroom to increase the level of student interaction
Prewitt, Robert R.; Oropeza, Katherine B. (Monterey, California. Naval Postgraduate School, 2008-12);Interaction is crucial in classrooms because increased interaction is linked to increased learning. Past studies report that students learn by a myriad of methods, and that it is up to the instructor to promote as many ...