EVALUATING SBIR PROPOSALS: A COMPARATIVE ANALYSIS USING ARTIFICIAL INTELLIGENCE AND STATISTICAL PROGRAMMING IN THE DOD ACQUISITIONS PROCESS
Loading...
Authors
Tores, Cullen G.
Subjects
Small Business Innovation Research
SBIR
generative text artificial intelligence
AI
large language model
LLM
OpenAI
Department of Defense
DOD
SBIR
generative text artificial intelligence
AI
large language model
LLM
OpenAI
Department of Defense
DOD
Advisors
Mortlock, Robert F.
Massenkoff, Maxim
Date of Issue
2024-06
Date
Publisher
Monterey, CA; Naval Postgraduate School
Language
Abstract
The Small Business Innovation Research (SBIR) program is a tool that the Department of Defense (DOD) uses to encourage industry development in technology that the market is otherwise not demanding. This helps to drive innovation and facilitate competition for government contracts. However, the source selection process within the SBIR program could be improved. It currently takes too long and is riddled with inconsistencies. Given this application and the rising interest in artificial intelligence (AI), it is worth exploring ways to augment the source selection process with AI. This study assesses the effectiveness of using large language models (LLMs) to automate classification of acquisition proposals as either competitive or noncompetitive. This study used R to extract text from the proposals, interact with OpenAI’s models, and then iteratively loop through all of the proposals until completion. The intent was to establish a faster, more consistent, and objective evaluation system when compared to subjective human assessments. The final analysis indicated an emerging capability with vast potential, but one that is not reliable enough for immediate application into the SBIR program. This study emphasizes the importance of accuracy and reliability in DOD’s initiatives and highlights the potential roles of AI in optimizing DOD acquisitions.
Type
Thesis
Description
Series/Report No
Department
Organization
Identifiers
NPS Report Number
Sponsors
Funder
Format
Citation
Distribution Statement
Distribution Statement A. Approved for public release: Distribution is unlimited.
Rights
This publication is a work of the U.S. Government as defined in Title 17, United States Code, Section 101. Copyright protection is not available for this work in the United States.