Algorithms for efficient intelligence collection

Abstract

Modern intelligence techniques have drastically increased the rate at which communications data can be intercepted. The increased ability to collect and store this data poses a significant processing problem for intelligence agencies. We develop a software library, implementing a previously developed mathematical model of the information selection problem facing these agencies: given a time constraint, which items should be screened in order to maximize the relevant information obtained. Using our software, we analyze the performance of several screening strategies on a variety of representative intercepted intelligence networks, which we construct using real world data sets. We show the model consistently outperforms more naive approaches on networks with clusters of relevant sources, and highlight the importance of exploration in robust screening strategies.