Boot Camp for Cognitive Systems
Abstract
The Defense Advanced Research Projects Agency
(DARPA) has implemented a program to build the first
instance of a complete cognitive agent. The program,
called Personalized Assistant that Learns (PAL), is
expected to yield new cognitive technology of significant
value not only to the military, but also to the business and
academic sectors. (Gunning 2004)
With traditional engineering projects, evaluation can be
done in a straightforward manner determining if the
documented requirements of the system have been met.
Agent-based capabilities and other network centric
capabilities (e.g., web services) complicate matters because
the environment that they will operate under constantly
changes. Add to that complication, the ability to learn new
capabilities, and testing whether or not a new agent is
ready to be deployed becomes a problem beyond the
current state of art and practice.
In this paper an initial experiment design is discussed as
well as a description of a broader approach for evaluation
in transitioning cognitive systems that learn into an
operational environment.