Automatic Classification of Objects in Captioned Depictive Photographs for Retrieval
Rowe, Neil C.
MetadataShow full item record
We investigate the robust classification of objects within photographs in a large and varied picture library of natural photographs. We assume the photographs have captions describing and locating only imprecisely some of the objects present in the picture, as is common in libraries. Our approach does not match to shape templates nor do full image picture understanding, neither of which works well for natural photographs where appearance varies considerably with lighting and perspective. Instead, we strike a robust compromise by statistically characterizing photograph regions with 17 key domain-independent parameters covering shape, color, texture, and contrast. We explored two ways to use the parameters to classify picture regions, case-based reasoning and a neural network, both of which require training. We found the neural network outperformed case-based reasoning, especially when we included caption information and a separate neuron inferring likelihood that a region was the "visual focus" of the picture. Then 25- category shape classification succeeded 48.1% of the time on a set of pictures randomly selected from a large picture library currently in use. Our work represents good progress on the difficult problem of retrieval by content from large real-world picture libraries.
This article is Chapter 4 in Intelligent Multimedia Information Retrieval, ed. M. Maybury, pp. 65-79, Cambridge, MA: AAAI Press, 1997
Showing items related by title, author, creator and subject.
Borden Davis, Susan C. (Monterey, California. Naval Postgraduate School, 1996-06);The Naval Postgraduate School has actively explored the design and implementation of NPSNET, a real-time three-dimensional simulator on low-cost, readily accessible workstations. NPSNET involves a tremendous amount of ...
Classification, search, and retrieval in a multi-variable, multi-level taxonomy: application to DecisionNet Corgnati, Christopher M (Monterey, California. Naval Postgraduate School, 1997-09);The explosion of information available on global computer networks underlines the need for effective repositories that facilitate organization of, and search for, information. These digital repositories may contain simple ...
Rowe, Neil C. (2002-07);Finding multimedia objects to meet some need is considerably harder on the World Wide Web than finding text because content-based retrieval of multimedia is much harder than text retrieval and caption text is inconsistently ...