Automatic Classification of Objects in Captioned Depictive Photographs for Retrieval
Loading...
Authors
Rowe, Neil C.
Frew, Brian
Subjects
Advisors
Date of Issue
1997
Date
1997
Publisher
Monterey, California. Naval Postgraduate School
Language
Abstract
We investigate the robust classification of objects within photographs in a large and varied picture library of natural photographs. We
assume the photographs have captions describing and locating only imprecisely some of the objects present in the picture, as is
common in libraries. Our approach does not match to shape templates nor do full image picture understanding, neither of which works
well for natural photographs where appearance varies considerably with lighting and perspective. Instead, we strike a robust
compromise by statistically characterizing photograph regions with 17 key domain-independent parameters covering shape, color,
texture, and contrast. We explored two ways to use the parameters to classify picture regions, case-based reasoning and a neural
network, both of which require training. We found the neural network outperformed case-based reasoning, especially when we
included caption information and a separate neuron inferring likelihood that a region was the "visual focus" of the picture. Then 25-
category shape classification succeeded 48.1% of the time on a set of pictures randomly selected from a large picture library currently
in use. Our work represents good progress on the difficult problem of retrieval by content from large real-world picture libraries.
Type
Book Chapter
Description
This article is Chapter 4 in Intelligent Multimedia Information Retrieval, ed. M. Maybury, pp. 65-79, Cambridge, MA: AAAI Press, 1997
Series/Report No
Department
Computer Science (CS)
Organization
Identifiers
NPS Report Number
Sponsors
sponsored by DARPA as part of the I3 Project under AO 8939, and by the U.S. Army Artificial Intelligence Center
Funder
Format
Citation
This article is Chapter 4 in Intelligent Multimedia Information Retrieval, ed. M. Maybury, pp. 65-79, Cambridge, MA: AAAI Press, 1997
Distribution Statement
Approved for public release; distribution is unlimited.