Automatic Classification of Objects in Captioned Depictive Photographs for Retrieval
Rowe, Neil C.
MetadataShow full item record
We investigate the robust classification of objects within photographs in a large and varied picture library of natural photographs. We assume the photographs have captions describing and locating only imprecisely some of the objects present in the picture, as is common in libraries. Our approach does not match to shape templates nor do full image picture understanding, neither of which works well for natural photographs where appearance varies considerably with lighting and perspective. Instead, we strike a robust compromise by statistically characterizing photograph regions with 17 key domain-independent parameters covering shape, color, texture, and contrast. We explored two ways to use the parameters to classify picture regions, case-based reasoning and a neural network, both of which require training. We found the neural network outperformed case-based reasoning, especially when we included caption information and a separate neuron inferring likelihood that a region was the "visual focus" of the picture. Then 25- category shape classification succeeded 48.1% of the time on a set of pictures randomly selected from a large picture library currently in use. Our work represents good progress on the difficult problem of retrieval by content from large real-world picture libraries.
This article is Chapter 4 in Intelligent Multimedia Information Retrieval, ed. M. Maybury, pp. 65-79, Cambridge, MA: AAAI Press, 1997
Showing items related by title, author, creator and subject.
Rowe, Neil C. (Monterey, California. Naval Postgraduate School, 1999);The MARIE project has explored knowledge-based information retrieval of captioned images of the kind found in picture libraries and on the Internet. It exploits the idea that images are easier to understand with context, ...
Retrieving Captioned Pictures Using Statistical Correlations and a Theory of Caption-Picture Co-reference Rowe, Neil C. (Monterey, California. Naval Postgraduate School, 1995-04);The MARIE project is investigating new methods for efficient information retrieval of captioned multimedia from multimedia libraries. Captions are essential to understanding multimedia and to finding relevant examples ...
Ozdemir, Dogan (Monterey, California. Naval Postgraduate School, 1992-12);Rapid prototyping with automated retrieval of reusable software components is a software development method to construct software systems expeditiously. This thesis describes a tool to enhance the practice of software ...