Automatic Classification of Objects in Captioned Depictive Photographs for Retrieval
Abstract
We investigate the robust classification of objects within photographs in a large and varied picture library of natural photographs. We
assume the photographs have captions describing and locating only imprecisely some of the objects present in the picture, as is
common in libraries. Our approach does not match to shape templates nor do full image picture understanding, neither of which works
well for natural photographs where appearance varies considerably with lighting and perspective. Instead, we strike a robust
compromise by statistically characterizing photograph regions with 17 key domain-independent parameters covering shape, color,
texture, and contrast. We explored two ways to use the parameters to classify picture regions, case-based reasoning and a neural
network, both of which require training. We found the neural network outperformed case-based reasoning, especially when we
included caption information and a separate neuron inferring likelihood that a region was the "visual focus" of the picture. Then 25-
category shape classification succeeded 48.1% of the time on a set of pictures randomly selected from a large picture library currently
in use. Our work represents good progress on the difficult problem of retrieval by content from large real-world picture libraries.
Description
This article is Chapter 4 in Intelligent Multimedia Information Retrieval, ed. M. Maybury, pp. 65-79, Cambridge, MA: AAAI Press, 1997
Collections
Related items
Showing items related by title, author, creator and subject.
-
Effects of Mission Rehearsal Simulation on Air-to-Ground Target Acquisition
Krebs, William K.; McCarley, Jason S.; Bryant, Eric V. (Sage Journals, 1999-12-01);Traditionally military aviators have prepared for air-to-ground bombing missions with maps and aerial photographs of their targets. Mission rehearsal systems augment these media by allowing pilots to view simulated ingress ... -
NPSNET vehicle database: an object-oriented database in a real-time vehicle simulation
Borden Davis, Susan C. (Monterey, California. Naval Postgraduate School, 1996-06);The Naval Postgraduate School has actively explored the design and implementation of NPSNET, a real-time three-dimensional simulator on low-cost, readily accessible workstations. NPSNET involves a tremendous amount of ... -
Classification, search, and retrieval in a multi-variable, multi-level taxonomy: application to DecisionNet
Corgnati, Christopher M (Monterey, California. Naval Postgraduate School, 1997-09);The explosion of information available on global computer networks underlines the need for effective repositories that facilitate organization of, and search for, information. These digital repositories may contain simple ...