Natural-Language Retrieval of Images Based on Descriptive Captions
Guglielmo, Eugene J.
Rowe, Neil C.
MetadataShow full item record
We describe a prototype intelligent information retrieval system that uses natural-language understanding to efficiently locate captioned data. Multimedia data generally require captions to explain their features and significance. Such descriptive captions often rely on long nominal compounds (strings of consecutive nouns) which create problems of disambiguating word sense. In our system, captions and user queries are parsed and interpreted to produce a logical form, using a detailed theory of the meaning of nominal compounds. A fine-grain match can then compare the logical form of the query to the logical forms for each caption. To improve system efficiency, we first perform a coarse-grain match with index files, using nouns and verbs extracted from the query. Our experiments with randomly selected queries and captions from an existing image library show an increase of 30% in precision and 50% in recall over the keyphrase approach currently used. Our processing times have a median of seven seconds as compared to eight minutes for the existing system, and our system is much easier to use.
ACM Transactions on Information Systems, Vol. 14, No. 3, July 1996, Pages 237–267.The article of record as published may be located at http://dx.doi.org/10.1145/230538.230539
Showing items related by title, author, creator and subject.
Guglielmo, Eugene J. (Monterey, California. Naval Postgraduate School, 1992-09);This report describes an intelligent information retrieval system, MARIE, that employs natural language processing techniques for indexing and retrieving multimedia data. Captions describe photographs from the Naval Air ...
Guglielmo, Eugene J.; Rowe, Neil C. (Monterey, California. Naval Postgraduate School, 1992);This paper briefly describes the current implementation status of an intelligent information retrieval system, MARIE, that employs natural language processing techniques. Descriptive captions are used to identify photographic ...
Rowe, Neil C.; Guglielmo, Eugene J. (Monterey, California. Naval Postgraduate School, 1992-07); NPS-CS-92-011Descriptive natural-language captions can help organize multimedia data. We described our MARIE system that interprets English queries directing the fetch of media objects. it is novel in the extent to which it exploits ...