Evaluation of the impact of multispectral image fusion on human performance in global scene processing

Abstract

An observer extracts local and global information from a natural scene to form a visual perception. Neisser and Treisman demonstrated that a natural scene contains different types of features, i.e., color, edges, luminance, and orientation to aid visual search. Infrared and visible sensors present nighttime images to an observer to aid target detection. These sensors present the observer an adequate representation of a nighttime scene, but sometimes fail to provide quality features for accurate visual perception. The purpose of this thesis is to investigate whether color features (combining an infrared and visible sensor image) improve visual scene comprehension compared to single band grayscale features during a signal detection task. Twenty three scenes were briefly presented in four different sensor formats (infrared, visible, fused monochrome, and fused color) to measure subjects global visual ability to detect whether a natural scene was right side up or upside down. Subjects are significantly more accurate at detecting scene orientation for an infrared and fused color scene compared to a fused monochrome and visible scene. Both the infrared and fused color sensor formats provide enough essential features to allow an observer to perceptually organize a complex nighttime scene.