Fusion of multiple sensor types in computer vision systems

Abstract

This research provides analysis of several approaches to the fusion of multiple dissimilar sensors to supplement simple color vision detection and recognition. Non-visible sensor systems can enhance computer vision systems. Our research investigates using thermal infrared (IR) sensors in combination with color data for object detection and recognition. We analyze several types of high-level and low-level sensor fusion to compare error rates with raw color and raw IR error rates in detection and recognition of vehicles in a scene. Principal components analysis is used to reduce the dimensionality of sensor input data in order to discard non-essential data, while preserving data important to classification. One recognition method showing promise is to exploit the strength of non-visible information (low light, shadows, etc.) to reduce the search space for color data by replacing the V channel in the HSV color sensor data with IR. For detection, one method showing promise is replacement or averaging of the dominant color channel with IR.