Vulnerability analysis of HD photo image viewer applications

Abstract

The introduction of Microsoft's new graphics file format, Windows Media Photo, into the mainstream market in 2006 has been one of the most interesting developments in the digital world. The file format, which has since been renamed to HD Photo in November of 2006, is being touted as the successor to the ubiquitous JPEG image format, as well as the eventual de facto standard in the digital photography market. With massive efforts already underway to increase the software support of this file format, to make available support for digital camera makers to incorporate it into their products, and to propose the file format to the Joint Photography Experts Group in order to make HD Photo as a standard itself, HD Photo is poised to become as widespread as any of the common image file formats today. This provides the motivation into studying whether the HD Photo file format can be used as a vehicle to compromise a user s system. This work addresses the security of handling the HD Photo file format as it pertains to image viewer applications. Whenever an application is updated to accommodate a new file format, it is possible that the application in question can be vulnerable to exploitation. This is a concern, especially if a malformed instance of that file format can make the application to deviate from its specified behavior and cause the execution of arbitrary code. This thesis investigates if some of the existing applications today that render image files are susceptible to compromise by opening a malformed HD Photo image file. The goal of this thesis is to test the security of various image viewer applications compatible with the HD Photo file format. We modified MiniFuzz, an automated fuzzing tool, to conduct mutation-based smart fuzzing and generation-based fuzzing. The test instrumentation worked correctly, but the test cases did not reveal any security vulnerabilities.