Detection and classification of Web robots with honeypots
Loading...
Authors
McKenna, Sean F.
Subjects
World Wide Web
crawlers
robots
malicious
logs
honeypots
crawlers
robots
malicious
logs
honeypots
Advisors
Rowe, Neil
Date of Issue
2016-03
Date
Mar-16
Publisher
Monterey, California: Naval Postgraduate School
Language
Abstract
Web robots are automated programs that systematically browse the Web, collecting information. Although Web robots are valuable tools for indexing content on the Web, they can also be malicious through phishing, spamming, or performing targeted attacks. In this thesis, we study an approach to Web-robot detection that uses honeypots in the form of hidden resources on Web pages. Our detection model is based upon the observation that malicious Web robots do not consider a resource’s visibility when gathering information. We performed a test on an academic website and analyzed the honeypots’ performance using Web logs from the site’s server. Our results did detect Web robots, but did not adequately detect the more sophisticated robots, such as those using deep-crawling algorithms with query generation.
Type
Thesis
Description
Series/Report No
Department
Computer Science
Computer Science
Organization
Identifiers
NPS Report Number
Sponsors
Funder
Format
Citation
Distribution Statement
Approved for public release; distribution is unlimited.
Rights
Copyright is reserved by the copyright owner.