User Tools

Site Tools


ongoing

Ongoing projects

Web Crawlers Behaving Like Humans: Are We There Yet!?

Student: Dmitry Drinfeld

Supervisor: Natalija Vlajic

Required Background: CSE4480 prerequisites

Description

Distributed Denial of Service (DDoS) attacks are recognized as one of the most serious threats to today's Internet due to the relative simplicity of their execution and their ability to severely degrade the quality at which Web-based services are offered to the end users. An especially challenging form of DDoS attacks are the so-called Application-Layer DDoS attacks. Namely:

  1. In Application-Layer DDoS attacks, the attackers utilize a flood of legitimate-looking Layer-7 network sessions (i.e., sessions that are generally hard to detect and/or filter out by a firewall or an IDS system)
  2. Increasingly, these sessions comprise HTML requests generated by a cleverly programmed crawler that executes a semi-random walk over the web site links, thereby attempting to appear as a legitimate human visitor.

The goal of this project is to investigate the state of the art in malicious web crawler design. In particular, the project will look into the challenges of designing a smart-DDoS-crawler from the attacker point of view - one of these challenges being the estimation of web-page popularity assuming no a priori access to the web-logs of the victim web-site.

ongoing.txt · Last modified: 2011/10/14 12:57 by bil