Web proxy script in Perl using HTTP::Daemon

The purpose of the task is to write a robust and lightweight Web proxy script in Perl using HTTP::Daemon class. Please read "Detailed requirements" below for full details. ## Deliverables The purpose of the task is to write a robust and lightweight Web proxy script in Perl using HTTP::Daemon class. Please read on for full details. BACKGROUND Web proxy is a Web application that is used to browse anonymously. Basically the script resides on a Web server somewhere and is used do download requested Web pages and displaying them to the user. To get an idea of how it works, please visit [login to view URL] . They differ from standard HTTP or SOCKS proxies in that they do not require the user to reconfigure their browser settings - they work right in the browser itself. The main purpose of such application is to get around Web filters placed by the network administrator or the government. For example, these proxies are used extensively in oppressed countries such as Iran or China, where the Internet is heavily censored. There are several so-called Web proxy scripts in use today, the most popular ones being Glype, PHPproxy and CGIproxy. They work as a normal Web script, written in PHP or Perl. They require a Web server (most usually Apache) to work. AIM OF THIS PROJECT I want to create a Web proxy script that is a standalone application - that is, to eliminate Apache (or other Web server) from the equation. This should reduce the overhead associated with processing user requests, and thus result in lower CPU and memory usage in the server. My choice is Perl with HTTP::Daemon class. Perl is a language specifically designed to parse text (and this is what such application does most of the time, for details read on) and has tons of useful ready-made modules that can be applied to this task. MY CURRENT SOLUTION Attached is the script that I started to write as a proof of concept. I spent two days on it. It kind of works, but has many deficiencies. You are not required to improve this script; rather look at it as a description and example of what I need to be done. The script uses HTTP::Daemon as an application framework, and each client request is handled using a separate Perl thread. Because the server does not fork, it is quite efficient. Upon receiving a request from the client, the requested page is fetched with LWP::UserAgent. If it is a HTML page (recognized by MIME-type), it is further parsed so that all links to external objects are converted to be relayed by the proxy. This way, if the user clicks any links on "proxified" page, he will still use the proxy to view the link; also all images and other objects will be displayed using the proxy. For speed, HTML::TagReader is used because I found it to be the fastest HTML parser available for Perl. The script has many problems though, which would need to be corrected, for example: * The requested URLs are not encrypted, so the network administrator will see easily what sites the user visits. * Cookies are handled in a wrong way - all cookies for all visited domains are sent with every request, which is impractical and breaks many websites. * Popular websites such as Facebook or YouTube do not work (in stock Web proxies they get special treatment using plugins to rewrite requests, because of high usage of JavaScript and complexity). You can run the script on your server in the shell, and then go to http://yourserver:11111/g/?url=[login to view URL] to see how it works. As you cna see, at the top of the page is a small form loaded from a static file that shows the current URL and allows to change it easily. YOUR TASK You are required to write the complete application in a proper way. Generally the basic structure with HTTP::Daemon and Perl threads should remain. The code must be well documented and modularized (mine unfortunately is not) so that I will be able to maintain and modify it further. There must be mechanisms for protecting from overloading the server, such as limiting the size of requested files to prevent memory exhaustion (I did that in my script, as you can see), limiting number of requests per IP (this I didn't do), good handling of errors etc. A small, configurable cache of static files (such as images) would be very welcome, because it will further reduce the server load. Care must be taken in handling cookies, which must be passed back and forth between user's browser and the destination server. I did only very rudimentary cookie handling in my script which is not sufficient. You must make sure that these popular websites work in the proxy: Facebook, Myspace, Youtube, Bebo, Orkut. You can look at plugins for Glype how it is done (for example, for Facebook they redirect all requests to mobile site because the original site is so bloated with JavaScript).