imedo Development Blog

I don’t know why we didn’t try that earlier, but haproxy is a far better proxy solution then all the others we tried. Ok, we didn’t try that many but the most common ones on the Rails deployment landscape:

• Apache 2.2 mod_proxy_balancer
• nginx standard proxy module
• nginx with fair proxy module
• haproxy

So why is haproxy so much better? It is better because it actually can limit the requests per mongrel to one at a time. This important because otherwise you get behavour like this: One mongrel has a long running request and through round-robin, gets another request while other mongrels are idleing. Also mongrels with a request queue bigger then one start to eat memory like hell.

The same thing is possible with Apache’s mod_proxy_balancer. We tried and failed to get it working. And it seems as if we are not alone with that problem

The plain nginx balancer has the same problem. This is where the fair proxy module comes in. It’s supposed to send requests only to idleing mongrels. But we had the same “mongrels with many requests while others are ideling” problem again.

We finally tried haproxy which is in use for Rails deployments for quite some time but got a lot of buzz recently. Ilya Grigorik wrote a nice article about load balancing QoS with haproxy and Alexander Staubo posted a performance comparison of nginx and haproxy which got the attention of William Tarreau (the haproxy author). They found some haproxy bugs which got fixed and resulted in an even better performance. Details can be found in the second comparison of haproxy and nginx

So we are quite happy with haproxy at the moment and hope it stays this way.

Popularity: 1% [?]

The nginx fair proxy module is useful if you have some actions that take longer and you want to avoid request queueing on the mongrel side. Ezra Zygymtowicz wrote an article about the module and here are some performance testing results for the nginx fair proxy module in combination with thin

If you want to use the module on a debian system you have to build your own package because it’s not included in the standard package and the standard package is outdated anyway. Here’s a short step-by-step howto:

1. Install build debian tools

$ sudo apt-get install build-essential dpkg-dev

2. Download the latest nginx tarball

$ wget http://sysoev.ru/nginx/nginx-0.6.31.tar.gz # 0.6.31 is the latest stable version

3. Get latest debian package

$ apt-get source nginx

4. Unpack tarball

$ tar xvzf nginx-0.6.31.tar.gz

5. Copy debian package dir to the new directory

$ cp -r nginx-0.4.23/debian nginx-0.6.31

6. Geht fair proxy module

$ mkdir /somedir/ngx_http_upstream_fair_module
$ cd /somedir/ngx_http_upstream_fair_module
$ curl -o ngx_http_upstream_fair_module.c  http://github.com/gnosek/nginx-upstream-fair/tree/master%2Fngx_http_upstream_fair_module.c?raw=true
$ curl -o config  http://github.com/gnosek/nginx-upstream-fair/tree/master%2Fconfig?raw=true

7. Add configure option in debian/rules

--add-module=/somedir/ngx_http_upstream_fair_module

8. Change debian changelog file in debian/changelog

nginx (0.6.31-1) unstable; urgency=low

  * added fair proxy module

 -- Your Name   Mon, 26 May 2008 07:12:46 +0200

You have to stick to exactly this format for the changelog otherwise the package builder will complain.

8. Build the package

$ dpkg-buildpackage -b

9. Install the package and have fun with your brand new fair proxying nginx

$ dpkg -i nginx_0.6.31-1_i386.deb

Don’t forget to enable the fair proxy module using the “fair” directive in the “upstream” section of your nginx.conf. I recommend the use of the nginx config file generator to generate the nginx.conf file.

Popularity: 1% [?]