Nginx vs. Apache2 - in Rail...

First a word on my methods.

The intent was not to strip down the servers to only Apache vs. Nginx. Instead I wanted to find out which server gave better performance with respect to the Rails app it was serving up. This particular Rails app is running 4 mongrels, ferret server, memcache, mysql, firebird and monit. The testing server was minimal and in no way represents a true production machine, but as long as the environment was the same between test runs I assume the results should be valid. This particular server is a VM with 512 mem. Both Nginx and Apache2 were straight installs from the Debian Lenny distro, no additional modifications were made beyond the necessary steps to get the app running. The performance tests were run from a seperate machine on our local network.

Next I wanted to test 3 different types of traffic to the application. First I wanted to test the full Rails stack, that is - a request that used sessions, logging and some template rendering. Second was a stripped Rails request - no session, logging or template rendering. A simple request that replied a bit of text but required the mongrels to run the Rails code. The last test, mostly for baseline purposes, was a run on the 404.html file - to see how the two servers match up when serving static content.

I ran tests using Apache Bench and Httperf as I wanted to try out the two and see if one was better. The result was that neither was necessarily better. AB was more reliable to return a result set, but Htpf gave more data. Probably I'll keep working with both and learn more about how to configure them. For now I find them equally useful. For the test results I'll indicate which tool was used.

The Results

First is the applications index page to test the full Rails stack. This was an AB test. As can be seen in the graph below - the results are very close. I interpret this result to mean that the Rails app is keeping the performance from either web server from being a factor.



Next is the simplified call to the Rails stack. This is also an AB test. Now we're seeing something. Clearly Nginx is doing it's business at a far faster rate than Apache.

What is also interesting, but not seen since the data was not capture by either test tool, was the memory usage. Apache used mass quantities of memory - in fact there were several occasions when the entire swap was consumed and I needed to reboot the server. Nginx did grow in memory footprint under load but it generally hit a max load point and never overloaded the server.



Finally we have the static content performance tested vi AB. Now we see the knock-out blow. Nginx takes the gloves off and handily beats Apache in this round serving over 6 times the number of requests.



At this point I was able to convince people that giving Nginx a trial run was a good idea. But I didn't stop there, since I like making graphs. I made this chart of the transfer rates as measured by AB during the static content tests.



The last graph I have is from Htpf of the minimal Rails test. The nice part about Htpf is that it gives you the standard deviation, which I appreciate. This really just underscores Nginx's victory. Not only does Nginx provide better reply rates but the std dev is down near zero.



So there you have it, make of it what you will. I don't intend to mean this is the end-all comparison. Apache has some nice features and I'll continue to use it. For example, getting PHP running on Apache is much easier than on Nginx. On the other hand, configuring Nginx for Rails is easier than with Apache.

Of course this shouldn't replace your own testing. YMMV. But for my part, I'll recommend Nginx for running Rails until a better option is presented.