Apache2 proxy with Lighttpd - FastCGI for Rails

To you visitor, an elated 'Hello' from "Lighty" - as it is so fondly referred to. Thats right, my server greeted you not just with the standard apache service, but instead with both apache and Lighttpd. How could this be? And most importantly, why do I have apache2 and lighttpd running on the same server? Well, listen close and I will divulge the recent happenings surrounding my server and my websites.

Well it all started with this innocent periodically_call_remote function I had on my main page here. It was set to call back for a snippet of data every 8 seconds. Nothing too complex was ever returned (at most about 3-4 lines of plain text). Things were going great until I started to get a few visitors on my site, and some of them seemed to leave the tab on my main page. After a few dozen people had forthecode.com open in their browser, all with the periodically_call_remote playing pitch and catch with my server, things started to really slow down. Why?

I like to think that baking cookies would have a correlation to the way my apache2 configuration handled my Rails apps. Each request for page (a pan of delicious chocolate chip cookies) required that Apache heat up the oven (load the cgi processor), bake and serve the cookies (send the processed page), then cool the oven completely down before cooking another batch (unload the cgi processor). What a waste of time and processing power.

I took a screenshot of my processes at one of the slow hours- right when I really didn't have a whole lot of hits. In most scenarios my load was around 10+... quite frightening:

So, what did I do to fix the problem? Well in the simple world I would've just installed Lighttpd or Mongrel and ditched Apache (and probably added more memory to my machine). However, it wasn't so simple as a few sites on my server were quite happy with Apache, and I didn't feel like making a complete swap. Instead I thought it best I try running both packages side by side - one controlling all the initial connections (Apache2) and the other taking over where it is better suited (Lighttpd for Rails - FastCGI!).

I took the liberty of wget 'ing Lighttpd and installing it following the instructions on their site www.lighttpd.net. The simple portion of the experience at that moment passed away. For quite some time (several hours spread out over 3 days) I fought with Apache's apache.conf and vhost file - and likewise tangled with Lighty's lighttpd.conf file.

I spent quite a bit of time trying to find how to setup Apache2 as a front-end proxy that could serve requests to Lighttpd without breaking anything. So, let me quickly divulge what I learned:

Where do I start - easy... at the top of the list. For brevity's sake - check the top 3 out yourself - they are handy resources, with the first being perhaps the most particular to what I am discussing here. If you follow Duncan's essay to a T then you will most likely run into very few (if any) gotchas.

Number 4 and 5 are quite similar in nature. These are just things to keep in mind so that you dont overlook any potential small problems that seem to gum everything up. Make certain that your errorlog for fastcgi is available and writeable, also make certain that your dispatch.fcgi (found in your public directory -- my case /web) is correct and also executable (chmod 655).

Number 6 on the list really requires a bit more detail. There are several things you will have to do for Apache to correctly work with Lighttpd - let me hit the highlights by giving you generous snippets from my configs:

ProxyRequests should be turned off for added protection from undesired proxying (others using your machine to hide themselves as they route to other locations). We are configuring a Reverse Proxy, so no need in it being on - so set it to Off to be certain. The one snippet that gave me problems when left out was the < Proxy *>...</Proxy> settings. Be sure to include them.

In your virtual hosts file (generally found either in a vhosts folder, or in /etc/apache2/sites-enabled) you will need to make a few domain specific changes. An example out of my configs:

The parts you will add are in bold (excluding the file path at the top) - easy enough.. Apache is now configured - and once we have lighttpd configured and running we can restart Apache to see it all work!

So on to point number 7 - be aware that the "server.error-handler-404" directive in your Lighttpd config is especially important. When Lighty receives the proxied (forwarded) request to respond to '/' or the root index for the site - it isn't much of a problem. Lighttpd simply calls on dispatch.fcgi to pull out whatever you have routed for your index page. However when the request to Lighttpd is something like:

It does something quite practical behind the scenes to try and match the address to a file / page. Lighty trys to match the GET with a physical address on your server. When using the debug.* directives you get to see this interesting sequence of events - one that seems to end in disaster for our rails app:

As you can see - Lighty takes the GET request for /user/comment/1 and trys to match it with the physical directory structure presented on the server... unfortunately it doesn't even talk to rails and instead decides to bomb out with a 404 Not Found error page. So, here is where the configuration command I just mentioned is so important. Instead of allowing the 404 not found page to display- you set this in your config to make things happen:

This tells Lighty to use the dispatch.fcgi found in the virtual document root to handle all 404 errors. And in this case - the 404 error is stifled as dispatch.fcgi knows exactly what to do with this GET request - hand it over to your Rails app to process. Thus to continue on the request above, after setting the error-handler-404 in your config you should see this result:

Congrats - it appears to be working (once you have gotten to this point). So without further ado - allow me to post my current Lighttpd config as a guide to what you need to setup multiple domains behind the Apache2 proxy:

Thanks to this configuration my server is running much faster than it was before. Pages are displayed in a fraction of the time - my personal tests showed that before with Apache2 alone page access was about 15 seconds, now with Apache2 proxying to Lighttpd each page can be accessed in under 3 seconds! Now that is a major productivity increase!