Jump To Content

Nginx + HaProxy usage and a gotcha

I’ve recently switched from using a straight up Apache mod_proxy + mongrel setup to Nginx + HaProxy +mongrel. Basically, mod_proxy is dumb as shit, and should never be used with mongrel + rails, especially with better options like Passenger and HaProxy.

Why does mod_proxy suck? Take a look for yourself:

[8030/5/247]: handling 127.0.0.1: GET
[8031/6/134]: handling 127.0.0.1: GET
[8032/2/160]: handling 127.0.0.1: GET
[8033/0/93]: idle
[8034/10/128]: handling 127.0.0.1: GET
[8036/2/124]: handling 127.0.0.1: GET
[8035/0/110]: idle
[8037/0/101]: idle

Look at this garbage. There are three idle mongrels even though the other five have 25 requests to be processed.

Whats great about HaProxy is that it will queue up requests and only distribute them to idle mongrels. It translates into way better response times under load. Add Nginx, which is way easier to configure that Apache, allowing me to easily setup things like sending known ‘slow’ requests to separate mongrels and it’s a big win. As a bonus, haproxy has a great status tool, which gives a quick overview of your proxy clusters.

Photo 5926

One little problem …

The gotcha when I initially switched over was that exception_notifier stopped sending me error emails, in fact the errors were being displayed to the user. What the hell?

Exception notifier looks at the incoming IP address. If it’s local (aka: 127.0.0.1), it’s assumed that the user is a developer, so it won’t email the exception to you. haproxy was sending nginx’s IP address, rather than the forwarded client address. The problem was that I had configured both nginx and haproxy to forward headers.

haproxy.conf: 
option          forwardfor    # enable insert of X-Forwarded-For headers
nginx.conf 
proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;

Removing the header forwarding in the haproxy configuration solved the problem. You only need to set the X-Forwarded header in Nginx (or Apache or Pound or whatever front end you choose). Setting it in Haproxy as well will over write the header with one containing the IP of the front end. Since I have Nginx and Haproxy running on the same machine, that IP is 127.0.0.1, which broke the exception_notifier.

  • Your comment will be modifiable for 10 minutes after posted.

Page Author

Avatar
wmoxam
Name
wmoxam

From Here You Can…

Information

Most Recent Related Content

Published In…

Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.