Back in the old world of real infrastructure, we used Netscalers or F5′s and we were happy. Now in the cloud, you have several options all of which seem to have problems.
1. Open source. But once you want SSL, and redundancy, and HTTP compression, you get people saying with a straight face “nginx (for HTTP compression) –> Varnish cache (for caching) –> HTTP level load balancer (HAProxy, or nginx, or the Varnish built-in) –> webservers.” (Quoted from Server Fault). Like four levels, often with the same software twice in it. And don’t forget some kind of heartbeat between the two front-ends. Oh look I’ve spent $150/mo on just machines to run my load balancing. And I really want to load balance/failover between all my tiers not just the front end. It’s a lot of software parts to go wrong.
2. Zeus. For some reason none of the other LB vendors have gotten off their happy asses and delivered a good software load balancer you can use in Amazon. I got tired of talking to our Netscaler reps about it after the first couple years. They’re more interested in selling their hardware to the cloud data centers than helping real people load balance their apps. Zeus is the only one – and it’s really quite expensive
3. Amazon ELBs. These just have a lot of problems under the hood. We’ve been engaged with Amazon ELB product management on them – large files serve out super slow; users get hits refused due to throttling/changes during ELB scaling – basically if you want 100% of your hits to come through you can’t use them.
4. Geo-IP load balancing, through Dyn or whoever. They claim to have the failover problem fixed, but it still only works for the front end tier of what is a multitier architecture. I certainly don’t want to have to advertise every internal IP in external DNS to make load balancing work.
And really the frustrating part is there seems to have been no headway on any of this stuff in a decade. Same old open source options, same old techniques. Can someone come up with a way to load balance on the cloud that a) doesn’t lose any hits, b) is one thing not 4 things, and c) is useful for front and back end balancing? Seems like a necessary part of oh say every system ever, why is it still so hard?
Ernest is DevOps Platform Manager for Bazaarvoice in Austin, TX.
nginx now allows you to have compression, load-balancing, SSL and now cache. Of course, cache functionality is pretty basic compared to varnish and load-balancing functionality is pretty light compared to haproxy.
nginx is awesome… as a web server. That’s what it was designed for. As a proxy, it lacks a lot of the key things that the author mentions in terms of proper HA, rich load balancing, etc. To your point, HAproxy is a better choice if you want an open source load balancer.
If only someone would resurrect Resonate! It was a pioneer in the load balancing industry and unfortunately got eaten up by the .com crash. It was software based which meant you can install anywhere and spread the load of your traffic. Also it would allow non-http load balancing for the odd ball apps out there. Maybe need to look at the latest nginx version as its proper successor.
Resonate used a networking hack that would not fly on AWS.
I use Amazon ELB and run the monitoring tool from Pingdom on a 1-minute check. I have yet to have any down time as a result of ELB.
That’s way too low of an interval to find something like the problems I saw. It’s not continuous downtime, it’s 1 in 1000 hits being dropped. Very hard to find.
Has anyone everhad duplicate info transmission due to a load balancing issue?
We had that, not load balancing itself per se but from our network redundancy around our front end, ended up repeating packets on us. I forget what we did about it, mainly yelled at our network admins…
Have you tested Zen Load Balancer? It is still in development, but it can be a very good open source load balancer solution. It is moving with the suggestions of their users, and the develpment team is very active. If you don’t find the features you need, you can suggest it to the mailing list.
NetScaler announced a “tech preview” (which means beta in their world) for AWS here: https://www.citrix.com/English/ne/news/news.asp?newsID=2324093. NetScaler is already available on a few other clouds, most notably you can get it as a self-serve at SoftLayer.
Huh. Is it really AWS native? I’ve gone through the whole snipe hunt of rooting through their marketing talk every time they claim “we have netscaler for AWS” and it always turns out to be something lame and on premise. Well, I hope it livens up. At NI we used Netscalers, at BV we’re using F5 at Rackspace and HAProxy in AWS regions with UltraDNS GTM in front of it, works decent but automating the changes to HAproxy config is a PITA. I’m open to new solutions that actually work… I get frustrated at the FUD around cloud load balancing, I had to finally tell A10 to just stop calling me. “But do you have something for pure AWS systems?” “Sure! Let’s get you on a call with an engineer.” “Hello engineer, so will this work on a pure AWS system?” “Well no…” Now sadly my attitude towards vendors is “I’m going to wait till someone reputable I know is actually using it before bothering to listen much.”
And don’t forget the ELB slowness – up to 1.5s – on the SSL negotiation when using big SSL keys.
With Direct Connect you can use your BigIP LBs from your datacenter to your cloud instances. http://aws.amazon.com/directconnect/
I’m really surprised with you having issues with Amazon ELB, have you logged a support case? Netflix is open sourcing some of the tools that they use, but the main thing is that Amazon has no interest in SSL or compression because they would rather you do simple ELB and handle CPU intensive stuff in the cluster. We often get asked at Loadbalancer.org about wizzy layer 7 stuff in our Amazon EC2 product which is based on HAProxy. Our answer tends to be “Don’t even think about it…” our product is great for things Amazon ELB doesn’t do like maintenance mode , extended health checks, rdp cookies etc… But we would not recommend it for really high loads unless you clustered it behind ELB… Much like Zeus aka. Riverbed recommend….
Yes, not only support but as I said in the article we worked with the actual product manager for a while. We currently use GTM + HAProxy on the front end at BV. (Well, and Akamai, and Apache, and nginx.)
In my quest for a Internal Secured ELB (multi-tiered) architecture, I now have to use AWS VPC.
Us too. Victor Trac from Bazaarvoice did a presentation at last month’s Austin Cloud User Group about our newest implementation at BV, basically a whole “virtual data center” in Amazon for our dev teams to use. VPCs, a CloudFormation template to bring up DNS, NAT, ENI attachers, VPN tunnels… I’ll ping him on posting about it/us open sourcing the github project.
Why not spend on some Cisco ASA to LB? Its not free but for me it works very good see http://www.asavirtual.org there is some articles and info i use Cisco, juniper and at main DC all is Extreme networks equipment exept 4x Cisco for LB and firewall we have some older HP servers running esx for honey/IDS that “repport” to firewall.
In lab we play with pf (pfsence) CARP for HA the idé is to setup multi openvpn or ipsec tunnels true diffrent locations (we have HP blade full and half racks in .se/.de/.uk, USA/CA and .ru/.ua) any suggestions other then pfsence? Ofcourse free open source would be good. Dont get me wrong i love Extreme, cisco etc but i belive its time to support good open source projects (donate) insted of feeding “extreme” big companys like cisco (long time cosumers 10+years should get some discount…NOT!) Oh and i almost for got thanks for article and also comments good reading!