Users might balk at 1,000 TPS. After all, 1,000 SSL TPS just doesn’t sound like a lot. So how much is it, really?

To answer this, just do a simple calculation: Take the average page size and calculate the maximum number of TPS possible through a given connection (T1, 10 Meg line, DS3). I think you’ll find that the numbers are a bit surprising.

Let’s take a page size of 20K. That’s 20,000 Bytes, or 160,000 bits. Add about 10% for SSL overhead, and that gets you 176,000 bits. Take the example of a T1 line, at 1.5 Megabits per second. Divide 1,500,000 bits by 176,000 bits, and you get about 8.5. So in a T1 line, the maximum number of 8.5 TPS in a T1 line without over-subscribing the line.

• T1: 8.5 TPS
• 10 Mbps: 56.8 TPS
• DS3 (45 Mbps): 255.7
• 100 Mbps/Fast Ethernet: 568.1
• OC3 (155 Mbps): 880.7
• Gigabit Ethernet (1,00Mbps): 5681.8

So, as you can see, to get to really insane levels of TPS you either have to have a huge pipe, or you have to have insanely small pages.

Three of the vendors had their features compared on the load-balancer.org site, and a thread on the load balancing mailing list as well as my own research have brought up a few more:

• Barracuda
• CoyotePoint Systems
• KEMP Technologies
• BalanceNG
• XGForce
• Load Balancer.org (Not to be confused with load-balancer.org, this is a different site)

Some of them do just Layer 4 load balancing, some do more (cookie persistence, URL parsing, SSL acceleration/offloading), so if you’re in the market for a load balancer, check ‘em out.

A Brief History of Load Balancing Time

Load balancing was created, and it was good. The market was hot, and the pick-and-shovel of the web play created valuations in the billions. ArrowPoint sold for $6.7 Billion to Cisco, Alteon sold to Nortel for $8 Billion.

Then there was a crash, and it was bad. Established load balancing companies, like many tech suppliers, found their orders and pipelines drying up. Compounding this was that companies that were buying load balancers were buying used, or in their capacity glut, found they had more load balancers than they knew what to do with. Not only were they competing with other load balancing companies, they were effectively competing with their highly successful selves from the past. It was like a bad Star Trek plot.

Some vendors all but abandoned their products, letting them atrophy without much development. Some vendors virtually left load balancing altogether and jumped into security. Some just plain went under. Everyone seemed tired of load balancing, and ready to move onto something else. Load balancers had created a head-splitting hangover.

This was the doldrums time for load balancing.

But recently, things have turned around. While companies like F5 and Citrix have for a while been doing robust business in high-end load balancing (I’ve heard both F5 and Citrix mentioned on Jim Cramer’s Mad Money), there is a new market exploding, one that seems to have gone largely under the radar for the past couple of years. While companies like F5 and Citrix have dominated the high end enterprise market (and well serving them), this new market has developed to accommodate a need for more Spartan load balancers.

The customers of this market are typically small to medium sized businesses, companies that don’t have the money necessarily to buy the higher-end load balancers, but also don’t really need all the fancy features they have either (TCP multiplexing, on-the-fly compression, etc.). They just need load balancing, maybe some cookie persistence, and perhaps even a little bit of SSL acceleration.

After the Dotcom Bust, two markets emerged, one catering to the high-end enterprise, and another, less evolved, to support the companies that didn’t have the budgets to buy the products that the enterprise was spending on, even after the bust subsided.

At first, this market resorted to the flotsam from Dot-com busts on eBay listings. You could pick up a full-featured load balancer on the cheap. But there was a problem with this: These products come with no support or ability to get code updates.

Load balancers have traditionally been voracious consumers of code updates, especially during the feature arms race between the vendors in the late 90’s. This has subsided somewhat, but many of the eBay products are children of this era. Most are so old they aren’t even eligible for a (typically) expensive recertification/support contract from the vendors that made them. (This was highlighted in my article “Buyer Beware” from 2003).

So here you have this huge market of scrappy, under-funded startups, as well as established small and medium sized businesses, which have a need for server load balancing, but either can’t the full-featured prices or can’t rely on the eBay hardware they purchase.

A couple of vendors — some old, some new — have recognized this emerging and maturing market, and load balancing has become exciting again. These are server load balancers going for less than $10,000 per unit, and in some cases, less than $3,000. Some even offer advanced features beyond that of simple IP and port, such as cookie-based persistence and SSL acceleration/termination.

Regular Layer 4 load balancing has been commoditized; you can get it free with the Linux Virtual Server project, as well as BSD’s PF firewall code. LVS is actually the core of many of the low-cost load balancers in the market today.

The more advanced features are becoming less expensive as well. Cookie-persistence and SSL termination used to be far more costly, and can now be had for less than $3,000. Moore’s Law has had an interesting effect on all this too. Products that sell for $5,000 today can handle traffic levels beyond what top of the line, $100,000+ product could do just 5 years ago.

If you need full-blown features of the F5’s and Citrix’s of the world, then you’re going to need to pony up. But if you’re got the need for the same type of load balancing that was high-end 5 years ago (layer 4, cookie persistence, SSL), then you don’t need to pay a lot.

To fully understand load balancers require skills from career paths that don’t typically converge: Those of a server administrator (and application developer) and those of a network engineer.

This strange convergence in skill requirements often means that load balancers are misunderstood. It’s one of the reasons why “It’s always the load balancer“.

I’ve found though that there are a few concepts and tricks that if learned by opposing skill sets can go a long way into understanding/dealing with/troubleshooting load balancing technology.

What a network engineer load balancer administrator needs to know about servers

• Default routes of the servers. I don’t keep statistics on the hundreds of issues I’ve diagnosed over the years, but I’d guess fully 50% involved default routes on the servers as either the primary problem, or a related problem. This is true for every vendor I’ve worked with (or for).

• Learn to use liveHTTP headers for Firefox and/or ieHTTPheaders for Internet Explorer. This shows you exactly what the server is doing, and can help you answer questions like “why is the load balancer redirecting me to Google?” (The answer is no, it’s not, the application is doing that, and here’s the HTTP 302 in the header to prove it.)

• If you’re setting up a non-transparent setup (known as full-NAT) where the IP address of the web clients are hidden, make sure that your server people are aware of this (and OK with this). Otherwise, when the server people call asking where the IP addresses of their clients are, and you have to explain to them that the IPs are lost forever, and that the load balancer doesn’t log them, it’s going to get unpleasant.

• Setup MRTG for your load balancer. Pull as many stats as you can get your grubby little paws on.

What a server admin/application developer needs to know about load balancers

• If you’ve got a tricky problem that can be solved by a solution in a load balancer and in an application, do it in an application. It may seem seductive to have the load balancer take care of specialized cases with specialized URL parsing or redirects, but it can greatly complicate your setup. For one thing, you have to go to a different group in order to make a change. This doesn’t include the core functionality of load balancers, such as cookie persistence of course.

• Know how to do a network capture, using TCPDump or WindDump. This is a language network engineers understand, and even if you don’t understand what it means, the network group can often pick through it and figure it out. It would help greatly to learn to parse it yourself, as well. It would help to know how to read Ethereal (available for Linux/BSD/Unix and Windows).

• Load balancers don’t log client connections. There’s only one load balancer that I know of that could even configure such logging, but it wouldn’t be recommended and would likely impact performance.

Have a tip? Leave a comment.