SSL performance is a tricky thing to measure.  There are a lot of different aspects to consider, and on top of that different vendors use slightly different definitions for the same terms. I was talking with a vendor the other day, and I realized as we were talking performance numbers we had different definitions for the term TPS (Transactions Per Second) and CPS (Connections Per Second).

TPS is probably one of the more ill-advised acronyms (boy do we love acronyms) for load balancers, because it’s open to debate what the “transaction” in transactions per second means.

SSL Basics

Before we talk about CPS versus TPS, let’s review some SSL basics.  In a new SSL connection, two things happen: An asymmetric exchange, then a transition to symmetric encryption.

The asymmetric part is why load balancers from F5 to KEMP use SSL accelerator chips.  It’s incredibly CPU intensive; so much so that a server that is capable of tens of thousands of connections per second of regular TCP connections is only capable of several hundred SSL connections.  To keep this from crippling the CPU, SSL accelerator processors are used to offload the cryptographic functions from the main CPU.

So when we measure SSL performance of a load balancer, we typically want to measure three things:

1: How many new SSL connections per second can a device handle, that is how many of the expensive RSA operations per second are possible.

2: How much bandwidth of SSL traffic can the device push.  This is symmetric encryption (such as AES), and is much easier on a CPU.

3: How many HTTP requests per second can the device handle after an SSL connection is established.  In HTTP 1.1, a client is allowed to make multiple HTTP requests off a single TCP/SSL connection.  This is far more efficient than the old HTTP 1.0 standard, that required a separate TCP connection for each object.  Many sites will have pages with literally hundreds of objects (I’m looking at you http://vg.no).

The trick is how many HTTP requests per SSL connection?  Personally, I’d say anywhere between 10 to 20 is a pretty good place to start, but that’s not what everyone uses.  In an epically flawed fight that o3 magazine picked with F5, author John Buswell claimed that an Nginx-based box he built could handle 25,000 TPS.

The system had no problems handling over 26,590 TPS, the test lab ran out of capacity to generate additional transactions. Compare that to the F5 Networks Big-IP 6900 which handles a maximum of 25,000 TPS but carries a starting price tag of $55,000.

The box he used had 2 quad core Opteron CPUs that were in no way capable of doing 26,000+ new SSL connections per second without an SSL accelerator card (which he didn’t use).  A few thousand 1024-bit RSA operations per second at the most, but definitely not 26,000.

He likely made the mistake of opening a couple of SSL connection, and running 25,000+ HTTP request per second off those open TCP/SSL connections (which is not even close to a real world scenario).   25,000 HTTP requests per second is within the capabilities of his box.  This is not how F5 measures TPS, nor how any other vendor measures TPS.  Of course, I’m only guessing at his methodology, as he never bothered to share his testing methodology.  (F5 and other vendors share their methodology).

TPS versus CPS

While SSL throughput is pretty easy to measure and the definition is identical for all vendors, the terms TPS and CPS aren’t.  So what do they mean?  CPS (Connections per Second) can mean a couple of things.  For SSL, it would typically mean new SSL connections, requiring the asymmetric operation.  Some vendors use TPS for this definition (the T standing for an asymmetric transaction), while other vendors use CPS for the SSL portion, and TPS for the HTTP-within-an-SSL connection.

So it’s important to understand what’s being measured.  Different vendors have different ideas of what that means, although they are honest differences of opinions (things reasonable people can disagree on).  Most vendors are reputable and will outline the methodology they used to come up with their numbers (03 magazine did not).

Benefiting from Moore’s Law to a great extent are load balancers/ADCs, where the lowest end device from just about every vendor can handle traffic loads in the 50-100 Mbps range.  Of course, throughput isn’t a terrible way of measuring performance capability of a load balancer (100 Mbps of large file downloads is a heckuva lot easier than 100 Mbps of tiny file connections), but it does relate well to one very important factor in web site serving:

How big is your pipe?

Businesses of all sizes have seen steady increases in their available bandwidth to be sure, but while the growth may be steady, for the most part it tends to be a slower growth rate.  Much slower than doubling every 18 months, like in Moore’s Law.   The result is that increasingly that the lower end offerings from vendors are more than sufficient to run a larger share of web sites out there.

There are a number of reasons for this.  We’re no longer experiencing the exponential accross-the-board growth rates in users that occurred during the dot-com boom.  While there are some sites going through a growth explosion, for most websites in this economic environment, growth rates of any kind are fairly extraordinary.  So extra bandwidth isn’t in as high demand. Secondly, if you’re hosting your own data center as many large businesses do, getting extra bandwidth is often time consuming.  Moving from a single DS3 (45 Mbps) to an OC-3 (155 Mbps) is going to take some time to get that order fulfilled.

Cisco’s ACE 4710 appliance comes with a default license of 1 Gbps of throughput.  F5′s entry-level BIG-IP 1600 LTM maxes out at 1 Gbps.   In the Enterprise market, 500-Mbps to 1 Gbps is about the rock bottom in terms of performance capability.  Yet many of the high-end clients of these vendors don’t push nearly that much traffic.

Companies that aren’t media or mega-content providers (such as Google, Youtube, Yahoo!, Facebook, etc.) that have web applications serving customers or businesses typically don’t go above 100-200 Mbps in traffic, even for some Fortune 500 companies.  Of course, there are exceptions, and there are quite a few factors involved in determining the traffic characteristics of a site. Companies that are offering media such as streaming video or audio often use third-party content providers, such as Youtube or Akami, so as to keep that bandwidth off their own pipe.

So we’ve got all this idle CPU time, so why not make use of it?  That’s what many vendors are doing, in both the enterprise and value markets. With the steady rise in CPU power while bandwidth consumption lags behind, vendors are throwing more and more capabilities into these devices to take advantage of the unused CPU cycles, such as caching, compression, Layer-7 inspection, etc.  Vendors are offering more functionality with the greater power they have available to them.

Prices have come down in Gigabit to the point where Gigabit is almost the default. However, in the purpose built hardware that the value vendors use, Fast Ethernet is still more common than Gigabit, especially in the entry level units.

One of the cost considerations isn’t so much the load balancer itself, but the cost of the actually getting connectivity at levels greater than 100 Mbps. (Remember, the whole point of Gigabit is to have the ability to push 101 Mbps, not necessarily 800 Mbps). A couple of quick searches on Google show that prices for raw bandwidth in a colo facility in the US starts around $12 per Mbit, give or take, making 100 Mbps $1,200 USD per month. Combined with the colo costs, server rental (or upfront capital expenditure), that’s not an insignificant cost when compared with the prices of the devices in the value market.

At that level of traffic, hopefully you have a revenue source sufficient to pay for a Gigabit-capable valuemarket load balancer. If not, you may want to consider bandwidth reducing measures, such as putting company videos up on Youtube.

But the point of this post isn’t cultural references, but rather the addictive nature of a particular feature of high-end load balancers/application delivery. I’m talking about control languages, such as iRules for F5 or aRules for A10 networks.

They allow scripted inspection and manipulation of both the headers and payload of inbound and outbound HTTP, which is an extremely useful feature. Usually based on a scripting language such as Python or Perl, they allow for some pretty sophisticated actions.

However, not that many vendors have a control language. A10 networks, F5, and Zeus are the ones I can think of off the top of my head. And their feature sets vary pretty widely.

F5 has a great community for iRule development, and they’ve got some example iRules on devcentral such as a credit card number scrubber, cookie encryption (decrypt it on the server with a shared key), and cookie persistence logger.

Vendors benefit from the exclusivity of this feature, in that there isn’t a lot of competition. But the biggest benefit to vendors is that using the control language is so freaking addictive. It’s sort of like the heated car seats of the slb industry. Sure, you did just fine without it. But once you’ve used it, it’s hard to go back.

Depending on the feature set, it may be possible to switch from one vendor’s control language to another’s, but it’s non-trivial. Going to a non-control language vendor would very likely be a deal breaker.

Can you get a client to give up their control language? Sure, but the hand you pry it out of will probably need to be cold and dead.

Cisco CSS users specifically have an interesting choice when it comes to choosing a new product: They can take the high road, or they can take the low road.

Feature-wise, the Cisco CSS is roughly in parity with much of the value market (KEMP, Barracuda, Coyote Point). If a site is looking to keep the same level of functionality along with saving a ton of cash, then the value market may be the way to go.

More advanced load balancers, such as F5′s BIG-IP, A10 networks, and of course, Cisco’s ACE platform, offer a lot more features than the CSS. F5 for example offers their iRule platform, which allows the load balancer to do some pretty sophisticated app-level functions (such as authentication) before ever hitting an application.  They all offer a more flexible network implementation, with multiple in-bound and out-bound routes.  Of course, the prices are significantly more than the value market products.

The good news is, if you’re looking to replace your Cisco CSS load balancers, there are over a dozen appropriate vendors that can replace your infrastructure while giving you similar functionality.  The bad news is, there is over a dozen vendors to choose from.  So don’t be afraid to get picky, and start looking into either saving a lot of money, or getting a lot more features.

• A10 Networks
• Array Networks (corrected)
• Barracuda Networks
• BalanceNG
• CAI Networks Web Mux
• Celestix
• Cisco, makers of LocalDirector, Distributed Director, CSS, CSM, ACE load balancers
• Citrix/Nescaler
• Coyote Point Systems
• Extreme Networks
• F5 Networks, makers of BIG-IP/3DNS
• Foundry Networks
• Juniper DX
• KEMP Technologies
• Linux Virtual Server project
• Load Balancer.org
• Nortel, makers of the Alteon line
• OpenBSD’s PF
• Pound (Open Source)
• Radware
• XGForce
• Zeus

That’s over 20 different vendors/projects that involve load balancing. If you know of even more, shoot me an email (tony at lb digest dot com).

And as always, don’t be afraid to shoot me a question about load balancing. I’ve been getting quite a few lately, and I’ve posted some of the answers so that others may hopefully benefit. You can email them, or simply throw them in a comment section of a post.

But what you don’t know much about is load balancers. So here’s a bit of a primer on load balancing for those involved with application development.

Persistence

The first thing you need to know about is persistence, and specifically, do you require it. If your application is stateful, where information regarding a session is stored on only one server, you’ll need persistence. Virtually all load balancers support this, but you’ll need to know to turn it on (or ask your load balancer administrator to turn it on).

Most applications are stateful, so it’s a fair bet you will. A quick way to test is to start a session on one server, then change the hostname or IP address in your browser to point to another server with the same application installed. Does it break, act freaky, or otherwise malfunction? Then you’ll need persistence.

As I’ve said several times before, you’ll probably want cookie persistence.

What The Load Balancer Passes On To The Server

Sometimes I’m asked what the load balancer changes in the client request to the server.

The answer is: absolutely nothing.

Load balancers will pass all HTTP headers that it receives onto the server. It may add a few items, such as a persistence cookie, but in most configurations, the load balancer won’t change anything (and with many vendors, the load balancer just doesn’t have the ability to change anything).

A load balancer might give out an HTTP 302 redirect. A very common example is redirecting from HTTP to HTTPS.

Virtual Hosting: The Host Header

This is often called software virtual hosting, virtual hosting, etc. Basically, it’s running more than one URL off the same IP address.

Let’s say you’ve got two URLs: www.domain1.com and www.domain2.com. In DNS, they both point to the same IP address, yet when you go to the sites with a browser, two separate web pages come up. How come? It’s all in the HTTP host header.

When the browser makes a request, it includes a “Host:” entry, telling the web server what host it’s looking for. The web server looks at this host, and serves up the appropriate page.

The load balancer will forward this host request along with the entire request. Most load balancers don’t have the ability to even change this.

SSL Termination

If you utilize SSL on your web site, you may want to consider having the load balancer terminate the SSL connection.

You’ll want to check to see if the load balancer has hardware acceleration, which is a special card that removes the SSL encryption/decryption operations from the general CPU and onto a specialized processor.

The two main benefits to SSL termination are the performance benefit by having the load balancer handle the SSL instead of your servers, and by terminating the SSL connection on the load balancer, you can use cookie persistence.

Header Dump

It helps to have a method, in either a standalone page or in your own library as a quick function call, to dump all the HTTP header variables.

In PHP, you can use the built-in phpinfo() function.

<?php
phpinfo();
?>

Have this page/function handy, in case a problem arises.  Point your load balancer administrator there, and they may be able to point out the problem.

So, here’s a few tips to help whittle down the vendors.

The first thing to consider is what type of company you are. Are you a huge mega-corporation? A financial? A scrappy upstart? That by itself will dramatically reduce the number of viable vendors. If you’re a Fortune 500, you’re going to want to go with one of the established premium market players. If you’re a scrappy upstart, your budget is probably very limited, so you’ll want a value vendor.

Next, consider the focus of the companies you’re evaluating. Is their primary focus load balancing? Is their core competency networking in general, with a tiny subset dealing with load balancing? There are companies that are intently focused on load balancing, and companies that are generalists with networking products. F5 and Cisco are great examples of each. F5 is intently focused on load balancing, and generally have the best in terms of features and technology. Cisco hasn’t been as focused, and seem to always be playing catch-up in terms of features, but they do have a widely known, highly regarded reputation. Just about everyone has heard of Cisco, few outside of IT have heard of F5. I tend to prefer the better technology, but both aspects have merit, and which has precedence depends highly upon your corporate directives and culture.

Of course, there’s also the new versus used question, but I think you know my answer on that one: Always buy new. Not because of the coveted “new load balancer smell” (although it is delightful), but because new boxes are supported by the vendor in terms of software updates and hardware failures, and used typically aren’t.

The rest of the selection process depends highly upon whether you’re going for the value market or you’re going for the premium market. I will be updating shortly with the process I recommend.

Send an email to tony @ lb digest dot com

They are three things I don’t buy used. Underwear is obvious. With hard drives, they’re the component in a PC that is almost guaranteed to fail before any other component involved in a server, desktop, or workstation, so getting them new means you’ve got a rough idea how long it’s going to last (I try to move off hard drives older than 4 years).

With networking gear, as I’ve said before, if it operates above Layer 3, I don’t buy it used unless it comes with a support contract from the original vendor. The reason is that with increasingly complex code associated with operating on Layers 4-7, you’ll need access to the original vendor’s code updates. No vendor I know of offers these code updates for free, so you’ll need a support contract.

So while a used Alteon might seem like a great deal on eBay, you’re completely on your own when it comes to software updates. If the units don’t include a recent version, that could mean trouble in terms of security, stability, or bug fixes (or all three).

With Value vendors marketing directly to the SMB (KEMP, Coyote Point, Barracuda), there’s little reason not to go the new route with an affordable yet feature rich load balancer.