In the previous post, I talked about the o3 article, and where I think they may have gotten it wrong (but it’s impossible to tell, as he didn’t publish any details on his testing methodologies, which is pretty lame).

But that he may have used the wrong terminology for the performance testing he did (saying it was TPS instead of HTTP requests per second) shows that there’s a lot of confusion on benchmark terminology, so I’m going to go over some of the basics.

TPS

In the load balancing world, TPS (Transactions per second) refer to the number of new SSL connections initiated.  The new part is important, because each new SSL connection requires a relatively CPU-expensive asymmetric encryption operation.  This is why most load balancers that do SSL have a separate chip for SSL processing (SSL ASIC), which offloads the SSL functionality from the main CPU.

Once the first step of an SSL connection is completed, the encryption then shifts to the much more CPU-friendly symmetric encryption, which is often referred to as “bulk encryption”.  Pushing bulk encryption throughput is relatively easy for a load balancer, even without an SSL accelerator chip.

However, HTTP/HTTPS typically involve short-lived connections, so there is relatively little throughput, and a lot of connection setup/teardown. Hence the need to know the TPS rating of a given device.

Many vendors will offer tiered licensing for SSL TPS.  So keep in mind when they mean TPS, they usually reffer to *new* SSL connections per second.

Connection Rate versus Request Rate

There are two ways to measure “rates” with load balancers:  Connection rate, and request rate.  While they sound similar, when you get right down to it, they’re actually quite different.

Connection rate refers to the number of TCP connections per second a device can handle.  HTTP request rate refers to the number of HTTP requests the device can handle.  How are they different? You can have multiple HTTP requests in a single TCP connection.

When your browser goes to a web site, it firsts initiates a TCP connection to the server (or in our case, a load balancer load balancing traffic for servers).  In that TCP connection, your browser will typically make several HTTP requests over that connection.

Making multiple requests over a single TCP connection is a lot easier than making a TCP connection for every single request.  In fact, the original HTTP 1.0 specification required on TCP connection per request.  The HTTP 1.1 specification fixed that, by allowing the multiple requests per TCP stream.

Layer-4 versus Layer-7

When a load balancer operates in Layer-4 mode, it’s functioning a lot like a router.  In fact, it’s not doing much more than your wireless access point at home.  Very little memory is consumed with each new connection, and only the TCP/IP header information is evaluated.

When a load balancer operates in Layer-7 mode, it’s functioning more like a server.  The TCP session is terminated at the load balancer, and a new TCP connection is initated to the server.  HTTP requests are buffered in the load balancer’s memory in order to be evaluated.  This requires a lot more processing power and a lot more memory.

Obviously, a load balancer can handle more Layer-4 workload than Layer-7, so it’s important to know which mode you plan on using when it comes to performance.