I’d read the O3 Magazine article by John Buswell with great interest, as well as Lori MacVittie’s response article. I thought they both made great points as I said in my previous post, and I was content to leave it at that.
Then I read the follow-up response over at the 03 blog. And I got that pang. You know the one.
Someone is wrong on the Internet.
He made a number of errors about F5′s capabilities, and they were cleared up by Lori in her response. But there are a couple of items I wanted to address. Take this quote from his blog post:
She claims that L7 is expensive, sure it takes extra processing, but if you read the article, you’d see that I drop hints that Nginx has a very superior way of handling I/O.
Maybe it’s just me, but “oh, I dropped hints!” seems to be talking down to your audience. At best, it’s condescending to your audience. At worst, it’s a shallow and transparent attempt to show a depth of knowledge in an area you actually know only superficially. Either detail the “very superior way of handling I/O”, or post a link to something detailing as such, or get off the pot.
There’s also his claim of being able to do over 25,000 TPS, which Lori rightfully called into question.
My personal favourite is that she quotes a 3 year old article to try to claim that the Opteron can handle “around 1500″ 1024-bit RSA operations per second, I don’t think she understands what is written in that report, as she has mis-quoted it, and picked a report thats over 3 years old. Lets play far shall we, I know marketing people are used to trying to skew reports, but you try that with me, I’m going to call you on it. Yet we have a running test that shows that machine is handling requests on-par with the F5 solution.
I found a more updated article on a processor very similar to the one used in the original O3 article. For a 4096-bit operation, the value was 343.12 decrypt RSA operations for 8 cores (dual quad-core Opteron 2356, a similar CPU to the one he used). 4096-bit RSA of course being much more expensive than 1024-bit, but by a predictable amount. Mulitply by about 32, and you get the number of 1024-bit operations (used in most SSL certificates). The value comes to about 11,000 for 8-cores, which is inline with what she states, and a lot less than his stated 26,000 connections per second. And as Lori pointed out, 11,000 would be possible if the system were only doing this SSL work. He also didn’t refute that he used 512-bit certs.
It’d be easy enough for him to test. The command to test the system’s SSL capabilities with 8 cores is:
openssl speed rsa -multi 8
So he either really did use 512-bit certs, or he’s actually not measuring TPS correctly. When an SSL vendor measures SSL, they typically use the term TPS, or transactions per second. This typically refers to the rate at which the system can accept new SSL sessions, with each new connection requiring an asymmetric encryption operation.  SSL/TLS uses two encryption technologies: Symmetric and Asymmetric. The asymmetric encryption is relatively expensive on a general purpose CPU like an Opteron (about 1,000 times as CPU intensive as symmetric encryption). That’s why devices like the F5 and other vendors include SSL accelerator cards, which are special processors that keep the encryption operations off the main CPU. A device with an SSL accelerator won’t “feel” the impact of an SSL connection any different than a non-SSL connection.
What I’m guessing (and it’s just a guess, since he didn’t state how he did his tests) is that he measured HTTP requests per second, which is a bit different than TPS. If he used HTTP 1.1 connection persistence, he could do 10 or more requests with only one asymmetric operation.  While that’s an absolutely fair measure of HTTP/S performance, it is not TPS, at least not in the generally accepted way. If you measured an F5 the same way (multiple HTTP requests through a single SSL connection), the F5 (or any other vendor advertising in TPS) would be able to push far beyond 25,000.
If that is the case, then he could do a comparative TPS test by using HTTP 1.0 mode, where each HTTP request required it’s own TCP connection (and thus asymmetric operation).
And finally, we have this.
1. No offense but covering the beat doesn’t exactly equate to 9 years experience with the technology. Sure you look at the trends, products and evaluations, but this is within a sandbox, not day to day real world experience. I’ve got 2 years experience working with Alteon as a customer back in 1998, working on the bleeding edge at the time of L4 switching. I kept F5 out of the customer site where I was working, simply because Alteon offered a much better and more innovative hardware platform. The web guys liked F5 because it did fancy graphs, Alteon got the job done in terms of performance and scalability. Following that, I spent over 6 years working as a Sustaining Engineer for Nortel / Alteon, responsible for thousands of bug fixes, and beating F5 on many occasions. After that I spent about 18 months working on Open Source App Delivery before returning to Nortel to work on their next-gen platform and help Sustain the Application Switch line. So as you can see, not all experience is equal.
He should really have done more research on Lori. There aren’t a lot of people in the world with a wider breadth of knowledge in the Layer-7/ADC/load balancing world as Lori (and he ain’t among them). He makes the mistake of dismissing her as “covering the beat”, referring to her time at Network World Computing, as if her job there only involved sitting through power point presentations and hitting the occasional power switch. Just looking at her F5 devcentral posts, she has an impressive knowledge from such aspects of the technology as HTTP security, application acceleration, and finer aspects of the HTTP protocol, to application-specific issues such as SOA, XML security, and APIs.  I’ve never met her, but I’ve spoken with her on a number of occasions, and she’s the real deal.
