I’m a big fan of virtualization.  There’s a lot to like about it, including consolidation (getting rid of space-heater servers that do nothing running 1% CPU and sucking up electricity and throwing off heat), flexibility, and management.  I’ve even gone and got my VCP4 (VMware Ceritified Professional 4) certification.  (Hear that ladies? I’m certified.)

One aspect of virtualization I’m a fan of is the appliances.  Vendors are taking physical appliances (such as a Vyatta router) and turning it into a VM appliance.  I don’t have to worry about an underlying operating system (and the requisite patches), the appliance vendor handles the software and the OS.

Several load balancing vendors have gotten into that virtualization game.  Vendors that have traditionally offered hardware appliances now have virtual appliances (some for years).  From From F5 to KEMP, from Coyote Point to loadbalancer.org, there are a number of virtual load balancers/ADCs to choose from.   And for the most part, they offer the same features as their hardware brethren.

Their throughput and performance is hampered somewhat by the fact that they’re all software and no silicon. Everything is done in the x86 virtualized CPU(s).  Still, depending on how you provision them, they can generally handle several thousand HTTP requests per second even in Layer 7 mode.

The one caveat to virtual load balancers is that their SSL performance is severely limited.  Even value-market load balancers that do most of their functions in a general purpose CPU will still use SSL ASICs for the asymmetric crypto (even using the general processor for the less CPU-intensive symmetric crypto).

The first part of every new SSL connection is a very CPU-intensive asymmetric operation (about 1000x more CPU intensive than symmetric operations).    CPUs that can normally handle tens of thousand of regular TCP connections per second can only handle a few thousand SSL connections at the most.

It is technically possible to do hardware SSL acceleration on a VM load balancers however.  It requires that the virtual machine host (like VMWare ESXi) have an SSL card installed, and VM Passthrough enabled (where the virtual machine can have direct access to physical hardware).

Unfortunately, these SSL cards are tough to come by.  Cavium is probably the most notable vendor, but cards from them aren’t exactly easy to come by, and they’re fairly expensive.  You may not have the option if you’re using blade systems.   And if you you want to leverage features like HA and DRS (using ESX hosts in a cluster), then every machine in the cluster would need to have such a card.  I’m not aware of any virtual load balancer vendor that even supports this configuration.

There are a lot of situations where virtual load balancers make a lot of sense, but keep in mind that the SSL performance capability is going to be fairly constrained.

Typically, Apache Bench (ab) is installed with the base Apache install, from at least Apache 1.3 on.  This includes when Apache is installed on Windows.

You can check all of the available options on the ab documentation page, but here’s a (very) quick reference to using it.

Two of the most important options are “-n” for the number of total connections, and “-c” for how many concurrent connections are done at the same time.

For instance, using the option “-n 1000″ will do 1,000 requests, one at a time, to a target URL.

ab -n 1000 http://website.com/

One at a time is rarely an effective test, so it’s best to use the “-c” option to specify a high number of concurrent connections, such as 100.

ab -n 1000 -c 100 http://website.com/

If you use concurrency, ab will split the total number of requests up amongst the concurrent settings.  For instance, using the option “-n 1000″ will do 1,000 connections, but “-n 2000 -c 100″ will only do 20 requests from 100 different connections (2,000 / 100 = 20).  So it’s best to use a much larger number of total connections if you’re doing concurrency.

ab -n 100000 -c 100 http://website.com/

When ab is finished running, it will spit out a performance report, including such info as the time taken for tests, requests per second, wait time, etc.

Finished 1000 requests

Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        45 bytes

Concurrency Level:      10
Time taken for tests:   0.427 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      320640 bytes
HTML transferred:       45090 bytes
Requests per second:    2341.45 [#/sec] (mean)
Time per request:       4.271 [ms] (mean)
Time per request:       0.427 [ms] (mean, across all concurrent requests)
Transfer rate:          733.17 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   0.4      2       3
Processing:     0    2   0.5      2       7
Waiting:        0    2   0.5      2       6
Total:          0    4   0.7      4       8

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      5
  80%      5
  90%      5
  95%      5
  98%      5
  99%      5
 100%      8 (longest request)

The ab utility defaults to one request per TCP connection (KeepAlive turned off).  If you want to use KeepAlive, where multiple requests are made through a TCP connection, use the “-K” option, open up as many TCP connections as you specify in concurrency (“-c”) and make the total number of quests through those few open TCP connections.

The utility is a simple but power tool for testing load balancers and web servers.  It doesn’t tend to reflect real-world usage, but it can be useful for baseline testing and troubleshooting.  I’ve found it quite useful over the years.

So now Gmail will allow you to do all SSL, all the time.  This isn’t just a gmail problem, but one that affects all logged-in sessions.  I’m guessing gmail has a pretty high-end SSL accelerator in operation for this.

I was on a run in my new home of Portland, Oregon when I came upon an AMC Pacer parked on the street.  And I thought, how like the old Alteons the AMC Pacer is.  They’re both rather odd looking, and have a rather fanatical fan following (as evidenced in it being in a starring role in the Wayne’s World movies).

There was a post on the lb-l mailing list recently about them, and it’s interesting to see how the old Alteons are still going strong in the market.

On Slashdot today was a posting about the infrastructure for the super-site Wikipedia.  And what load balancer do they use?  Why, Linux Virtual Server of course.