Typically, Apache Bench (ab) is installed with the base Apache install, from at least Apache 1.3 on.  This includes when Apache is installed on Windows.

You can check all of the available options on the ab documentation page, but here’s a (very) quick reference to using it.

Two of the most important options are “-n” for the number of total connections, and “-c” for how many concurrent connections are done at the same time.

For instance, using the option “-n 1000″ will do 1,000 requests, one at a time, to a target URL.

ab -n 1000 http://website.com/

One at a time is rarely an effective test, so it’s best to use the “-c” option to specify a high number of concurrent connections, such as 100.

ab -n 1000 -c 100 http://website.com/

If you use concurrency, ab will split the total number of requests up amongst the concurrent settings.  For instance, using the option “-n 1000″ will do 1,000 connections, but “-n 2000 -c 100″ will only do 20 requests from 100 different connections (2,000 / 100 = 20).  So it’s best to use a much larger number of total connections if you’re doing concurrency.

ab -n 100000 -c 100 http://website.com/

When ab is finished running, it will spit out a performance report, including such info as the time taken for tests, requests per second, wait time, etc.

Finished 1000 requests

Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        45 bytes

Concurrency Level:      10
Time taken for tests:   0.427 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      320640 bytes
HTML transferred:       45090 bytes
Requests per second:    2341.45 [#/sec] (mean)
Time per request:       4.271 [ms] (mean)
Time per request:       0.427 [ms] (mean, across all concurrent requests)
Transfer rate:          733.17 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   0.4      2       3
Processing:     0    2   0.5      2       7
Waiting:        0    2   0.5      2       6
Total:          0    4   0.7      4       8

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      5
  80%      5
  90%      5
  95%      5
  98%      5
  99%      5
 100%      8 (longest request)

The ab utility defaults to one request per TCP connection (KeepAlive turned off).  If you want to use KeepAlive, where multiple requests are made through a TCP connection, use the “-K” option, open up as many TCP connections as you specify in concurrency (“-c”) and make the total number of quests through those few open TCP connections.

The utility is a simple but power tool for testing load balancers and web servers.  It doesn’t tend to reflect real-world usage, but it can be useful for baseline testing and troubleshooting.  I’ve found it quite useful over the years.

So what’s a part-time regex’r to do? Like any good professional, I cheat. I use a great little tool called Regex Coach.

It’s great little graphical tool that lets you take an example string and test (in real time) your regex statement.

It handles splits, fields, and all of the other regex madness that you might need help with.

Update: Reader Jason Williams pointed out another great little Regex tool called Kodos.  Similar in function, it’s got some neat features like automatic code generation.

To fully understand load balancers require skills from career paths that don’t typically converge: Those of a server administrator (and application developer) and those of a network engineer.

This strange convergence in skill requirements often means that load balancers are misunderstood. It’s one of the reasons why “It’s always the load balancer“.

I’ve found though that there are a few concepts and tricks that if learned by opposing skill sets can go a long way into understanding/dealing with/troubleshooting load balancing technology.

What a network engineer load balancer administrator needs to know about servers

• Default routes of the servers. I don’t keep statistics on the hundreds of issues I’ve diagnosed over the years, but I’d guess fully 50% involved default routes on the servers as either the primary problem, or a related problem. This is true for every vendor I’ve worked with (or for).

• Learn to use liveHTTP headers for Firefox and/or ieHTTPheaders for Internet Explorer. This shows you exactly what the server is doing, and can help you answer questions like “why is the load balancer redirecting me to Google?” (The answer is no, it’s not, the application is doing that, and here’s the HTTP 302 in the header to prove it.)

• If you’re setting up a non-transparent setup (known as full-NAT) where the IP address of the web clients are hidden, make sure that your server people are aware of this (and OK with this). Otherwise, when the server people call asking where the IP addresses of their clients are, and you have to explain to them that the IPs are lost forever, and that the load balancer doesn’t log them, it’s going to get unpleasant.

• Setup MRTG for your load balancer. Pull as many stats as you can get your grubby little paws on.

What a server admin/application developer needs to know about load balancers

• If you’ve got a tricky problem that can be solved by a solution in a load balancer and in an application, do it in an application. It may seem seductive to have the load balancer take care of specialized cases with specialized URL parsing or redirects, but it can greatly complicate your setup. For one thing, you have to go to a different group in order to make a change. This doesn’t include the core functionality of load balancers, such as cookie persistence of course.

• Know how to do a network capture, using TCPDump or WindDump. This is a language network engineers understand, and even if you don’t understand what it means, the network group can often pick through it and figure it out. It would help greatly to learn to parse it yourself, as well. It would help to know how to read Ethereal (available for Linux/BSD/Unix and Windows).

• Load balancers don’t log client connections. There’s only one load balancer that I know of that could even configure such logging, but it wouldn’t be recommended and would likely impact performance.

Have a tip? Leave a comment.