Have you ever received the following error?
[error] server reached MaxClients setting, consider raising the MaxClients setting.

If so, this article might be for you. It can help you understand what MaxClients does, and how to estimate the correct value for it.

Setting MaxClients: Erlang models and Apache performance

Banks, call centres, and telephone operators frequently employ Erlang models to study the relationship between resource use and service levels. This page introduces the Erlang model to Apache administrators trying to correctly set parameters such as MaxClients. It should also be a useful guide for adminstrators of other servers.

It's 9AM. How much memory is your Apache server using?

Apache allows an administrator to modify the maximum number of simultaneous connections that can be maintained by a single server. Like many things in life, there is a trade-off involved. Increasing the number of simultaneous connections causes Apache to use extra memory. This means that increasing the number threads to a large number is a bad idea, unless you have an awful lot of free memory. You might try increasing the number by trial and error. But that leaves a few questions- how can you predict how a well known application will scale? How much memory will the server need?

Using a better model obviously has its advantages. Apache will spend less time thrashing and more time serving.

Optimising Apache's memory use is more important as your Apache instances become larger. To see the how much non-swapped physical memory Apache uses on your machine, execute the following command:

 
[cje@host ~]$ ps -ylC httpd --sort:rss

 
S   UID   PID  PPID  C PRI  NI  RSS    SZ WCHAN  TTY       TIME CMD
S    99  7043  9409  0  75   0 25680 7369 semtim ?   00:00:01 httpd
S    99  6757  9409  0  75   0 25712 7358 semtim ?   00:00:01 httpd
S    99  6761  9409  0  75   0 25736 7361 semtim ?   00:00:01 httpd
S    99  6763  9409  0  76   0 25792 7397 semtim ?   00:00:01 httpd
S    99  6756  9409  0  75   0 25804 7470 semtim ?   00:00:01 httpd
S    99  6752  9409  0  75   0 25872 7396 411315 ?   00:00:01 httpd

The RSS column is the one you are looking for. You can see that on this machine, around 25Mb is required for each new thread. It's important to get your traffic right. As you can imagine, there is a big difference between setting a maximum number of connections at 100 and 130.

When is an Erlang model applicable?

Apache serialises inbound connections. Through configuration parameters, it limits the total number of inbound connections. An erlang model is appropriate for dimensioning the traffic to a single Apache server, or many of them, provided each connection is independent of the last connection. An example of this scenario would be a server which loads independent banner advertisements from a large number of clients.

When is an Erlang model inapplicable?

When the nature of inbound connections becomes statistically dependent, the Erlang model will not apply. An example of this would be a client that opens a large number of connections at once, or makes a series of related connections in series. These scenarios distort the underlying statistical assumptions of the model, and must be handled by a separate calculation process. It may also be necessary to consider the impact of HTTP/1.1 pipelining. From the Apache page on maxKeepAliveRequests.

"The maximum number of HTTP requests which can be pipelined until the connection is closed by the server. Setting this attribute to 1 will disable HTTP/1.0 keep-alive, as well as HTTP/1.1 keep-alive and pipelining. Setting this to -1 will allow an unlimited amount of pipelined or keep-alive HTTP requests. If not specified, this attribute is set to 100.

How much traffic am I handling?

The Erlang is a dimensionless quantity. For a given server, it can be calculated in a straightforward manner. Let's start with some assumptions:

You have traffic on a single server.
The server is not configured for redundant operation.

How many requests are you serving? Let's say that altogether, your servers handle 1000 distinct requests per second.
How long do your requests take to serve? 0.15 seconds is a typical amount of time. Don't be alarmed by how long a request may take. Remember that a given request may be blocked for much of the total service time.
What's your mean arrival rate? What's your mean call holding time? If you don't know, you can find out by inserting an entry like the following into your Apache configuration file.
```
 
 LogFormat "%D %h %l %u %t \"%r\" %>s %b" combined
```
The trick is the %D parameter. It logs the time taken to serve the request in microseconds. Leave it log over a busy period. The mean arrival rate is simply the number of seconds elapsed from start to finish, divided by the number of lines in the file. (You can find that you using wc -l). The average holding time is simply the sum of all of the numbers in the first column, divided by the total number of lines. Remember to convert the answer, which will be in microseconds .

Let's assume that you have calculated the average holding time (average request time) as 0.15 seconds. Let's assume you have around 1000 requests/second arriving. The volume of traffic is simply the product of the last two numbers: 150 erlang.

Okay, I know how many Erlangs I'm handling. What's next?

If you think a little, you'll appreciate that at any given time, our server is probably handling around 150 requests. A naive approach might be to make Apache handle a total of 150 connections. However, that wouldn't make sense- new connections don't just arrive when we're finished serving the last connection! They arrive randomly, and we need to take that into account.

To do so, we use an Erlang B calculator , to see that for an offered traffic of 150 Erlang, with a GoS of 0.1%, it will be necessary to allow for 183 simultaneous connections. That means setting maxThreads to at least 183, or possibly higher if clients open multiple persistent connections to your server. (ie: maxKeepAliveRequests is greater than 1).

Advanced logging

If you would like to drive Apache further, and get more accurate results, you'll need better information. The following LogFormat command gives some useful additional information:

LogFormat "%D %P %X %h %l %u %t \"%r\" %>s %b" combined

The %P command gives the process ID of the serving thread. This is the first step toward fine grained information about the utilization of each thread.

The %X command is relevant to connections which serve multiple requests. The key is as follows: X means the connection aborted, + means the connection may be kept alive after the response is sent, - means the connection will be closed after the response is sent.