"old" connections in pen using apache 2.2 as backend

Hello!

For quite some time i am running pen (0.18) under linux 2.6 with seven
backend servers running apache 1.3 without any problems. Recently i
upgraded one of these servers to apache 2.2 and since this time
recognized a strong imbalance of client distribution to this upgraded
apache 2.2 server.

Configuration of Apache 2.2 is - as possible - the same as for apache
1.3. Meaning MPM prefork, same KeepAlive (10s), Timeout, MaxClients etc.

After some digging i set up an test environment with server0 using
apache2.2 and server1 using apache1.3.

penctl mode output:
no block no delayed_forward hash no roundrobin no stubborn weight no prio

Looking at penctl's "server" output after a few minutes, load seems to
be equal.
server0: conn 24 max 170 hard 220 weight 20 prio 0 sx 182436 rx 158731
server1: conn 24 max 170 hard 220 weight 20 prio 0 sx 167685 rx 928783

But after some hours load on server0 drops and looking at the
server-status of apache 2.2 shows me only a few request each minute,
whereas the load on server1 is in increasing accordingly.
Output of "penctl server" still shows nearly equal "con"-numbers. Even
for server0 - which has according to the Apache "Server-Status" nearly
nothing to do - shows still a high number of active connections in pen.
I forgot to mention, that when i restart pen at this point, without
touching one of the apache servers, load is distributed equally again ..
for some hours only, of course.

I patched pen.c to change the output of the "server" command to show me
the number of hits and connections a server got through pen, which -
beside other changes - looks like this:

server0: status 0 conn 28 max 170 hard 220 weight 20 prio 0 sx 2216KB rx
18MB tcon 944 hits 2631
server1: status 0 conn 27 max 170 hard 220 weight 20 prio 0 sx 2117KB rx
14MB tcon 958 hits 2394

where tcon is the total number of connections a server were assigned and
hits the total number of hits.

Again, after some hours:

server0: status 0 conn 93 max 170 hard 220 weight 20 prio 0 sx 385MB rx
4114MB tcon 184295 hits 484555
server1: status 0 conn 93 max 170 hard 220 weight 20 prio 0 sx 636MB rx
6072MB tcon 297695 hits 788039

I modified pen.c one more time to output the struct conns and clients
(like webstats). This showed me, that after some hours most of the
connections to the server running apache 1.3 have a low value for the
last time a connection was used/accessed, where most of the connections
to the server running apache 2.2 have a high value, 100 seconds and up
sometimes up to 900. ( computed using
(now-clients[conns[n].clt].last/.lastaccess )

Any ideas why these "old" connections remain open that long time in pen
using apache 2.2 and not with apache 1.3?

Thomas
Received on Thu Jan 29 2009 - 18:59:52 CET