RE: pen performance with poll on large number of simultaneous connections

From: Torsten Valentin (winsock2@musiker.de)
Date: Wed Sep 18 2002 - 17:49:39 CEST


> > From what I've seen so far,
> > poll() should make a huge difference to select(). Isn't that so?
> It all depends on the implementation. A properly implemented select
can
> be more efficient than poll, but I believe that Linux emulates select
> through poll. Using poll on a huge number of file descriptors will
also
> use huge amounts of memory.

I'm on OpenBSD, I'm not sure, but I think select() is select() there and
no emulation of poll(), but I can be wrong. At the end all that
interests me is performance and as far as I can see now you've got to
throw a huge machine at pen if you want to use pen with a large number
of simultaneous connections, and there's no way around this.

> The tun/tap driver (http://vtun.sourceforge.net) is available for
Linux,
> Solaris and FreeBSD. I believe it would allow us to get good
performance
> for very large numbers of connections, while at the same time allowing
> features that are not possible with either select, poll or kqueue.

I took a look at it, but I cannot see where this should help pen with
replacing select() or poll(). But that's probably just my poor knowledge
about these structures. Anyway, my personal opinion is that kqueue is so
superior to anything else here that I doubt it's worth searching for
alternatives. If I regard it economically, it would make sense to
implement kqueue into pen and then convince the Linux-users who want to
use pen with a very large number of simultaneous connections, that they
should use BSD. I think one needs to keep in mind that such a large
number of simultaneous connections is a rather seldom requirement. Most
users probably go quite well with their Linux-box and pen compiled to
use select(). My opinion is that if you have special demands, you need
special solutions. One of those could be xxxBSD with pen and kqueue. :-)

> That is
> not to say I would reject a patch that adds kqueue functionality.

:-) OK, like I said, if I could do it, I'd do it, but I can't. Seems
like I've got to find someone as keen into this as me with proper
knowledge of programming this kind of stuff. How many users are there on
the pen ML at the moment? :-)

> > You're pretty much into Linux, aren't you? :-)
> More importantly, *all* of our customers who use pen run it on Linux,
so
> it has to be rock solid there.

I got your point and agree with you here. Nevertheless I'd suggest
anyone who wants to use pen as a loadbalancer for a very large number of
simultaneous connections to use a BSD OS if there was a patch for pen to
use kqueue. That's just because it could be a lot more powerful with
reasonable effort. :-)

> Ulric

If I find someone who enhances pen to use kqueue, I'll let you know. :-)

Regards,
Torsten



This archive was generated by hypermail 2.1.2 : Wed Sep 18 2002 - 18:02:48 CEST