RE: pen performance with poll on large number of simultaneous connections

From: Ulric Eriksson (ulric@siag.nu)
Date: Wed Sep 18 2002 - 17:20:22 CEST


On Wed, 18 Sep 2002, Torsten Valentin wrote:

> What does that mean? Is poll() used when compiling with --with-poll ?
> Are there known instabilities will poll() ? Or has there just not been
> enough testing?

--with-poll causes a bunch of code to be compiled that uses poll() instead
of select(). It still won't be used, unless the -P option is specified.
The only reason anybody would want to do that is to play with the code
(and hack on it).

> It's surely a lot better than select(), as far as I know. I don't know
> which performance measures you have made. All my knowledge is based upon
> theory and from that website I mentioned. From what I've seen so far,
> poll() should make a huge difference to select(). Isn't that so?

It all depends on the implementation. A properly implemented select can
be more efficient than poll, but I believe that Linux emulates select
through poll. Using poll on a huge number of file descriptors will also
use huge amounts of memory.

> > An alternative would be to get at the raw frames (ip or ethernet)
> instead.
> > It would also
> > require a major rewrite of the low-level plumbing in pen. On the other
> > hand, it would be easier to do things, such as modifying frame
> headers. It
> > is difficult to say what it would do to performance, although I
> believe it
> > would be very fast.
>
> Yes, maybe, but why reinventing the wheel again. Kqueue should not be
> too hard to implement and then you have it. Even raw coding would not
> provide a much better performance than kqueue, besides, that's even not
> necessary at all, because kqueue performs perfectly. In my eyes I
> wouldn't recommend anyone with the need for a large number of
> simultaneous connections to use Linux as OS. Those people should use a
> BDS-system because of the kqueue advantage. That's it. On Linux even
> /dev/poll is in experimental state. I cannot see a sense in trying to
> make a tool as portable as possible, while some OS limitations just make
> it unusable for certain purposes on certain OSs.

The tun/tap driver (http://vtun.sourceforge.net) is available for Linux,
Solaris and FreeBSD. I believe it would allow us to get good performance
for very large numbers of connections, while at the same time allowing
features that are not possible with either select, poll or kqueue. That is
not to say I would reject a patch that adds kqueue functionality.

> You're pretty much into Linux, aren't you? :-)

More importantly, *all* of our customers who use pen run it on Linux, so
it has to be rock solid there. I sometimes test it at home on Solaris and
Windows NT4, but that's sissy stuff compared to a major web site with
real traffic.

Ulric



This archive was generated by hypermail 2.1.2 : Wed Sep 18 2002 - 17:20:47 CEST