High-performance server: epoll or a thread pool? - Unix

This is a discussion on High-performance server: epoll or a thread pool? - Unix ; Hi all, I need to build a performance-critical TCP server, and trying to make a decision of whether to use IO multiplexing (with epoll) or just dispatch requests to one of worker threads (from the thread pool), each worker doing ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: High-performance server: epoll or a thread pool?

  1. High-performance server: epoll or a thread pool?

    Hi all,

    I need to build a performance-critical TCP server, and trying to make
    a decision of whether to use IO multiplexing (with epoll) or just
    dispatch requests to one of worker threads (from the thread pool),
    each worker doing synchronous IO and processing. We will probably
    have a limited number of connections reused for multiple requests.

    My initial test show that epoll-based system is about 30% faster (and
    utilizes processor much better), but I suspect that when comparing
    performance of such systems, a lot needs to be taken into account --
    network bandwidth, the number of processors, usage pattern, IO to
    processing ratio, worker threads, etc.

    (for the test I read 50 byte request, perform search in std::map 50
    times, and send 250 byte response. I have two processors)

    Do you think it is possible to say that this test data reflect the
    reality more or less accurate, and the epoll-based server would be
    generally more efficient?

    Thanks in advance,
    Arkadiy




  2. Re: High-performance server: epoll or a thread pool?

    Arkadiy writes:
    > I need to build a performance-critical TCP server, and trying to make
    > a decision of whether to use IO multiplexing (with epoll) or just
    > dispatch requests to one of worker threads (from the thread pool),
    > each worker doing synchronous IO and processing. We will probably
    > have a limited number of connections reused for multiple requests.
    >
    > My initial test show that epoll-based system is about 30% faster (and
    > utilizes processor much better), but I suspect that when comparing
    > performance of such systems, a lot needs to be taken into account --
    > network bandwidth, the number of processors, usage pattern, IO to
    > processing ratio, worker threads, etc.
    >
    > (for the test I read 50 byte request, perform search in std::map 50
    > times, and send 250 byte response. I have two processors)
    >
    > Do you think it is possible to say that this test data reflect the
    > reality more or less accurate, and the epoll-based server would be
    > generally more efficient?


    It will generally be more efficient to distribute task among about as
    many threads as one has processor to run them using some
    multiplexeing-scheme than to have signficantly more runnable threads
    than processor, reason: context-switching overhead.

  3. Re: High-performance server: epoll or a thread pool?

    Arkadiy wrote:
    > Hi all,
    >
    > I need to build a performance-critical TCP server, and trying to make
    > a decision of whether to use IO multiplexing (with epoll) or just
    > dispatch requests to one of worker threads (from the thread pool),
    > each worker doing synchronous IO and processing. We will probably
    > have a limited number of connections reused for multiple requests.


    I suspect a hybrid approach would be best. Use one thread per available
    cpu core, with epoll to track readable sockets.

    Chris

  4. Re: High-performance server: epoll or a thread pool?

    Arkadiy wrote:
    > Do you think it is possible to say that this test data reflect the
    > reality more or less accurate, and the epoll-based server would be
    > generally more efficient?


    In broad terms yes because your epoll-based code probably has a much
    lower "context switch" overhead than a thread switch would.

    If you can get all the work done you need to get done with the
    services of a single core, then I prefer the epoll-esque
    approach. Now, if in servicing one client, your epoll-esque code could
    block for a while that might not be good and would be an indication of
    thread-per-something.

    If you need the services of more than one core, then as pointed-out,
    something more hybrid would be good - either thread or if you don't
    need as tight a coupling, process per core.

    rick jones
    --
    a wide gulf separates "what if" from "if only"
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

+ Reply to Thread