Shared sockets - TCP-IP

This is a discussion on Shared sockets - TCP-IP ; On Dec 27, 1:00 am, Emil Naepflein wrote: > >You usually have to work to make things unfair. You will generally get > >fairness automatically. > Just think about how you scan the ready file descriptors. If you do it ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 27 of 27

Thread: Shared sockets

  1. Re: Shared sockets

    On Dec 27, 1:00 am, Emil Naepflein wrote:

    > >You usually have to work to make things unfair. You will generally get
    > >fairness automatically.


    > Just think about how you scan the ready file descriptors. If you do it
    > sequentially starting at the lowest number the file descriptors at the
    > beginning always get the fastest service. If you handle thousands of
    > them then there will be a significant difference.


    They won't get the fastest service. You can argue that they'll get the
    slowest service just as easily as you can argue that they get the
    slowest service because the time interval between when data is sent to
    them and when you check if they've sent a reply is the longest.

    Every socket is checked just as often as every other socket.

    > You also will get a high variance of response times.


    Sure, but that's just a fact of life. Low load, fast response. High
    load, slow response. However, the interval between successive
    operations will be the same for all descriptors, so in a sense, you
    get more fairness.

    It depends upon whether you think equally punishing all connections
    for one slow one is fair. Sometimes that is fair and sometimes it is
    not.

    > To avoid this you would have to reimplement a kind of "fair" scheduling
    > just like it is done for processes/threads in the OS. So it may be
    > easier to just use one native process/thread to handle one connection
    > and let the OS do the work. Of course, it also depends how long living
    > your connections are. If you have a very high connection rate with short
    > living connections then the select/poll implemention may be the way to
    > go (beside the multiprocessor issue). If there are very long living
    > connections then the overhead of creating processes/threads is
    > amortisized over time.


    If you care about a notion of fairness you have to do this anyway, for
    two reasons:

    1) Processes and threads are usually not scheduled "fairly". In fact,
    POSIX does not provide any guarantees about thread fairness.

    2) The notion of fairness that schedulers provide is generally a
    technical one and very unlikely to align with the practical notion of
    fairness that applies to your particular task.

    You always want to control what work your threads do such that it's
    the work that makes the most sense to your overall application goal.
    Let the implementation choose the most efficient execution vehicle to
    run, but you write its code so it does the most useful work. That's
    how the split of responsibility goes.

    DS

  2. Re: Shared sockets

    On Dec 27, 1:55 pm, Barry Margolin wrote:

    > Is that (emulated threading) really true on any high-performance flavor
    > of Unix these days?


    Not on any platform that claims POSIX support. The emulation tends to
    break down in horrible ways. How do you make disk I/O not block all
    threads?

    DS

  3. Re: Shared sockets

    In article
    <29441737-46d3-486e-8761-18a24ba42ead@s12g2000prg.googlegroups.com>,
    David Schwartz wrote:

    > On Dec 27, 1:55 pm, Barry Margolin wrote:
    >
    > > Is that (emulated threading) really true on any high-performance flavor
    > > of Unix these days?

    >
    > Not on any platform that claims POSIX support. The emulation tends to
    > break down in horrible ways. How do you make disk I/O not block all
    > threads?


    By implementing threads in the kernel, rather than emulating them in
    user mode. Isn't that how threads are done in most Unix systems these
    days?

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  4. Re: Shared sockets

    In article ,
    Barry Margolin wrote:
    ><29441737-46d3-486e-8761-18a24ba42ead@s12g2000prg.googlegroups.com>,
    > David Schwartz wrote:


    >> > Is that (emulated threading) really true on any high-performance flavor
    >> > of Unix these days?

    >>
    >> Not on any platform that claims POSIX support.


    No one with an honest idea of what POSIX thread support means would
    say that.

    >> The emulation tends to
    >> break down in horrible ways. How do you make disk I/O not block all
    >> threads?


    Disk I/O is generally special, because it is usually non-blocking until
    you push the limits of the buffer cache or take a page fault. Disk I/O
    is also complicated in the kernel by the paging system. Even on systems
    with kernel support for threads and even paging/swapping unified with
    the buffer cache, the prudent or experienced will assume that all threads
    of a process might block when any thread is blocked for disk I/O and,
    if it matters enough, write obvious countermeasures.


    >By implementing threads in the kernel, rather than emulating them in
    >user mode. Isn't that how threads are done in most Unix systems these
    >days?


    Based on my continuing experience flogging DCC or Distributed Checksum
    Clearinghouses, a spam filter that runs on many flavors of UNIX, I think
    the answer to that question is still "no." Perhaps the reason for that
    surprising state of affairs is that putting your threading machinery
    into the kernel risks violating the reason for using "lightweight theads"
    in the first place. If switching between threads requires a switch
    from user space to kernel space and back, and so many of the CPU and
    RAM costs of a process context switch, then your threads might not be
    very "lightweight." However, I suspect the reason is simply lack of
    attention. "Emulated" POSIX threads generally work well enough.


    Some responses have pointed out that a single process using POSIX
    compliant threads might not be as fast on a multi-processor as separate
    processes. Depending on the particulars of the multi-processor hardware,
    operating system, and the application, that can be true. However, it
    might not be. When optimizing for a particular system, you have to
    pay attention to the particular system.

    Please note that I am *not* an advocate for POSIX or any flavor of
    threads. In some cases they are a reasonable choice, but they have
    never been the panacea that their advocates claimed they would be.


    Vernon Schryver vjs@rhyolite.com

  5. Re: Shared sockets


    Looking past my killfile I see

    In article ,
    David Schwartz wrote:

    >1) Processes and threads are usually not scheduled "fairly". In fact,
    >POSIX does not provide any guarantees about thread fairness.


    Anyone who has actually bothered to read the right manual pages
    knows that is wrong, if the implementation has all of the POSIX
    threads bells and whistles.

    >2) The notion of fairness that schedulers provide is generally a
    >technical one and very unlikely to align with the practical notion of
    >fairness that applies to your particular task.


    That is nonsense.
    It is true that various notions of process scheduling have technical
    definitions. How could it be otherwise?
    It is also true that the notions provided by the target platform's
    process or POSIX threads might or might not be best for your
    application. Such is life.


    >You always want to control what work your threads do such that it's
    >the work that makes the most sense to your overall application goal.
    >Let the implementation choose the most efficient execution vehicle to
    >run, but you write its code so it does the most useful work. That's
    >how the split of responsibility goes.


    Willful ignorance of how significant parts of the system work is rarely
    a recipe for producing the best application, except when the definition
    of "best" is "quickest coding and generally down to Microsoft standard."


    Vernon Schryver vjs@rhyolite.com

  6. Re: Shared sockets

    On Dec 29 2007, 7:39 am, v...@calcite.rhyolite.com (Vernon Schryver)
    wrote:

    > Based on my continuing experience flogging DCC or Distributed Checksum
    > Clearinghouses, a spam filter that runs on many flavors of UNIX, I think
    > the answer to that question is still "no." Perhaps the reason for that
    > surprising state of affairs is that putting your threading machinery
    > into the kernel risks violating the reason for using "lightweight theads"
    > in the first place. If switching between threads requires a switch
    > from user space to kernel space and back, and so many of the CPU and
    > RAM costs of a process context switch, then your threads might not be
    > very "lightweight." However, I suspect the reason is simply lack of
    > attention. "Emulated" POSIX threads generally work well enough.


    If you care about context switch time, you are doing something
    *horribly* wrong. If there is no work to do, you should not care how
    long context switch time is. If there is work to do, the thread that
    is running should be doing it, not switching to another thread.

    If you are switching threads for any reason other than time slice
    exhaustion when there is lots of work to do, your design is broken.
    When you are under load, threads should be using out their timeslices
    doing the work.

    The scheduler's job is to figure out the most efficient way to
    schedule your threads. Your job is to take whatever thread the
    scheduler schedules and make it do the work you need done. If you need
    to keep switching threads while there's still work to be done, you've
    defaulted on your job.

    Note that you can certainly write threading implementations with
    different rules. I'm talking about a UNIX platform's native POSIX
    support.

    DS

  7. Re: Shared sockets

    On Dec 29 2007, 7:52 am, v...@calcite.rhyolite.com (Vernon Schryver)
    wrote:

    > >2) The notion of fairness that schedulers provide is generally a
    > >technical one and very unlikely to align with the practical notion of
    > >fairness that applies to your particular task.


    > That is nonsense.
    > It is true that various notions of process scheduling have technical
    > definitions. How could it be otherwise?
    > It is also true that the notions provided by the target platform's
    > process or POSIX threads might or might not be best for your
    > application. Such is life.


    Let me give an example. It's hard to provide a typical example, but I
    hope I can at least provide an exemplary one.

    I have 12 sources of queries. They each produce about the same query
    load. Ten of them are against one object, two of them are against
    another. Queries are basically CPU limited. My application fairness
    goal is to have about the same latency for all queries.

    Now, I could devote one thread to queries to the first object and one
    to queries to the second object. This would minimize contention among
    the threads and would provide near perfect performance on a single CPU
    machine.

    Can I count on the scheduler to meet my application fairness goal by
    giving the thread handling five times as many query sources five times
    the CPU? Of course not.

    Why? Because the scheduler is most likely to be fair to threads. This
    may stop my threads from filing union grievances, but it's not going
    to help me meet my application fairness goal.

    There are a few ways I could handle this. One would be to teach my
    scheduler my application fairness requirements. This can actually be
    done on Linux 2.6.23, since it has scheduler classes that actually map
    to CPU ratios. I would then tie my application's fairness to my
    scheduler's fairness. And, of course, if the query ratio is off, so is
    the fairness. If the scheduler implementation changes, my fairness
    goes away.

    Oh, and now my application isn't portable.

    I am saying this is why you should not, in the vast majority of cases,
    try to turn whatever scheduler fairness you have into the application
    fairness you need. You should instead *code* the application fairness
    you need relying on as little of the scheduler implementation as you
    possibly can.

    You could do that in this case by putting all requests in a queue,
    having some threads that pull requests from the queue, and blocking if
    the object our query is against is in use. This will provide the
    required fairness no matter what the scheduler does (within reason).

    > Willful ignorance of how significant parts of the system work is rarely
    > a recipe for producing the best application, except when the definition
    > of "best" is "quickest coding and generally down to Microsoft standard."


    While there's no benefit to ignorance, there's a huge benefit to not
    making your design reliant on your detailed knowledge of a particular
    system's operation. Well-designed robust applications result from
    knowledge of what is guaranteed to be there, not what happens to be
    there in this version.

    Trying to teach the scheduler what fairness means to your application
    is a recipe for disaster in the majority of cases. The cases where it
    makes sense are the exception.

    DS

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2