SMP and Linux network protocol stack - Linux

This is a discussion on SMP and Linux network protocol stack - Linux ; Hi, When Linux is running on a SMP system, every processor can receive NIC's interrupts and receive packets into it's own queue. When protocol's own xx_rcv() functions (rx softirq) starts to run, it could only receives the packets from its ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: SMP and Linux network protocol stack

  1. SMP and Linux network protocol stack

    Hi,

    When Linux is running on a SMP system, every processor can receive
    NIC's interrupts and receive packets into it's own queue. When
    protocol's own xx_rcv() functions (rx softirq) starts to run, it could
    only receives the packets from its own processor's queue. So it seems
    that if rx softirq in one processor runs more, then the protocol could
    only receive parts of packets, as other packets are still in the other
    processor's packets queue, is that possible the protocol will finaly
    think the rest packets are lost?


  2. Re: SMP and Linux network protocol stack


    cyril.li@gmail.com wrote:

    > When Linux is running on a SMP system, every processor can receive
    > NIC's interrupts and receive packets into it's own queue. When
    > protocol's own xx_rcv() functions (rx softirq) starts to run, it could
    > only receives the packets from its own processor's queue. So it seems
    > that if rx softirq in one processor runs more, then the protocol could
    > only receive parts of packets, as other packets are still in the other
    > processor's packets queue, is that possible the protocol will finaly
    > think the rest packets are lost?


    There are several ways this problem can be solved, and I don't recall
    offhand which method Linux uses, but I'm 99% sure it's one of these
    two:

    1) A protocol receive function is always scheduled on the same
    processor an interrupt is handled on. If two CPUs each get an
    interrupt, each will eventually run the protocol receive function.

    2) The queues are not per-CPU, so no matter which CPU the protocol
    receive function runs on, it will see packets received by interrupts
    handled by any CPU.

    DS


  3. Re: SMP and Linux network protocol stack

    It seems that NIC interrupts always routes to one CPU in current Linux
    implementation.Though there's irq-balance if one CPU is overload, in
    certain period, only one CPU can receive NIC interrupts.

    davids@webmaster.com 写道:

    > cyril.li@gmail.com wrote:
    >
    > > When Linux is running on a SMP system, every processor can receive
    > > NIC's interrupts and receive packets into it's own queue. When
    > > protocol's own xx_rcv() functions (rx softirq) starts to run, it could
    > > only receives the packets from its own processor's queue. So it seems
    > > that if rx softirq in one processor runs more, then the protocol could
    > > only receive parts of packets, as other packets are still in the other
    > > processor's packets queue, is that possible the protocol will finaly
    > > think the rest packets are lost?

    >
    > There are several ways this problem can be solved, and I don't recall
    > offhand which method Linux uses, but I'm 99% sure it's one of these
    > two:
    >
    > 1) A protocol receive function is always scheduled on the same
    > processor an interrupt is handled on. If two CPUs each get an
    > interrupt, each will eventually run the protocol receive function.
    >
    > 2) The queues are not per-CPU, so no matter which CPU the protocol
    > receive function runs on, it will see packets received by interrupts
    > handled by any CPU.
    >
    > DS



+ Reply to Thread