Any way to eliminate bonding interface dropped packets? - Networking

This is a discussion on Any way to eliminate bonding interface dropped packets? - Networking ; We're using two e1000 interfaces bonded to one pseudo interface. When the network traffic is heavy, we see dropped packets. Is there any way to improve this? /etc/modprobe.conf (bond0 omitted here because it has much less traffic and works fine): ...

+ Reply to Thread
Results 1 to 5 of 5

Thread: Any way to eliminate bonding interface dropped packets?

  1. Any way to eliminate bonding interface dropped packets?

    We're using two e1000 interfaces bonded to one pseudo interface. When
    the network traffic is heavy, we see dropped packets. Is there any way
    to improve this?

    /etc/modprobe.conf (bond0 omitted here because it has much less
    traffic and works fine):
    alias eth2 e1000
    alias eth3 e1000
    install bond1 /sbin/modprobe bonding -o bond1 miimon=100 mode=1
    primary=eth2
    options e1000 FlowControl=1,1

    $ uname -a
    Linux 2.6.9-55.ELsmp #1 SMP Fri Apr 20 16:36:54 EDT 2007
    x86_64 x86_64 x86_64 GNU/Linux

    Note the 52 packets dropped since bootup. It always happens to
    receiving.
    $ ifconfig bond1
    bond1 Link encap:Ethernet HWaddr 00:1B:78:58:F4:9C
    inet addr:192.168.2.11 Bcast:192.168.2.255 Mask:
    255.255.255.0
    inet6 addr: fe80::21b:78ff:fe58:f49c/64 Scope:Link
    UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
    RX packets:150304549 errors:0 dropped:52 overruns:0 frame:0
    TX packets:148948403 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:111541084012 (103.8 GiB) TX bytes:109717011822
    (102.1 GiB)

    eth2 is the primary or active slave (eth3 is passive).
    $ ifconfig eth2 | grep dropped
    RX packets:150487351 errors:0 dropped:52 overruns:0 frame:0
    TX packets:149129271 errors:0 dropped:0 overruns:0 carrier:0

    Is there any magic we can add here?
    $ grep -v ^# /etc/sysctl.conf
    net.ipv4.ip_forward = 0
    net.ipv4.conf.default.rp_filter = 1
    net.ipv4.conf.default.accept_source_route = 0
    kernel.sysrq = 0
    kernel.core_uses_pid = 1
    net.ipv4.ip_local_port_range=1024 65000
    fs.file-max=1515611
    kernel.sem=250 32000 100 128
    kernel.shmmax=8589934592
    kernel.shmall=4194304
    kernel.shmmni=4096
    net.core.rmem_default = 1048576
    net.core.rmem_max = 1048576
    net.core.wmem_default = 1048576
    net.core.wmem_max = 1048576
    vm.nr_hugepages = 5120

    Receiving flow control is on:
    # ethtool -a eth2
    Pause parameters for eth2:
    Autonegotiate: on
    RX: on
    TX: off

    Thanks.

    Yong Huang

  2. Re: Any way to eliminate bonding interface dropped packets?

    On Mar 13, 3:36 pm, Yong Huang wrote:

    > We're using two e1000 interfaces bonded to one pseudo interface. When
    > the network traffic is heavy, we see dropped packets. Is there any way
    > to improve this?


    I'm sorry you are seeing a .000000345% packet loss rate. Ethernet is
    best effort, and this seems pretty darned good to me. If you can't
    deal with a packet loss rate this low, something is *very* wrong with
    how you are using it.

    DS

  3. Re: Any way to eliminate bonding interface dropped packets?


    Sorry, should be .0000346%, but the point stands. That's close enough
    to perfect as had better make no difference.

    DS

  4. Re: Any way to eliminate bonding interface dropped packets?

    On Thu, 13 Mar 2008 15:36:32 -0700 (PDT), Yong Huang
    wrote:

    >$ ifconfig bond1
    >bond1 Link encap:Ethernet HWaddr 00:1B:78:58:F4:9C
    > inet addr:192.168.2.11 Bcast:192.168.2.255 Mask:
    >255.255.255.0
    > inet6 addr: fe80::21b:78ff:fe58:f49c/64 Scope:Link
    > UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
    > RX packets:150304549 errors:0 dropped:52 overruns:0 frame:0
    > TX packets:148948403 errors:0 dropped:0 overruns:0 carrier:0
    > collisions:0 txqueuelen:0
    > RX bytes:111541084012 (103.8 GiB) TX bytes:109717011822
    >(102.1 GiB)
    >Thanks.
    >
    >Yong Huang


    I know nothing of bonding, but can you set txqueuelen (anywhere, not
    just on bond1)? The zero size says there is no buffer. This probably
    won't help because RX is the problem, but I've had some (small)
    success with other devices that have no buffer by setting txqueuelen
    to 1000.
    --
    buck


  5. Re: Any way to eliminate bonding interface dropped packets?

    On Mar 14, 11:52 am, buck wrote:
    > On Thu, 13 Mar 2008 15:36:32 -0700 (PDT), Yong Huang
    >
    > wrote:
    > >$ ifconfig bond1
    > >bond1 Link encap:Ethernet HWaddr 00:1B:78:58:F4:9C
    > > inet addr:192.168.2.11 Bcast:192.168.2.255 Mask:
    > >255.255.255.0
    > > inet6 addr: fe80::21b:78ff:fe58:f49c/64 Scope:Link
    > > UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
    > > RX packets:150304549 errors:0 dropped:52 overruns:0 frame:0
    > > TX packets:148948403 errors:0 dropped:0 overruns:0 carrier:0
    > > collisions:0 txqueuelen:0
    > > RX bytes:111541084012 (103.8 GiB) TX bytes:109717011822
    > >(102.1 GiB)
    > >Thanks.

    >
    > >Yong Huang

    >
    > I know nothing of bonding, but can you set txqueuelen (anywhere, not
    > just on bond1)? The zero size says there is no buffer. This probably
    > won't help because RX is the problem, but I've had some (small)
    > success with other devices that have no buffer by setting txqueuelen
    > to 1000.
    > --
    > buck


    Thanks, buck. I did 'ifconfig bond1 txqueuelen 1000' (eth2 and eth3
    are already 1000). The rate of dropped packets increase remains the
    same.

    To David, thanks for your comments. I think I have to agree with you
    we're already near perfection so we'll probably live with it. We're
    stress testing this 8-node Oracle RAC (cluster).

    Yong Huang

+ Reply to Thread