Problem in traffic control with HTB + PRIO - Networking

This is a discussion on Problem in traffic control with HTB + PRIO - Networking ; Hi Everyone, I met some problems in traffic control with TC. My purpose: 1. Restrict the overall bandwidth. 2. Starve the flows in the lower bands. Based on the following link, I develope the script bellow. http://linux.derkeiler.com/Newsgroup...4-12/0624.html ####################script################## IPTABLES=/usr/local/sbin/iptables MODPROBE=/sbin/modprobe ...

+ Reply to Thread
Results 1 to 11 of 11

Thread: Problem in traffic control with HTB + PRIO

  1. Problem in traffic control with HTB + PRIO

    Hi Everyone,
    I met some problems in traffic control with TC.
    My purpose:
    1. Restrict the overall bandwidth.
    2. Starve the flows in the lower bands.
    Based on the following link, I develope the script bellow.
    http://linux.derkeiler.com/Newsgroup...4-12/0624.html



    ####################script##################
    IPTABLES=/usr/local/sbin/iptables
    MODPROBE=/sbin/modprobe
    IP=/usr/sbin/ip
    TC=/usr/sbin/tc
    RATE=10

    $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.1 -j MARK --set-
    mark 2 &>/dev/null
    $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.2 -j MARK --set-
    mark 3 &>/dev/null
    $IPTABLES -t mangle -D POSTROUTING -m length --length 0:64 -j MARK --
    set-mark 1 &>/dev/null

    $TC qdisc del dev eth0 root &>/dev/null


    $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.1 -j MARK --set-
    mark 2
    $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.2 -j MARK --set-
    mark 3
    $IPTABLES -t mangle -A POSTROUTING -m length --length 0:64 -j MARK --
    set-mark 1


    $TC qdisc add dev eth0 root handle 1:0 htb default 0
    $TC class add dev eth0 parent 1:0 classid 1:1 htb rate ${RATE}mbit
    $TC qdisc add dev eth0 parent 1:1 handle 2: prio

    $TC qdisc add dev eth0 parent 2:1 handle 10:0 bfifo limit 64k
    $TC qdisc add dev eth0 parent 2:2 handle 20:0 bfifo limit 64k
    $TC qdisc add dev eth0 parent 2:3 handle 30:0 bfifo limit 64k

    $TC filter add dev eth0 parent 1:0 prio 0 protocol ip handle 1 fw
    flowid 1:1
    $TC filter add dev eth0 parent 1:0 prio 1 protocol ip handle 2 fw
    flowid 1:1
    $TC filter add dev eth0 parent 1:0 prio 2 protocol ip handle 3 fw
    flowid 1:1

    $TC filter add dev eth0 parent 2:0 prio 0 protocol ip handle 1 fw
    flowid 2:1
    $TC filter add dev eth0 parent 2:0 prio 1 protocol ip handle 2 fw
    flowid 2:2
    $TC filter add dev eth0 parent 2:0 prio 2 protocol ip handle 3 fw
    flowid 2:3

    ##########################script################## ##########

    The problem is that if I set RATE to some value less than 20, it works
    perfectly. But if I set RATE to some value bigger than 20, It fails
    to work. Is there anything wrong with the script?
    My test environment:
    MIPS (400HZ)
    linux 2.6.18
    iproute2-2.6.15

    Any help would be appreciated.
    Banyeer.

  2. Re: Problem in traffic control with HTB + PRIO

    "banyeer" wrote in message
    news:e402d9c4-ec35-4d7f-aef9-7601e8699912@x16g2000prn.googlegroups.com...
    > Hi Everyone,
    > I met some problems in traffic control with TC.
    > My purpose:
    > 1. Restrict the overall bandwidth.
    > 2. Starve the flows in the lower bands.
    > Based on the following link, I develope the script bellow.
    >

    http://linux.derkeiler.com/Newsgroup...4-12/0624.html
    > ####################script##################
    > IPTABLES=/usr/local/sbin/iptables
    > MODPROBE=/sbin/modprobe
    > IP=/usr/sbin/ip
    > TC=/usr/sbin/tc
    > RATE=10
    >
    > $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.1 -j MARK --set-
    > mark 2 &>/dev/null
    > $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.2 -j MARK --set-
    > mark 3 &>/dev/null
    > $IPTABLES -t mangle -D POSTROUTING -m length --length 0:64 -j MARK --
    > set-mark 1 &>/dev/null
    >
    > $TC qdisc del dev eth0 root &>/dev/null
    >
    >
    > $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.1 -j MARK --set-
    > mark 2
    > $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.2 -j MARK --set-
    > mark 3
    > $IPTABLES -t mangle -A POSTROUTING -m length --length 0:64 -j MARK --
    > set-mark 1
    >
    >
    > $TC qdisc add dev eth0 root handle 1:0 htb default 0
    > $TC class add dev eth0 parent 1:0 classid 1:1 htb rate ${RATE}mbit
    > $TC qdisc add dev eth0 parent 1:1 handle 2: prio
    >
    > $TC qdisc add dev eth0 parent 2:1 handle 10:0 bfifo limit 64k
    > $TC qdisc add dev eth0 parent 2:2 handle 20:0 bfifo limit 64k
    > $TC qdisc add dev eth0 parent 2:3 handle 30:0 bfifo limit 64k
    >
    > $TC filter add dev eth0 parent 1:0 prio 0 protocol ip handle 1 fw
    > flowid 1:1
    > $TC filter add dev eth0 parent 1:0 prio 1 protocol ip handle 2 fw
    > flowid 1:1
    > $TC filter add dev eth0 parent 1:0 prio 2 protocol ip handle 3 fw
    > flowid 1:1
    >
    > $TC filter add dev eth0 parent 2:0 prio 0 protocol ip handle 1 fw
    > flowid 2:1
    > $TC filter add dev eth0 parent 2:0 prio 1 protocol ip handle 2 fw
    > flowid 2:2
    > $TC filter add dev eth0 parent 2:0 prio 2 protocol ip handle 3 fw
    > flowid 2:3
    >
    > ##########################script################## ##########
    >
    > The problem is that if I set RATE to some value less than 20, it works
    > perfectly. But if I set RATE to some value bigger than 20, It fails
    > to work. Is there anything wrong with the script?
    > My test environment:
    > MIPS (400HZ)
    > linux 2.6.18
    > iproute2-2.6.15


    1) By "failing to work," do you mean that the rules are completely ignored
    (i.e. the class counters are NOT incremented), or do you mean that the rules
    are used but the bandwidth used exceeds 20Mbit?
    2) In your filter rules, the selection criteria are all the same for each
    set of 3 rules.
    3) What happens if you upgrade to the current versions (2.6.27.4 kernel and
    2.6.26 iproute2)?



  3. Re: Problem in traffic control with HTB + PRIO

    On Nov 7, 12:15*pm, "D. Stussy" wrote:
    > "banyeer" wrote in message
    >
    > news:e402d9c4-ec35-4d7f-aef9-7601e8699912@x16g2000prn.googlegroups.com...> Hi Everyone,
    > > I met some problems in traffic control with TC.
    > > My purpose:
    > > 1. Restrict the overall bandwidth.
    > > 2. Starve the flows in the lower bands.
    > > Based on the following link, I develope the script bellow.

    >
    > http://linux.derkeiler.com/Newsgroup...working/2004-1...
    >
    >
    >
    >
    >
    > > ####################script##################
    > > IPTABLES=/usr/local/sbin/iptables
    > > MODPROBE=/sbin/modprobe
    > > IP=/usr/sbin/ip
    > > TC=/usr/sbin/tc
    > > RATE=10

    >
    > > $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.1 -j MARK --set-
    > > mark 2 &>/dev/null
    > > $IPTABLES -t mangle -D POSTROUTING --dst 192.168.0.2 -j MARK --set-
    > > mark 3 &>/dev/null
    > > $IPTABLES -t mangle -D POSTROUTING -m length --length 0:64 -j MARK --
    > > set-mark 1 &>/dev/null

    >
    > > $TC qdisc del dev eth0 root &>/dev/null

    >
    > > $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.1 -j MARK --set-
    > > mark 2
    > > $IPTABLES -t mangle -A POSTROUTING --dst 192.168.0.2 -j MARK --set-
    > > mark 3
    > > $IPTABLES -t mangle -A POSTROUTING -m length --length 0:64 -j MARK --
    > > set-mark 1

    >
    > > $TC qdisc add dev eth0 root handle 1:0 htb default 0
    > > $TC class add dev eth0 parent 1:0 classid 1:1 htb rate ${RATE}mbit
    > > $TC qdisc add dev eth0 parent 1:1 handle 2: prio

    >
    > > $TC qdisc add dev eth0 parent 2:1 handle 10:0 bfifo limit 64k
    > > $TC qdisc add dev eth0 parent 2:2 handle 20:0 bfifo limit 64k
    > > $TC qdisc add dev eth0 parent 2:3 handle 30:0 bfifo limit 64k

    >
    > > $TC filter add dev eth0 parent 1:0 prio 0 protocol ip handle 1 fw
    > > flowid 1:1
    > > $TC filter add dev eth0 parent 1:0 prio 1 protocol ip handle 2 fw
    > > flowid 1:1
    > > $TC filter add dev eth0 parent 1:0 prio 2 protocol ip handle 3 fw
    > > flowid 1:1

    >
    > > $TC filter add dev eth0 parent 2:0 prio 0 protocol ip handle 1 fw
    > > flowid 2:1
    > > $TC filter add dev eth0 parent 2:0 prio 1 protocol ip handle 2 fw
    > > flowid 2:2
    > > $TC filter add dev eth0 parent 2:0 prio 2 protocol ip handle 3 fw
    > > flowid 2:3

    >
    > > ##########################script################## ##########

    >
    > > The problem is that if I set RATE to some value less than 20, it works
    > > perfectly. *But if I set RATE to some value bigger than 20, It fails
    > > to work. *Is there anything wrong with the script?
    > > My test environment:
    > > MIPS (400HZ)
    > > linux 2.6.18
    > > iproute2-2.6.15

    >
    > 1) *By "failing to work," do you mean that the rules are completely ignored
    > (i.e. the class counters are NOT incremented), or do you mean that the rules
    > are used but the bandwidth used exceeds 20Mbit?
    > 2) *In your filter rules, the selection criteria are all the same for each
    > set of 3 rules.
    > 3) *What happens if you upgrade to the current versions (2.6.27.4 kernel and
    > 2.6.26 iproute2)?- Hide quoted text -
    >
    > - Show quoted text -


    1) The rules works when RATE is set to 10, while fails when RATE is
    set to 50.
    Bellow is the detailed discription.
    A) set RATE to 10.
    Create 3 flows, 9M(High priority),2M(Medium priority), 2M(Low
    priority).
    After traffic control, 9M(H), 1M(M), 0M(L). This is just I want.
    #tc -s qdis ls dev eth0
    qdisc htb 1: r2q 10 default 0 direct_packets_stat 243
    Sent 3851532 bytes 3867 pkt (dropped 748, overlimits 4331 requeues 0)
    rate 0bit 0pps backlog 0b 137p requeues 0
    qdisc prio 2: parent 1:1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1
    1 1
    Sent 3609504 bytes 3624 pkt (dropped 748, overlimits 0 requeues 0)
    rate 0bit 0pps backlog 0b 137p requeues 0
    qdisc bfifo 10: parent 2:1 limit 64Kb
    Sent 2970072 bytes 2982 pkt (dropped 0, overlimits 0 requeues 0)
    rate 0bit 0pps backlog 8964b 9p requeues 0
    qdisc bfifo 20: parent 2:2 limit 64Kb
    Sent 565728 bytes 568 pkt (dropped 166, overlimits 0 requeues 0)
    rate 0bit 0pps backlog 62748b 63p requeues 0
    qdisc bfifo 30: parent 2:3 limit 64Kb
    Sent 73704 bytes 74 pkt (dropped 582, overlimits 0 requeues 0)
    rate 0bit 0pps backlog 64740b 65p requeues 0

    B)
    set RATE to 50.
    Create 3 flows, 40M(High priority),20M(Medium priority), 20M(Low
    priority).
    After traffic control, 35M(H), 1M(M), 0M(L). It does not meet my
    request.
    My anticipate is 40M(H), 20M(M), 0M(L).

    #tc -s qdis ls dev eth0
    qdisc htb 1: r2q 10 default 0 direct_packets_stat 258
    Sent 874083006 bytes 877602 pkt (dropped 975747, overlimits 1375210
    requeues 335962)
    rate 0bit 0pps backlog 0b 122p requeues 335962
    qdisc prio 2: parent 1:1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1
    1 1
    Sent 873834624 bytes 877344 pkt (dropped 975747, overlimits 0
    requeues 335962)
    rate 0bit 0pps backlog 0b 122p requeues 335962
    qdisc bfifo 10: parent 2:1 limit 64Kb
    Sent 842449668 bytes 845833 pkt (dropped 89972, overlimits 0 requeues
    309487)
    rate 0bit 0pps backlog 0b 0p requeues 309487
    qdisc bfifo 20: parent 2:2 limit 64Kb
    Sent 24020532 bytes 24117 pkt (dropped 434613, overlimits 0 requeues
    26474)
    rate 0bit 0pps backlog 56772b 57p requeues 26474
    qdisc bfifo 30: parent 2:3 limit 64Kb
    Sent 7364424 bytes 7394 pkt (dropped 451162, overlimits 0 requeues 1)
    rate 0bit 0pps backlog 64740b 65p requeues 1

    2) The rules are set according to
    http://linux.derkeiler.com/Newsgroup...-12/0624..html.
    3) I have not upgraded the kernel and iproute2.
    Thanks.

  4. Re: Problem in traffic control with HTB + PRIO

    banyeer wrote:
    > On Nov 7, 12:15 pm, "D. Stussy" wrote:
    >> "banyeer" wrote in message
    >>
    >> news:e402d9c4-ec35-4d7f-aef9-7601e8699912@x16g2000prn.googlegroups.com...> Hi Everyone,
    >>> I met some problems in traffic control with TC.
    >>> My purpose:
    >>> 1. Restrict the overall bandwidth.
    >>> 2. Starve the flows in the lower bands.


    A bit mean but if that's what you need.
    You can do prio with htb classes and let the lower classes have a small
    amount of bandwidth.

    >>> Based on the following link, I develope the script bellow.

    >> http://linux.derkeiler.com/Newsgroup...working/2004-1...


    2004 - One thing I've learned since then -


    >>> $TC filter add dev eth0 parent 1:0 prio 0 protocol ip handle 1 fw
    >>> flowid 1:1


    On filters prio 0 isn't the highest. It doesn't really matter much in
    this case but you should use 1 2 3. For htb classes prio 0 is the top.

    64k for buffers is also quite small if you are going to increase the
    bandwidth from the 5mbit in that example.


    > set RATE to 50.
    > Create 3 flows, 40M(High priority),20M(Medium priority), 20M(Low
    > priority).
    > After traffic control, 35M(H), 1M(M), 0M(L). It does not meet my
    > request.
    > My anticipate is 40M(H), 20M(M), 0M(L).


    40 10 0

    >
    > #tc -s qdis ls dev eth0
    > qdisc htb 1: r2q 10 default 0 direct_packets_stat 258
    > Sent 874083006 bytes 877602 pkt (dropped 975747, overlimits 1375210
    > requeues 335962)
    > rate 0bit 0pps backlog 0b 122p requeues 335962
    > qdisc prio 2: parent 1:1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1
    > 1 1
    > Sent 873834624 bytes 877344 pkt (dropped 975747, overlimits 0
    > requeues 335962)


    Hmm requeues, on a 400MHz mips box maybe 40mbit of small packets, 78k
    pps is just too much for it to handle.

    Generally when testing with small packets you also need to consider how
    accurate htb/qos is. On eth htb sees packets as ip length + 14 and there
    is also up to a 7 byte error in the rate lookup tables. In the past the
    error fell on the over rate side, more recently it's been changed to
    fall on the under rate.

  5. Re: Problem in traffic control with HTB + PRIO

    On Nov 8, 12:57*am, Andy Furniss wrote:
    > banyeer wrote:
    > > On Nov 7, 12:15 pm, "D. Stussy" wrote:
    > >> "banyeer" wrote in message

    >
    > >>news:e402d9c4-ec35-4d7f-aef9-7601e8699912@x16g2000prn.googlegroups.com....> Hi Everyone,
    > >>> I met some problems in traffic control with TC.
    > >>> My purpose:
    > >>> 1. Restrict the overall bandwidth.
    > >>> 2. Starve the flows in the lower bands.

    >
    > A bit mean but if that's what you need.
    > You can do prio with htb classes and let the lower classes have a small
    > amount of bandwidth.

    But how to restrict the overall bandwith if using prio with htb
    classes?
    >
    > >>> Based on the following link, I develope the script bellow.
    > >>http://linux.derkeiler.com/Newsgroup...working/2004-1....

    >
    > 2004 - One thing I've learned since then -
    >
    > >>> $TC filter add dev eth0 parent 1:0 prio 0 protocol ip handle 1 fw
    > >>> flowid 1:1

    >
    > On filters prio 0 isn't the highest. It doesn't really matter much in
    > this case but you should use 1 2 3. For htb classes prio 0 is the top.
    >
    > 64k for buffers is also quite small if you are going to increase the
    > bandwidth from the 5mbit in that example.

    I try to increase it but it seems no effect.
    >
    > > set RATE to 50.
    > > * * Create 3 flows, 40M(High priority),20M(Medium priority), 20M(Low
    > > priority).
    > > * * After traffic control, 35M(H), 1M(M), 0M(L). *It does not meet my
    > > request.
    > > * * My anticipate is *40M(H), 20M(M), 0M(L).

    >
    > 40 10 0

    Yes, I made a mistake here.

    >
    >
    >
    > > #tc -s qdis ls dev eth0
    > > qdisc htb 1: r2q 10 default 0 direct_packets_stat 258
    > > *Sent 874083006 bytes 877602 pkt (dropped 975747, overlimits 1375210
    > > requeues 335962)
    > > *rate 0bit 0pps backlog 0b 122p requeues 335962
    > > qdisc prio 2: parent 1:1 bands 3 priomap *1 2 2 2 1 2 0 0 1 1 1 1 1 1
    > > 1 1
    > > *Sent 873834624 bytes 877344 pkt (dropped 975747, overlimits 0
    > > requeues 335962)

    >
    > Hmm requeues, on a 400MHz mips box maybe 40mbit of small packets, 78k
    > pps is just too much for it to handle.
    >
    > Generally when testing with small packets you also need to consider how
    > accurate htb/qos is. On eth htb sees packets as ip length + 14 and there
    > is also up to a 7 byte error in the rate lookup tables. In the past the
    > error fell on the over rate side, more recently it's been changed to
    > fall on the under rate.


    I try with large packets instead of small packets. In the 50M case,
    without qos, I can get 40M(H), 20M(M), 20M(L).
    I also notice that requeues is very large. What does this mean? Are
    there any parameters to set to reduce the value?
    Thanks a lot.


  6. Re: Problem in traffic control with HTB + PRIO

    banyeer wrote:

    > But how to restrict the overall bandwith if using prio with htb
    > classes?


    Something like -

    $TC qdisc add dev eth0 root handle 1: htb
    $TC class add dev eth0 parent 1: classid 1:1 htb rate 50mbit

    $TC class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 47mbit
    ceil 50mbit

    $TC class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 2mbit
    ceil 50mbit

    $TC class add dev eth0 parent 1:1 classid 1:30 htb prio 2 rate 1mbit
    ceil 50mbit

    Each class will get its rate always, but will be able to borrow spare
    bandwidth from the higher priority classes.


    > I try with large packets instead of small packets. In the 50M case,
    > without qos, I can get 40M(H), 20M(M), 20M(L).
    > I also notice that requeues is very large. What does this mean? Are
    > there any parameters to set to reduce the value?
    > Thanks a lot.


    Hmm, I tested on a 2.6.19 and for me it works and I get no requeues.

    I think the reason could be to do with Hz and the way htb works.

    If you do

    tc -s -d class ls dev eth0

    You will see burst and cburst and quantum values. These are calculated
    from the rate/ceil and assume Hz 100. Quantum is to do with sharing and
    I don't think will matter in this case.

    You can specify all these as cvars rather than use the defaults.

    If your mips kernel really is Hz 100 then that could be a problem as
    burst bytes x Hz needs to be big enough for the bandwidth you need.

    I suspect why it works for me is that on my nic/nic driver there is a
    big enough buffer to absorb the burst. Perhaps yours is smaller (maybe
    tweakable?).

    If Hz can be set to/is 1000 on mips then just specify burst/cburst 10x
    smaller.

    If you can't get it to work with htb it is possible to attach prio to
    tbf or hfsc I suppose. Without testing I am not sure if the Hz/burst
    issue arises with these.



  7. Re: Problem in traffic control with HTB + PRIO

    On Nov 11, 5:49*am, Andy Furniss wrote:
    > banyeer wrote:
    > > But how to restrict the overall bandwith if using prio with htb
    > > classes?

    >
    > Something like -
    >
    > $TC qdisc add dev eth0 root handle 1: htb
    > $TC class add dev eth0 parent 1: classid 1:1 htb rate 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 47mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 2mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:30 htb prio 2 rate 1mbit
    > ceil 50mbit

    In this way low priority traffic always consumes some bandwidth,
    that's not meet my request.
    I'd like something like:
    $TC qdisc add dev eth0 root handle 1: htb
    $TC class add dev eth0 parent 1: classid 1:1 htb rate 50mbit
    $TC class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 50mbit
    ceil 50mbit
    $TC class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 0mbit
    ceil 50mbit
    $TC class add dev eth0 parent 1:1 classid 1:30 htb prio 2 rate 0mbit
    ceil 50mbit

    But the 0mbit parameter does not work.


    >
    > Each class will get its rate always, but will be able to borrow spare
    > bandwidth from the higher priority classes.
    >
    > > I try with large packets instead of small packets. In the 50M case,
    > > without qos, I can get 40M(H), *20M(M), 20M(L).
    > > I also notice that requeues is very large. What does this mean? Are
    > > there any parameters to set to reduce the value?
    > > Thanks a lot.

    >
    > Hmm, I tested on a 2.6.19 and for me it works and I get no requeues.
    >
    > I think the reason could be to do with Hz and the way htb works.
    >
    > If you do
    >
    > tc -s -d class ls dev eth0
    >
    > You will see burst and cburst and quantum values. These are calculated
    > from the rate/ceil and assume Hz 100. Quantum is to do with sharing and
    > I don't think will matter in this case.
    >
    > You can specify all these as cvars rather than use the defaults.

    I've adjusted the cvars, but till now it does not work.

    >
    > If your mips kernel really is Hz 100 then that could be a problem as
    > burst bytes x Hz needs to be big enough for the bandwidth you need.

    HZ is 100 for my kernel, and it's better not to change it.
    >
    > I suspect why it works for me is that on my nic/nic driver there is a
    > big enough buffer to absorb the burst. Perhaps yours is smaller (maybe
    > tweakable?).

    I'll try to check the buffer.
    >
    > If Hz can be set to/is 1000 on mips then just specify burst/cburst 10x
    > smaller.
    >
    > If you can't get it to work with htb it is possible to attach prio to
    > tbf or hfsc I suppose. Without testing I am not sure if the Hz/burst
    > issue arises with these.

    It seems that pure htb , or attaching prio to tbf, can not meet the
    request.I'll try hfsc then.
    Thanks!


  8. Re: Problem in traffic control with HTB + PRIO

    On Nov 11, 5:49*am, Andy Furniss wrote:
    > banyeer wrote:
    > > But how to restrict the overall bandwith if using prio with htb
    > > classes?

    >
    > Something like -
    >
    > $TC qdisc add dev eth0 root handle 1: htb
    > $TC class add dev eth0 parent 1: classid 1:1 htb rate 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 47mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 2mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:30 htb prio 2 rate 1mbit
    > ceil 50mbit
    >
    > Each class will get its rate always, but will be able to borrow spare
    > bandwidth from the higher priority classes.
    >
    > > I try with large packets instead of small packets. In the 50M case,
    > > without qos, I can get 40M(H), *20M(M), 20M(L).
    > > I also notice that requeues is very large. What does this mean? Are
    > > there any parameters to set to reduce the value?
    > > Thanks a lot.

    >
    > Hmm, I tested on a 2.6.19 and for me it works and I get no requeues.
    >
    > I think the reason could be to do with Hz and the way htb works.
    >
    > If you do
    >
    > tc -s -d class ls dev eth0


    Here's the output for the 50M case:
    #tc -s -d class ls dev eth0
    class htb 1:1 root leaf 2: prio 0 quantum 200000 rate 50000Kbit ceil
    50000Kbit burst 2000Kb/8 mpu 0b overhead 0b cburst 64100b/8 mpu
    0b overhead 0b level 0
    Sent 2710334124 bytes 2721219 pkt (dropped 3548985, overlimits 0
    requeues 0)
    rate 50263Kbit 6308pps backlog 0b 11142p requeues 0
    lended: 3618022 borrowed: 0 giants: 0
    tokens: 314491 ctokens: -2933

    class prio 2:1 parent 2: leaf 10:

    class prio 2:2 parent 2: leaf 20:

    class prio 2:3 parent 2: leaf 30:


    I'm puzzled why it shows " rate 50263Kbit " , while in smartbits I
    only get 37M.


    >
    > You will see burst and cburst and quantum values. These are calculated
    > from the rate/ceil and assume Hz 100. Quantum is to do with sharing and
    > I don't think will matter in this case.
    >
    > You can specify all these as cvars rather than use the defaults.
    >
    > If your mips kernel really is Hz 100 then that could be a problem as
    > burst bytes x Hz needs to be big enough for the bandwidth you need.
    >
    > I suspect why it works for me is that on my nic/nic driver there is a
    > big enough buffer to absorb the burst. Perhaps yours is smaller (maybe
    > tweakable?).
    >
    > If Hz can be set to/is 1000 on mips then just specify burst/cburst 10x
    > smaller.
    >
    > If you can't get it to work with htb it is possible to attach prio to
    > tbf or hfsc I suppose. Without testing I am not sure if the Hz/burst
    > issue arises with these.



  9. Re: Problem in traffic control with HTB + PRIO

    On Nov 11, 5:49*am, Andy Furniss wrote:
    > banyeer wrote:
    > > But how to restrict the overall bandwith if using prio with htb
    > > classes?

    >
    > Something like -
    >
    > $TC qdisc add dev eth0 root handle 1: htb
    > $TC class add dev eth0 parent 1: classid 1:1 htb rate 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:10 htb prio 0 rate 47mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:20 htb prio 1 rate 2mbit
    > ceil 50mbit
    >
    > $TC class add dev eth0 parent 1:1 classid 1:30 htb prio 2 rate 1mbit
    > ceil 50mbit
    >
    > Each class will get its rate always, but will be able to borrow spare
    > bandwidth from the higher priority classes.
    >
    > > I try with large packets instead of small packets. In the 50M case,
    > > without qos, I can get 40M(H), *20M(M), 20M(L).
    > > I also notice that requeues is very large. What does this mean? Are
    > > there any parameters to set to reduce the value?
    > > Thanks a lot.

    >
    > Hmm, I tested on a 2.6.19 and for me it works and I get no requeues.
    >
    > I think the reason could be to do with Hz and the way htb works.
    >
    > If you do
    >
    > tc -s -d class ls dev eth0
    >
    > You will see burst and cburst and quantum values. These are calculated
    > from the rate/ceil and assume Hz 100. Quantum is to do with sharing and
    > I don't think will matter in this case.
    >
    > You can specify all these as cvars rather than use the defaults.
    >
    > If your mips kernel really is Hz 100 then that could be a problem as
    > burst bytes x Hz needs to be big enough for the bandwidth you need.
    >
    > I suspect why it works for me is that on my nic/nic driver there is a
    > big enough buffer to absorb the burst. Perhaps yours is smaller (maybe
    > tweakable?).
    >
    > If Hz can be set to/is 1000 on mips then just specify burst/cburst 10x
    > smaller.
    >
    > If you can't get it to work with htb it is possible to attach prio to
    > tbf or hfsc I suppose. Without testing I am not sure if the Hz/burst
    > issue arises with these.


    Another problem , different packet size causes different result:
    50M case:
    40(H), 20(M), 20(L), packet size:500byte--> 30(H), 15(M), 3(L).
    40(H), 20(M), 20(L), packet size:1000byte--> 39(H), 0.5(M), 3(L).

    It's so strange. Is it possibe that some variable for HTB overflows?



  10. Re: Problem in traffic control with HTB + PRIO

    banyeer wrote:

    > It seems that pure htb , or attaching prio to tbf, can not meet the
    > request.I'll try hfsc then.


    You will need to set up a default class for hfsc (or filter arp to a
    class) as it behaves opposite to htb in that it drops unclassified traffic.

    What does top say about CPU usage when running these tests?

  11. Re: Problem in traffic control with HTB + PRIO

    banyeer wrote:

    > Here's the output for the 50M case:
    > #tc -s -d class ls dev eth0
    > class htb 1:1 root leaf 2: prio 0 quantum 200000 rate 50000Kbit ceil
    > 50000Kbit burst 2000Kb/8 mpu 0b overhead 0b cburst 64100b/8 mpu
    > 0b overhead 0b level 0


    That's strange, unless you specified burst. I get the same burst as cburst.

    > Sent 2710334124 bytes 2721219 pkt (dropped 3548985, overlimits 0
    > requeues 0)
    > rate 50263Kbit 6308pps backlog 0b 11142p requeues 0
    > lended: 3618022 borrowed: 0 giants: 0
    > tokens: 314491 ctokens: -2933
    >
    > class prio 2:1 parent 2: leaf 10:
    >
    > class prio 2:2 parent 2: leaf 20:
    >
    > class prio 2:3 parent 2: leaf 30:
    >
    >
    > I'm puzzled why it shows " rate 50263Kbit " , while in smartbits I
    > only get 37M.


    htb uses quite a long average - but smartbits could be wrong.

    The drops/ huge backlog show you are hitting it hard (which wouldn't
    happen so much with normal tcp)

    You could add

    .... estimator 1 4 ...

    to the bfifos and see what the rates on those say with tc -s qdisc ls ....

    If there are still discrepancies then you could always do a tcpdump on
    the sink box to see what it's really getting.

+ Reply to Thread