multiple classes with target CLASSIFY - Networking

This is a discussion on multiple classes with target CLASSIFY - Networking ; Hi, I have a router where I use a CLASSIFY target on FORWARD chain to do QoS. Now I need a packet to be classified to 2 classes. The following: iptables -t mangle -A FORWARD -s $SOME_IP -j CLASSIFY --set-class ...

+ Reply to Thread
Results 1 to 8 of 8

Thread: multiple classes with target CLASSIFY

  1. multiple classes with target CLASSIFY

    Hi,

    I have a router where I use a CLASSIFY target on FORWARD chain to do
    QoS.
    Now I need a packet to be classified to 2 classes. The following:
    iptables -t mangle -A FORWARD -s $SOME_IP -j CLASSIFY --set-class
    1:$FIRST
    iptables -t mangle -A FORWARD -s $SOME_IP -j CLASSIFY --set-class
    1:$SECOND
    doesn't work - only the last rule takes effect.

    Intuitively I wrote two classes seperated with a comma (no space):
    iptables -t mangle -A FORWARD -s $SOME_IP -j CLASSIFY --set-class
    1:$FIRST,1:$SECOND
    no errors... but only the 1:$FIRST class receives packets.

    Couldn't find anything about it but it seems to be wrong, doesn't it?
    Could someone test this rule? (should be easy - just replace an
    existing rule with two classes and see if the second class works)

    --
    Tomek Gruca

  2. Re: multiple classes with target CLASSIFY

    Tomek Gruca wrote:
    > Hi,
    >
    > I have a router where I use a CLASSIFY target on FORWARD chain to do
    > QoS.
    > Now I need a packet to be classified to 2 classes. The following:


    I can't see that this makes sense - you can't have a packet in two
    queues at once.

    What are you trying to do that needs different classes?

    Andy.

  3. Re: multiple classes with target CLASSIFY

    On 9 Lut, 21:41, Andy Furniss wrote:
    > What are you trying to do that needs different classes?


    I have 2 ISPs and per port load balancing. I want my users to have
    specified downstream (let's say 2Mb/s) no matter what ports they use.
    So I set a class for each user where root class has unlimited bitrate.
    But also I need queues on both links to avoid uncontrolled overload.

    One solution is to use IMQ but I don't want to patch the kernel.

    Another is to CLASSIFY in FORWARD and then in POSTROUTING (and it
    works) but I need the "-i $DEV" match...

    --
    Tomek

  4. Re: multiple classes with target CLASSIFY

    On 10 Lut, 02:00, Tomek Gruca wrote:
    > Another is to CLASSIFY in FORWARD and then in POSTROUTING (and it
    > works) but I need the "-i $DEV" match...


    Sorry, in POSTROUTING it doesn't work... it overrides FORWARD
    CLASSIFY.

    "you can't have a packet in two queues at once"
    you mean it's technically impossible or you don't see a reason for
    this?

    --
    Tomek

  5. Re: multiple classes with target CLASSIFY

    Tomek Gruca wrote:

    > "you can't have a packet in two queues at once"
    > you mean it's technically impossible or you don't see a reason for
    > this?


    I mean technically impossible.

    You can using imq/ifb and set things up so the packet goes through more
    than one queue, though.

    IFB has been in 2.6s for quite a time now as a replacement for imq. I
    don't know if all distros enable it - try modprobe ifb.

    It's not quite the same as imq as on ingress it sees the traffic before
    netfilter whereas imq is after prerouting/de-nat. You can use it on
    egress aswell as ingress.

    Depending on exactly how complicated you need your classification/qos
    policied to be, one option I would consider is to just use policers.

    Shaping from the wrong end of a wan is never perfect and you need to
    sacrifice some bandwidth whether you police or shape. The nice thing
    about policers is that they don't add latency.

    You could police the wans at say 80% of their link speed with one
    policer for each and have a 2meg per user policer on the lan interface.

    It all really depends on exactly what you are trying to acheive, what
    bandwidth you have, how many users, what they do etc etc

    Andy.


  6. Re: multiple classes with target CLASSIFY

    On 10 Lut, 14:06, Andy Furniss wrote:
    > I mean technically impossible.
    >
    > You can using imq/ifb and set things up so the packet goes through more
    > than one queue, though.
    >
    > IFB has been in 2.6s for quite a time now as a replacement for imq. I
    > don't know if all distros enable it - try modprobe ifb.
    >
    > It's not quite the same as imq as on ingress it sees the traffic before
    > netfilter whereas imq is after prerouting/de-nat. You can use it on
    > egress aswell as ingress.
    >


    Thanks,

    I have IFB as kernel module (modprobe ifb works), bo I'm not sure how
    to test it..
    I have two classes:
    tc class add dev eth0 parent 1:0 classid 1:1 htb rate 100kbit
    tc class add dev ifb0 parent 2:0 classid 2:1 htb rate 100kbit

    how could I make a packet to be classified to both of them?
    I'm not sure if I understand actions correctly (probably not), can't
    find any tutorial:
    Let's say the packet is maked by iptables with mark=1001 in FORWARD
    tc filter add dev eth0 protocol ip handle 1001 fw flowid 1:1 \
    action ipt -j MARK --set-mark 2001 \
    action mirred egress redirect dev ifb0
    tc filter add dev ifb0 protocol ip handle 2001 fw flowid 2:1
    Does it make sense? The first rule classifies it to 1:1 and passes on?

    > Depending on exactly how complicated you need your classification/qos
    > policied to be, one option I would consider is to just use policers.
    >


    Well... I've always used HTB for fair per user distribution of
    bandwidth..
    Don't know anything about policers but gonna read sth

    > Shaping from the wrong end of a wan is never perfect and you need to
    > sacrifice some bandwidth whether you police or shape. The nice thing
    > about policers is that they don't add latency.
    >


    Could you explain a bit these 3 sentences ? Each seems to be very
    interesting..

    > You could police the wans at say 80% of their link speed with one
    > policer for each and have a 2meg per user policer on the lan interface.
    >
    > It all really depends on exactly what you are trying to acheive, what
    > bandwidth you have, how many users, what they do etc etc
    >


    I have one 15/0.8M (for asymmeric services: http,pop3 etc) and 6/6M
    for the rest.
    150 users, bandwidth per user varies.
    I'd need to be able to specify CIR for a user on each link..

    --
    Tomek

  7. Re: multiple classes with target CLASSIFY

    Tomek Gruca wrote:

    I will try and test/post some real rules soon, but this is a bit of an
    untypical setup that I haven't done before, so I'll be in
    trial/error/test/repeat mode and I won't have time for a couple of days.

    > I have IFB as kernel module (modprobe ifb works), bo I'm not sure how
    > to test it..
    > I have two classes:
    > tc class add dev eth0 parent 1:0 classid 1:1 htb rate 100kbit
    > tc class add dev ifb0 parent 2:0 classid 2:1 htb rate 100kbit
    >
    > how could I make a packet to be classified to both of them?
    > I'm not sure if I understand actions correctly (probably not), can't
    > find any tutorial:
    > Let's say the packet is maked by iptables with mark=1001 in FORWARD
    > tc filter add dev eth0 protocol ip handle 1001 fw flowid 1:1 \
    > action ipt -j MARK --set-mark 2001 \
    > action mirred egress redirect dev ifb0
    > tc filter add dev ifb0 protocol ip handle 2001 fw flowid 2:1
    > Does it make sense? The first rule classifies it to 1:1 and passes on?


    I am guessing that you are doing nat on the wan interfaces, if so this
    complicates things somewhat. If you are not doing nat then the easy
    solution would be to direct the packets to ifb as soon as they arrive at
    the wan nic(s). If you are doing nat and have the need to seperate users
    then this will not work because you won't be able to see the local IP
    addresses or use any iptables rules to aid your classification.

    So assuming this, you are going to have to try and do everything as the
    packets leave you lan facing interface - this is a bit harder as in
    order to prevent loops, packets redirected to ifb when about to leave an
    interface can't directly be reclassified by another qdisc on that
    interface.

    What should work is to classify them before the redirect and then when
    they return from ifb I beleive any previous classification will be
    reinstated. So if you both classify and mark it may be possible to use
    the marks on the two ifbs for the link specific qdiscs and the
    classification for the third qdisc on the real lan facing interface.

    >
    >> Depending on exactly how complicated you need your classification/qos
    >> policied to be, one option I would consider is to just use policers.
    >>

    >
    > Well... I've always used HTB for fair per user distribution of
    > bandwidth..
    > Don't know anything about policers but gonna read sth


    Policers don't queue packets they just drop or let pass depending on
    whether the configured rate is exceeded (oversimplification). To make
    this work they keep a virtual buffer whose length is set with the burst
    parameter.

    They do have a little used feature called shared meters which allows for
    more complicated setups. Personally I think shared meter is a bit of a
    misnomer shared policer would probably be better as they don't allow
    different actions for different traffic that hits them. You may be able
    to do a simple CIR with them, but on 2.6 kernels when used on ingress
    they will see the traffic before it's been de-natted. On 2.4s they do
    hook after de-nat. You can use them on egress aswell as ingress. So you
    may be able to use them for your setup with ifb.


    >
    >> Shaping from the wrong end of a wan is never perfect and you need to
    >> sacrifice some bandwidth whether you police or shape. The nice thing
    >> about policers is that they don't add latency.


    You have 15 & 6mbit that's not too bad. The less bandwidth you have the
    harder things get when shaping from the wrong end of the bottleneck.

    Take your 6/6 link as an example. For traffic headed from you to the wan
    you have near enough total control - you put a qdisc on the interface
    and set it to 6mbit and you have on your box any queue that builds up.
    Your classification can do fairness and allow any interactive packets to
    skip past the lower prio packets so they don't get delayed.

    So what about traffic coming from the wan to your net - you can catch it
    somewhere, but what rate do you set on the qdisc. it's never going to
    come in faster than 6mbit so if you set 6mbit as the rate nothing will
    happen, you will never build up a queue to be able to do fairness/prio
    for some.

    Exactly what happens will depend on what your ISP does - there may be a
    big tail drop buffer that fills up so latency for all traffic suffers,
    or maybe they police rate so latency is not so bad but you will get a
    bit of loss and will be unable to do fairness. The way tcp works means
    it will try and fill any buffer at the far end of your wan.

    In practice what you do is have to compromise by setting a rate somewhat
    lower than link speed and understanding that say at 80% the qdisc is
    only going to fill up slowly as in the time 10 packets come in at 6meg
    you will have dequeued 8. The way tcp slowstart works means that until
    you get loss it will ramp up speed - so there is an incentive to use
    shortish queues, but if you set them too short you hurt throughput, and
    if you have lots of classes you can't be too mean to each class yet if
    they are all active the length may add up to a queue that's a bit too
    long. ESFQ can be handy here, but requires patching currently - sfq is
    currently being enhanced aswell, but it's not mainstream yet.

    Policers have another advantage for ingress - if you buffer then drop
    it's a tail drop, so the tcp sender doesn't know to backoff until the
    packets ahead in the queue have been dequeued - if in slowstart the
    speed will still be rising and your ISP buffer still filling.

    All of the above is a bit over simplistic but should give an idea of the
    problems. As I said the lower the bandwidth the worse it is and you have
    a reasonable amount, so all is not lost - even at lower rates it's
    alot better than doing nothing, but it does mean sacrificing some
    bandwidth and dropping packets that have already come in on your line.

    Andy.


  8. Re: multiple classes with target CLASSIFY

    Andy Furniss wrote:

    > I will try and test/post some real rules soon, but this is a bit of an
    > untypical setup that I haven't done before, so I'll be in
    > trial/error/test/repeat mode and I won't have time for a couple of days.


    Well I tried and it seems to work - I only tested for classification and
    I did it on my desktop so you would have to change things for a your
    router.

    I couldn't get -j CLASSIFY to work with ifb so you'll have to use marks
    or tc filters direct or a combination. I used marks and masks just as an
    example of using marks in a more complicated than normal way. I also
    show one way to control queue length - you can use bfifos or sfq with
    limit as well.

    If I had to do this for real I would consider hashing or policers
    instead of htb - but then I don't know exactly your requirements.

    I just tested by adding the two addresses to my eth and used
    my gateway 192.168.0.1 running netserver as the target

    ping -I 192.168.0.101 192.168.0.1
    ping -I 192.168.0.102 192.168.0.1

    netperf -H 192.168.0.1 -l 2 -- -L 192.168.0.101
    netperf -H 192.168.0.1 -l 2 -- -L 192.168.0.102

    and looked at the counters with

    tc -s class ls dev eth0/ifb0/ifb1

    Looked OK to me but don't take my word that it works.

    FWIW if you ever use 100s of netperfs in a script to generate traffic
    you'll need to add all the addresses you use and 0.0.0.0 to /etc/hosts
    or it will try and do loads of DNS lookups which will mess things up
    (well it does for me).

    The script - bound to be lame in many ways - I would be messing around
    for days/weeks if it were for real, but it should show you how to use
    ifb on egress at least.

    I hope your distro enabled the u32 mark match!
    Change the paths & mend any wrapping.

    Andy.

    #set -x
    IP=/usr/sbin/ip
    TC=/usr/sbin/tc
    IPT=/usr/local/sbin/iptables

    $TC qdisc del dev ifb0 root &>/dev/null
    $TC qdisc del dev ifb1 root &>/dev/null
    $TC qdisc del dev eth0 root &>/dev/null
    $IP link set ifb0 down &>/dev/null
    $IP link set ifb1 down &>/dev/null
    modprobe -r ifb &>/dev/null
    $IPT -F OUTPUT -t mangle


    if [ "$1" = "stop" ]
    then
    exit
    fi

    modprobe ifb
    $IP link set ifb0 up
    $IP link set ifb1 up

    # this is just a way to test on my host with a single eth
    # icmp goes to what would be wan 2 all other wan 1
    # my fictional 2 users then ORed so the wan marking remains
    # note the length of the "u32 match mark" masks mean the
    # filters on the ifbs don't see the wan marks

    $IPT -A OUTPUT -t mangle -o eth0 -j MARK --set-mark 0x10000
    $IPT -A OUTPUT -t mangle -o eth0 -p icmp -j MARK --set-mark 0x20000
    $IPT -A OUTPUT -t mangle -o eth0 -s 192.168.0.101 -j MARK --or-mark 0x1
    $IPT -A OUTPUT -t mangle -o eth0 -s 192.168.0.102 -j MARK --or-mark 0x2

    # traffaic from 6mbit wan
    # you would have lots of users here
    # so rates will be 5mbit/num users
    # the limit param on the pfifo controls queue length

    $TC qdisc add dev ifb0 root handle 1: htb
    $TC class add dev ifb0 parent 1: classid 1:ffff htb rate 5mbit

    # user 1
    $TC class add dev ifb0 parent 1:ffff classid 1:1 htb rate 2.5mbit ceil 5mbit
    $TC qdisc add dev ifb0 parent 1:1 pfifo limit 10
    $TC filter add dev ifb0 parent 1: protocol ip prio 1 \
    u32 match mark 1 0xffff flowid 1:1

    # user 2
    $TC class add dev ifb0 parent 1:ffff classid 1:2 htb rate 2.5mbit ceil 5mbit
    $TC qdisc add dev ifb0 parent 1:2 pfifo limit 10
    $TC filter add dev ifb0 parent 1: protocol ip prio 1 \
    u32 match mark 2 0xffff flowid 1:2

    # traffic from 15mbit wan
    # it's just a test so I didn't bother doing it properly
    # as above - just wanted the counters to see
    # where the traffic went

    $TC qdisc add dev ifb1 root handle 1: htb

    $TC class add dev ifb1 parent 1: classid 1:1 htb rate 100kbit
    $TC filter add dev ifb1 parent 1: protocol ip prio 1 \
    u32 match mark 1 0xffff flowid 1:1

    $TC class add dev ifb1 parent 1: classid 1:2 htb rate 100kbit
    $TC filter add dev ifb1 parent 1: protocol ip prio 1 \
    u32 match mark 2 0xffff flowid 1:2


    # 2mbit per user no sharing
    # classids are set before redirect to ifbs they get saved
    # and reinstated when the packets return - they do not get seen
    # by the queue on the ifb

    $TC qdisc add dev eth0 root handle 1: htb
    $TC class add dev eth0 parent 1: classid 1:1 htb rate 2mbit
    $TC class add dev eth0 parent 1: classid 1:2 htb rate 2mbit

    # user 1
    $TC filter add dev eth0 parent 1:0 \
    protocol ip prio 1 u32 match mark 0x10001 0xfffff classid 1:1 \
    action mirred egress redirect dev ifb0
    $TC filter add dev eth0 parent 1:0 \
    protocol ip prio 2 u32 match mark 0x20001 0xfffff classid 1:1 \
    action mirred egress redirect dev ifb1

    #user 2
    $TC filter add dev eth0 parent 1:0 \
    protocol ip prio 1 u32 match mark 0x10002 0xfffff classid 1:2 \
    action mirred egress redirect dev ifb0
    $TC filter add dev eth0 parent 1:0 \
    protocol ip prio 2 u32 match mark 0x20002 0xfffff classid 1:2 \
    action mirred egress redirect dev ifb1


+ Reply to Thread