PF inadequacy: queue download - BSD

This is a discussion on PF inadequacy: queue download - BSD ; Why can't you queue download traffic on an interface? The reason openbsd.org's FAQ gives is: "Note that queueing is only useful for packets in the outbound direction. Once a packet arrives on an interface in the inbound direction it's already ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: PF inadequacy: queue download

  1. PF inadequacy: queue download

    Why can't you queue download traffic on an interface? The reason
    openbsd.org's FAQ gives is:

    "Note that queueing is only useful for packets in the outbound
    direction. Once a packet arrives on an interface in the inbound
    direction it's already too late to queue it -- it's already consumed
    network bandwidth to get to the interface that just received it. The
    only solution is to enable queueing on the adjacent router or, if the
    host that received the packet is acting as a router, to enable queueing
    on the internal interface where packets exit the router."

    But this is wrong. It's not too late to queue it; by queueing it and
    dropping some packets of inbound traffic the sending host slows down
    the speed at which it sends.

    I'm using pf to do NAT on my box, and I can shape download traffic
    using the 'queueing on the internal interface' hack; so why can't I do
    it elegantly on one interface?
    Shaping NAT traffic downloads works fine with this hack, but I also run
    some services on the external interface. With downloads queued on the
    internal interface there's no way to queue the services' download
    traffic, which means an external service can hog up all the bandwidth
    and I can't do anything.


    I know this is possible because IPFW with dummynet doesn't have any
    problems. If everyone loves PF because of its elegance why can't it do
    something as simple as queue download traffic?

    Regards,
    Kestas


  2. Re: PF inadequacy: queue download

    * kestas.j.k@gmail.com [2006-04-29]:
    > Why can't you queue download traffic on an interface? The reason
    > openbsd.org's FAQ gives is:
    >
    > "Note that queueing is only useful for packets in the outbound
    > direction. Once a packet arrives on an interface in the inbound
    > direction it's already too late to queue it -- it's already consumed
    > network bandwidth to get to the interface that just received it. The
    > only solution is to enable queueing on the adjacent router or, if the
    > host that received the packet is acting as a router, to enable queueing
    > on the internal interface where packets exit the router."
    >
    > But this is wrong. It's not too late to queue it; by queueing it and
    > dropping some packets of inbound traffic the sending host slows down
    > the speed at which it sends.
    >

    Are you sure about that? Perhaps a well behaved sending host
    would, but what if it isn't? Also, if you're being DDOSd, will
    this even matter?

    > Shaping NAT traffic downloads works fine with this hack, but I also run
    > some services on the external interface. With downloads queued on the
    > internal interface there's no way to queue the services' download
    > traffic, which means an external service can hog up all the bandwidth
    > and I can't do anything.
    >

    I haven't heard of any firewall that successfully could. If you're
    being DDOSd, you're being DDOSd. No firewall with any special set
    of rules can improve your bandwidth in that case. If the pipe is
    filled, it's filled.
    >
    > I know this is possible because IPFW with dummynet doesn't have any
    > problems.


    I don't use IPFW, so I can't speak on its specific technical merits.
    Maybe you should ask the IPFW devs how they're able to perform this
    magic. Last I heard, the PF devs were technicians, engineers, and
    scientists, not a single magic-user among them. (IRL anyhow. :-))

    If everyone loves PF because of its elegance why can't it do
    > something as simple as queue download traffic?
    >

    I think that you _can_ configure PF to do this, but I believe that
    what the developers are trying to say is, `What's the point'? We're
    talking about trying to control traffic _before_ it hits your
    interface. Even if the remote sending host is well-behaved enough
    to slow down its sending rate, it still has to interact with PF
    before PF can decide whether to pass the packets, drop the packets,
    or tell the sending host to `bugger off'.

    So, you can queue the download traffic, but that really has a minor
    to no effect on traffic outside of your firewall. The queuing
    actually occurs on _your_ side of the external interface.

    My $0.02 CDN. Take it for what it's worth, or exchange it for
    better currency. :-)

    --
    W. Steven Schneider

  3. Re: PF inadequacy: queue download

    > I haven't heard of any firewall that successfully could. If you're
    > being DDOSd, you're being DDOSd. No firewall with any special set
    > of rules can improve your bandwidth in that case. If the pipe is
    > filled, it's filled.

    Yes, if you're being DDoSed then incoming traffic shaping won't do
    anything, but if you're using TCP streams from cooperative hosts you
    can shape incoming traffic very effectively; you drop packets, sender
    realises packets are getting lost, sender slows down sending packets.
    It works when you use the hack of queueing on the internal interface
    when you're using NAT, it clearly works, so why can't you do it on a
    single interface?

    > I don't use IPFW, so I can't speak on its specific technical merits.
    > Maybe you should ask the IPFW devs how they're able to perform this
    > magic. Last I heard, the PF devs were technicians, engineers, and
    > scientists, not a single magic-user among them. (IRL anyhow. :-))

    There's no magic about it :-P

    > I think that you _can_ configure PF to do this, but I believe that
    > what the developers are trying to say is, `What's the point'? We're
    > talking about trying to control traffic _before_ it hits your
    > interface. Even if the remote sending host is well-behaved enough
    > to slow down its sending rate, it still has to interact with PF
    > before PF can decide whether to pass the packets, drop the packets,
    > or tell the sending host to `bugger off'.


    > So, you can queue the download traffic, but that really has a minor
    > to no effect on traffic outside of your firewall. The queuing
    > actually occurs on _your_ side of the external interface.

    As I explained above shaping download traffic works great, but only
    using the two interface hack. People on other boards are telling me I
    should put another box beetween the LAN and the internet so I can use
    the two interface hack for all traffic, but this seems stupid; if it
    works over two interfaces so it can certainly work on one interface, so
    why doesn't it?

    Kesas


  4. Re: PF inadequacy: queue download

    * kestas.j.k@gmail.com [2006-04-29]:
    >> I haven't heard of any firewall that successfully could. If you're
    >> being DDOSd, you're being DDOSd. No firewall with any special set
    >> of rules can improve your bandwidth in that case. If the pipe is
    >> filled, it's filled.

    > Yes, if you're being DDoSed then incoming traffic shaping won't do
    > anything, but if you're using TCP streams from cooperative hosts you
    > can shape incoming traffic very effectively; you drop packets, sender
    > realises packets are getting lost, sender slows down sending packets.
    > It works when you use the hack of queueing on the internal interface
    > when you're using NAT, it clearly works, so why can't you do it on a
    > single interface?
    >

    Obviously, I misunderstood. It's still true that the queuing
    actually occurs on your side of the firewall, and that it has to be
    on the outbound traffic.

    Now I might still be wrong, but I think that you want to control
    your machine's ack rate. Have you looked at
    http://www.benzedrine.cx/ackpri.html? This guy has some ideas that
    might be applicable to your situation.

    You may also want to look at
    http://www.bgnett.no/~peter/pf/en/long-firewall.html.

    I've found both of these sites to be sources of useful PF info. :-)

    --
    W. Steven Schneider

  5. Re: PF inadequacy: queue download

    Thanks for the help, but I still don't think you understand. This isn't
    a limitation in my understanding of PF and ALTQ, this is a limitation
    of PF and ALTQ itself; you cannot shape traffic coming in on an
    interface. There's no good reason why you shouldn't be able to do this.
    I want to know how I can get this added to a todo list, everyone who
    has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this
    is an important feature which PF lacks, and should be implemented asap.

    Kestas


  6. Re: PF inadequacy: queue download

    kestas.j.k@gmail.com writes:

    > Thanks for the help, but I still don't think you understand. This isn't
    > a limitation in my understanding of PF and ALTQ, this is a limitation
    > of PF and ALTQ itself; you cannot shape traffic coming in on an
    > interface. There's no good reason why you shouldn't be able to do this.


    No, there is plenty of good reason to not do so. Spending extra
    processor cycles queuing (i.e. delaying) packets that have already
    been received, e.g. have already used network bandwidth, is not a
    good thing. What you are asking is "please make my network slower
    so well-behaved peers will throttle their long lived TCP connections"
    The cost of doing that will slow *everything else* down, too. It buys
    you nothing with UDP, ICMP, and mis-behaved TCP hosts (of which there
    are all too many). It exacerbates DOS scenarios. It makes things
    like popular web servers slower -- http requests are too short
    lived to get much benefit from TCP congestion control.

    > I want to know how I can get this added to a todo list, everyone who
    > has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this
    > is an important feature which PF lacks, and should be implemented asap.


    Since you appear to be the only person who is eager to slow down your
    network processing for minimal benefit (if any at all) you will probably
    have to do the work yourself.

    // marc

  7. Re: PF inadequacy: queue download

    Most hosts are well behaved, and UDP also throttles if the protocol is
    intelligent. The fact that this is useful is obvious because so many
    people do it on their gateway using two interfaces. I know it's useful
    because before implementing it one person downloading would hog up the
    whole network; with incoming traffic shaping the person hogging up the
    network immidiately gets throttled down and other queues can get a
    chance.

    Extra processor cycles? Downloading at full pace my Pentium II 350MHz
    isn't using 5% of it processor.

    Finally I'm not the only person who wants this. From the thread in
    bit.listserv.openbsd-pf:
    > It's a question of what's "useful". Not useful against
    > a malicious agent, but useful for traffic shaping regardless.


    > It surely has its caveats, but has its use, too. I'm stuck with ipfw on
    > one last machine, because I can't limit bulk TCP traffic with pf.


    > I can speak for myself - I can't afford both the hardware and the
    > electricity bill for a separate machine. Maybe downstream limiting isn't
    > very robust, but IMO is the biggest thing pf/altq lacks.


    I think most people who use pf for anything more than the most trivial
    traffic shaping applications realise that this is a major feature which
    is lacking.

    Kestas


  8. Re: PF inadequacy: queue download

    * kestas.j.k@gmail.com [2006-04-30]:
    > Thanks for the help, but I still don't think you understand.


    I spent a fair bit of today researching the issue to make sure that
    I understand you better. I believe I do now.

    > This isn't
    > a limitation in my understanding of PF and ALTQ, this is a limitation
    > of PF and ALTQ itself; you cannot shape traffic coming in on an
    > interface.


    Yes, I was aware that it can't and that this is a limitation of
    PF/ALTQ. I'm not entirely convinced that it's serious one, however.

    Do you have access to any documents that show measurable advantages
    that queuing incoming traffic would have over not queuing it?
    Personally, I've been finding mostly "ad hoc" accounts being cited
    as evidence for shaping incoming traffic.

    > There's no good reason why you shouldn't be able to do this.
    > I want to know how I can get this added to a todo list, everyone who
    > has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this
    > is an important feature which PF lacks, and should be implemented asap.
    >

    Your statement assumes that anyone who disagrees with you has a poor
    grasp of PF/ALTQ, and is a zealot. That polarizes things just a
    bit, don't you think?

    Keeping in mind that I'm not a developer myself, the following
    points might be helpful.

    If you could demonstrate the importance of shaping incoming traffic
    to Theo and the developers. Better yet, submit a patch, that might
    go further than simply telling them that it, "should be implemented
    asap."

    Generous donations may also help. I hear that they're rather fond
    of a good beer. :-)

    --
    W. Steven Schneider

  9. Re: PF inadequacy: queue download

    > I spent a fair bit of today researching the issue to make sure that
    > I understand you better. I believe I do now.

    Thanks, I appreciate that you'd spend some time trying to understand
    this.

    > Do you have access to any documents that show measurable advantages
    > that queuing incoming traffic would have over not queuing it?
    > Personally, I've been finding mostly "ad hoc" accounts being cited
    > as evidence for shaping incoming traffic.

    Hmm, okay.
    These images show me downloading two files on two different hosts. I
    want one host to recieve 30KB/s, and the other to recieve the rest. But
    most importantly I want it to happen immidiately (so the priority queue
    can be used to quickly get web pages). I run a download on the low
    priority host, and start a download on the high priority host, and run
    date before and after, and see the time it takes to stabilise at a fair
    speed for each.
    Here it is without download queueing:
    http://img151.imagevenue.com/img.php...2445_asdf2.JPG
    It takes over 20 seconds for the speeds to stabilise at around 25KB/s
    each. This means if someone's downloading it'll take over 20 seconds to
    download a fairly lightweight webpage.

    And with download queueing:
    http://img41.imagevenue.com/img.php?...82210_asdf.JPG
    It takes 2 seconds.

    Remember people do use download queueing using the two interface hack I
    use, but this functionality is limited only to NAT or dedicated traffic
    shaping boxes; I've shown you why it's useful, and lots of people use
    it with NAT, so why can't you do it on one interface?

    > Your statement assumes that anyone who disagrees with you has a poor
    > grasp of PF/ALTQ, and is a zealot. That polarizes things just a
    > bit, don't you think?

    I don't know how a statement can assume anything, or what is polarized,
    but yes that's true. It's not a personal attack to say that I doubt
    you're familiar with PF's traffic shaping capabilities, I'm not
    familiar with lots of things. I think anyone who understands PF's
    traffic shaping capabilities must understand its limitations, and this
    is a severe limitation.

    > If you could demonstrate the importance of shaping incoming traffic
    > to Theo and the developers. Better yet, submit a patch, that might
    > go further than simply telling them that it, "should be implemented
    > asap."

    I've done the former, not that it needs to be done as I'm sure the
    developers understand this download shaping limitation intimately, and
    unfortunately I can't do the latter as I don't know C very well, and I
    think working on something as complex and mission critical as PF+ALTQ
    would be daunting to someone who knew C well.

    > Generous donations may also help. I hear that they're rather fond
    > of a good beer. :-)

    I said I'd be happy to donate in another thread, to help get this moved
    along faster. Maybe a couple of days wages, but the only person who
    responded suggested a fee of $20,000. I'm a student earning minimum
    wage, I'm the last person who could splash out /$20,000/ to fund a
    hackathon to get this feature re-implemented.

    Kestas


+ Reply to Thread