Userspace packet queuing with libipq: ip_conntrack does not defragment? - Networking

This is a discussion on Userspace packet queuing with libipq: ip_conntrack does not defragment? - Networking ; Hi all, I'm using libipq to pass certain packets to my userspace application on Fedora 6 / Kernel 2.6.21.1 and ipTables 1.3.5. I do: modprobe iptable_filter modprobe ip_queue modprobe ip_conntrack iptables -A INPUT -p tcp -j QUEUE Works fine. However, ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: Userspace packet queuing with libipq: ip_conntrack does not defragment?

  1. Userspace packet queuing with libipq: ip_conntrack does not defragment?

    Hi all,

    I'm using libipq to pass certain packets to my userspace application
    on Fedora 6 / Kernel 2.6.21.1 and ipTables 1.3.5.

    I do:
    modprobe iptable_filter
    modprobe ip_queue
    modprobe ip_conntrack
    iptables -A INPUT -p tcp -j QUEUE

    Works fine. However, since ip_conntrack is loaded I would expect that
    the packets are defragmented before they are passed to my userspace
    appliation (as indicated here for example:
    http://lists.netfilter.org/pipermail...y/034006.html).
    This does not seem the case, i.e., the maximum size of the packets
    which I get (through ->data_len) is 1500 bits.

    The len parameter of the ipq_read method is well over 1500, as is the
    buffer size.

    Any suggestions what I'm doing wrong?

    Many thanks,
    Michael

    ps: I also tried to use "OUTPUT" in my rule since I read somewhere
    that connection tracking only works in OUTPUT and PREROUTING: Same
    result - maximum packet size is 1500, i.e., no defragmentation :-(


  2. Re: Userspace packet queuing with libipq: ip_conntrack does not defragment?

    Hi again,

    I would like to use an example to clarify what I want to achieve:

    Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
    What I see is that each picture of the video is send as a single TCP/
    IP packet (?) - usually with a size of around 10 KB. This TCP/IP
    packet is fragmented according to the MTU and therefore I see the
    following fragments on the wire:
    Fragment 1: application data
    Fragment 2: application data
    Fragment 3: application data
    ....

    On my router/firewall I would like to inspect the video picture as a
    whole. In my current (windows) application I have to reassemble the
    fragments manually into the original TCP/IP packet. I hoped that
    netfilter would make my life easier by performing the reassembling for
    me (and potentially also the recalculation of the checksum(s) if the
    video picture is changed).

    Is there support in netfilter for this kind of stuff?

    Many thanks again for your help,
    Michael


  3. Re: Userspace packet queuing with libipq: ip_conntrack does not defragment?

    Hello,

    I do not know about packet queuing and will reply only about connection
    tracking and packet reassembly.

    Daneel a écrit :
    >
    > I'm using libipq to pass certain packets to my userspace application
    > on Fedora 6 / Kernel 2.6.21.1 and ipTables 1.3.5.
    >
    > I do:
    > modprobe iptable_filter
    > modprobe ip_queue
    > modprobe ip_conntrack
    > iptables -A INPUT -p tcp -j QUEUE
    >
    > Works fine. However, since ip_conntrack is loaded I would expect that
    > the packets are defragmented before they are passed to my userspace
    > appliation


    Yes, but actually you do not need connection tracking for that.
    Reassembly occurs between the input routing decision and the INPUT
    chains even without the connection tracking. So the INPUT chains always
    see complete datagrams.

    > (as indicated here for example:
    > http://lists.netfilter.org/pipermail...y/034006.html).


    Err.. your URL is incomplete.

    > This does not seem the case, i.e., the maximum size of the packets
    > which I get (through ->data_len) is 1500 bits.


    Did you send fragmented datagrams with a size > 1500 bytes ?

    > ps: I also tried to use "OUTPUT" in my rule since I read somewhere
    > that connection tracking only works in OUTPUT and PREROUTING: Same
    > result - maximum packet size is 1500, i.e., no defragmentation :-(


    Same question, did you send datagrams with a size > 1500 bytes ?

    Notes :
    1) Connection tracking does not *work* in OUTPUT and PREROUTING ; it
    *takes places* there. Actually it is the packet classification, which is
    one part of the connection tracking process, which occurs before the
    PREROUTING and OUTPUT chains of the 'mangle' table (but after the
    PREROUTING and OUTPUT chains of the 'raw' table). Fragment reassembly is
    required for proper connection tracking operation, so it is performed
    just before packet classification.

    2) In recent 2.6 kernels at least, including 2.6.21, fragmentation of
    locally generated packets occurs - if needed - only after the OUTPUT
    chains, so these chains do not see fragmentation even without the
    connection tracking.

  4. Re: Userspace packet queuing with libipq: ip_conntrack does not defragment?

    Daneel a écrit :
    >
    > Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
    > What I see is that each picture of the video is send as a single TCP/
    > IP packet (?) - usually with a size of around 10 KB. This TCP/IP
    > packet is fragmented according to the MTU and therefore I see the
    > following fragments on the wire:
    > Fragment 1: application data
    > Fragment 2: application data
    > Fragment 3: application data


    This sounds highly unusual. Usually TCP connections limit the MSS
    (maximum segment size) according to the MTU size in order to avoid IP
    fragmentation.

  5. Re: Userspace packet queuing with libipq: ip_conntrack does not defragment?

    On May 9, 9:07 pm, Pascal Hambourg
    wrote:
    > Daneel a écrit :
    >
    >
    >
    > > Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
    > > What I see is that each picture of the video is send as a single TCP/
    > > IP packet (?) - usually with a size of around 10 KB. This TCP/IP
    > > packet is fragmented according to the MTU and therefore I see the
    > > following fragments on the wire:
    > > Fragment 1: application data
    > > Fragment 2: application data
    > > Fragment 3: application data

    >
    > This sounds highly unusual. Usually TCP connections limit the MSS
    > (maximum segment size) according to the MTU size in order to avoid IP
    > fragmentation.


    So what you say is that I should (!) only see:
    application data
    without any
    application data
    in-between for a TCP stream (given a proper TCP/IP stack)?

    Thanks,
    Michael


  6. Re: Userspace packet queuing with libipq: ip_conntrack does not defragment?

    Daneel a écrit :
    >>
    >>>Let's say I'm downloading a video from a website (over HTTP/TCP/IP).
    >>>What I see is that each picture of the video is send as a single TCP/
    >>>IP packet (?) - usually with a size of around 10 KB. This TCP/IP
    >>>packet is fragmented according to the MTU and therefore I see the
    >>>following fragments on the wire:

    [...]
    >>This sounds highly unusual. Usually TCP connections limit the MSS
    >>(maximum segment size) according to the MTU size in order to avoid IP
    >>fragmentation.

    >
    > So what you say is that I should (!) only see:
    > application data
    > without any
    > application data
    > in-between for a TCP stream (given a proper TCP/IP stack)?


    Indeed this is what I would expect to see, unless the path MTU is lower
    than the interface MTU and the hosts do not use path MTU discovery
    (PMTUd), thus causing fragmentation on some router along the path.
    However I would not expect to see TCP segments bigger than the interface
    MTU. But I am not a TCP/IP specialist at all, so my view of what
    actually exists may be limited. Someone with better knowledge and/or
    experience may correct me if I'm wrong.

    Could you please provide a capture of the beginning of a connection,
    showing at least the SYN packets with the MSS option and one ~10kB
    segment ? This will not help you, but I'm curious. ;-)

+ Reply to Thread