Understanding path MTU discovery - Networking

This is a discussion on Understanding path MTU discovery - Networking ; Can someone confirm or correct my understanding of path MTU discovery regarding packets passing through a linux router/firewall to a server on a DMZ? As far as I can tell from my reading, if a computer on the internet accesses ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Understanding path MTU discovery

  1. Understanding path MTU discovery

    Can someone confirm or correct my understanding of path MTU discovery
    regarding packets passing through a linux router/firewall to a server on
    a DMZ? As far as I can tell from my reading, if a computer on the
    internet accesses our web server, but the reply from the server is too
    big (for example, the client computer is using a PPPoE link with an MTU
    of 1492), the client's ISP's gateway router will send an ICMP package
    back to our router. The router will see this as a "related" packet, and
    pass it on to the web server in the DMZ, and the web server will use the
    smaller packet size for the rest of the connection.

    Will the web server machine remember that lower MTU as being connected
    to the client's IP address, so that future connections by the client
    will avoid the overhead of the path MTU discovery, or will the discovery
    be needed for each new connection? I presume the lower MTU will not be
    needed for connections to/from other IP addresses.

    What will happen if our router has more than one route back to the
    client (i.e., two DSL links)? I understand that I could mark incoming
    packets from clients so that replies are sent out through the same
    interface they came in, but I would prefer to balance the output packets
    (downstream ADSL has more than enough bandwidth for web server requests,
    but it would be best to take advantage of two upstream links for
    replies). Suppose the client's original request comes in via eth0, gets
    passed on to the server on eth2, and the reply happens to go out on eth1
    via the other ADSL line. The ICMP packet complaining about the MTU size
    is going to be sent back to the eth1 address - will that still count as
    "related" and therefore be passed on to the web server?

    Thanks,

    David

  2. Re: Understanding path MTU discovery

    David Brown wrote:
    > Can someone confirm or correct my understanding of path MTU
    > discovery regarding packets passing through a linux router/firewall
    > to a server on a DMZ? As far as I can tell from my reading, if a
    > computer on the internet accesses our web server, but the reply from
    > the server is too big (for example, the client computer is using a
    > PPPoE link with an MTU of 1492), the client's ISP's gateway router
    > will send an ICMP package back to our router. The router will see
    > this as a "related" packet, and pass it on to the web server in the
    > DMZ, and the web server will use the smaller packet size for the
    > rest of the connection.


    Assuming that there wasn't anything filtering the requisite ICMP
    messages in the name of security.

    > Will the web server machine remember that lower MTU as being
    > connected to the client's IP address, so that future connections by
    > the client will avoid the overhead of the path MTU discovery, or
    > will the discovery be needed for each new connection? I presume the
    > lower MTU will not be needed for connections to/from other IP
    > addresses.


    Typically a host route for that client's IP address is created in the
    server's routing table. This route has a timer associated with it to
    a) reset the PathMTU back to link-local and later b) remove the route
    entirely if idle. So, it will be rememebered so long as the new
    connections to the same client IP happen within the timeouts.

    For the second question, indeed the lower PathMTU will not be used for
    connections to/from other IP addresses.

    > What will happen if our router has more than one route back to the
    > client (i.e., two DSL links)? I understand that I could mark
    > incoming packets from clients so that replies are sent out through
    > the same interface they came in, but I would prefer to balance the
    > output packets (downstream ADSL has more than enough bandwidth for
    > web server requests, but it would be best to take advantage of two
    > upstream links for replies). Suppose the client's original request
    > comes in via eth0, gets passed on to the server on eth2, and the
    > reply happens to go out on eth1 via the other ADSL line. The ICMP
    > packet complaining about the MTU size is going to be sent back to
    > the eth1 address - will that still count as "related" and therefore
    > be passed on to the web server?


    On that one I've no idea, other than to say that the ICMP messages
    will be send based on the source IP address in the TCP segment which
    triggered it, and the source IP (as well as destination IP and the
    local/remote port numbers) will be invariant for a connection
    regardless of the path it takes.

    rick jones
    --
    No need to believe in either side, or any side. There is no cause.
    There's only yourself. The belief is in your own precision. - Jobert
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  3. Re: Understanding path MTU discovery

    Rick Jones wrote:
    > David Brown wrote:
    >> Can someone confirm or correct my understanding of path MTU
    >> discovery regarding packets passing through a linux router/firewall
    >> to a server on a DMZ? As far as I can tell from my reading, if a
    >> computer on the internet accesses our web server, but the reply from
    >> the server is too big (for example, the client computer is using a
    >> PPPoE link with an MTU of 1492), the client's ISP's gateway router
    >> will send an ICMP package back to our router. The router will see
    >> this as a "related" packet, and pass it on to the web server in the
    >> DMZ, and the web server will use the smaller packet size for the
    >> rest of the connection.

    >
    > Assuming that there wasn't anything filtering the requisite ICMP
    > messages in the name of security.
    >


    I gather that this is becoming a problem, with many firewalls being set
    to filter out the relevant ICMP messages "in the name of security". Our
    current firewall/router (an old ZyWall 10, to be replaced by a linux or
    possibly bsd box) blocks all ICMP messages, so I've had no choice but to
    force the server's Ethernet port to use an MTU of 1492 for
    compatibility with some clients. That works, but is not really ideal.
    As far as I can see, allowing "related" ICMP packets through should
    cover the required messages while blocking any other ICMP's.

    >> Will the web server machine remember that lower MTU as being
    >> connected to the client's IP address, so that future connections by
    >> the client will avoid the overhead of the path MTU discovery, or
    >> will the discovery be needed for each new connection? I presume the
    >> lower MTU will not be needed for connections to/from other IP
    >> addresses.

    >
    > Typically a host route for that client's IP address is created in the
    > server's routing table. This route has a timer associated with it to
    > a) reset the PathMTU back to link-local and later b) remove the route
    > entirely if idle. So, it will be rememebered so long as the new
    > connections to the same client IP happen within the timeouts.
    >
    > For the second question, indeed the lower PathMTU will not be used for
    > connections to/from other IP addresses.
    >


    Excellent - thanks for that information.

    >> What will happen if our router has more than one route back to the
    >> client (i.e., two DSL links)? I understand that I could mark
    >> incoming packets from clients so that replies are sent out through
    >> the same interface they came in, but I would prefer to balance the
    >> output packets (downstream ADSL has more than enough bandwidth for
    >> web server requests, but it would be best to take advantage of two
    >> upstream links for replies). Suppose the client's original request
    >> comes in via eth0, gets passed on to the server on eth2, and the
    >> reply happens to go out on eth1 via the other ADSL line. The ICMP
    >> packet complaining about the MTU size is going to be sent back to
    >> the eth1 address - will that still count as "related" and therefore
    >> be passed on to the web server?

    >
    > On that one I've no idea, other than to say that the ICMP messages
    > will be send based on the source IP address in the TCP segment which
    > triggered it, and the source IP (as well as destination IP and the
    > local/remote port numbers) will be invariant for a connection
    > regardless of the path it takes.
    >


    The source IP will not be invariant for packets coming out of the router
    towards the internet - such packets will have different SNAT addresses
    depending on which of the two ADSL lines it passes through (each line
    has its own Ethernet port on the router, and they have different public
    IP addresses). I think I'm going to have to read a bit more about
    exactly how SNAT is implemented and how "related" states are determined,
    so that I can see what happens here.

    Thanks,

    David


    > rick jones


  4. Re: Understanding path MTU discovery

    David Brown wrote:
    > The source IP will not be invariant for packets coming out of the
    > router towards the internet - such packets will have different SNAT
    > addresses depending on which of the two ADSL lines it passes through
    > (each line has its own Ethernet port on the router, and they have
    > different public IP addresses). I think I'm going to have to read a
    > bit more about exactly how SNAT is implemented and how "related"
    > states are determined, so that I can see what happens here.


    NAT - ah, yes, that wonderful thing. The four-tuple of local/remote
    IP, local/remote port is what uniquely "names" a TCP connection. When
    there is only one NAT, there remains a 1-1 mapping of the TCP
    connection name between its "intenal" and "external" persona.
    Introduce the possibility that a given TCP connection may have _two_
    external personas (personi?) then I suspect all Heck breaks loose and
    Prince Phil will run rufshod over your connectivity.

    rick jones

    Longing for the day when a stake can be driven through the heart of
    NAT.

    NAT, routing, firewall - all different things...even if they are in
    the same box...

    --
    portable adj, code that compiles under more than one compiler
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  5. Re: Understanding path MTU discovery

    Rick Jones wrote:
    > David Brown wrote:
    >> The source IP will not be invariant for packets coming out of the
    >> router towards the internet - such packets will have different SNAT
    >> addresses depending on which of the two ADSL lines it passes through
    >> (each line has its own Ethernet port on the router, and they have
    >> different public IP addresses). I think I'm going to have to read a
    >> bit more about exactly how SNAT is implemented and how "related"
    >> states are determined, so that I can see what happens here.

    >
    > NAT - ah, yes, that wonderful thing. The four-tuple of local/remote
    > IP, local/remote port is what uniquely "names" a TCP connection. When
    > there is only one NAT, there remains a 1-1 mapping of the TCP
    > connection name between its "intenal" and "external" persona.
    > Introduce the possibility that a given TCP connection may have _two_
    > external personas (personi?) then I suspect all Heck breaks loose and
    > Prince Phil will run rufshod over your connectivity.
    >


    It could be that I'll limit outgoing reply packets to match the
    interface the original request came in on - it's not going to be a big
    issue, and may be better than fighting MTU issues. The only case were
    it could really make a difference is when using rsync to an offsite
    location as part of our backup routines - doubling the outward speed
    with two lines would be useful here. But in that case, there will be no
    MTU telegrams to worry about, since the MTU is the same at each end.

    > rick jones
    >
    > Longing for the day when a stake can be driven through the heart of
    > NAT.
    >
    > NAT, routing, firewall - all different things...even if they are in
    > the same box...
    >


    I can certainly agree that NAT is a pain for these sorts of issues,
    certain other kinds of routing problems, and it negates any sort of
    IP-based authentication or tracking on external servers. But I think
    NAT is a useful part of a firewalling and security strategy - it makes
    the machines on the inside invisible to the outside. That's an
    inconvenience when you *want* the inside machines to be visible, but in
    most cases you do not want them visible. If you do not have NAT, then
    all machines on the inside that can connect to the net have globally
    visible addresses - that means that they are targets. Sure, your
    firewall should block any incoming telegrams aimed at them - but NAT
    means that incoming telegrams cannot be aimed in the first place.
    That's an extra layer of protection if something should go wrong (like
    configuration errors in the firewall, or bugs of some sort). I think in
    most cases, that makes NAT a useful security aid. The other type of
    NAT, DNAT, is also useful for its flexibility - it allows traffic to be
    aimed at the static router, and redirected to different servers as
    needed, without the clients needing to know anything about the
    configuration (DNS can do similar things in a different way, of course).

  6. Re: Understanding path MTU discovery

    David Brown wrote:
    > I can certainly agree that NAT is a pain for these sorts of issues,
    > certain other kinds of routing problems, and it negates any sort of
    > IP-based authentication or tracking on external servers. But I
    > think NAT is a useful part of a firewalling and security strategy -
    > it makes the machines on the inside invisible to the outside.
    > That's an inconvenience when you *want* the inside machines to be
    > visible, but in most cases you do not want them visible. If you do
    > not have NAT, then all machines on the inside that can connect to
    > the net have globally visible addresses - that means that they are
    > targets. Sure, your firewall should block any incoming telegrams
    > aimed at them - but NAT means that incoming telegrams cannot be
    > aimed in the first place. That's an extra layer of protection if
    > something should go wrong (like configuration errors in the
    > firewall, or bugs of some sort). I think in most cases, that makes
    > NAT a useful security aid. The other type of NAT, DNAT, is also
    > useful for its flexibility - it allows traffic to be aimed at the
    > static router, and redirected to different servers as needed,
    > without the clients needing to know anything about the configuration
    > (DNS can do similar things in a different way, of course).


    Add little to little and soon there will be a great pile

    rick jones
    --
    firebug n, the idiot who tosses a lit cigarette out his car window
    these opinions are mine, all mine; HP might not want them anyway...
    feel free to post, OR email to rick.jones2 in hp.com but NOT BOTH...

  7. Re: Understanding path MTU discovery

    Rick Jones wrote:
    > David Brown wrote:
    >> I can certainly agree that NAT is a pain for these sorts of issues,
    >> certain other kinds of routing problems, and it negates any sort of
    >> IP-based authentication or tracking on external servers. But I
    >> think NAT is a useful part of a firewalling and security strategy -
    >> it makes the machines on the inside invisible to the outside.
    >> That's an inconvenience when you *want* the inside machines to be
    >> visible, but in most cases you do not want them visible. If you do
    >> not have NAT, then all machines on the inside that can connect to
    >> the net have globally visible addresses - that means that they are
    >> targets. Sure, your firewall should block any incoming telegrams
    >> aimed at them - but NAT means that incoming telegrams cannot be
    >> aimed in the first place. That's an extra layer of protection if
    >> something should go wrong (like configuration errors in the
    >> firewall, or bugs of some sort). I think in most cases, that makes
    >> NAT a useful security aid. The other type of NAT, DNAT, is also
    >> useful for its flexibility - it allows traffic to be aimed at the
    >> static router, and redirected to different servers as needed,
    >> without the clients needing to know anything about the configuration
    >> (DNS can do similar things in a different way, of course).

    >
    > Add little to little and soon there will be a great pile
    >
    > rick jones


    There is something to be said for that. But at the moment, NAT is the
    only practical solution - perhaps one day IPv6 will give us an
    alternative. But even within IPv6, I am far from convinced that we
    should not use NAT even then for standard clients on a LAN. For servers
    in the DMZ, direct addressing would be easier. Of course, maybe my
    opinions will change as my network gets more complicated!

    mvh.,

    David


+ Reply to Thread