TCP Tuning question for LFN (High bandwidth, high latency) links - TCP-IP

This is a discussion on TCP Tuning question for LFN (High bandwidth, high latency) links - TCP-IP ; Hi I hope this hasn't been asked before, but I am after some advice about tuning an LFN network my company has been asked to administrate. The network is a worldwide fibre ring running from London-America-Malaysia-Middle East-London and has a ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: TCP Tuning question for LFN (High bandwidth, high latency) links

  1. TCP Tuning question for LFN (High bandwidth, high latency) links

    Hi

    I hope this hasn't been asked before, but I am after some advice about
    tuning an LFN network my company has been asked to administrate.

    The network is a worldwide fibre ring running from
    London-America-Malaysia-Middle East-London and has a bandwidth of
    155Mbps and a latency of between 155ms and 355ms depending on which
    node you are trying to contact. This latency can and does change though
    it a fibre goes down or is taken down for maintenance.

    I have been doing a lot of reading up on TCP/IP and have implemented
    the tcp1323opts registry key (set it to 3, timestamping and dynamic
    windowing) and increased the tcp window size to around 8 Megs (this is
    the window size that the bandwidth delay product gives me).

    When I use iperf to test these settings, I get a throughput of 64Mbps,
    which is nice, but I would have expected things to be a bit better.

    To make matters worth, the application developers say that their apps
    are very sensitive to window size and therefore we are not allowed to
    change it from the default (which I think is 64k, but could be wrong)
    and they are therefore thinking of implementing some kind of hardware
    appliance to increase the performance of the link. I have looked at F5,
    McData and Packateer and they look good, but I would like to know what
    other paramaters (if any) have been implemented in the WinXP/2003
    TCP/IP stack that I could experiment with to improve the performance of
    the link without having to resort to a hardware solution.

    Also, are any of these able to self adjust to take into account the
    possible latency increase mentioned above?

    We are running Windows XP and Server 2003 if that helps

    Any advice/ideas would be very much appreciated.

    Thanks for your time

    Cheers

    Phil


  2. Re: TCP Tuning question for LFN (High bandwidth, high latency) links


    philip.teale@gmail.com wrote:
    > Hi
    >
    > I hope this hasn't been asked before, but I am after some advice about
    > tuning an LFN network my company has been asked to administrate.
    >
    > The network is a worldwide fibre ring running from
    > London-America-Malaysia-Middle East-London and has a bandwidth of
    > 155Mbps and a latency of between 155ms and 355ms depending on which
    > node you are trying to contact. This latency can and does change though
    > it a fibre goes down or is taken down for maintenance.
    >
    > I have been doing a lot of reading up on TCP/IP and have implemented
    > the tcp1323opts registry key (set it to 3, timestamping and dynamic
    > windowing) and increased the tcp window size to around 8 Megs (this is
    > the window size that the bandwidth delay product gives me).
    >
    > When I use iperf to test these settings, I get a throughput of 64Mbps,
    > which is nice, but I would have expected things to be a bit better.
    >
    > To make matters worth, the application developers say that their apps
    > are very sensitive to window size and therefore we are not allowed to
    > change it from the default (which I think is 64k, but could be wrong)
    > and they are therefore thinking of implementing some kind of hardware
    > appliance to increase the performance of the link. I have looked at F5,
    > McData and Packateer and they look good, but I would like to know what
    > other paramaters (if any) have been implemented in the WinXP/2003
    > TCP/IP stack that I could experiment with to improve the performance of
    > the link without having to resort to a hardware solution.
    >
    > Also, are any of these able to self adjust to take into account the
    > possible latency increase mentioned above?
    >
    > We are running Windows XP and Server 2003 if that helps
    >
    > Any advice/ideas would be very much appreciated.



    If you can't increase the window size beyond 64KB, don't bother doing
    anything else. On a link with a round trip time of 155ms, a 64KB TCP
    window limits you to about 4.5mb/s. If the latency you quoted is
    one-way, then halve that number.


  3. Re: TCP Tuning question for LFN (High bandwidth, high latency) links

    In article <1155812502.958312.263750@i42g2000cwa.googlegroups. com>,
    wrote:
    >Hi
    >
    >I hope this hasn't been asked before, but I am after some advice about
    >tuning an LFN network my company has been asked to administrate.
    >
    >The network is a worldwide fibre ring running from
    >London-America-Malaysia-Middle East-London and has a bandwidth of
    >155Mbps and a latency of between 155ms and 355ms depending on which
    >node you are trying to contact. This latency can and does change though
    >it a fibre goes down or is taken down for maintenance.

    [..]

    Given the network you might consider WAN optimization
    equipment. For example Riverbed (Steelhead) and Juniper.

    http://www.riverbed.com/
    http://www.juniper.net/products/appaccel/wan/wxc/

    The buzz word to search on is WAFS (Wide Area File Systems)

    alan

  4. Re: TCP Tuning question for LFN (High bandwidth, high latency) links

    philip.teale@gmail.com wrote:
    > I have been doing a lot of reading up on TCP/IP and have implemented
    > the tcp1323opts registry key (set it to 3, timestamping and dynamic
    > windowing) and increased the tcp window size to around 8 Megs (this is
    > the window size that the bandwidth delay product gives me).
    >
    > When I use iperf to test these settings, I get a throughput of 64Mbps,
    > which is nice, but I would have expected things to be a bit better.


    Have you determined why the throughput isn't better than this?
    Possibilities include poor performance on the endpoints, packet loss
    (the tubes are full), and higher than expected latency.

    If you upload a network capture somewhere, maybe someone would look at
    it for you...

    > To make matters worth, the application developers say that their apps
    > are very sensitive to window size and therefore we are not allowed to
    > change it from the default (which I think is 64k, but could be wrong)


    This is a curious thing for developers to say. If the application is
    heavily request/response then I agree that larger windows might not
    _help_, but I can't see how they'd _hurt_. Can someone explain how
    windows could be too big? Memory is cheap, even wasted memory.

    Further, if the window size really is critical, then the application
    should be setting the desired size on its own. There should be no
    issue of "not allowed to change it from the default". You should be
    able to do whatever you choose with the default, and the application
    should be overriding the defaults (up to the configured ceiling).

    > and they are therefore thinking of implementing some kind of hardware
    > appliance to increase the performance of the link. I have looked at F5,
    > McData and Packateer and they look good, but I would like to know what
    > other paramaters (if any) have been implemented in the WinXP/2003
    > TCP/IP stack that I could experiment with to improve the performance of
    > the link without having to resort to a hardware solution.
    >
    > Also, are any of these able to self adjust to take into account the
    > possible latency increase mentioned above?


    Alan mentioned some other hardware WAN optimizers in his reply
    elsewhere.

    These things are news to me. What do they do? A quick perusal of the
    cited pages gives the impression that they are one of:
    A) clever application proxies
    B) inline compression devices

    'A' is only useful if there is a module for your application. I'm
    guessing by the conversations you've had with developers that the
    application is homegrown and not an industry standard. Likely no
    module supports it in that case.

    'B' can only make latency worse, while possibly improving throughput.

    I'm highly suspicious of the application. The things they've told you
    so far don't make sense to me. If the problem boils down to
    request/response performance then the developers need to tag and batch
    their requests, and implement a windowing system for outstanding
    requests, just like TCP does. ...Or they could cheat and use several
    sockets.

    If the requests *must* be performed sequentially because response #1
    determines request #2, then you're just screwed. Need to get these
    servers on the same continent.

    /chris


+ Reply to Thread