Performance question - Websphere

This is a discussion on Performance question - Websphere ; Hi, we are thinking of migrating our transport layer to MQ Series. one issue we need to understand is the following: - sender is in Asia - receiver is in Europe - a ping between these 2 locations takes approximately ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Performance question

  1. Performance question

    Hi,

    we are thinking of migrating our transport layer to MQ Series.
    one issue we need to understand is the following:

    - sender is in Asia
    - receiver is in Europe
    - a ping between these 2 locations takes approximately 350 ms
    - we intend to use distributed point to point queueing, and have the sender send messages to a server located in Asia, which will automatically forward the message to the server located in Europe, where the receiver will listen for messages
    - according to beliefs in our neighbourhood, messages in a specific queue are forwarded sequentially, that is MQ Server waits for a message to be acknowledged before sending the next one
    - we could not find in the docs any element to invalidate this belief
    - with the above latency, this would result in a maximum throughput per queue of 3 messages per second which is far from our needs

    Questions:
    - are we missing something ?
    - is there a way to tell MQ to parallelize message forwarding ?
    - if there is, does MQ preserve message sequence ?
    - can anyone share experience on this subject ?

    Regards,

    Eric

  2. Re: Performance question

    ext-eric.vergnaud@hsbc.fr wrote:
    > Hi,
    >
    > we are thinking of migrating our transport layer to MQ Series.
    > one issue we need to understand is the following:
    >
    > - sender is in Asia
    > - receiver is in Europe
    > - a ping between these 2 locations takes approximately 350 ms
    > - we intend to use distributed point to point queueing, and have the sender send messages to a server located in Asia, which will automatically forward the message to the server located in Europe, where the receiver will listen for messages
    > - according to beliefs in our neighbourhood, messages in a specific queue are forwarded sequentially, that is MQ Server waits for a message to be acknowledged before sending the next one
    > - we could not find in the docs any element to invalidate this belief
    > - with the above latency, this would result in a maximum throughput per queue of 3 messages per second which is far from our needs
    >
    > Questions:
    > - are we missing something ?


    You can send messages in batches, the whole batch will be committed or
    rolled-back as one unit, so you will only incur the turnaround cost at
    the end of each batch. The whole batch must be resent if any of it is
    missing or corrupted however, so don't make your batches too big.

    For non-persistent messages you can use NPMSPEED(FAST) on the channel to
    significantly reduce the latency of non-persistent messages at the cost
    of losing them if something goes wrong while a non-persistent message is
    in transit. Persistent messages will still be sent in syncpoint and
    committed at the batchinterval (and fast non-persistent messages do
    count against the batch interval).

    > - is there a way to tell MQ to parallelize message forwarding ?


    You can define more than one channel from one queue manager to the
    other, all channels can run in parallel and thus one will be able to use
    the available network bandwidth while the other is waiting for a
    line-turnaround.

    > - if there is, does MQ preserve message sequence ?


    Message ordering is in general not guaranteed across channels; if you
    need to get messages in the order in which they were put you should use
    message groups.

    Regards,

    Phil Willoughby
    --
    Staff Software Engineer - IBM WebSphere MQ for z/OS
    Senior Inventor

    IBM Certified System Administrator - WebSphere MQ V6.0
    IBM Certified Solution Designer - WebSphere MQ V6.0
    IBM Certified SOA Solution Designer

  3. Re: Performance question

    MQ "batches" messages for transmission across channels. Read the WebSphere MQ Intercommunication manual.

  4. Re: Performance question

    Hi, thanks for paying attention.

    Here is what I read in the manual:

    The batch size is the maximum number of messages to be sent before a syncpoint is taken. The batch size does not affect the way the channel transfers messages; messages are always transferred individually, but are committed or backed out as a batch.

    I understand that the batch improves the overall transmission time by reducing the number of commits, but it seem sto still require 1 transmission per message.

  5. Re: Performance question

    350ms turn-around latency does not occur on every single message.

    If you need better response, get a better network.

  6. Re: Performance question

    350 ms is NOT a bad performance.

    Due to the wave transmission speed limit, the minimum theorical time for a turn-around between Hong-Kong and Paris (via London in our case) is roughly 100 ms ( 2 * 15.000km / 300.000 km/s ).

    In a perfect world (that is without any electronical equipment between the 2 points), an MQ transmission queue would have a maximum throughput of 10 messages per second.

    I can hardly believe MQ does not provide any means to improve this throughput.

  7. Re: Performance question

    MQ can operate at full link speed to sequentially transmit messages, no matter how far away the destination is. It only experiences the turn-around latency after each "batch" of messages, to perform the commit handshake. Non-persisent messages don't do this, so they are faster, at the higher risk of being lost. See the NPMSPEED attribute of MQ channels.

  8. Re: Performance question

    Non-persistent messages are not acceptable in our solution.

    So, correct me if I'm wrong:
    - MQ forwards message sequentially
    - the channel runs over TCP
    - when MQ sends a packet it takes 350ms to receive the TCP ack
    - meanwhile the sender is blocked by the TCP stack
    - so MQ cannot send the next message until the TCP ack is received

    In this context, can you explain:

    "MQ can operate at full link speed to sequentially transmit messages, no matter how far away the destination is"

    Eric

  9. Re: Performance question

    ext-eric.vergnaud@hsbc.fr wrote:
    > Non-persistent messages are not acceptable in our solution.
    >
    > So, correct me if I'm wrong:
    > - MQ forwards message sequentially


    Yes.

    > - the channel runs over TCP


    Yes.

    > - when MQ sends a packet it takes 350ms to receive the TCP ack


    At least; probably more as the TCP stack in the receiver will delay it
    waiting for a data packet on which it can piggy-back.

    > - meanwhile the sender is blocked by the TCP stack


    No. That's not how TCP works. The sender can send more bytes up to the
    receiver's TCP receive window size, which is anything up to 1GiByte on
    modern systems.

    The required windows size to use the available bandwidth is the product
    of the bandwidth in bytes/second and the latency in seconds; if your
    link is 2MBit/sec then the window size required to achieve that with MQ
    is 87.5KBytes. Some systems and routing hardware restrict the window
    size to 64KiByte; if you find yourself in such a situation the MQ
    solution I would recommend is to put one or more forwarding queue
    managers between the two endpoints: this will improve throughput at the
    cost of some small additional latency.

    Regards,

    Phil Willoughby
    --
    Staff Software Engineer, WebSphere MQ for z/OS

+ Reply to Thread