explanation of GAB (Global Atomic Broadcast) and LLT - Veritas Cluster Server

This is a discussion on explanation of GAB (Global Atomic Broadcast) and LLT - Veritas Cluster Server ; Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast) and LLT? What is it and how is it used in relation to VCS? I'm in the midst of writing a High Availability Test document, and one of ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: explanation of GAB (Global Atomic Broadcast) and LLT

  1. explanation of GAB (Global Atomic Broadcast) and LLT


    Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast)
    and LLT?
    What is it and how is it used in relation to VCS? I'm in the midst of writing
    a High Availability Test document, and one of the tests will involve ensuring
    that the "GAB ports being used are working". I'd like to include some generic
    text describing what this is. (The specific test procedure executes \sbin\gabconfig
    -a command,... if this helps!).

    Thanks in advance!

    Bob Davis

    I

  2. Re: explanation of GAB (Global Atomic Broadcast) and LLT

    The cluster configuration is shared between all nodes in the cluster.
    This means not only the resources, but also the stae of the resources (I
    presume you know what a resource is)

    This implies that there must be some form of communication between teh
    nodes in a cluster.

    In general, communication on the internet happens via TCP/IP

    That is, IP is used to address and transport the packets, and TCP is the
    part that actually contains the "messages" that needs to be carried.

    TCP/IP is not fast enough for cluster communications !!!! (There are
    several reasons, but mainly because TCP has a self-correcting,
    connection-orientated nature).

    So what transport does VCS (Veritas Clutser Server) use ? and what are
    the "messages" carried around ?


    This is where LLT and GAB come in. Basically, LLT is the transport
    protocol and GAB is the "message".

    LLT was written by Veritas because other transports were too slow and
    there was not enough control over it. LLT "talks" directly to the DLPI
    interfaces of the network layers. This enables LLT to send messages
    really fast. LLT is also resonsible for the heartbeats between nodes in
    the cluster.

    GAB is used to carry the state of the cluster, the configration of the
    cluster and also makes sure that the correct membership is maintained.
    Membership comes into play with products like RAC and Veritas Cluster
    Volume Manager (CVM).

    OK, so when will which kind of message be sent by GAB ?

    When the nodes in a cluster starts up, 1 node is selected to read the
    configuration from disk. (This is normally the first node to start).
    Once it has read the configuration from disk, it keeps it in memory and
    it will then also start getting the states of resources in the cluster
    (well on the single machine that starts). You can specify GAB not to
    start, unless a certain amount of members (or nodes) are present (this
    is specified by the "-n" option for "gabconfig -c".

    So, the machine reads the config, keeps it in memory, and then as soon
    as any other machine starts (or joins the cluster), the configuration is
    passed to that machine (via GAB messages). The joining machine will then
    also go and find out what the state of resources are (on that machine)
    and report them back to the rest of the cluster (yep, GAB again). The
    state of resources are communicated regularly to other nodes in the cluster.

    Like TCP, GAB also have different ports. These ports are used by
    different programs to "talk" to GAB.

    GAB to GAB (thus the communications between nodes in the cluster)
    happens on port "a".

    HAD (the main VCS program that runs), actually does all the monitoring
    (well it tells Agents to monitor), and then the state of resources
    should be communicated to GAB (so that GAB can tell the rest of the
    nodes). The "had" process "talks" to GAB on port "h".

    So, in a VCS cluster, you should always see GAB ports "a" and "h".

    GAB also has other ports available :

    Port "b" is used for SCSI III fencing. (Fencing is a whole new chapter)

    Port "d" is used by Oracle RAC (Well actually ODM)

    Port "f" is used by fsckd (part of the Veritas Cluster Filesystem)


    Port "o" is used by Oracle RAC (cache fusion)

    Port "q" is used by the Veritas Cluster Filesystem Quick Log

    Port "u" is an exclusive port to serialize joiners to Cluster Volume
    Manager. Any node that wants to join the CVM, will try to get port "u"
    exclusively, and then once it has joined the cluster, it will let go of
    port "u" again. So, in a normal situation (where nodes have joined), you
    should not see port "u"

    Port "v" is part of CVM. It is opened after port "u"

    Port "w" is also used in CVM.



    -----

    That should explain most of it. If you want to know more about either
    LLT or GAB, just let us know. Obviously, some of the stuff is Veritas
    (or Symantec now) intellectual property, but I will try to explain as
    much as I can.

    bob davis wrote:
    > Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast)
    > and LLT?
    > What is it and how is it used in relation to VCS? I'm in the midst of writing
    > a High Availability Test document, and one of the tests will involve ensuring
    > that the "GAB ports being used are working". I'd like to include some generic
    > text describing what this is. (The specific test procedure executes \sbin\gabconfig
    > -a command,... if this helps!).
    >
    > Thanks in advance!
    >
    > Bob Davis
    >
    > I


  3. Re: explanation of GAB (Global Atomic Broadcast) and LLT


    You have mentioned about Port "d" . ( Port "d" is used by Oracle RAC (Well
    actually ODM) ) . Does it means that if Oracle Disk Manager (ODM) is enabled,
    gabconfig -a should show port d also ? I have ODM enabled in two clusters,
    but is not shoding port d.

    thanks


    Me wrote:
    >The cluster configuration is shared between all nodes in the cluster.
    >This means not only the resources, but also the stae of the resources (I


    >presume you know what a resource is)
    >
    >This implies that there must be some form of communication between teh
    >nodes in a cluster.
    >
    >In general, communication on the internet happens via TCP/IP
    >
    >That is, IP is used to address and transport the packets, and TCP is the


    >part that actually contains the "messages" that needs to be carried.
    >
    >TCP/IP is not fast enough for cluster communications !!!! (There are
    >several reasons, but mainly because TCP has a self-correcting,
    >connection-orientated nature).
    >
    >So what transport does VCS (Veritas Clutser Server) use ? and what are
    >the "messages" carried around ?
    >
    >
    >This is where LLT and GAB come in. Basically, LLT is the transport
    >protocol and GAB is the "message".
    >
    >LLT was written by Veritas because other transports were too slow and
    >there was not enough control over it. LLT "talks" directly to the DLPI
    >interfaces of the network layers. This enables LLT to send messages
    >really fast. LLT is also resonsible for the heartbeats between nodes in


    >the cluster.
    >
    >GAB is used to carry the state of the cluster, the configration of the
    >cluster and also makes sure that the correct membership is maintained.
    >Membership comes into play with products like RAC and Veritas Cluster
    >Volume Manager (CVM).
    >
    >OK, so when will which kind of message be sent by GAB ?
    >
    >When the nodes in a cluster starts up, 1 node is selected to read the
    >configuration from disk. (This is normally the first node to start).
    >Once it has read the configuration from disk, it keeps it in memory and


    >it will then also start getting the states of resources in the cluster
    >(well on the single machine that starts). You can specify GAB not to
    >start, unless a certain amount of members (or nodes) are present (this
    >is specified by the "-n" option for "gabconfig -c".
    >
    >So, the machine reads the config, keeps it in memory, and then as soon
    >as any other machine starts (or joins the cluster), the configuration is


    >passed to that machine (via GAB messages). The joining machine will then


    >also go and find out what the state of resources are (on that machine)
    >and report them back to the rest of the cluster (yep, GAB again). The
    >state of resources are communicated regularly to other nodes in the cluster.
    >
    >Like TCP, GAB also have different ports. These ports are used by
    >different programs to "talk" to GAB.
    >
    >GAB to GAB (thus the communications between nodes in the cluster)
    >happens on port "a".
    >
    >HAD (the main VCS program that runs), actually does all the monitoring
    >(well it tells Agents to monitor), and then the state of resources
    >should be communicated to GAB (so that GAB can tell the rest of the
    >nodes). The "had" process "talks" to GAB on port "h".
    >
    >So, in a VCS cluster, you should always see GAB ports "a" and "h".
    >
    >GAB also has other ports available :
    >
    >Port "b" is used for SCSI III fencing. (Fencing is a whole new chapter)
    >
    >Port "d" is used by Oracle RAC (Well actually ODM)
    >
    >Port "f" is used by fsckd (part of the Veritas Cluster Filesystem)
    >
    >
    >Port "o" is used by Oracle RAC (cache fusion)
    >
    >Port "q" is used by the Veritas Cluster Filesystem Quick Log
    >
    >Port "u" is an exclusive port to serialize joiners to Cluster Volume
    >Manager. Any node that wants to join the CVM, will try to get port "u"
    >exclusively, and then once it has joined the cluster, it will let go of


    >port "u" again. So, in a normal situation (where nodes have joined), you


    >should not see port "u"
    >
    >Port "v" is part of CVM. It is opened after port "u"
    >
    >Port "w" is also used in CVM.
    >
    >
    >
    >-----
    >
    >That should explain most of it. If you want to know more about either
    >LLT or GAB, just let us know. Obviously, some of the stuff is Veritas
    >(or Symantec now) intellectual property, but I will try to explain as
    >much as I can.
    >
    >bob davis wrote:
    >> Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast)
    >> and LLT?
    >> What is it and how is it used in relation to VCS? I'm in the midst of

    writing
    >> a High Availability Test document, and one of the tests will involve ensuring
    >> that the "GAB ports being used are working". I'd like to include some

    generic
    >> text describing what this is. (The specific test procedure executes \sbin\gabconfig
    >> -a command,... if this helps!).
    >>
    >> Thanks in advance!
    >>
    >> Bob Davis
    >>
    >> I



  4. Re: explanation of GAB (Global Atomic Broadcast) and LLT

    port "d" is used by Veritas ODM

    ODM is a specification written by Oracle of how IO needs to be done.
    This was done because developers of filesystem software wanted to speed
    up Oracle access to filesystems.

    The big issue with that is that Veritas were the only people to ever
    develop their own ODM libraries.

    So, if you have the Veritas database edition installed (or RAC edition),
    you can use the Veritas ODM (by overwriting the Oracle default provided
    stuff).

    A fast way to check if you have Veritas ODM installed is to see if there
    is an ODM device ("df -k" and you should see the /dev/odm device mounted)

    Biju K Joseph wrote:
    > You have mentioned about Port "d" . ( Port "d" is used by Oracle RAC (Well
    > actually ODM) ) . Does it means that if Oracle Disk Manager (ODM) is enabled,
    > gabconfig -a should show port d also ? I have ODM enabled in two clusters,
    > but is not shoding port d.
    >
    > thanks
    >
    >
    > Me wrote:
    >
    >>The cluster configuration is shared between all nodes in the cluster.
    >>This means not only the resources, but also the stae of the resources (I

    >
    >
    >>presume you know what a resource is)
    >>
    >>This implies that there must be some form of communication between teh
    >>nodes in a cluster.
    >>
    >>In general, communication on the internet happens via TCP/IP
    >>
    >>That is, IP is used to address and transport the packets, and TCP is the

    >
    >
    >>part that actually contains the "messages" that needs to be carried.
    >>
    >>TCP/IP is not fast enough for cluster communications !!!! (There are
    >>several reasons, but mainly because TCP has a self-correcting,
    >>connection-orientated nature).
    >>
    >>So what transport does VCS (Veritas Clutser Server) use ? and what are
    >>the "messages" carried around ?
    >>
    >>
    >>This is where LLT and GAB come in. Basically, LLT is the transport
    >>protocol and GAB is the "message".
    >>
    >>LLT was written by Veritas because other transports were too slow and
    >>there was not enough control over it. LLT "talks" directly to the DLPI
    >>interfaces of the network layers. This enables LLT to send messages
    >>really fast. LLT is also resonsible for the heartbeats between nodes in

    >
    >
    >>the cluster.
    >>
    >>GAB is used to carry the state of the cluster, the configration of the
    >>cluster and also makes sure that the correct membership is maintained.
    >>Membership comes into play with products like RAC and Veritas Cluster
    >>Volume Manager (CVM).
    >>
    >>OK, so when will which kind of message be sent by GAB ?
    >>
    >>When the nodes in a cluster starts up, 1 node is selected to read the
    >>configuration from disk. (This is normally the first node to start).
    >>Once it has read the configuration from disk, it keeps it in memory and

    >
    >
    >>it will then also start getting the states of resources in the cluster
    >>(well on the single machine that starts). You can specify GAB not to
    >>start, unless a certain amount of members (or nodes) are present (this
    >>is specified by the "-n" option for "gabconfig -c".
    >>
    >>So, the machine reads the config, keeps it in memory, and then as soon
    >>as any other machine starts (or joins the cluster), the configuration is

    >
    >
    >>passed to that machine (via GAB messages). The joining machine will then

    >
    >
    >>also go and find out what the state of resources are (on that machine)
    >>and report them back to the rest of the cluster (yep, GAB again). The
    >>state of resources are communicated regularly to other nodes in the cluster.
    >>
    >>Like TCP, GAB also have different ports. These ports are used by
    >>different programs to "talk" to GAB.
    >>
    >>GAB to GAB (thus the communications between nodes in the cluster)
    >>happens on port "a".
    >>
    >>HAD (the main VCS program that runs), actually does all the monitoring
    >>(well it tells Agents to monitor), and then the state of resources
    >>should be communicated to GAB (so that GAB can tell the rest of the
    >>nodes). The "had" process "talks" to GAB on port "h".
    >>
    >>So, in a VCS cluster, you should always see GAB ports "a" and "h".
    >>
    >>GAB also has other ports available :
    >>
    >>Port "b" is used for SCSI III fencing. (Fencing is a whole new chapter)
    >>
    >>Port "d" is used by Oracle RAC (Well actually ODM)
    >>
    >>Port "f" is used by fsckd (part of the Veritas Cluster Filesystem)
    >>
    >>
    >>Port "o" is used by Oracle RAC (cache fusion)
    >>
    >>Port "q" is used by the Veritas Cluster Filesystem Quick Log
    >>
    >>Port "u" is an exclusive port to serialize joiners to Cluster Volume
    >>Manager. Any node that wants to join the CVM, will try to get port "u"
    >>exclusively, and then once it has joined the cluster, it will let go of

    >
    >
    >>port "u" again. So, in a normal situation (where nodes have joined), you

    >
    >
    >>should not see port "u"
    >>
    >>Port "v" is part of CVM. It is opened after port "u"
    >>
    >>Port "w" is also used in CVM.
    >>
    >>
    >>
    >>-----
    >>
    >>That should explain most of it. If you want to know more about either
    >>LLT or GAB, just let us know. Obviously, some of the stuff is Veritas
    >>(or Symantec now) intellectual property, but I will try to explain as
    >>much as I can.
    >>
    >>bob davis wrote:
    >>
    >>>Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast)
    >>>and LLT?
    >>>What is it and how is it used in relation to VCS? I'm in the midst of

    >
    > writing
    >
    >>>a High Availability Test document, and one of the tests will involve ensuring
    >>>that the "GAB ports being used are working". I'd like to include some

    >
    > generic
    >
    >>>text describing what this is. (The specific test procedure executes \sbin\gabconfig
    >>>-a command,... if this helps!).
    >>>
    >>>Thanks in advance!
    >>>
    >>>Bob Davis
    >>>
    >>>I

    >
    >


  5. Re: explanation of GAB (Global Atomic Broadcast) and LLT


    Hi Bob or Hi to all
    If any body written a TEST Document how the VCS failover.
    If you have any good link that is also fine
    If so can you share.
    You can send to my email id "vishwakumarbg@gmail.com"

    thanks in advance
    Rgds
    Vishwa
    Me wrote:
    >The cluster configuration is shared between all nodes in the cluster.
    >This means not only the resources, but also the stae of the resources (I


    >presume you know what a resource is)
    >
    >This implies that there must be some form of communication between teh
    >nodes in a cluster.
    >
    >In general, communication on the internet happens via TCP/IP
    >
    >That is, IP is used to address and transport the packets, and TCP is the


    >part that actually contains the "messages" that needs to be carried.
    >
    >TCP/IP is not fast enough for cluster communications !!!! (There are
    >several reasons, but mainly because TCP has a self-correcting,
    >connection-orientated nature).
    >
    >So what transport does VCS (Veritas Clutser Server) use ? and what are
    >the "messages" carried around ?
    >
    >
    >This is where LLT and GAB come in. Basically, LLT is the transport
    >protocol and GAB is the "message".
    >
    >LLT was written by Veritas because other transports were too slow and
    >there was not enough control over it. LLT "talks" directly to the DLPI
    >interfaces of the network layers. This enables LLT to send messages
    >really fast. LLT is also resonsible for the heartbeats between nodes in


    >the cluster.
    >
    >GAB is used to carry the state of the cluster, the configration of the
    >cluster and also makes sure that the correct membership is maintained.
    >Membership comes into play with products like RAC and Veritas Cluster
    >Volume Manager (CVM).
    >
    >OK, so when will which kind of message be sent by GAB ?
    >
    >When the nodes in a cluster starts up, 1 node is selected to read the
    >configuration from disk. (This is normally the first node to start).
    >Once it has read the configuration from disk, it keeps it in memory and


    >it will then also start getting the states of resources in the cluster
    >(well on the single machine that starts). You can specify GAB not to
    >start, unless a certain amount of members (or nodes) are present (this
    >is specified by the "-n" option for "gabconfig -c".
    >
    >So, the machine reads the config, keeps it in memory, and then as soon
    >as any other machine starts (or joins the cluster), the configuration is


    >passed to that machine (via GAB messages). The joining machine will then


    >also go and find out what the state of resources are (on that machine)
    >and report them back to the rest of the cluster (yep, GAB again). The
    >state of resources are communicated regularly to other nodes in the cluster.
    >
    >Like TCP, GAB also have different ports. These ports are used by
    >different programs to "talk" to GAB.
    >
    >GAB to GAB (thus the communications between nodes in the cluster)
    >happens on port "a".
    >
    >HAD (the main VCS program that runs), actually does all the monitoring
    >(well it tells Agents to monitor), and then the state of resources
    >should be communicated to GAB (so that GAB can tell the rest of the
    >nodes). The "had" process "talks" to GAB on port "h".
    >
    >So, in a VCS cluster, you should always see GAB ports "a" and "h".
    >
    >GAB also has other ports available :
    >
    >Port "b" is used for SCSI III fencing. (Fencing is a whole new chapter)
    >
    >Port "d" is used by Oracle RAC (Well actually ODM)
    >
    >Port "f" is used by fsckd (part of the Veritas Cluster Filesystem)
    >
    >
    >Port "o" is used by Oracle RAC (cache fusion)
    >
    >Port "q" is used by the Veritas Cluster Filesystem Quick Log
    >
    >Port "u" is an exclusive port to serialize joiners to Cluster Volume
    >Manager. Any node that wants to join the CVM, will try to get port "u"
    >exclusively, and then once it has joined the cluster, it will let go of


    >port "u" again. So, in a normal situation (where nodes have joined), you


    >should not see port "u"
    >
    >Port "v" is part of CVM. It is opened after port "u"
    >
    >Port "w" is also used in CVM.
    >
    >
    >
    >-----
    >
    >That should explain most of it. If you want to know more about either
    >LLT or GAB, just let us know. Obviously, some of the stuff is Veritas
    >(or Symantec now) intellectual property, but I will try to explain as
    >much as I can.
    >
    >bob davis wrote:
    >> Can anyone provide me a concise explanation of GAB (Global Atomic Broadcast)
    >> and LLT?
    >> What is it and how is it used in relation to VCS? I'm in the midst of

    writing
    >> a High Availability Test document, and one of the tests will involve ensuring
    >> that the "GAB ports being used are working". I'd like to include some

    generic
    >> text describing what this is. (The specific test procedure executes \sbin\gabconfig
    >> -a command,... if this helps!).
    >>
    >> Thanks in advance!
    >>
    >> Bob Davis
    >>
    >> I



  6. Re: explanation of GAB (Global Atomic Broadcast) and LLT

    You are Wizard man........ great many thanks...... for this tutorial.
    Last edited by proton; 08-17-2009 at 05:07 PM. Reason: a small mistake

+ Reply to Thread