Concurrency Violation:CurrentCount increased - Veritas Cluster Server

This is a discussion on Concurrency Violation:CurrentCount increased - Veritas Cluster Server ; HI! Because of an SYBASE-Server crash we found the following entries in the engine.log: 2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount increased above 1 for failover group sybasegr 2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for group sybasegr on ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Concurrency Violation:CurrentCount increased

  1. Concurrency Violation:CurrentCount increased


    HI!

    Because of an SYBASE-Server crash we found the following entries in the engine.log:

    2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    increased above 1 for failover group sybasegr
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    group sybasegr on all nodes
    2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    was started due to statechange of resource Sybase from OFFLINE to
    ONLINEe
    2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    group sybasegr on system kon430
    2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    -offline sybasegr kon430 from 127.0.0.1
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    group sybasegr on system kon430
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    Sybase (Owner: unknown, Group: sybasegr) on System kon430
    2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    -online kondorgr kon430 from 127.0.0.1
    2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    /opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    /opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    successfully
    2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    for resource(Sybase) because the resource became OFFLINE unexpectedly, on

    its own.
    2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    of the completed operation (clean)
    2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    completed successfully.
    2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    Group: sybasegr) is offline on kon410 (Not initiated by VCS)


    sybasegr is normally online on KON410.

    Does anyone know something about the meaning of the first entry in the logfile?
    For us it seems that sybasegr is started more than 1 time???!!!
    In the last entry one can see that VCS hadn't shutdown the SYBASE-Server.

    Any hints?

    Thanks
    Heinz

  2. Re: Concurrency Violation:CurrentCount increased

    OK, the log shows the sybasegr did go online (no indication why)

    At this stage VCS saw that a group that is supposed to be online on only
    1 machine, is actually online already !! (maybe fell over and someone
    tried to online it ?)

    My guess is that some user did not check if the group was online already
    in the cluster, then assumed that it should be online on kon430, and did
    a hagrp -online sybasegr -sys kon430 , which cause the violation.


    What you need to look for in the engine_A.log (part of which you pasted
    here), is just above the piece that you pasted, to see where the
    sybasegr actually went online (so, search backwards to see what happened
    to sybasegr). I bet you will see it go online on another machine in the
    cluster. I bet you will also see something like :

    VCS INFO V-16-1-50135 User root fired command: hagrp -online sybasegr
    kon430 from 127.0.0.1

    This is an indication that it was onlined manually.


    If you want, opst more of the engine log and we can see

    onkelheinz wrote:
    > HI!
    >
    > Because of an SYBASE-Server crash we found the following entries in the engine.log:
    >
    > 2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    > increased above 1 for failover group sybasegr
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    > group sybasegr on all nodes
    > 2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    > was started due to statechange of resource Sybase from OFFLINE to
    > ONLINEe
    > 2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    > group sybasegr on system kon430
    > 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    > -offline sybasegr kon430 from 127.0.0.1
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    > group sybasegr on system kon430
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    > Sybase (Owner: unknown, Group: sybasegr) on System kon430
    > 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    > -online kondorgr kon430 from 127.0.0.1
    > 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    > /opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    > 2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    > was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    > 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    > /opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    > successfully
    > 2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > 2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    > for resource(Sybase) because the resource became OFFLINE unexpectedly, on
    >
    > its own.
    > 2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    > of the completed operation (clean)
    > 2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    > completed successfully.
    > 2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > 2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    > Group: sybasegr) is offline on kon410 (Not initiated by VCS)
    >
    >
    > sybasegr is normally online on KON410.
    >
    > Does anyone know something about the meaning of the first entry in the logfile?
    > For us it seems that sybasegr is started more than 1 time???!!!
    > In the last entry one can see that VCS hadn't shutdown the SYBASE-Server.
    >
    > Any hints?
    >
    > Thanks
    > Heinz


  3. Re: Concurrency Violation:CurrentCount increased


    Hi Me,

    I haven't found something like this:

    >VCS INFO V-16-1-50135 User root fired command: hagrp -online sybasegr
    >kon430 from 127.0.0.1





    But just other things:



    2006/01/09 04:55:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 04:56:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 04:57:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 04:58:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 04:59:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    Group: sybasegr) is online on kon430 (Not initiated by VCS)
    2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    increased above 1 for failover group sybasegr
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    group sybasegr on all nodes
    2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    was started due to statechange of resource Sybase from OFFLINE to
    ONLINEe
    2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    group sybasegr on system kon430
    2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    -offline sybasegr kon430 from 127.0.0.1
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    group sybasegr on system kon430
    2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    Sybase (Owner: unknown, Group: sybasegr) on System kon430
    2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    -online kondorgr kon430 from 127.0.0.1
    2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    /opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    /opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    successfully
    2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    Msg 6005, Level 14, State 1:
    Server 'KPLUS_PROD':
    A SHUTDOWN is in progress.
    Msg 4002, Level 14, State 1:
    Server 'KPLUS_PROD':
    Login failed.
    CT-LIBRARY error:
    ct_connect(): protocol specific layer: external error: The attempt
    to connect to the server failed.

    2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    for resource(Sybase) because the resource became OFFLINE unexpectedly, on
    its own.
    2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    of the completed operation (clean)
    Password:
    A SHUTDOWN command is already in progress. Please log off.
    2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    completed successfully.
    2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    of isql:
    Msg 6005, Level 14, State 1:
    Server 'KPLUS_PROD':
    A SHUTDOWN is in progress.
    Msg 4002, Level 14, State 1:
    Server 'KPLUS_PROD':
    Login failed.
    CT-LIBRARY error:
    ct_connect(): protocol specific layer: external error: The attempt
    to connect to the server failed.

    2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    Group: sybasegr) is offline on kon410 (Not initiated by VCS)






    What makes me a little bit confused is:

    2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    of isql:
    (1 row affected)

    2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    Group: sybasegr) is online on kon430 (Not initiated by VCS)


    At 05:00:01 the cluster probed the group sybasegr against kon430!! Why???
    What mechanism "told" him, to do that?

    Ok, what I can see is that resuorce SYBASE was not initiated by VCS
    on kon430!!!

    What I have learned is, probing SYBASE is done via isql against vcsdb
    and a special table. But SYBASE wasn't online on kon430!! I've checked the
    SYABSE logfile on kon430 and no entry since 8 month was generated

    In SYBASE logfile on kon410 there are entries:

    03:00000:00476:2006/01/08 14:28:06.83 server Beginning REORG REBUILD of
    'WeekHolidays'.
    03:00000:00476:2006/01/08 14:28:06.92 server REORG REBUILD of 'WeekHolidays'
    completed.
    00:00000:00233:2006/01/09 05:00:12.26 kernel XP Server stopped successfully.
    00:00000:00233:2006/01/09 05:00:12.69 server SQL Server shutdown by request.

    00:00000:00233:2006/01/09 05:00:12.69 kernel ueshutdown: exiting
    00:00000:00000:2006/01/09 06:53:33.58 kernel Use license file /usr/sybase/sybase_prod/SYSAM-1_0/licenses/license.dat.
    00:00000:00000:2006/01/09 06:53:33.58 kernel Checked out license ASE_SERVER
    00:00000:00000:2006/01/09 06:53:33.58 kernel Adaptive Server Enterprise
    Edition


    But nothing like "SYBASE has started", only shutdown.
    (Info: At 6:53 we started SYBASE again).

    Hmmmm... STRANGE!!!

    regards
    Heinz


  4. Re: Concurrency Violation:CurrentCount increased

    VCS has a job of :

    1. Bringing stuff online (that is resources)
    2. Making sure that they are online or not (monitor)
    3. Taking stuff offline
    4. Cleaning the mess if the offline does not work


    As soon as VCS sees that you have a certain resource enabled, it will
    start monitoring it on ALL NODES in the cluster. This is for finding
    potential sources of corruption (say you manualy import the same
    diskgroup on 2 machines, or mount the same file system on 2 machines,
    you will get corruption.) VCS will try to save your butt and protect
    itself from doing that by monitoring on all nodes in the cluster.


    So, next the question. Why was it online on 2 machines ? Is it a
    parallel thing (like Oracle RAC or a parallel capable database) ?

    If it is not the "same thing", how are they diofferent ? and how should
    VCS know they are different ? What did you tell VCS (in the
    configuration) about the resources (stuff like database identifier or
    application lock file location or ......) ? VCS does what you tell it to
    do. It looks for the resource to match what you specified. So if you did
    not tell it about the differences between the stuff that was online on
    another box and then also came online on kon430


    Only other places to start looking (as you have seen that VCS did not
    bring SyBase online), is to look at the sybase logs, see if some idiot
    did it by hand on the machine ?

    onkelheinz wrote:
    > Hi Me,
    >
    > I haven't found something like this:
    >
    >
    >>VCS INFO V-16-1-50135 User root fired command: hagrp -online sybasegr
    >>kon430 from 127.0.0.1

    >
    >
    >
    >
    >
    > But just other things:
    >
    >
    >
    > 2006/01/09 04:55:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 04:56:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 04:57:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 04:58:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 04:59:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    > Group: sybasegr) is online on kon430 (Not initiated by VCS)
    > 2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    > increased above 1 for failover group sybasegr
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    > group sybasegr on all nodes
    > 2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    > was started due to statechange of resource Sybase from OFFLINE to
    > ONLINEe
    > 2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    > group sybasegr on system kon430
    > 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    > -offline sybasegr kon430 from 127.0.0.1
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    > group sybasegr on system kon430
    > 2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    > Sybase (Owner: unknown, Group: sybasegr) on System kon430
    > 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    > -online kondorgr kon430 from 127.0.0.1
    > 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    > /opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    > 2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    > was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    > 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    > /opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    > successfully
    > 2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > Msg 6005, Level 14, State 1:
    > Server 'KPLUS_PROD':
    > A SHUTDOWN is in progress.
    > Msg 4002, Level 14, State 1:
    > Server 'KPLUS_PROD':
    > Login failed.
    > CT-LIBRARY error:
    > ct_connect(): protocol specific layer: external error: The attempt
    > to connect to the server failed.
    >
    > 2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    > for resource(Sybase) because the resource became OFFLINE unexpectedly, on
    > its own.
    > 2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    > of the completed operation (clean)
    > Password:
    > A SHUTDOWN command is already in progress. Please log off.
    > 2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    > completed successfully.
    > 2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    > of isql:
    > Msg 6005, Level 14, State 1:
    > Server 'KPLUS_PROD':
    > A SHUTDOWN is in progress.
    > Msg 4002, Level 14, State 1:
    > Server 'KPLUS_PROD':
    > Login failed.
    > CT-LIBRARY error:
    > ct_connect(): protocol specific layer: external error: The attempt
    > to connect to the server failed.
    >
    > 2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    > Group: sybasegr) is offline on kon410 (Not initiated by VCS)
    >
    >
    >
    >
    >
    >
    > What makes me a little bit confused is:
    >
    > 2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    > of isql:
    > (1 row affected)
    >
    > 2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    > Group: sybasegr) is online on kon430 (Not initiated by VCS)
    >
    >
    > At 05:00:01 the cluster probed the group sybasegr against kon430!! Why???
    > What mechanism "told" him, to do that?
    >
    > Ok, what I can see is that resuorce SYBASE was not initiated by VCS
    > on kon430!!!
    >
    > What I have learned is, probing SYBASE is done via isql against vcsdb
    > and a special table. But SYBASE wasn't online on kon430!! I've checked the
    > SYABSE logfile on kon430 and no entry since 8 month was generated
    >
    > In SYBASE logfile on kon410 there are entries:
    >
    > 03:00000:00476:2006/01/08 14:28:06.83 server Beginning REORG REBUILD of
    > 'WeekHolidays'.
    > 03:00000:00476:2006/01/08 14:28:06.92 server REORG REBUILD of 'WeekHolidays'
    > completed.
    > 00:00000:00233:2006/01/09 05:00:12.26 kernel XP Server stopped successfully.
    > 00:00000:00233:2006/01/09 05:00:12.69 server SQL Server shutdown by request.
    >
    > 00:00000:00233:2006/01/09 05:00:12.69 kernel ueshutdown: exiting
    > 00:00000:00000:2006/01/09 06:53:33.58 kernel Use license file /usr/sybase/sybase_prod/SYSAM-1_0/licenses/license.dat.
    > 00:00000:00000:2006/01/09 06:53:33.58 kernel Checked out license ASE_SERVER
    > 00:00000:00000:2006/01/09 06:53:33.58 kernel Adaptive Server Enterprise
    > Edition
    >
    >
    > But nothing like "SYBASE has started", only shutdown.
    > (Info: At 6:53 we started SYBASE again).
    >
    > Hmmmm... STRANGE!!!
    >
    > regards
    > Heinz
    >


  5. Re: Concurrency Violation:CurrentCount increased


    Hi Me,

    I've checfked the sybase logs and there is no entrry that sybase
    has taken online on the second node (kon430).

    I think we will never know what occured this error.

    thank you!

    regards
    heinz



    Me wrote:
    >VCS has a job of :
    >
    >1. Bringing stuff online (that is resources)
    >2. Making sure that they are online or not (monitor)
    >3. Taking stuff offline
    >4. Cleaning the mess if the offline does not work
    >
    >
    >As soon as VCS sees that you have a certain resource enabled, it will
    >start monitoring it on ALL NODES in the cluster. This is for finding
    >potential sources of corruption (say you manualy import the same
    >diskgroup on 2 machines, or mount the same file system on 2 machines,
    >you will get corruption.) VCS will try to save your butt and protect
    >itself from doing that by monitoring on all nodes in the cluster.
    >
    >
    >So, next the question. Why was it online on 2 machines ? Is it a
    >parallel thing (like Oracle RAC or a parallel capable database) ?
    >
    >If it is not the "same thing", how are they diofferent ? and how should


    >VCS know they are different ? What did you tell VCS (in the
    >configuration) about the resources (stuff like database identifier or
    >application lock file location or ......) ? VCS does what you tell it to


    >do. It looks for the resource to match what you specified. So if you did


    >not tell it about the differences between the stuff that was online on
    >another box and then also came online on kon430
    >
    >
    >Only other places to start looking (as you have seen that VCS did not
    >bring SyBase online), is to look at the sybase logs, see if some idiot
    >did it by hand on the machine ?
    >
    >onkelheinz wrote:
    >> Hi Me,
    >>
    >> I haven't found something like this:
    >>
    >>
    >>>VCS INFO V-16-1-50135 User root fired command: hagrp -online sybasegr


    >>>kon430 from 127.0.0.1

    >>
    >>
    >>
    >>
    >>
    >> But just other things:
    >>
    >>
    >>
    >> 2006/01/09 04:55:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 04:56:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 04:57:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 04:58:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 04:59:04 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    >> Group: sybasegr) is online on kon430 (Not initiated by VCS)
    >> 2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    >> increased above 1 for failover group sybasegr
    >> 2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute

    for
    >> group sybasegr on all nodes
    >> 2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    >> was started due to statechange of resource Sybase from OFFLINE to
    >> ONLINEe
    >> 2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    >> group sybasegr on system kon430
    >> 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >> -offline sybasegr kon430 from 127.0.0.1
    >> 2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline

    of
    >> group sybasegr on system kon430
    >> 2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    >> Sybase (Owner: unknown, Group: sybasegr) on System kon430
    >> 2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >> -online kondorgr kon430 from 127.0.0.1
    >> 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger

    executed
    >> /opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    >> 2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    >> was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    >> 2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger

    executed
    >> /opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    >> successfully
    >> 2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> Msg 6005, Level 14, State 1:
    >> Server 'KPLUS_PROD':
    >> A SHUTDOWN is in progress.
    >> Msg 4002, Level 14, State 1:
    >> Server 'KPLUS_PROD':
    >> Login failed.
    >> CT-LIBRARY error:
    >> ct_connect(): protocol specific layer: external error: The attempt
    >> to connect to the server failed.
    >>
    >> 2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    >> for resource(Sybase) because the resource became OFFLINE unexpectedly,

    on
    >> its own.
    >> 2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    >> of the completed operation (clean)
    >> Password:
    >> A SHUTDOWN command is already in progress. Please log off.
    >> 2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) -

    clean
    >> completed successfully.
    >> 2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >> of isql:
    >> Msg 6005, Level 14, State 1:
    >> Server 'KPLUS_PROD':
    >> A SHUTDOWN is in progress.
    >> Msg 4002, Level 14, State 1:
    >> Server 'KPLUS_PROD':
    >> Login failed.
    >> CT-LIBRARY error:
    >> ct_connect(): protocol specific layer: external error: The attempt
    >> to connect to the server failed.
    >>
    >> 2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    >> Group: sybasegr) is offline on kon410 (Not initiated by VCS)
    >>
    >>
    >>
    >>
    >>
    >>
    >> What makes me a little bit confused is:
    >>
    >> 2006/01/09 05:00:01 VCS NOTICE V-16-20013-9 (kon430) Sybase:Sybase:monitor:Output
    >> of isql:
    >> (1 row affected)
    >>
    >> 2006/01/09 05:00:01 VCS INFO V-16-1-10299 Resource Sybase (Owner: unknown,
    >> Group: sybasegr) is online on kon430 (Not initiated by VCS)
    >>
    >>
    >> At 05:00:01 the cluster probed the group sybasegr against kon430!! Why???
    >> What mechanism "told" him, to do that?
    >>
    >> Ok, what I can see is that resuorce SYBASE was not initiated by VCS
    >> on kon430!!!
    >>
    >> What I have learned is, probing SYBASE is done via isql against vcsdb
    >> and a special table. But SYBASE wasn't online on kon430!! I've checked

    the
    >> SYABSE logfile on kon430 and no entry since 8 month was generated
    >>
    >> In SYBASE logfile on kon410 there are entries:
    >>
    >> 03:00000:00476:2006/01/08 14:28:06.83 server Beginning REORG REBUILD

    of
    >> 'WeekHolidays'.
    >> 03:00000:00476:2006/01/08 14:28:06.92 server REORG REBUILD of 'WeekHolidays'
    >> completed.
    >> 00:00000:00233:2006/01/09 05:00:12.26 kernel XP Server stopped successfully.
    >> 00:00000:00233:2006/01/09 05:00:12.69 server SQL Server shutdown by request.
    >>
    >> 00:00000:00233:2006/01/09 05:00:12.69 kernel ueshutdown: exiting
    >> 00:00000:00000:2006/01/09 06:53:33.58 kernel Use license file /usr/sybase/sybase_prod/SYSAM-1_0/licenses/license.dat.
    >> 00:00000:00000:2006/01/09 06:53:33.58 kernel Checked out license ASE_SERVER
    >> 00:00000:00000:2006/01/09 06:53:33.58 kernel Adaptive Server Enterprise
    >> Edition
    >>
    >>
    >> But nothing like "SYBASE has started", only shutdown.
    >> (Info: At 6:53 we started SYBASE again).
    >>
    >> Hmmmm... STRANGE!!!
    >>
    >> regards
    >> Heinz
    >>



  6. Re: Concurrency Violation:CurrentCount increased


    VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount increased above
    1 for failover group ClusterService

    What is the meaning of above error, any one explain me please..

    regards,
    saran

    "onkelheinz" wrote:
    >
    >HI!
    >
    >Because of an SYBASE-Server crash we found the following entries in the

    engine.log:
    >
    >2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    >increased above 1 for failover group sybasegr
    >2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    >group sybasegr on all nodes
    >2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    >was started due to statechange of resource Sybase from OFFLINE to
    >ONLINEe
    >2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    >group sybasegr on system kon430
    >2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >-offline sybasegr kon430 from 127.0.0.1
    >2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    >group sybasegr on system kon430
    >2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    >Sybase (Owner: unknown, Group: sybasegr) on System kon430
    >2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >-online kondorgr kon430 from 127.0.0.1
    >2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    >/opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    >2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    >was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    >2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    >/opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    > successfully
    >2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >of isql:
    >2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    >for resource(Sybase) because the resource became OFFLINE unexpectedly, on
    >
    >its own.
    >2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    >of the completed operation (clean)
    >2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    >completed successfully.
    >2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >of isql:
    >2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    >Group: sybasegr) is offline on kon410 (Not initiated by VCS)
    >
    >
    >sybasegr is normally online on KON410.
    >
    >Does anyone know something about the meaning of the first entry in the

    logfile?
    >For us it seems that sybasegr is started more than 1 time???!!!
    >In the last entry one can see that VCS hadn't shutdown the SYBASE-Server.
    >
    >Any hints?
    >
    >Thanks
    >Heinz



  7. Re: Concurrency Violation:CurrentCount increased

    Some resource in the Cluster Service group (which is a fail-over group
    which should be online only on 1 machine in the cluster) is also online
    on another node in the cluster.

    Check the /var/VRTSvcs/log/engine_A.log file to find out which resource



    saran wrote:
    > VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount increased above
    > 1 for failover group ClusterService
    >
    > What is the meaning of above error, any one explain me please..
    >
    > regards,
    > saran
    >
    > "onkelheinz" wrote:
    >
    >>HI!
    >>
    >>Because of an SYBASE-Server crash we found the following entries in the

    >
    > engine.log:
    >
    >>2006/01/09 05:00:01 VCS ERROR V-16-1-10214 Concurrency Violation:CurrentCount
    >>increased above 1 for failover group sybasegr
    >>2006/01/09 05:00:01 VCS NOTICE V-16-1-10233 Clearing Restart attribute for
    >>group sybasegr on all nodes
    >>2006/01/09 05:00:01 VCS WARNING V-16-6-9997 (kon430) resstatechange:resstatechange
    >>was started due to statechange of resource Sybase from OFFLINE to
    >>ONLINEe
    >>2006/01/09 05:00:01 VCS WARNING V-16-6-15034 (kon430) violation:Offlining
    >>group sybasegr on system kon430
    >>2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >>-offline sybasegr kon430 from 127.0.0.1
    >>2006/01/09 05:00:01 VCS NOTICE V-16-1-10167 Initiating manual offline of
    >>group sybasegr on system kon430
    >>2006/01/09 05:00:01 VCS NOTICE V-16-1-10300 Initiating Offline of Resource
    >>Sybase (Owner: unknown, Group: sybasegr) on System kon430
    >>2006/01/09 05:00:01 VCS INFO V-16-1-50135 User root fired command: hagrp
    >>-online kondorgr kon430 from 127.0.0.1
    >>2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    >>/opt/VRTSvcs/bin/triggers/violation kon430 sybasegr successfully
    >>2006/01/09 05:00:01 VCS INFO V-16-6-9998 (kon430) resstatechange:kondorgr
    >>was started due to statechange of resource Sybase from OFFLINE to ONLINEe
    >>2006/01/09 05:00:01 VCS INFO V-16-6-15002 (kon430) hatrigger:hatrigger executed
    >>/opt/VRTSvcs/bin/triggers/resstatechange kon430 Sybase OFFLINE ONLINE
    >>successfully
    >>2006/01/09 05:00:05 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >>of isql:
    >>2006/01/09 05:00:05 VCS ERROR V-16-2-13067 (kon410) Agent is calling clean
    >>for resource(Sybase) because the resource became OFFLINE unexpectedly, on
    >>
    >>its own.
    >>2006/01/09 05:00:06 VCS INFO V-16-2-13001 (kon410) Resource(Sybase): Output
    >>of the completed operation (clean)
    >>2006/01/09 05:00:06 VCS INFO V-16-2-13068 (kon410) Resource(Sybase) - clean
    >>completed successfully.
    >>2006/01/09 05:00:07 VCS NOTICE V-16-20013-9 (kon410) Sybase:Sybase:monitor:Output
    >>of isql:
    >>2006/01/09 05:00:07 VCS INFO V-16-1-10307 Resource Sybase (Owner: unknown,
    >>Group: sybasegr) is offline on kon410 (Not initiated by VCS)
    >>
    >>
    >>sybasegr is normally online on KON410.
    >>
    >>Does anyone know something about the meaning of the first entry in the

    >
    > logfile?
    >
    >>For us it seems that sybasegr is started more than 1 time???!!!
    >>In the last entry one can see that VCS hadn't shutdown the SYBASE-Server.
    >>
    >>Any hints?
    >>
    >>Thanks
    >>Heinz

    >
    >


+ Reply to Thread