How to reduce iSCSI MPIO failover time? - Storage

This is a discussion on How to reduce iSCSI MPIO failover time? - Storage ; I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This Windows 2003 server with one NIC accesses an iSCSI target LUN through two Cisco MDS 9509/IPS paths. I was able to force a failover by disabling the network interface ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: How to reduce iSCSI MPIO failover time?

  1. How to reduce iSCSI MPIO failover time?

    I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    Windows 2003 server with one NIC accesses an iSCSI target LUN through two
    Cisco MDS 9509/IPS paths. I was able to force a failover by disabling the
    network interface (NIC) on one of the MDS 9509/IPS Directors . My issue on
    the failover is the failover time. I was getting the following results (with
    each step about a few minutes apart):

    1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled MDS(A) NIC
    2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled MDS(A) NIC
    5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC

    So I am getting a failover time of 56, 54, 35, and 35 seconds for the MDS(A)
    path
    and
    a failover time of 35 and 35 for the MDS(B) path.

    What parameter(s) determine this failover timing and how can I reduce it to
    15 seconds.

    Thank you,

    Hon


  2. Re: How to reduce iSCSI MPIO failover time?

    There are a number of ways to reduce this time but all of them depend on
    your storage provider. You should contact your storage provider and ask
    them what their recommended settings are.

    BTW, what target are you using?

    ~kenny

    "cdj" wrote in message
    news:13C472B8-F5A3-4BB4-859A-9EA3DF7F40E2@microsoft.com...
    >I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    > Windows 2003 server with one NIC accesses an iSCSI target LUN through two
    > Cisco MDS 9509/IPS paths. I was able to force a failover by disabling the
    > network interface (NIC) on one of the MDS 9509/IPS Directors . My issue
    > on
    > the failover is the failover time. I was getting the following results
    > (with
    > each step about a few minutes apart):
    >
    > 1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled MDS(A) NIC
    > 2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > 3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > 4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled MDS(A) NIC
    > 5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    > 6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    >
    > So I am getting a failover time of 56, 54, 35, and 35 seconds for the
    > MDS(A)
    > path
    > and
    > a failover time of 35 and 35 for the MDS(B) path.
    >
    > What parameter(s) determine this failover timing and how can I reduce it
    > to
    > 15 seconds.
    >
    > Thank you,
    >
    > Hon
    >



  3. Re: How to reduce iSCSI MPIO failover time?

    I am using Cisco MDS 9509 Directors with IPS module (iSCSI to Fibre Channel
    bridge/router) and an EMC Symmetrix F/C storage array. I have the same setup
    on a Red Hat Linux server and was able to reduce the failover time to 10
    seconds by changing the iSCSI Initiator parameter ConnFailTimeout to 10 in
    /etc/iscsi.conf file. This parameter specifies the session timeout value
    before failing the SCSI commands. I was hoping there is a similar timeout
    Registry entry for Microsoft's iSCSI Initiator.

    Thank you,

    Hon

    "Kenny Speer" wrote:

    > There are a number of ways to reduce this time but all of them depend on
    > your storage provider. You should contact your storage provider and ask
    > them what their recommended settings are.
    >
    > BTW, what target are you using?
    >
    > ~kenny
    >
    > "cdj" wrote in message
    > news:13C472B8-F5A3-4BB4-859A-9EA3DF7F40E2@microsoft.com...
    > >I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    > > Windows 2003 server with one NIC accesses an iSCSI target LUN through two
    > > Cisco MDS 9509/IPS paths. I was able to force a failover by disabling the
    > > network interface (NIC) on one of the MDS 9509/IPS Directors . My issue
    > > on
    > > the failover is the failover time. I was getting the following results
    > > (with
    > > each step about a few minutes apart):
    > >
    > > 1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled MDS(A) NIC
    > > 2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > > 3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > > 4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled MDS(A) NIC
    > > 5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    > > 6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    > >
    > > So I am getting a failover time of 56, 54, 35, and 35 seconds for the
    > > MDS(A)
    > > path
    > > and
    > > a failover time of 35 and 35 for the MDS(B) path.
    > >
    > > What parameter(s) determine this failover timing and how can I reduce it
    > > to
    > > 15 seconds.
    > >
    > > Thank you,
    > >
    > > Hon
    > >

    >
    >


  4. Re: How to reduce iSCSI MPIO failover time?

    I found this article

    http://technet.microsoft.com/en-us/l.../aa997069.aspx

    I will try around with this registry value

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Servic es\Disk\TimeOutValue

    Thanks,

    "cdj" wrote:

    > I am using Cisco MDS 9509 Directors with IPS module (iSCSI to Fibre Channel
    > bridge/router) and an EMC Symmetrix F/C storage array. I have the same setup
    > on a Red Hat Linux server and was able to reduce the failover time to 10
    > seconds by changing the iSCSI Initiator parameter ConnFailTimeout to 10 in
    > /etc/iscsi.conf file. This parameter specifies the session timeout value
    > before failing the SCSI commands. I was hoping there is a similar timeout
    > Registry entry for Microsoft's iSCSI Initiator.
    >
    > Thank you,
    >
    > Hon
    >
    > "Kenny Speer" wrote:
    >
    > > There are a number of ways to reduce this time but all of them depend on
    > > your storage provider. You should contact your storage provider and ask
    > > them what their recommended settings are.
    > >
    > > BTW, what target are you using?
    > >
    > > ~kenny
    > >
    > > "cdj" wrote in message
    > > news:13C472B8-F5A3-4BB4-859A-9EA3DF7F40E2@microsoft.com...
    > > >I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    > > > Windows 2003 server with one NIC accesses an iSCSI target LUN through two
    > > > Cisco MDS 9509/IPS paths. I was able to force a failover by disabling the
    > > > network interface (NIC) on one of the MDS 9509/IPS Directors . My issue
    > > > on
    > > > the failover is the failover time. I was getting the following results
    > > > (with
    > > > each step about a few minutes apart):
    > > >
    > > > 1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled MDS(A) NIC
    > > > 2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > > > 3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled MDS(B) NIC
    > > > 4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled MDS(A) NIC
    > > > 5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    > > > 6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled MDS(A) NIC
    > > >
    > > > So I am getting a failover time of 56, 54, 35, and 35 seconds for the
    > > > MDS(A)
    > > > path
    > > > and
    > > > a failover time of 35 and 35 for the MDS(B) path.
    > > >
    > > > What parameter(s) determine this failover timing and how can I reduce it
    > > > to
    > > > 15 seconds.
    > > >
    > > > Thank you,
    > > >
    > > > Hon
    > > >

    > >
    > >


  5. Re: How to reduce iSCSI MPIO failover time?

    The Disk class timeout value is the most misunderstood timer in windows
    storage. If you change it, there is a very good chance you will lose
    support from your storage vendor. If you make this value arbitrarily low,
    you will cause more problems than you are trying to solve.

    Again, without specific knowledge of what these timers do, you will most
    likely create a scenario where your host will no longer survive faults.

    To point you in the right direction, you need this key:
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\0001\Parameters

    Although, the 0001 towards the end may be different on your system. This is
    the SCSI Bus controller class and if you have more than 1 bus, this number
    changes.

    The only parameter that you might have luck with is: LinkDownTime, change
    that lower for your cable pull test.


    "cdj" wrote in message
    news:A48920FE-B829-4BEC-A811-B0E64EB0861F@microsoft.com...
    >I found this article
    >
    > http://technet.microsoft.com/en-us/l.../aa997069.aspx
    >
    > I will try around with this registry value
    >
    > HKEY_LOCAL_MACHINE\System\CurrentControlSet\Servic es\Disk\TimeOutValue
    >
    > Thanks,
    >
    > "cdj" wrote:
    >
    >> I am using Cisco MDS 9509 Directors with IPS module (iSCSI to Fibre
    >> Channel
    >> bridge/router) and an EMC Symmetrix F/C storage array. I have the same
    >> setup
    >> on a Red Hat Linux server and was able to reduce the failover time to 10
    >> seconds by changing the iSCSI Initiator parameter ConnFailTimeout to 10
    >> in
    >> /etc/iscsi.conf file. This parameter specifies the session timeout value
    >> before failing the SCSI commands. I was hoping there is a similar
    >> timeout
    >> Registry entry for Microsoft's iSCSI Initiator.
    >>
    >> Thank you,
    >>
    >> Hon
    >>
    >> "Kenny Speer" wrote:
    >>
    >> > There are a number of ways to reduce this time but all of them depend
    >> > on
    >> > your storage provider. You should contact your storage provider and
    >> > ask
    >> > them what their recommended settings are.
    >> >
    >> > BTW, what target are you using?
    >> >
    >> > ~kenny
    >> >
    >> > "cdj" wrote in message
    >> > news:13C472B8-F5A3-4BB4-859A-9EA3DF7F40E2@microsoft.com...
    >> > >I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    >> > > Windows 2003 server with one NIC accesses an iSCSI target LUN through
    >> > > two
    >> > > Cisco MDS 9509/IPS paths. I was able to force a failover by
    >> > > disabling the
    >> > > network interface (NIC) on one of the MDS 9509/IPS Directors . My
    >> > > issue
    >> > > on
    >> > > the failover is the failover time. I was getting the following
    >> > > results
    >> > > (with
    >> > > each step about a few minutes apart):
    >> > >
    >> > > 1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled
    >> > > MDS(A) NIC
    >> > > 2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled
    >> > > MDS(B) NIC
    >> > > 3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled
    >> > > MDS(B) NIC
    >> > > 4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled
    >> > > MDS(A) NIC
    >> > > 5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled
    >> > > MDS(A) NIC
    >> > > 6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled
    >> > > MDS(A) NIC
    >> > >
    >> > > So I am getting a failover time of 56, 54, 35, and 35 seconds for the
    >> > > MDS(A)
    >> > > path
    >> > > and
    >> > > a failover time of 35 and 35 for the MDS(B) path.
    >> > >
    >> > > What parameter(s) determine this failover timing and how can I reduce
    >> > > it
    >> > > to
    >> > > 15 seconds.
    >> > >
    >> > > Thank you,
    >> > >
    >> > > Hon
    >> > >
    >> >
    >> >



  6. Re: How to reduce iSCSI MPIO failover time?

    Thank you Kenny,

    I will try the LinkDownTime entry, it is currently set at f (or 15) seconds.

    Thanks again,

    Hon

    "Kenny Speer" wrote:

    > The Disk class timeout value is the most misunderstood timer in windows
    > storage. If you change it, there is a very good chance you will lose
    > support from your storage vendor. If you make this value arbitrarily low,
    > you will cause more problems than you are trying to solve.
    >
    > Again, without specific knowledge of what these timers do, you will most
    > likely create a scenario where your host will no longer survive faults.
    >
    > To point you in the right direction, you need this key:
    > HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Contro l\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\0001\Parameters
    >
    > Although, the 0001 towards the end may be different on your system. This is
    > the SCSI Bus controller class and if you have more than 1 bus, this number
    > changes.
    >
    > The only parameter that you might have luck with is: LinkDownTime, change
    > that lower for your cable pull test.
    >
    >
    > "cdj" wrote in message
    > news:A48920FE-B829-4BEC-A811-B0E64EB0861F@microsoft.com...
    > >I found this article
    > >
    > > http://technet.microsoft.com/en-us/l.../aa997069.aspx
    > >
    > > I will try around with this registry value
    > >
    > > HKEY_LOCAL_MACHINE\System\CurrentControlSet\Servic es\Disk\TimeOutValue
    > >
    > > Thanks,
    > >
    > > "cdj" wrote:
    > >
    > >> I am using Cisco MDS 9509 Directors with IPS module (iSCSI to Fibre
    > >> Channel
    > >> bridge/router) and an EMC Symmetrix F/C storage array. I have the same
    > >> setup
    > >> on a Red Hat Linux server and was able to reduce the failover time to 10
    > >> seconds by changing the iSCSI Initiator parameter ConnFailTimeout to 10
    > >> in
    > >> /etc/iscsi.conf file. This parameter specifies the session timeout value
    > >> before failing the SCSI commands. I was hoping there is a similar
    > >> timeout
    > >> Registry entry for Microsoft's iSCSI Initiator.
    > >>
    > >> Thank you,
    > >>
    > >> Hon
    > >>
    > >> "Kenny Speer" wrote:
    > >>
    > >> > There are a number of ways to reduce this time but all of them depend
    > >> > on
    > >> > your storage provider. You should contact your storage provider and
    > >> > ask
    > >> > them what their recommended settings are.
    > >> >
    > >> > BTW, what target are you using?
    > >> >
    > >> > ~kenny
    > >> >
    > >> > "cdj" wrote in message
    > >> > news:13C472B8-F5A3-4BB4-859A-9EA3DF7F40E2@microsoft.com...
    > >> > >I am running Microsoft iSCSI Initiator 2.0 with MPIO configured. This
    > >> > > Windows 2003 server with one NIC accesses an iSCSI target LUN through
    > >> > > two
    > >> > > Cisco MDS 9509/IPS paths. I was able to force a failover by
    > >> > > disabling the
    > >> > > network interface (NIC) on one of the MDS 9509/IPS Directors . My
    > >> > > issue
    > >> > > on
    > >> > > the failover is the failover time. I was getting the following
    > >> > > results
    > >> > > (with
    > >> > > each step about a few minutes apart):
    > >> > >
    > >> > > 1. Disabled MDS(A) NIC -> I/O stopped for 56 seconds -> Enabled
    > >> > > MDS(A) NIC
    > >> > > 2. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled
    > >> > > MDS(B) NIC
    > >> > > 3. Disabled MDS(B) NIC -> I/O stopped for 35 seconds -> Enabled
    > >> > > MDS(B) NIC
    > >> > > 4. Disabled MDS(A) NIC -> I/O stopped for 54 seconds -> Enabled
    > >> > > MDS(A) NIC
    > >> > > 5. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled
    > >> > > MDS(A) NIC
    > >> > > 6. Disabled MDS(A) NIC -> I/O stopped for 35 seconds -> Enabled
    > >> > > MDS(A) NIC
    > >> > >
    > >> > > So I am getting a failover time of 56, 54, 35, and 35 seconds for the
    > >> > > MDS(A)
    > >> > > path
    > >> > > and
    > >> > > a failover time of 35 and 35 for the MDS(B) path.
    > >> > >
    > >> > > What parameter(s) determine this failover timing and how can I reduce
    > >> > > it
    > >> > > to
    > >> > > 15 seconds.
    > >> > >
    > >> > > Thank you,
    > >> > >
    > >> > > Hon
    > >> > >
    > >> >
    > >> >

    >
    >


  7. Re: How to reduce iSCSI MPIO failover time?

    Looks like Microsoft iSCSI Initiator (iSCSI-2.0-x86fre.exe) add this
    parameter to the registry:

    ADD LinkDownTime=dword:0000000f


    *** Info from McAfee Site Advisor:

    http://www.siteadvisor.com/sites/eas...loads/3051368/

    I will leave it alone.

    Thank you,

  8. Re: How to reduce iSCSI MPIO failover time?

    Right, when you install the initiator, it creates that entire hive. It's a
    new SCSI BUS so it creates all of those parameters.

    That page is just listing what changes were made to the registry (i.e. a
    whole bunch of keys were added).

    Leaving them all alone is recommended unless your storage vendor suggests
    you change them. But if you want to experiment then you can safely lower
    that timeout. I recommend 5 seconds (long enough for someone who
    accidentally pulled the cable to put it back).

    ~kenny

    "cdj" wrote in message
    news:93A20525-929E-406B-9E46-30F9A795C187@microsoft.com...
    > Looks like Microsoft iSCSI Initiator (iSCSI-2.0-x86fre.exe) add this
    > parameter to the registry:
    >
    > ADD LinkDownTime=dword:0000000f
    >
    >
    > *** Info from McAfee Site Advisor:
    >
    > http://www.siteadvisor.com/sites/eas...loads/3051368/
    >
    > I will leave it alone.
    >
    > Thank you,



  9. Re: How to reduce iSCSI MPIO failover time?

    Thank you Kenny,

    I will try it.

    Hon

    "Kenny Speer" wrote:

    > Right, when you install the initiator, it creates that entire hive. It's a
    > new SCSI BUS so it creates all of those parameters.
    >
    > That page is just listing what changes were made to the registry (i.e. a
    > whole bunch of keys were added).
    >
    > Leaving them all alone is recommended unless your storage vendor suggests
    > you change them. But if you want to experiment then you can safely lower
    > that timeout. I recommend 5 seconds (long enough for someone who
    > accidentally pulled the cable to put it back).
    >
    > ~kenny
    >
    > "cdj" wrote in message
    > news:93A20525-929E-406B-9E46-30F9A795C187@microsoft.com...
    > > Looks like Microsoft iSCSI Initiator (iSCSI-2.0-x86fre.exe) add this
    > > parameter to the registry:
    > >
    > > ADD LinkDownTime=dword:0000000f
    > >
    > >
    > > *** Info from McAfee Site Advisor:
    > >
    > > http://www.siteadvisor.com/sites/eas...loads/3051368/
    > >
    > > I will leave it alone.
    > >
    > > Thank you,

    >
    >


+ Reply to Thread