Having trouble identifying bottleneck - Storage

This is a discussion on Having trouble identifying bottleneck - Storage ; ello, Basically, my users are complaining about the speed of SMB (they're devs, but don't use a DB, but flat-files). There's definitely an issue of over utilization, but I am having trouble identifying where the bottleneck resides. I'm attempting to ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Having trouble identifying bottleneck

  1. Having trouble identifying bottleneck

    ello,

    Basically, my users are complaining about the speed of SMB (they're
    devs, but don't use a DB, but flat-files). There's definitely an
    issue of over utilization, but I am having trouble identifying where
    the bottleneck resides.

    I'm attempting to follow this guide on finding bottlenecks, but there
    are some very strange issues with paging and memory:
    http://www.computerperformance.co.uk...isk_Health.htm

    I then found this page from Microsoft, but it isn't very helpful:
    http://technet.microsoft.com/en-us/l.../cc779038.aspx

    Here are some results from a full days worth logging:

    HDD Testing

    Counter Average/Min/Max
    PhysicalDisk: Avg. Disk Read Queue Length .235/0/16.298
    PhysicalDisk: Avg. Disk Write Queue Length .182/0/19.967
    PhysicalDisk: % Disk Time 41.731/0/2009.201

    Memory Systems (RAM, VMM, Cache) Testing

    Counter Average/Min/Max
    Memory: Available MBytes 2519/2044/2593
    Memory: Pages/sec 190.427/0/5850.220
    Memory: Page Faults/sec 972.779/11.4/15263.598
    Memory: Committed Bytes .664805579GB/.643325806GB/.78294754GB
    PageFile: % Usage 1.923/1.766/2.495
    Cache: Data Map Hits % 99.832/75.747/100

    Processor Testing

    Counter Average/Min/Max
    Processor: % Processor Time 2.349/0/14.062
    System: Processor Queue Length 0/0/1
    Processor: DPCs Queue/sec 89/0/679
    Processor: Interrupts/sec 7733.899/284.409/40280.277
    Processor: % Privileged Time 389.572/309.994/400.084
    Processor: %DPC Time ???

    Network Testing
    Counter Average/Min/Max
    Network: Bytes Total/sec 2385302/913.629/67946320
    Current Bandwidth 134217728 bytes/sec (1Gbps)

    http://support.microsoft.com/kb/139609 states that Memory: Pages/sec
    may be high due to "an application sequentially reading a memory
    mapped file." This is the method that the devs use to read files,
    usually. If this is occurring over SMB, is it safe to assume that the
    RAM on the LANManServer is being utilized, or is it just the
    LANManClient?

    I'm obviously a little overwhelmed and lost. Any help is greatly
    appreciated.

    Thanks!

    Matt Brown

  2. Re: Having trouble identifying bottleneck

    LogicalDisk:% Disk Time and PhysicalDisk:% Disk Time

    This is the percentage of time the disk drive is busy with read and write
    requests. Note that these two counters are selected on an Instance basis
    (physical hard disk for PhysicalDisk and partitions for LogicalDisk). It is
    important this counter value remain low (under 50 percent could be
    considered an acceptable range).

    > HDD Testing
    >
    > Counter Average/Min/Max
    > PhysicalDisk: Avg. Disk Read Queue Length .235/0/16.298
    > PhysicalDisk: Avg. Disk Write Queue Length .182/0/19.967
    > PhysicalDisk: % Disk Time 41.731/0/2009.201


    Max of 2009.201 seems off.

    Try looking at the physical disk counters trasnfers/sec and sec/transfer.
    They'll tell you a lot more about disk bottlenecks than queues will.

    > Memory: Pages/sec 190.427/0/5850.220


    memory mapped IO does add to the pages/sec on the host where the memory
    mapped IO occurs (the client). To see max 5850 pages/sec on the server is
    odd. Look at the physical disk counters for the drive where the page file
    is located to sort it out.



    "mbrownnyc" wrote in message
    news:EFC3DD29-3C40-48C8-A6FA-CAE64A1EF7C7@microsoft.com...
    > ello,
    >
    > Basically, my users are complaining about the speed of SMB (they're
    > devs, but don't use a DB, but flat-files). There's definitely an
    > issue of over utilization, but I am having trouble identifying where
    > the bottleneck resides.
    >
    > I'm attempting to follow this guide on finding bottlenecks, but there
    > are some very strange issues with paging and memory:
    > http://www.computerperformance.co.uk...isk_Health.htm
    >
    > I then found this page from Microsoft, but it isn't very helpful:
    > http://technet.microsoft.com/en-us/l.../cc779038.aspx
    >
    > Here are some results from a full days worth logging:
    >
    > HDD Testing
    >
    > Counter Average/Min/Max
    > PhysicalDisk: Avg. Disk Read Queue Length .235/0/16.298
    > PhysicalDisk: Avg. Disk Write Queue Length .182/0/19.967
    > PhysicalDisk: % Disk Time 41.731/0/2009.201
    >
    > Memory Systems (RAM, VMM, Cache) Testing
    >
    > Counter Average/Min/Max
    > Memory: Available MBytes 2519/2044/2593
    > Memory: Pages/sec 190.427/0/5850.220
    > Memory: Page Faults/sec 972.779/11.4/15263.598
    > Memory: Committed Bytes .664805579GB/.643325806GB/.78294754GB
    > PageFile: % Usage 1.923/1.766/2.495
    > Cache: Data Map Hits % 99.832/75.747/100
    >
    > Processor Testing
    >
    > Counter Average/Min/Max
    > Processor: % Processor Time 2.349/0/14.062
    > System: Processor Queue Length 0/0/1
    > Processor: DPCs Queue/sec 89/0/679
    > Processor: Interrupts/sec 7733.899/284.409/40280.277
    > Processor: % Privileged Time 389.572/309.994/400.084
    > Processor: %DPC Time ???
    >
    > Network Testing
    > Counter Average/Min/Max
    > Network: Bytes Total/sec 2385302/913.629/67946320
    > Current Bandwidth 134217728 bytes/sec (1Gbps)
    >
    > http://support.microsoft.com/kb/139609 states that Memory: Pages/sec
    > may be high due to "an application sequentially reading a memory
    > mapped file." This is the method that the devs use to read files,
    > usually. If this is occurring over SMB, is it safe to assume that the
    > RAM on the LANManServer is being utilized, or is it just the
    > LANManClient?
    >
    > I'm obviously a little overwhelmed and lost. Any help is greatly
    > appreciated.
    >
    > Thanks!
    >
    > Matt Brown




  3. Re: Having trouble identifying bottleneck

    Thanks for the reply John!

    Note that the server is a RAID 5 with four drives, one logical drive
    with two partitions.

    I'll take a look at the:
    PhysicalDisk\Disk Transfers/sec
    PhysicalDisk\Avg. Disk sec/Transfer


    Which Physical Disk counters should I look at to analyze the page
    file, specifically?

    Any more tips you can give me?


    Thanks!

    Matt

    On Nov 8, 8:11*pm, "John Fullbright"
    wrote:
    > LogicalDisk:% Disk Time and PhysicalDisk:% Disk Time
    >
    > This is the percentage of time the disk drive is busy with read and write
    > requests. Note that these two counters are selected on an Instance basis
    > (physical hard disk for PhysicalDisk and partitions for LogicalDisk). It is
    > important this counter value remain low (under 50 percent could be
    > considered an acceptable range).
    >
    > > HDD Testing

    >
    > > * * * *Counter Average/Min/Max
    > > * * * *PhysicalDisk: Avg. Disk Read Queue Length * * * .235/0/16.298
    > > * * * *PhysicalDisk: Avg. Disk Write Queue Length * * *.182/0/19.967
    > > * * * *PhysicalDisk: % Disk Time * * * 41.731/0/2009.201

    >
    > Max of 2009.201 seems off.
    >
    > Try looking at the physical disk counters trasnfers/sec and sec/transfer.
    > They'll tell you a lot more about disk bottlenecks than queues will.
    >
    > > * * * *Memory: Pages/sec * * * 190.427/0/5850.220

    >
    > memory mapped IO does add to the pages/sec on the host where the memory
    > mapped IO occurs (the client). *To see max 5850 pages/sec on the serveris
    > odd. *Look at the physical disk counters for the drive where the page file
    > is located to sort it out.
    >
    > "mbrownnyc" wrote in message
    >
    > news:EFC3DD29-3C40-48C8-A6FA-CAE64A1EF7C7@microsoft.com...
    >
    > > ello,

    >
    > > Basically, my users are complaining about the speed of SMB (they're
    > > devs, but don't use a DB, but flat-files). *There's definitely an
    > > issue of over utilization, but I am having trouble identifying where
    > > the bottleneck resides.

    >
    > > I'm attempting to follow this guide on finding bottlenecks, but there
    > > are some very strange issues with paging and memory:
    > >http://www.computerperformance.co.uk...isk_Health.htm

    >
    > > I then found this page from Microsoft, but it isn't very helpful:
    > >http://technet.microsoft.com/en-us/l.../cc779038.aspx

    >
    > > Here are some results from a full days worth logging:

    >
    > > HDD Testing

    >
    > > * * * *Counter Average/Min/Max
    > > * * * *PhysicalDisk: Avg. Disk Read Queue Length * * * .235/0/16.298
    > > * * * *PhysicalDisk: Avg. Disk Write Queue Length * * *.182/0/19.967
    > > * * * *PhysicalDisk: % Disk Time * * * 41.731/0/2009.201

    >
    > > Memory Systems (RAM, VMM, Cache) Testing

    >
    > > * * * *Counter Average/Min/Max
    > > * * * *Memory: Available MBytes * * * *2519/2044/2593
    > > * * * *Memory: Pages/sec * * * 190.427/0/5850.220
    > > * * * *Memory: Page Faults/sec 972.779/11.4/15263.598
    > > * * * *Memory: Committed Bytes .664805579GB/.643325806GB/.78294754GB
    > > * * * *PageFile: % Usage * * * 1.923/1.766/2.495
    > > * * * *Cache: Data Map Hits % *99.832/75.747/100

    >
    > > Processor Testing

    >
    > > * * * *Counter Average/Min/Max
    > > * * * *Processor: % Processor Time * * 2.349/0/14.062
    > > * * * *System: Processor Queue Length *0/0/1
    > > * * * *Processor: DPCs Queue/sec * * * 89/0/679
    > > * * * *Processor: Interrupts/sec * * * 7733.899/284.409/40280.277
    > > * * * *Processor: % Privileged Time * *389.572/309.994/400.084
    > > * * * *Processor: %DPC Time * *???

    >
    > > Network Testing
    > > * * * *Counter Average/Min/Max
    > > * * * *Network: Bytes Total/sec * * * *2385302/913.629/67946320
    > > * * * *Current Bandwidth * * * 134217728 bytes/sec (1Gbps)

    >
    > >http://support.microsoft.com/kb/139609states that Memory: Pages/sec
    > > may be high due to "an application sequentially reading a memory
    > > mapped file." *This is the method that the devs use to read files,
    > > usually. *If this is occurring over SMB, is it safe to assume that the
    > > RAM on the LANManServer is being utilized, or is it just the
    > > LANManClient?

    >
    > > I'm obviously a little overwhelmed and lost. *Any help is greatly
    > > appreciated.

    >
    > > Thanks!

    >
    > > Matt Brown



  4. Re: Having trouble identifying bottleneck

    transfers/sec and sec/transfer.

    The page file resides on the boot partition unless you moved it.

    Generally, RAID 5 is a good fit for data that is read a lot more than it is
    written and that would include most file servers. If you want to make sure,
    you could gather reads/sec as well and calculate your read/write ratio. The
    basic rule of thumb is that if the write penalty of your RAID type is higher
    than the read/write ratio of your application, then it's a poor choice.




    "Matt Brown - nyc" wrote in message
    news:a764ea87-a566-4247-9018-5fa070d95e12@c22g2000prc.googlegroups.com...
    Thanks for the reply John!

    Note that the server is a RAID 5 with four drives, one logical drive
    with two partitions.

    I'll take a look at the:
    PhysicalDisk\Disk Transfers/sec
    PhysicalDisk\Avg. Disk sec/Transfer


    Which Physical Disk counters should I look at to analyze the page
    file, specifically?

    Any more tips you can give me?


    Thanks!

    Matt

    On Nov 8, 8:11 pm, "John Fullbright"
    wrote:
    > LogicalDisk:% Disk Time and PhysicalDisk:% Disk Time
    >
    > This is the percentage of time the disk drive is busy with read and write
    > requests. Note that these two counters are selected on an Instance basis
    > (physical hard disk for PhysicalDisk and partitions for LogicalDisk). It
    > is
    > important this counter value remain low (under 50 percent could be
    > considered an acceptable range).
    >
    > > HDD Testing

    >
    > > Counter Average/Min/Max
    > > PhysicalDisk: Avg. Disk Read Queue Length .235/0/16.298
    > > PhysicalDisk: Avg. Disk Write Queue Length .182/0/19.967
    > > PhysicalDisk: % Disk Time 41.731/0/2009.201

    >
    > Max of 2009.201 seems off.
    >
    > Try looking at the physical disk counters trasnfers/sec and sec/transfer.
    > They'll tell you a lot more about disk bottlenecks than queues will.
    >
    > > Memory: Pages/sec 190.427/0/5850.220

    >
    > memory mapped IO does add to the pages/sec on the host where the memory
    > mapped IO occurs (the client). To see max 5850 pages/sec on the server is
    > odd. Look at the physical disk counters for the drive where the page file
    > is located to sort it out.
    >
    > "mbrownnyc" wrote in message
    >
    > news:EFC3DD29-3C40-48C8-A6FA-CAE64A1EF7C7@microsoft.com...
    >
    > > ello,

    >
    > > Basically, my users are complaining about the speed of SMB (they're
    > > devs, but don't use a DB, but flat-files). There's definitely an
    > > issue of over utilization, but I am having trouble identifying where
    > > the bottleneck resides.

    >
    > > I'm attempting to follow this guide on finding bottlenecks, but there
    > > are some very strange issues with paging and memory:
    > >http://www.computerperformance.co.uk...isk_Health.htm

    >
    > > I then found this page from Microsoft, but it isn't very helpful:
    > >http://technet.microsoft.com/en-us/l.../cc779038.aspx

    >
    > > Here are some results from a full days worth logging:

    >
    > > HDD Testing

    >
    > > Counter Average/Min/Max
    > > PhysicalDisk: Avg. Disk Read Queue Length .235/0/16.298
    > > PhysicalDisk: Avg. Disk Write Queue Length .182/0/19.967
    > > PhysicalDisk: % Disk Time 41.731/0/2009.201

    >
    > > Memory Systems (RAM, VMM, Cache) Testing

    >
    > > Counter Average/Min/Max
    > > Memory: Available MBytes 2519/2044/2593
    > > Memory: Pages/sec 190.427/0/5850.220
    > > Memory: Page Faults/sec 972.779/11.4/15263.598
    > > Memory: Committed Bytes .664805579GB/.643325806GB/.78294754GB
    > > PageFile: % Usage 1.923/1.766/2.495
    > > Cache: Data Map Hits % 99.832/75.747/100

    >
    > > Processor Testing

    >
    > > Counter Average/Min/Max
    > > Processor: % Processor Time 2.349/0/14.062
    > > System: Processor Queue Length 0/0/1
    > > Processor: DPCs Queue/sec 89/0/679
    > > Processor: Interrupts/sec 7733.899/284.409/40280.277
    > > Processor: % Privileged Time 389.572/309.994/400.084
    > > Processor: %DPC Time ???

    >
    > > Network Testing
    > > Counter Average/Min/Max
    > > Network: Bytes Total/sec 2385302/913.629/67946320
    > > Current Bandwidth 134217728 bytes/sec (1Gbps)

    >
    > >http://support.microsoft.com/kb/139609states that Memory: Pages/sec
    > > may be high due to "an application sequentially reading a memory
    > > mapped file." This is the method that the devs use to read files,
    > > usually. If this is occurring over SMB, is it safe to assume that the
    > > RAM on the LANManServer is being utilized, or is it just the
    > > LANManClient?

    >
    > > I'm obviously a little overwhelmed and lost. Any help is greatly
    > > appreciated.

    >
    > > Thanks!

    >
    > > Matt Brown




+ Reply to Thread