Using 4GB of Memory on EV Server - Veritas

This is a discussion on Using 4GB of Memory on EV Server - Veritas ; Would be interested to know if anyone is using 4GB of memory out there. We do and also use the /3GB switch which I believe is required to ensure that normal user processes can use the memory (without this I ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Using 4GB of Memory on EV Server

  1. Using 4GB of Memory on EV Server


    Would be interested to know if anyone is using 4GB of memory out there.

    We do and also use the /3GB switch which I believe is required to ensure
    that normal user processes can use the memory (without this I believe that
    only 2GB will be available).

    The reason to ask is that I have some random index corruption going on and
    one line of thought (from support) is that they've seen issues on systems
    using this switch.

    Now, I'm dismissing this out of hand so thought I'd ask to see what the general
    EV admin population use; obviously an advisory note from the KVS developers
    would be nice - surely they must have some idea.

    Any more thoughts from anyone?

  2. Re: Using 4GB of Memory on EV Server


    Hi Glenn,

    we have 4 servers with the /3Gb switch and no problems. As you say, it forces
    the kernal into 1Gb istead of 2 (so you get an extra 1Gb addressable RAM).
    If you son't add the switch then you might as well chuck 2Gb of your RAM
    in the bin!

    I can't see anyway this can impact indexing except in a positive way...

    There is one downside to /3Gb, it will halve the abmount of Non-Paged Pool
    memory the server can use (128Mb instead of 256Mb). I've seen some leaks
    in WIndows Paged Pool recently (NTFS, backup and AV software all seem to
    misbehave a bit ) so it's worth monitoring. If it goes over 100 Mb and keeps
    climbing then worry.

    Other boot.ini switches

    /fastdetect - you'll want this
    /Userva=xxxx - according to M$ there should be a userva for all apps but
    KVS doesn't have one. We're undecided about SQL. I think it has one but
    our DBAs aren't worried about it so at the moment we don't use is. Exchange
    2003's is 3030. There is good technet on this (for once!)
    /nopae - fixes some issues with hotswappabhle RAM in the higher thresholds.
    We use it on some servers but I presonally think it's a read herring. Again,
    there is technet advice on this.

    Roll on 64bit Operating Systems!!

    Hope that helps a bit.


    David







  3. Re: Using 4GB of Memory on EV Server


    Thanks David

    This whole issue is deeper than I thought, there is lots of stuff available
    about the 3GB switch and there more I read the less I realise I know about
    memory management. But the definitive article I'll refer anyone to is the
    following one

    http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx

    It's very thorough and to be honest I'm still trying to figure out the real
    answer about the good old 3GB switch. In the end, as recomended by support,
    we booted without the switch and there's been no adverse effects. I should
    have mentioned last time that we use a single server for EV so obviously
    SQL soaks up memory (we limit it to 1500MB) and currently we're running about
    10 archive services (due to migrating from E2k to E2k3). So all in all we
    do need the physical memory but as you'll read above the 3GB limit is all
    about virtual memory. We have followed the MS advice and set the USERVA to
    3030 on our Exchange servers, but in the absence of advice from Veritas we
    didn't set one for our EV server.

    When reading all about the 3GB switch I found an article discussing that
    using it will reduce virtual memory for the kernel with particular impact
    on the System Page Table Entry (PTE). When I checked this on my system it
    was indeed low (in the 2500 range) compared to another 4GB system (without
    the 3Gb switch) that had about 180,000. Obviously a concern here, we had
    already logged a call with MS because we were trying to troubleshoot PerfOS
    system events. Interestingly our E2k3 boxes were also in the region of being
    quite low as well bu this is something I'll follow up with MS.

    An article that discusses System PTE's in more detail (quite a lot actually)
    can be found at the following address, it's also marked up as for internal
    MS use only so might well contain warts information.

    http://www.only4gurus.com/v3/downloadcount.asp?id=8167

    Meanwhile, I'll continue to expand my understanding of memory management
    and wait to see if I get any more index corruption but it's certainly a much
    wider subject than I first thought. I'll keep you updated with my progress.

  4. Re: Using 4GB of Memory on EV Server


    "Glenn Martin" wrote:
    >
    >Thanks David
    >
    >This whole issue is deeper than I thought, there is lots of stuff available
    >about the 3GB switch and there more I read the less I realise I know about
    >memory management. But the definitive article I'll refer anyone to is the
    >following one
    >
    >http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx
    >
    >It's very thorough and to be honest I'm still trying to figure out the real
    >answer about the good old 3GB switch. In the end, as recomended by support,
    >we booted without the switch and there's been no adverse effects. I should
    >have mentioned last time that we use a single server for EV so obviously
    >SQL soaks up memory (we limit it to 1500MB) and currently we're running

    about
    >10 archive services (due to migrating from E2k to E2k3). So all in all we
    >do need the physical memory but as you'll read above the 3GB limit is all
    >about virtual memory. We have followed the MS advice and set the USERVA

    to
    >3030 on our Exchange servers, but in the absence of advice from Veritas

    we
    >didn't set one for our EV server.


    I've run into issues trying to install the EV OWA extensions on our back
    end exchange servers. All of our exchange servers (20 servers in total in
    one site) all use the /3gb switch. In dev, when I modify the boot.ini file
    to use the /maxmem 1024 switch, the EV OWA extensions installed without a
    problem. The error I was getting was setup initialization error (error:111).
    After some investigation, I found a potential InstallShield bug http://community.installshield.com/a...?t-115911.html


    I've got a case open with support so lets see what they come back with.

  5. Re: Using 4GB of Memory on EV Server


    Glenn,

    you know what? I took a peek at my Free FTEs and nearly fell out of my chair!
    5,000 and falling fast...!! My NT boxes have 40,000 of these suckers (and
    M$ says an Exchange system is "unstable" (wha?) below 7,000....)

    I've taken the /3Gb switch off one of the boxes. It'll be slower but it
    might be more stable. Free FTEs @ 17-20,000 now. Interestingly NPP rose
    above the previous 128 max as well... I'll try to remember to update once
    I know for sure whether its the right way to go.

    Doesn't look like w2k3 scales quite properly does it?


    David


    "Glenn Martin" wrote:
    >
    >Thanks David
    >
    >This whole issue is deeper than I thought, there is lots of stuff available
    >about the 3GB switch and there more I read the less I realise I know about
    >memory management. But the definitive article I'll refer anyone to is the
    >following one
    >
    >http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx
    >
    >It's very thorough and to be honest I'm still trying to figure out the real
    >answer about the good old 3GB switch. In the end, as recomended by support,
    >we booted without the switch and there's been no adverse effects. I should
    >have mentioned last time that we use a single server for EV so obviously
    >SQL soaks up memory (we limit it to 1500MB) and currently we're running

    about
    >10 archive services (due to migrating from E2k to E2k3). So all in all we
    >do need the physical memory but as you'll read above the 3GB limit is all
    >about virtual memory. We have followed the MS advice and set the USERVA

    to
    >3030 on our Exchange servers, but in the absence of advice from Veritas

    we
    >didn't set one for our EV server.
    >
    >When reading all about the 3GB switch I found an article discussing that
    >using it will reduce virtual memory for the kernel with particular impact
    >on the System Page Table Entry (PTE). When I checked this on my system

    it
    >was indeed low (in the 2500 range) compared to another 4GB system (without
    >the 3Gb switch) that had about 180,000. Obviously a concern here, we had
    >already logged a call with MS because we were trying to troubleshoot PerfOS
    >system events. Interestingly our E2k3 boxes were also in the region of being
    >quite low as well bu this is something I'll follow up with MS.
    >
    >An article that discusses System PTE's in more detail (quite a lot actually)
    >can be found at the following address, it's also marked up as for internal
    >MS use only so might well contain warts information.
    >
    >http://www.only4gurus.com/v3/downloadcount.asp?id=8167
    >
    >Meanwhile, I'll continue to expand my understanding of memory management
    >and wait to see if I get any more index corruption but it's certainly a

    much
    >wider subject than I first thought. I'll keep you updated with my progress.



  6. Re: Using 4GB of Memory on EV Server


    Baldish,

    what EXC version? What O/S? have you set /USERVA 3030?

    Messy


    "Baldish" wrote:
    >
    >"Glenn Martin" wrote:
    >>
    >>Thanks David
    >>
    >>This whole issue is deeper than I thought, there is lots of stuff available
    >>about the 3GB switch and there more I read the less I realise I know about
    >>memory management. But the definitive article I'll refer anyone to is the
    >>following one
    >>
    >>http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx
    >>
    >>It's very thorough and to be honest I'm still trying to figure out the

    real
    >>answer about the good old 3GB switch. In the end, as recomended by support,
    >>we booted without the switch and there's been no adverse effects. I should
    >>have mentioned last time that we use a single server for EV so obviously
    >>SQL soaks up memory (we limit it to 1500MB) and currently we're running

    >about
    >>10 archive services (due to migrating from E2k to E2k3). So all in all

    we
    >>do need the physical memory but as you'll read above the 3GB limit is all
    >>about virtual memory. We have followed the MS advice and set the USERVA

    >to
    >>3030 on our Exchange servers, but in the absence of advice from Veritas

    >we
    >>didn't set one for our EV server.

    >
    >I've run into issues trying to install the EV OWA extensions on our back
    >end exchange servers. All of our exchange servers (20 servers in total

    in
    >one site) all use the /3gb switch. In dev, when I modify the boot.ini

    file
    >to use the /maxmem 1024 switch, the EV OWA extensions installed without

    a
    >problem. The error I was getting was setup initialization error (error:111).
    > After some investigation, I found a potential InstallShield bug http://community.installshield.com/a...?t-115911.html
    >
    >
    >I've got a case open with support so lets see what they come back with.



  7. Re: Using 4GB of Memory on EV Server


    "Messy" <1@2.com> wrote:
    >
    >Baldish,
    >
    >what EXC version? What O/S? have you set /USERVA 3030?
    >
    >Messy
    >


    I ran the setup.exe from the command line with the -z switch and that ignores
    the memory check so it worked beautifully. This was the recommendation from
    support so I'm assuming that this was a known issue and set up is not impacted
    in any way.

  8. Re: Using 4GB of Memory on EV Server


    It is indeed a deeply interesting area, we're still having stablility issues
    but I now think that having a system running with SQL,14 arc/ret services
    isn't a recipe for stability but perhaps the solution is hold on till next
    year when it will be the year that 64 bit servers finally arrive; or will
    it be the start of a different set of problems.?



    "David" <1@2.com> wrote:
    >
    >Glenn,
    >
    >you know what? I took a peek at my Free FTEs and nearly fell out of my

    chair!
    > 5,000 and falling fast...!! My NT boxes have 40,000 of these suckers (and
    >M$ says an Exchange system is "unstable" (wha?) below 7,000....)
    >
    >I've taken the /3Gb switch off one of the boxes. It'll be slower but it
    >might be more stable. Free FTEs @ 17-20,000 now. Interestingly NPP rose
    >above the previous 128 max as well... I'll try to remember to update once
    >I know for sure whether its the right way to go.
    >
    >Doesn't look like w2k3 scales quite properly does it?
    >
    >
    >David
    >
    >
    >"Glenn Martin" wrote:
    >>
    >>Thanks David
    >>
    >>This whole issue is deeper than I thought, there is lots of stuff available
    >>about the 3GB switch and there more I read the less I realise I know about
    >>memory management. But the definitive article I'll refer anyone to is the
    >>following one
    >>
    >>http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx
    >>
    >>It's very thorough and to be honest I'm still trying to figure out the

    real
    >>answer about the good old 3GB switch. In the end, as recomended by support,
    >>we booted without the switch and there's been no adverse effects. I should
    >>have mentioned last time that we use a single server for EV so obviously
    >>SQL soaks up memory (we limit it to 1500MB) and currently we're running

    >about
    >>10 archive services (due to migrating from E2k to E2k3). So all in all

    we
    >>do need the physical memory but as you'll read above the 3GB limit is all
    >>about virtual memory. We have followed the MS advice and set the USERVA

    >to
    >>3030 on our Exchange servers, but in the absence of advice from Veritas

    >we
    >>didn't set one for our EV server.
    >>
    >>When reading all about the 3GB switch I found an article discussing that
    >>using it will reduce virtual memory for the kernel with particular impact
    >>on the System Page Table Entry (PTE). When I checked this on my system

    >it
    >>was indeed low (in the 2500 range) compared to another 4GB system (without
    >>the 3Gb switch) that had about 180,000. Obviously a concern here, we had
    >>already logged a call with MS because we were trying to troubleshoot PerfOS
    >>system events. Interestingly our E2k3 boxes were also in the region of

    being
    >>quite low as well bu this is something I'll follow up with MS.
    >>
    >>An article that discusses System PTE's in more detail (quite a lot actually)
    >>can be found at the following address, it's also marked up as for internal
    >>MS use only so might well contain warts information.
    >>
    >>http://www.only4gurus.com/v3/downloadcount.asp?id=8167
    >>
    >>Meanwhile, I'll continue to expand my understanding of memory management
    >>and wait to see if I get any more index corruption but it's certainly a

    >much
    >>wider subject than I first thought. I'll keep you updated with my progress.

    >



  9. Re: Using 4GB of Memory on EV Server


    For me, the same rules apply:

    * Never use M$ version 1.0 of anything!! Inclusing 64bit O/Ses
    * Never install a SP from M$ if it's an even number

    ...I don't think Win2k or Win2k3 scale like they promised. The /3Gb is exactly
    what it looks like - a fudge - and M$ can't admit is 'cos they'd lose ground
    to the mighty Penguin...

    Mind you, I'm a Conspiricy Theorist.

    Is there a UNIX version of EV on the way I wonder


    "Glenn" wrote:
    >
    >It is indeed a deeply interesting area, we're still having stablility issues
    >but I now think that having a system running with SQL,14 arc/ret services
    >isn't a recipe for stability but perhaps the solution is hold on till next
    >year when it will be the year that 64 bit servers finally arrive; or will
    >it be the start of a different set of problems.?
    >
    >
    >
    >"David" <1@2.com> wrote:
    >>
    >>Glenn,
    >>
    >>you know what? I took a peek at my Free FTEs and nearly fell out of my

    >chair!
    >> 5,000 and falling fast...!! My NT boxes have 40,000 of these suckers

    (and
    >>M$ says an Exchange system is "unstable" (wha?) below 7,000....)
    >>
    >>I've taken the /3Gb switch off one of the boxes. It'll be slower but it
    >>might be more stable. Free FTEs @ 17-20,000 now. Interestingly NPP rose
    >>above the previous 128 max as well... I'll try to remember to update once
    >>I know for sure whether its the right way to go.
    >>
    >>Doesn't look like w2k3 scales quite properly does it?
    >>
    >>
    >>David
    >>
    >>
    >>"Glenn Martin" wrote:
    >>>
    >>>Thanks David
    >>>
    >>>This whole issue is deeper than I thought, there is lots of stuff available
    >>>about the 3GB switch and there more I read the less I realise I know about
    >>>memory management. But the definitive article I'll refer anyone to is

    the
    >>>following one
    >>>
    >>>http://blogs.msdn.com/oldnewthing/ar...22/218527.aspx
    >>>
    >>>It's very thorough and to be honest I'm still trying to figure out the

    >real
    >>>answer about the good old 3GB switch. In the end, as recomended by support,
    >>>we booted without the switch and there's been no adverse effects. I should
    >>>have mentioned last time that we use a single server for EV so obviously
    >>>SQL soaks up memory (we limit it to 1500MB) and currently we're running

    >>about
    >>>10 archive services (due to migrating from E2k to E2k3). So all in all

    >we
    >>>do need the physical memory but as you'll read above the 3GB limit is

    all
    >>>about virtual memory. We have followed the MS advice and set the USERVA

    >>to
    >>>3030 on our Exchange servers, but in the absence of advice from Veritas

    >>we
    >>>didn't set one for our EV server.
    >>>
    >>>When reading all about the 3GB switch I found an article discussing that
    >>>using it will reduce virtual memory for the kernel with particular impact
    >>>on the System Page Table Entry (PTE). When I checked this on my system

    >>it
    >>>was indeed low (in the 2500 range) compared to another 4GB system (without
    >>>the 3Gb switch) that had about 180,000. Obviously a concern here, we had
    >>>already logged a call with MS because we were trying to troubleshoot PerfOS
    >>>system events. Interestingly our E2k3 boxes were also in the region of

    >being
    >>>quite low as well bu this is something I'll follow up with MS.
    >>>
    >>>An article that discusses System PTE's in more detail (quite a lot actually)
    >>>can be found at the following address, it's also marked up as for internal
    >>>MS use only so might well contain warts information.
    >>>
    >>>http://www.only4gurus.com/v3/downloadcount.asp?id=8167
    >>>
    >>>Meanwhile, I'll continue to expand my understanding of memory management
    >>>and wait to see if I get any more index corruption but it's certainly

    a
    >>much
    >>>wider subject than I first thought. I'll keep you updated with my progress.

    >>

    >



+ Reply to Thread