dedicating task to specific cpu - SGI

This is a discussion on dedicating task to specific cpu - SGI ; Hi, Theoretical question about using an origin 3200 as a file server. We have a 4 cpu origin 3200 and we're beating the crap out of it, or at least we try. We are running 64 nfs processes, 128 samba. ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: dedicating task to specific cpu

  1. dedicating task to specific cpu

    Hi,

    Theoretical question about using an origin 3200 as a file server.

    We have a 4 cpu origin 3200 and we're beating the crap out of it, or
    at least we try. We are running 64 nfs processes, 128 samba. We have 4 x
    2GB hba card to the raid and 4 x tgn gigabit card for our 4 different vlans.

    My question is, would I gain in performance if I assign one cpu for
    samba, one for nfs, one for the hbas and one for the tgns?

    Because somehow, I find that I'm not getting enough throughput from
    my hba has I think I should :-)

    Cheers,

    Seb


  2. Re: dedicating task to specific cpu

    In article ,
    Sebastien Charland wrote:
    : Theoretical question about using an origin 3200 as a file server.

    : We have a 4 cpu origin 3200 and we're beating the crap out of it, or
    :at least we try. We are running 64 nfs processes, 128 samba. We have 4 x
    :2GB hba card to the raid and 4 x tgn gigabit card for our 4 different vlans.

    : My question is, would I gain in performance if I assign one cpu for
    :samba, one for nfs, one for the hbas and one for the tgns?

    : Because somehow, I find that I'm not getting enough throughput from
    :my hba has I think I should :-)

    First you need to measure whether you are cpu bound or memory bound
    or I/O bound.

    If you were to assign those processes to particular CPUs and isolated
    the CPUs, then you would have the theoretical possibility of faster
    interrupt handling, but you would also lose the ability of having
    different instances of the processes handled by different CPUs.

    And if I understand correctly, you wouldn't get faster interrupt
    handling unless you effectively control all the interrupts
    for a particular device; unless you want to get into writing your
    own device driver, you aren't going to be controlling all the
    interrupts to your RAID device as that's a regular filesystem.

    Look at your sar output and figure out the I/O access pattern.
    If you turn out to have a lot of large I/O requests, then increasing
    the XFS block size on the RAID filesystem might help. If you have
    a lot of very small I/O requests, decreasing the XFS block size
    might help.

    Are you using nfs3 ? You might be able to adjust the nfs parameters.
    nfs3 is a lot more efficient than nfs2. If you are running nfs2 then
    you aren't going to be able to get even close to full performance.


    Is this general-purpose file serving, or is a lot of the work relatively
    special purpose (e.g., serving up large data sets for rendering)? If
    it is special purpose, then -possibly- you could improve performance
    by tricks such as having a server-side process that read in the
    entire dataset so as to have the effect of placing it into filesystem
    cache ready for the nfs process to read it off.

    Have you installed lots of memory and tuned your filesystem cache
    parameters?
    --
    History is a pile of debris -- Laurie Anderson

  3. Re: dedicating task to specific cpu

    Sebastien Charland wrote:

    > Hi,
    >
    > Theoretical question about using an origin 3200 as a file server.
    >
    > We have a 4 cpu origin 3200 and we're beating the crap out of it, or
    > at least we try. We are running 64 nfs processes, 128 samba. We have 4 x
    > 2GB hba card to the raid and 4 x tgn gigabit card for our 4 different
    > vlans.
    >
    > My question is, would I gain in performance if I assign one cpu for
    > samba, one for nfs, one for the hbas and one for the tgns?


    No.
    >
    > Because somehow, I find that I'm not getting enough throughput from my
    > hba has I think I should :-)


    Ah -- I/O tuning? It usually takes me a day or two at any customer site
    .

    Rule #1: if they're not Tigon3 boards, NEVER put them in the same PCI
    bus as the HBAs. Even if they're Tigon3s, segregate the types of
    cards if you can.

    Rule #2: measure I/O patterns. PCP (or lacking that sar and par -k) are
    your friends. So are diskperf, lmdd and sio so you can characterize the
    RAID without Samba and NFS.

    Rule #3: Make sure the I/Os are aligned wrt the RAID layout -- esp. when
    you use 7+1 or 3+1 LUNs!

    Rule #4: Don't forget CTQ on any of the LUNs...




    --
    Alexis Cousein Senior Systems Engineer
    alexis@sgi.com SGI/Silicon Graphics Brussels




  4. Re: dedicating task to specific cpu

    Sebastien Charland wrote:
    > We have a 4 cpu origin 3200 and we're beating the crap out of it, or
    > at least we try. We are running 64 nfs processes, 128 samba. We have 4 x
    > 2GB hba card to the raid and 4 x tgn gigabit card for our 4 different

    vlans.

    Just a note: A long time ago we ran into trouble with our Origin 200 because
    we had too many nfs daemons running. You might consider reducing the number.

    greetings,



  5. Re: dedicating task to specific cpu

    Tarjei T. Jensen wrote:

    > Sebastien Charland wrote:
    >
    >> We have a 4 cpu origin 3200 and we're beating the crap out of it, or
    >>at least we try. We are running 64 nfs processes, 128 samba. We have 4 x
    >>2GB hba card to the raid and 4 x tgn gigabit card for our 4 different

    >
    > vlans.
    >
    > Just a note: A long time ago we ran into trouble with our Origin 200 because
    > we had too many nfs daemons running. You might consider reducing the number.
    >

    Bad advice *for him*, though it may have been good *for you*. Depends on
    whether you want your O200 to do things beside serving NFS/SMB.


    --
    Alexis Cousein Senior Systems Engineer
    alexis@sgi.com SGI/Silicon Graphics Brussels




  6. Re: dedicating task to specific cpu

    We did some internal tests recently that suggest that restricting the
    cpu with the tgns can definitely help out in some situations. In
    regards to your earlier QoS issue, have you considered giving the
    daemons different "nice" values?
    Michael

    Sebastien Charland wrote:
    > My question is, would I gain in performance if I assign one cpu for
    > samba, one for nfs, one for the hbas and one for the tgns?


    --
    Michael A. Raymond
    Core OS Scheduling Group Real-Time Lead

  7. Re: dedicating task to specific cpu

    Yes, we do a renice of all samba deamons every 10 minutes or so. That
    helps a bit, but not enough to counter act the effect of the renderfarm.

    Thanks,

    Seb

    Michael Raymond wrote:
    > We did some internal tests recently that suggest that restricting the
    > cpu with the tgns can definitely help out in some situations. In
    > regards to your earlier QoS issue, have you considered giving the
    > daemons different "nice" values?
    > Michael
    >
    > Sebastien Charland wrote:
    >
    >> My question is, would I gain in performance if I assign one cpu for
    >>samba, one for nfs, one for the hbas and one for the tgns?

    >
    >



+ Reply to Thread