Sharing a 3510 -- NFS vs. QFS? - SUN

This is a discussion on Sharing a 3510 -- NFS vs. QFS? - SUN ; Hello! We need to share a file-system residing on a StorageTek 3510 device between multiple servers (3 to 5). The plan was to make the LUN "native" to one of the server (server0) and export it to the rest via ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: Sharing a 3510 -- NFS vs. QFS?

  1. Sharing a 3510 -- NFS vs. QFS?

    Hello!

    We need to share a file-system residing on a StorageTek 3510 device between
    multiple servers (3 to 5).

    The plan was to make the LUN "native" to one of the server (server0) and
    export it to the rest via NFS.

    However, it appears, the same LUN can be made "native" to multiple servers
    using the QFS solution:

    http://www.sun.com/storagetek/manage...anagement/qfs/

    This may help performance (as data will travel to each consumer directly via
    fiber, instead of via fiber to the NFS-server, and then via CAT5 to the
    NFS-clients).

    But is it possible at all? I don't quite understand, how it work -- is 3510
    aware of file-system on it? If not, how will take care of the locking and
    other attempts by multiple servers to access the same data at the same
    time?

    Is QFS considered reliable? Is it worth the trouble of learning it for our
    case of very few servers?

    Can QFS be used to keep two 3510s in sync over LAN (not fiber)?

    Thanks for any advice. Yours,

    -mi

  2. Re: Sharing a 3510 -- NFS vs. QFS?

    On Sun, 03 Jun 2007 20:25:42 -0400, Mikhail Teterin
    wrote:
    > Hello!
    >
    > However, it appears, the same LUN can be made "native" to multiple servers
    > using the QFS solution:
    >
    > http://www.sun.com/storagetek/manage...anagement/qfs/
    >
    > This may help performance (as data will travel to each consumer directly via
    > fiber, instead of via fiber to the NFS-server, and then via CAT5 to the
    > NFS-clients).
    >
    > But is it possible at all? I don't quite understand, how it work -- is 3510
    > aware of file-system on it? If not, how will take care of the locking and
    > other attempts by multiple servers to access the same data at the same
    > time?


    QFS has to have an ethernet connection between the servers, as well as
    them all having fiber access to the disks. One of the serves will be a
    metadata server - all file access require a metadata call to the metadata
    server over the ethernet, then direct fiber access to the disk.
    >
    > Is QFS considered reliable? Is it worth the trouble of learning it for our
    > case of very few servers?


    As far as I know - it's designed for very large filesystems.
    >
    > Can QFS be used to keep two 3510s in sync over LAN (not fiber)?


    I don't believe so.

    Cheers, Liam
    >
    > Thanks for any advice. Yours,
    >
    > -mi


  3. Re: Sharing a 3510 -- NFS vs. QFS?

    Liam Greenwood wrote:

    > QFS has to have an ethernet connection between the servers, as well as
    > them all having fiber access to the disks. One of the serves will be a
    > metadata server - all file access require a metadata call to the metadata
    > server over the ethernet, then direct fiber access to the disk.


    Great, thank you very much -- it now makes sense to me. I now
    understand, at least, how it could work in principle -- without the 3510
    knowing the details of the filesystem, and concerning itself with just
    the device-level workings.

    Can you give a guesstimate, on how much space is required on the
    metadata server? For example, if the FS is 1.5Tb in size (12 140Gb disks
    inside a single 3510), can we use the 100Gb found inside one of the
    connected v490 servers for metadata?

    >> Is QFS considered reliable? Is it worth the trouble of learning it for our
    >> case of very few servers?

    >
    > As far as I know - it's designed for very large filesystems.
    >> Can QFS be used to keep two 3510s in sync over LAN (not fiber)?

    >
    > I don't believe so.


    Thanks!

    -mi

  4. Re: Sharing a 3510 -- NFS vs. QFS?

    On 2007-06-04 19:02:09 +0100, "Mikhail T." said:

    > Can you give a guesstimate, on how much space is required on the
    > metadata server? For example, if the FS is 1.5Tb in size (12 140Gb
    > disks inside a single 3510), can we use the 100Gb found inside one of
    > the connected v490 servers for metadata?


    Wouldn't it be more sensible to use some of the space in the 3510 as
    metadata? That would ensure that all the FS data was in one unit at
    least (of course, that's a bad thing for redundancy, but I don't think
    having the metadata on one set of spindles and the data on another
    increases redundancy...)


  5. Re: Sharing a 3510 -- NFS vs. QFS?

    Tim Bradshaw wrote:

    > On 2007-06-04 19:02:09 +0100, "Mikhail T."
    > said:


    >> Can you give a guesstimate, on how much space is required on the
    >> metadata server? For example, if the FS is 1.5Tb in size (12 140Gb
    >> disks inside a single 3510), can we use the 100Gb found inside one of
    >> the connected v490 servers for metadata?


    > Wouldn't it be more sensible to use some of the space in the 3510 as
    > metadata? That would ensure that all the FS data was in one unit at
    > least (of course, that's a bad thing for redundancy, but I don't think
    > having the metadata on one set of spindles and the data on another
    > increases redundancy...)


    Actually, I'm pretty sure, having the metadata and the data on different
    sets of spindles (in different enclosures) does increase redundancy...
    Metadata can be rebuilt from the actual data, and the data (or most of it?)
    can be recovered using an intact metadata (journal?).

    "Metadata separation" is, in fact, listed as one the "Key Features" of QFS:

    http://www.sun.com/storagetek/manage...anagement/qfs/

    Besides, each of the v490 servers connected to the 3510 has about 100Gb of
    (mirrored) spare capacity anyway :-) The v490s today come with two 140Gb
    drives in them, which are mirrored. The OS, optional software, and swap
    altogether take only about 40Gb...

    Performance should improve too, because the "metadata" server will be
    accessing the metadata partition locally, rather than troubling the
    external RAID5.

    The only question is, will the 100Gb be enough? It ought to be -- we are not
    going store 1.5Tb worth of 0-sized files, but getting a word from someone,
    who uses QFS already would be most interesting...

    -mi

  6. Re: Sharing a 3510 -- NFS vs. QFS?

    On 2007-06-05 01:46:29 +0100, Mikhail Teterin
    said:

    > Actually, I'm pretty sure, having the metadata and the data on different
    > sets of spindles (in different enclosures) does increase redundancy...
    > Metadata can be rebuilt from the actual data, and the data (or most of it?)
    > can be recovered using an intact metadata (journal?).


    That makes sense, if they can be recovered from each other or, really,
    if the metadata can be rebuilt from the data (If you've lost the data
    disks having the metadata won't help much I suspect). I'd not found
    enough technical info on QFS (is there any available?) to understand
    that, though I should have.

    > Performance should improve too, because the "metadata" server will be
    > accessing the metadata partition locally, rather than troubling the
    > external RAID5.


    Well, this depends where the bottleneck is I guess. I can see the
    trick QFS is doing - parallel access to data blocks is fine, so long as
    someone is in charge of the metadata to ensure thins remain consistent.
    And in normal usage (especially for db-type applications) that will
    not be a bottleneck - especially if the metadata is a mirror if the
    data is raid 5.


  7. Re: Sharing a 3510 -- NFS vs. QFS?

    On Tue, 5 Jun 2007 07:19:58 +0100, Tim Bradshaw wrote:
    > On 2007-06-05 01:46:29 +0100, Mikhail Teterin
    > said:
    >
    >> Actually, I'm pretty sure, having the metadata and the data on different
    >> sets of spindles (in different enclosures) does increase redundancy...
    >> Metadata can be rebuilt from the actual data, and the data (or most of it?)
    >> can be recovered using an intact metadata (journal?).

    >
    > That makes sense, if they can be recovered from each other or, really,
    > if the metadata can be rebuilt from the data (If you've lost the data
    > disks having the metadata won't help much I suspect). I'd not found
    > enough technical info on QFS (is there any available?) to understand
    > that, though I should have.


    QFS allows for the configuration of backup metadata servers, so that if
    the main one fail the filesystem can fail-over to the backup.

    Cheers, Liam

  8. Re: Sharing a 3510 -- NFS vs. QFS?

    On 2007-06-05 16:12:00 +0100, Liam Greenwood said:

    > QFS allows for the configuration of backup metadata servers, so that if
    > the main one fail the filesystem can fail-over to the backup.


    That sounds good. What's the coherency arrangement between them? This
    isn't a rhetorical question!

    Actually is there any good technical info (further than the stuff
    that's easily available at www.sun.com) on QFS? I admit to be being
    too lazy to search hard, sorry!

    --tim


  9. Re: Sharing a 3510 -- NFS vs. QFS?

    On Tue, 5 Jun 2007 20:48:24 +0100
    Tim Bradshaw wrote:

    > That sounds good. What's the coherency arrangement between them?
    > This isn't a rhetorical question!


    The metadata and catalog should be on devices accessible to all
    potential metadata servers through a SAN. If you're using SCSI you're
    limited to a single metadata server.

    --
    Stefaan A Eeckels
    --
    Sometimes I wonder whether the world is run by smart people who are
    putting us on or by imbeciles who really mean it. --Mark Twain

+ Reply to Thread