How to store big files contiguously on hd - Unix

This is a discussion on How to store big files contiguously on hd - Unix ; In article , "chevrot79@yahoo.fr" wrote: > First of all, my boss is great. The only problem is that he is an > electronic engineer. And in this field he does great job. However when > it comes to computer science ...

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 21 to 26 of 26

Thread: How to store big files contiguously on hd

  1. Re: How to store big files contiguously on hd

    In article <1157925977.220227.170130@q16g2000cwq.googlegroups. com>,
    "chevrot79@yahoo.fr" wrote:

    > First of all, my boss is great. The only problem is that he is an
    > electronic engineer. And in this field he does great job. However when
    > it comes to computer science field, he isn't as good.
    > He actually said : "raw data on hard disk store contiguously without
    > filesystem".
    > I think try to convince him to change his mind is a better thing than
    > do a boss upgrade.


    Try to get him to explain what he's trying to accomplish by storing the
    data contiguously. Requirements should be stated in terms of desired
    end results, not implementation techniques. If you can produce the
    results, it shouldn't matter how you get there.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  2. Re: How to store big files contiguously on hd

    chevrot79@yahoo.fr wrote:
    >
    > The only problem is that he is an
    > electronic engineer. And in this field he does great job. However when
    > it comes to computer science field, he isn't as good.
    > He actually said : "raw data on hard disk store contiguously without
    > filesystem".


    Sounds like he wishes to use a disk as tape.

    > I think try to convince him to change his mind is a better thing than
    > do a boss upgrade.


    If performance is what he wants, contiguous has not delivered
    the best performance in a very long time. Consider the old Berkeley
    article "A fast fat file system for UNIX" that came with the BSD 4.2
    docs in the 1980s. It showed statistics that proved that there are
    non-contiguous strategies that deliver better perfomance than
    contiguous.

    The disk technologies in use at the time of that article no longer
    apply, but each newer filesystem has used the same general
    concepts applied to how newer disks work to acheive the same
    result of faster performance than contiguous.

    If he wants performance, educate him starting with old articles.
    If he wants simplicity use tape.


  3. Re: How to store big files contiguously on hd


    Doug Freyburger a écrit :

    > chevrot79@yahoo.fr wrote:
    > >
    > > The only problem is that he is an
    > > electronic engineer. And in this field he does great job. However when
    > > it comes to computer science field, he isn't as good.
    > > He actually said : "raw data on hard disk store contiguously without
    > > filesystem".

    >
    > Sounds like he wishes to use a disk as tape.
    >
    > > I think try to convince him to change his mind is a better thing than
    > > do a boss upgrade.

    >
    > If performance is what he wants, contiguous has not delivered
    > the best performance in a very long time. Consider the old Berkeley
    > article "A fast fat file system for UNIX" that came with the BSD 4.2
    > docs in the 1980s. It showed statistics that proved that there are
    > non-contiguous strategies that deliver better perfomance than
    > contiguous.
    >
    > The disk technologies in use at the time of that article no longer
    > apply, but each newer filesystem has used the same general
    > concepts applied to how newer disks work to acheive the same
    > result of faster performance than contiguous.
    >
    > If he wants performance, educate him starting with old articles.
    > If he wants simplicity use tape.


    I have argued about that subject one more time but this time my boss
    has changed his mind !

    Thanks to you guys (jpd, Stefaan, Dave and the other ...).

    Now let's go to the next step : right filesystem choice and security
    problems ...

    --
    Cyrille CHEVROT


  4. Re: How to store big files contiguously on hd

    On 11 Sep 2006 14:46:07 -0700, chevrot79@yahoo.fr wrote:
    >
    > I have argued about that subject one more time but this time my boss
    > has changed his mind !


    Good to hear.

    > Thanks to you guys (jpd, Stefaan, Dave and the other ...).
    > Now let's go to the next step : right filesystem choice and security
    > problems ...


    (sigh). It depends. OK so what are the real requirements?


  5. Re: How to store big files contiguously on hd

    In article <1157717684.479714.266250@i42g2000cwa.googlegroups. com>,
    chevrot79@yahoo.fr wrote:
    >Hi,


    >I would like to know if there are a mean to store big files
    >contiguously on hd with linux.
    >I don't care about the hd space loss.
    >I don't care about the time to write the big files on the hd.
    >But, I DO CARE about the time to READ the big files.


    You might check into the xfs file system that was ported to Linux
    from SGI.

    It was designed for large files, and the last time I read specs you
    could have smallest allocations over 1 MB long - so that would give
    fast access. It was designed for large files and in particular
    video services.

    >For example, If I have 12 big files, 2 Go max, to place on my harddisk,
    >one solution would
    >be to allocate 24 Go on my hard disk, and associate 2 contiguous Go on
    >the disk for each file.


    >But I want to use a FILESYSTEM, I want to use
    >fopen/fclose/fread/fwrite to MANAGE this big files. I DON'T want
    >to use a raw hard disk with read/write sectors functions only.


    You first should read up on how to create filesystems and examine
    the options to blocksize.

    Bill

    --
    Bill Vermillion - bv @ wjv . com

  6. Re: How to store big files contiguously on hd

    In article <1157925977.220227.170130@q16g2000cwq.googlegroups. com>,
    chevrot79@yahoo.fr wrote:
    >Dave Hinz a =E9crit :
    >
    >> On 9 Sep 2006 13:27:23 -0700, chevrot79@yahoo.fr wro=

    >te:
    >> >
    >> > Dave Hinz a =E9crit :

    >>
    >> >> Yes. I think it might be a good idea if you can tell us what problem
    >> >> you're actually trying to solve, rather than ask us how to implement
    >> >> something that doesn't actually make any sense to do as an effort to
    >> >> solve whatever that is.

    >> (snip)
    >> >> No, his question still stands, you've explained a what, but you haven't
    >> >> defined a "why you're doing this in the first place". Helping someone
    >> >> with a lot of work to arrive at the wrong solution isn't helping them.

    >>
    >> > Because I must obey. And I need arguments to change my boss point of
    >> > view. That's it !

    >>
    >> Why don't you start there. What is your boss telling you needs to be
    >> solved. Did he actually say "find a way to contiguously write large
    >> files using Linux and make sure you map the bad blocks"? If so, you may
    >> need a boss upgrade. I just applied one and it's quite refreshing to do
    >> so.

    >
    >First of all, my boss is great. The only problem is that he is an
    >electronic engineer. And in this field he does great job. However when
    >it comes to computer science field, he isn't as good.
    >He actually said : "raw data on hard disk store contiguously without
    >filesystem".
    >I think try to convince him to change his mind is a better thing than
    >do a boss upgrade.


    What transfer rate do you need?

    There are a lot of options out there that will simply overwhelm
    most links unless you run GB links.



    --
    Bill Vermillion - bv @ wjv . com

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2