[9fans] Update on Fossil+Venti Stuff - Plan9

This is a discussion on [9fans] Update on Fossil+Venti Stuff - Plan9 ; I think everybody is a little confused about Fossil and Venti interaction. When I started this project, it was under the idea that Fossil was essentially a cache. It stored stuff, and when it either got too full or something ...

+ Reply to Thread
Results 1 to 17 of 17

Thread: [9fans] Update on Fossil+Venti Stuff

  1. [9fans] Update on Fossil+Venti Stuff

    I think everybody is a little confused about Fossil and Venti
    interaction. When I started this project, it was under the idea that
    Fossil was essentially a cache. It stored stuff, and when it either
    got too full or something else, it would start writing files out to
    Venti.

    As I've been copying files over to my machine, I've noticed Fossil get
    fuller and fuller, but not sync anything over to Venti. Upon
    attempting to find information about this, we came up with:

    http://www.mail-archive.com/9fans@cs.../msg03210.html

    Bruce doesn't sound too happy here, and since Plan 9 doesn't have any
    version control or any commit logs (that I'm aware of), I can't tell
    whether any of these bugs have been fixed. Or, if so, whether they've
    been released. As my Fossil fills from 15 to 30 GB out of 35GB (with
    180GB in my current Venti arena), I don't want to take this chance.

    My ideal usage of Fossil+Venti was to use it as expandable, persistent
    storage for my music. I'm finding out that this idea won't work
    because Venti exists as expandable archival storage. Which implies
    that files have to be deleted from Fossil before they'll end up in
    Venti. This would be fine with me, however, it seems that, by doing
    this, there's no way to get all of my mp3s listed in a single
    directory -- they would be archived by snapshot date / number, and I'd
    have to search through all of those to get a full list of music. This
    isn't really what I want.

    I could have probably saved myself a lot of suffering had I realized
    this sooner. Though, I just seem to recall a couple people doing this
    type of set-up, so I wonder what I'm doing wrong.

    Either way, I rarely delete files, and that seems to be the only way
    to get them into Venti. Or maybe I'm just missing things altogether.

    If there is a way to do what I'm wanting with Venti and Fossil, I'd
    love to see it. This isn't really a bitching and moaning and whining
    session, because this realization is probably more my fault for not
    researching enough, but regardless, I'd like to see something that can
    do what I'm looking for in the future, whether I have to write it
    myself or not. I guess this is sort of a probe as to how many people
    would find this useful versus not (versus already possible and I'm
    just dense).

    --Devon

  2. Re: [9fans] Update on Fossil+Venti Stuff

    Fossil is more like a write _buffer_ for venti. The interaction with venti
    takes place during the nightly dumps (at 5am for a default install; can be
    configured according to your needs). Fossil also regularly takes snapshots
    that aren't archived to venti. (Again, for a default setup snapshots every
    hour, kept for 8 days; can be configured otherwise.) The snapshots are taken
    for files that changed since the last snaptime; dumps are taken from files
    where the current epoch is greater than the last written to venti. Those
    procedures are quite well documented.

    The problems seem to arise when you write lots of new data between dumps
    (more than the fossil size; can be less when taking into account
    intermediate snapshots). I've been thinking about ways to cope with that
    lately, but wasn't too eager to post here until I'd actually have time
    to really delve into it (next month, or so). At the moment those are little
    more than some elaborations of possibilities already mentioned in the
    archives. I could do a write-up of what I could think of so far later
    tonight when I'm back home if there is wider interest in exploring possible
    enhancements (I'm not thinking of those asfixes) to fossil. I think it
    could be worthwhile, though.

    Martin

    P.S. I'm sure one could describe the current workings in more detail, but
    I'm too much in a hurry now.


  3. Re: [9fans] Update on Fossil+Venti Stuff

    On Wed Mar 21 16:24:59 EDT 2007, devon.odell@gmail.com wrote:
    > I think everybody is a little confused about Fossil and Venti
    > interaction. When I started this project, it was under the idea that
    > Fossil was essentially a cache. It stored stuff, and when it either
    > got too full or something else, it would start writing files out to
    > Venti.


    that "something else" is a snapshot archive which forces a flush to venti.
    read the section on setting up snapshots (expecially the -a option) of
    fossilcons(8). also, make sure fossil is really talking to venti.

    >
    > As I've been copying files over to my machine, I've noticed Fossil get
    > fuller and fuller, but not sync anything over to Venti. Upon
    > attempting to find information about this, we came up with:
    >
    > http://www.mail-archive.com/9fans@cs.../msg03210.html
    >
    > Bruce doesn't sound too happy here,


    if fossil fills between archival snapshots you are hosed. to the best of
    my knowledge, fossil still behaves this way.

    > and since Plan 9 doesn't have any
    > version control or any commit logs (that I'm aware of), I can't tell
    > whether any of these bugs have been fixed. Or, if so, whether they've
    > been released.


    while this is prima facia correct, sources keeps a archive of the sources
    tree for every day. after "9fs sourcesdump; cd /n/sourcesdump/2005/0612"
    would put you at the root of sources as it existed on 12 jun 2005.
    if you want to see the changes in a particular file on sources over time
    you can "9fs sources; cd /n/sources/plan9/sys/src/cmd/fossil; history 9srv.c".
    you will get

    Jan 28 11:32:21 EST 2006 9srv.c 3956 [rsc]
    Jan 28 11:32:21 EST 2006 /n/sourcesdump/2007/0321/plan9/sys/src/cmd/fossil/9srv.c 3956 [rsc]
    Feb 27 10:39:06 EST 2004 /n/sourcesdump/2006/0128/plan9/sys/src/cmd/fossil/9srv.c 3879 [jmk]
    Oct 13 22:21:37 EDT 2003 /n/sourcesdump/2004/0227/plan9/sys/src/cmd/fossil/9srv.c 3682 [jmk]
    Jun 15 15:02:14 EDT 2003 /n/sourcesdump/2003/1013/plan9/sys/src/cmd/fossil/9srv.c 3589 [rsc]
    Feb 18 15:26:58 EST 2003 /n/sourcesdump/2003/0615/plan9/sys/src/cmd/fossil/9srv.c 3291 [rsc]
    Jan 8 00:58:24 EST 2003 /n/sourcesdump/2003/0218/plan9/sys/src/cmd/fossil/9srv.c 3215 [rsc]

    all patches applied from external sources are also recorded in /n/sources/patch/applied.
    all pending external patches are in /n/sources/patch. internal labs changes
    don't have a patch directory associated with them, but history will still show the
    changes.

    - erik

  4. Re: [9fans] Update on Fossil+Venti Stuff

    Sounds to me like you want fossil to be a true cache --
    lazily writing to venti as more space is needed (or at
    regular intervals) and reading back from venti if something
    is needed and not in cache.

    I wonder if doing so would simplify fossil.... its size
    would become a function of peformance (to control data
    spill/fill rates). And one would have to always use
    fossil+venti and never just fossil. Not to mention venti
    then becomes the true file server and one can imagine
    building other front end cache filesystems.

    > This would be fine with me, however, it seems that, by doing
    > this, there's no way to get all of my mp3s listed in a single
    > directory -- they would be archived by snapshot date / number, and I'd
    > have to search through all of those to get a full list of music.


    You can always bind(1) all the snapshots under your music
    directory!

  5. Re: [9fans] Update on Fossil+Venti Stuff

    Devon - don't panic

    > that files have to be deleted from Fossil before they'll end up in
    > Venti.


    No, it's the other way round. A file's blocks can be deleted from
    fossil (automatically) after they have been archived to Venti,
    provided they haven't been modified since. The file's metadata
    will remain on fossil.

    The caveat is that blocks are archived to venti only when a snapshot
    is made (either periodically as scheduled by the snaptime command
    in fossilcons(8), or on request by the 'snap -a' command). So if
    the fossil partition fills with dirty (new or modified) blocks in
    between snapshots, bad things happen.

    Say you have a 4GB fossil partition. If you have snaptime set to
    take archival snapshots once a day, you can go on copying new mp3s
    to your fossil fs, without deleting anything, provided you never copy
    in more than 4GB per day. If you're impatient, you can copy 4GB,
    do a 'snap -a', copy another 4GB and so on. (The copying of snapshot
    blocks will go on in the background, so the 'snap' only freezes the
    fs very briefly.)

    If your fossil is getting "fuller and fuller", are you sure you
    have set it up to take periodic snapshots? You can check like this:

    % con -l /srv/fscons
    prompt: fsys main snaptime
    snaptime -a 0000 -s 60 -t 1440

    -- Richard


  6. Re: [9fans] Update on Fossil+Venti Stuff

    2007/3/22, Richard Miller <9fans@hamnavoe.com>:
    > Devon - don't panic


    Trying not to. I'm just really confused, and the sentiment seems to be
    ``WTF is wrong with you, it's plainly obvious how this works.'' But
    from what everyone says, it's not working that way for me.

    > The caveat is that blocks are archived to venti only when a snapshot
    > is made (either periodically as scheduled by the snaptime command
    > in fossilcons(8), or on request by the 'snap -a' command). So if
    > the fossil partition fills with dirty (new or modified) blocks in
    > between snapshots, bad things happen.
    >
    > Say you have a 4GB fossil partition. If you have snaptime set to
    > take archival snapshots once a day, you can go on copying new mp3s
    > to your fossil fs, without deleting anything, provided you never copy
    > in more than 4GB per day. If you're impatient, you can copy 4GB,
    > do a 'snap -a', copy another 4GB and so on. (The copying of snapshot
    > blocks will go on in the background, so the 'snap' only freezes the
    > fs very briefly.)


    I'm confused then. When I type snap -a, and then run fsys main df in
    fossilcons, I'm only noticing the usage going down by about 2GB/day.
    If I run snap -a, wait a bit, and then run fsys main df, I notice no
    change in disk usage.

    > If your fossil is getting "fuller and fuller", are you sure you
    > have set it up to take periodic snapshots? You can check like this:
    >
    > % con -l /srv/fscons
    > prompt: fsys main snaptime
    > snaptime -a 0000 -s 60 -t 1440


    Yep:

    10.0.0.10# con -l /srv/fscons
    prompt: fsys main snaptime
    snaptime -a 0500 -s 60 -t 2880

    I should note that I've manually modified this to happen a few times a
    day. Each time, my usage goes down a couple gigs. Why doesn't it just
    go ahead and sync everything?

    --dho

    > -- Richard
    >
    >


  7. Re: [9fans] Update on Fossil+Venti Stuff

    * Devon H. O'Dell (devon.odell@gmail.com) wrote:
    > 10.0.0.10# con -l /srv/fscons
    > prompt: fsys main snaptime
    > snaptime -a 0500 -s 60 -t 2880
    >
    > I should note that I've manually modified this to happen a few times a
    > day. Each time, my usage goes down a couple gigs. Why doesn't it just
    > go ahead and sync everything?


    It does sync everything active. There are probably still some intermediate
    snapshots there that are not written to venti. Those stay there until they
    expire after 2 days (2880 minutes). You can always do an explicit snapclean
    or try lowering the time snapshots are kept.

    Martin


  8. Re: [9fans] Update on Fossil+Venti Stuff

    2007/3/23, Martin Neubauer :
    > * Devon H. O'Dell (devon.odell@gmail.com) wrote:
    > > 10.0.0.10# con -l /srv/fscons
    > > prompt: fsys main snaptime
    > > snaptime -a 0500 -s 60 -t 2880
    > >
    > > I should note that I've manually modified this to happen a few times a
    > > day. Each time, my usage goes down a couple gigs. Why doesn't it just
    > > go ahead and sync everything?

    >
    > It does sync everything active. There are probably still some intermediate
    > snapshots there that are not written to venti. Those stay there until they
    > expire after 2 days (2880 minutes). You can always do an explicit snapclean
    > or try lowering the time snapshots are kept.


    I'm not disagreeing that this is supposed to happen, and I hate saying
    it's wrong, but it's not what I'm seeing, and I don't see anything
    invalid in my configuration. I can snap -a, and it won't go to venti.
    I ran snapclean and nothing went down. After that, I changed the
    snaptime so that it would run a Venti dump in 5 minutes. 10 minutes
    later, I see no difference.

    So I'm either missing something fundamental here, something's really
    broken, or I'm just dumb. I'm willing to concede the latter of the 3,
    but it would be really nice to be enlightened. Is there anything I can
    show you guys to help diagnose the issue further?

    > Martin


    -dho

  9. Re: [9fans] Update on Fossil+Venti Stuff

    * Devon H. O'Dell (devon.odell@gmail.com) wrote:
    > 2007/3/23, Martin Neubauer :
    > >* Devon H. O'Dell (devon.odell@gmail.com) wrote:
    > >> 10.0.0.10# con -l /srv/fscons
    > >> prompt: fsys main snaptime
    > >> snaptime -a 0500 -s 60 -t 2880
    > >>
    > >> I should note that I've manually modified this to happen a few times a
    > >> day. Each time, my usage goes down a couple gigs. Why doesn't it just
    > >> go ahead and sync everything?

    > >
    > >It does sync everything active. There are probably still some intermediate
    > >snapshots there that are not written to venti. Those stay there until they
    > >expire after 2 days (2880 minutes). You can always do an explicit snapclean
    > >or try lowering the time snapshots are kept.

    >
    > I'm not disagreeing that this is supposed to happen, and I hate saying
    > it's wrong, but it's not what I'm seeing, and I don't see anything
    > invalid in my configuration. I can snap -a, and it won't go to venti.
    > I ran snapclean and nothing went down. After that, I changed the
    > snaptime so that it would run a Venti dump in 5 minutes. 10 minutes
    > later, I see no difference.


    For snaptime, -a means ``time when to take archival dump.'' So you probably
    just made fossil dump 5 minutes after midnight. -s sets the interval (in
    minutes) between intermediate snapshots, -t the time (in minutes) after
    which old snapshots are discarded. If a snapshot of a file is referenced as
    the current version but isn't yet written to venti it is kept until the next
    dump.

    >
    > So I'm either missing something fundamental here, something's really
    > broken, or I'm just dumb. I'm willing to concede the latter of the 3,
    > but it would be really nice to be enlightened. Is there anything I can
    > show you guys to help diagnose the issue further?


    If it helps, it took some time for me to get a grasp of the workings, too.
    (And I'm sure there are still plenty of pitfalls for me to run into...)

    Martin


  10. Re: [9fans] Update on Fossil+Venti Stuff

    > If it helps, it took some time for me to get a grasp of the workings, too.
    > (And I'm sure there are still plenty of pitfalls for me to run into...)


    Each new user finds their own sharp edges to bump into I guess, the
    wiki and documentation, as well as this mailing list, serve the
    purpose of adding a bit of padding so it's a bit more comfortable from
    time to time, but no one is perfect, and people don't like to repeat
    themselves once it's been written.

    I know I appreciate all the help I've gotten with various issues here
    over the years. Sometimes I could have done more to self-educate,
    other times I'm just confused, and sometimes I've felt as frustrated
    as Devon with the whole thing.

    But for some silly reason I keep coming back to Plan 9, and keep
    re-loading it on *something*. Even when I only have time to boot
    that machine maybe 1 time a week to play around with it, it's still
    interesting :-)

  11. Re: [9fans] Update on Fossil+Venti Stuff

    2007/3/23, David Leimbach :
    > > If it helps, it took some time for me to get a grasp of the workings, too.
    > > (And I'm sure there are still plenty of pitfalls for me to run into...)

    >
    > Each new user finds their own sharp edges to bump into I guess, the
    > wiki and documentation, as well as this mailing list, serve the
    > purpose of adding a bit of padding so it's a bit more comfortable from
    > time to time, but no one is perfect, and people don't like to repeat
    > themselves once it's been written.
    >
    > I know I appreciate all the help I've gotten with various issues here
    > over the years. Sometimes I could have done more to self-educate,
    > other times I'm just confused, and sometimes I've felt as frustrated
    > as Devon with the whole thing.


    Yeah, and I still haven't found the issue. Contrary to Martin's
    suggestion, I did actually set the -a time correctly to N minutes
    after the current system time, though I guess I didn't imply this.
    This morning I'm still at 14GB. I mean, I should note that I'm not
    actually removing any files or anything, and I still have the 25GB of
    MP3s that were on there at the beginning. So. Still confused and
    frustrated.

    > But for some silly reason I keep coming back to Plan 9, and keep
    > re-loading it on *something*. Even when I only have time to boot
    > that machine maybe 1 time a week to play around with it, it's still
    > interesting :-)


    I'll give you that. It'll be nice when I figure out what it is and it
    gets fixed. But until then, *grrrr*

    --dho

  12. Re: [9fans] Update on Fossil+Venti Stuff

    Devon -

    > This morning I'm still at 14GB. I mean, I should note that I'm not
    > actually removing any files or anything, and I still have the 25GB of
    > MP3s that were on there at the beginning. So. Still confused and
    > frustrated.


    Why are you expecting your fossil partition to become more empty?
    It's a cache - so even when a block has been copied to venti, it
    is still useful to keep a copy in fossil as well, in case it's referenced
    again. It will be marked 'clean', so that it can be evicted from the
    cache instantly if fossil runs out of space.

    Are you still worried that blocks aren't being copied to venti at all?
    You can check by looking at the archive score which is printed
    on fossilcons right after a 'snap -a' is done. You can use vacfs
    to mount this score as a fs, and then explore that to see if everything
    is there.

    If you *really* want to empty your fossil partition, you can reformat
    it immediately after a 'snap -a', using the '-v score' option with the
    score of the snap you just made. (Take the usual precautions first,,
    of course.)

    -- Richard


  13. Re: [9fans] Update on Fossil+Venti Stuff

    2007/3/23, Richard Miller <9fans@hamnavoe.com>:
    > Devon -
    >
    > > This morning I'm still at 14GB. I mean, I should note that I'm not
    > > actually removing any files or anything, and I still have the 25GB of
    > > MP3s that were on there at the beginning. So. Still confused and
    > > frustrated.

    >
    > Why are you expecting your fossil partition to become more empty?
    > It's a cache - so even when a block has been copied to venti, it
    > is still useful to keep a copy in fossil as well, in case it's referenced
    > again. It will be marked 'clean', so that it can be evicted from the
    > cache instantly if fossil runs out of space.
    >
    > Are you still worried that blocks aren't being copied to venti at all?
    > You can check by looking at the archive score which is printed
    > on fossilcons right after a 'snap -a' is done. You can use vacfs
    > to mount this score as a fs, and then explore that to see if everything
    > is there.


    I'm not sure what you're referring to here -- when I type snap -a, I
    get no output back.

    > If you *really* want to empty your fossil partition, you can reformat
    > it immediately after a 'snap -a', using the '-v score' option with the
    > score of the snap you just made. (Take the usual precautions first,,
    > of course.)


    I'll read the fossilcons manpage on this again and see what the usage
    of this is. I guess I am somewhat worried, simply because I don't want
    to fill my fossil partition (I still have > 35GB data to copy over)
    and have it blow up.

    At this point, I think I've probably munged what's in Venti enough
    that I'm going to go ahead and reinstall with this `newfound'
    knowledge.

    > -- Richard


    Thanks for the information -- I'll definitely try it out this time.

    --dho

  14. Re: [9fans] Update on Fossil+Venti Stuff

    > I'm not sure what you're referring to here -- when I type snap -a, I
    > get no output back.


    Wait a while before disconnecting from fscons. It prints the score
    eventually after it finishes flushing blocks to disk.

    % con -l /srv/fscons
    prompt: fsys main
    main: snap -a
    main: archive vac:adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
    main: >>> q
    %

    (actual score edited for security reasons!)


  15. Re: [9fans] Update on Fossil+Venti Stuff

    2007/3/23, Richard Miller <9fans@hamnavoe.com>:
    > > I'm not sure what you're referring to here -- when I type snap -a, I
    > > get no output back.

    >
    > Wait a while before disconnecting from fscons. It prints the score
    > eventually after it finishes flushing blocks to disk.
    >
    > % con -l /srv/fscons
    > prompt: fsys main
    > main: snap -a
    > main: archive vac:adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
    > main: >>> q
    > %
    >
    > (actual score edited for security reasons!)


    Aha! Ok. Thanks!

    --dho

  16. Re: [9fans] Update on Fossil+Venti Stuff

    > I'll read the fossilcons manpage

    You might want to read /sys/doc/fossil.pdf as well.


  17. Re: [9fans] Update on Fossil+Venti Stuff

    > (actual score edited for security reasons!)

    Dam, and there was me ready to extrapolate your source tree, your bank
    account and shoe size from that score - Instead I seem to have ended up
    with the music collection of somone from the planet zarg...

    -Steve

+ Reply to Thread