[9fans] command line tool for storing / reading files on venti - Plan9

This is a discussion on [9fans] command line tool for storing / reading files on venti - Plan9 ; Hi folks, is there any command line tool for reading and writing files on venti ? The venti/read and venti/write commands just support single blocks. Vac seems fine as archive tool, but it only can store - I need some ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 22

Thread: [9fans] command line tool for storing / reading files on venti

  1. [9fans] command line tool for storing / reading files on venti


    Hi folks,


    is there any command line tool for reading and writing files
    on venti ?

    The venti/read and venti/write commands just support single blocks.
    Vac seems fine as archive tool, but it only can store - I need some
    "unvac" command. (mounting each single archive via vacfs is a bit
    too complicated for my project).


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  2. Re: [9fans] command line tool for storing / reading files on venti

    9fs dump
    command /n/dump/yyyy/mmdd/absolutepathtofile

    yyyy -> year
    mmdd -> month and date

    On Jun 13, 2008, at 3:50 PM, Enrico Weigelt wrote:

    >
    > Hi folks,
    >
    >
    > is there any command line tool for reading and writing files
    > on venti ?
    >
    > The venti/read and venti/write commands just support single blocks.
    > Vac seems fine as archive tool, but it only can store - I need some
    > "unvac" command. (mounting each single archive via vacfs is a bit
    > too complicated for my project).
    >
    >
    > cu
    > --
    > ----------------------------------------------------------------------
    > Enrico Weigelt, metux IT service -- http://www.metux.de/
    >
    > cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    > ----------------------------------------------------------------------
    > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    > ----------------------------------------------------------------------
    >




  3. Re: [9fans] command line tool for storing / reading files on venti

    Since you mention using vac for storage, I assume you're using venti
    directly, not via fossil, in which case the '9fs dump' suggestion will
    do nothing for you.

    I don't believe there is anything in Plan 9 that does what you want
    (certainly the BUGS section in venti(1) implies not), but you might
    look at vcat in p9p. It's not quite there - it seems to demand that
    you're working with only a single file, rather than directory trees
    (possibly containing only a single file) as vac creates - but does the
    redirection and may be close to what you want.
    Anthony



  4. Re: [9fans] command line tool for storing / reading files on venti

    * Pietro Gagliardi wrote:
    > 9fs dump
    > command /n/dump/yyyy/mmdd/absolutepathtofile


    but this still requires me to mount each single vac
    archive before reading, and I need to create a new one
    for each upload, right ?


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  5. Re: [9fans] command line tool for storing / reading files on venti

    I don't think so. 9fs dump gives you everything afaik.

    On Jun 13, 2008, at 4:29 PM, Enrico Weigelt wrote:

    > * Pietro Gagliardi wrote:
    >> 9fs dump
    >> command /n/dump/yyyy/mmdd/absolutepathtofile

    >
    > but this still requires me to mount each single vac
    > archive before reading, and I need to create a new one
    > for each upload, right ?
    >
    >
    > cu
    > --
    > ----------------------------------------------------------------------
    > Enrico Weigelt, metux IT service -- http://www.metux.de/
    >
    > cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    > ----------------------------------------------------------------------
    > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    > ----------------------------------------------------------------------
    >




  6. Re: [9fans] command line tool for storing / reading files on venti

    > The venti/read and venti/write commands just support single blocks.
    > Vac seems fine as archive tool, but it only can store - I need some
    > "unvac" command. (mounting each single archive via vacfs is a bit
    > too complicated for my project).


    I'm hesitant to bother replying, as we learned earlier that
    your idea of complicated does not match established norms.

    That said, here is an (untested) unvac command:

    #!/bin/rc
    if(! ~ $#* 2){
    echo 'usage: unvac file.vac directory'
    exit 1
    }
    rfork n
    vacfs $1 || exit
    mkdir -p $2 || exit
    dircp /n/vac $2

    Russ



  7. Re: [9fans] command line tool for storing / reading files on venti

    * a@9srv.net wrote:
    > Since you mention using vac for storage, I assume you're using venti
    > directly, not via fossil, in which case the '9fs dump' suggestion will
    > do nothing for you.


    ACK.

    > I don't believe there is anything in Plan 9 that does what you want
    > (certainly the BUGS section in venti(1) implies not), but you might
    > look at vcat in p9p. It's not quite there - it seems to demand that
    > you're working with only a single file, rather than directory trees
    > (possibly containing only a single file) as vac creates - but does the
    > redirection and may be close to what you want.


    Yes, that sounds good, but fails with:

    ./vcat: diskopenventi: not a single file

    Well, if I now just find the opposite - uploading a file -
    I'll be happy


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  8. Re: [9fans] command line tool for storing / reading files on venti

    * Russ Cox wrote:
    > > The venti/read and venti/write commands just support single blocks.
    > > Vac seems fine as archive tool, but it only can store - I need some
    > > "unvac" command. (mounting each single archive via vacfs is a bit
    > > too complicated for my project).

    >
    > I'm hesitant to bother replying, as we learned earlier that
    > your idea of complicated does not match established norms.
    >
    > That said, here is an (untested) unvac command:
    >
    > #!/bin/rc
    > if(! ~ $#* 2){
    > echo 'usage: unvac file.vac directory'
    > exit 1
    > }
    > rfork n
    > vacfs $1 || exit
    > mkdir -p $2 || exit
    > dircp /n/vac $2


    Well, this doesnt seem to run this way on p9p
    BTW: still leaves the problem that lot's of vacfs servers are
    started not terminated after use


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  9. Re: [9fans] command line tool for storing / reading files on venti

    > BTW: still leaves the problem that lot's of vacfs servers are
    > started not terminated after use


    no, it doesn't. on plan 9, when the shell script exits,
    nothing will have vacfs mounted anymore, so vacfs
    will get an eof on the 9p connection and exit.
    that's why i put an rfork n in the script.

    > Well, this doesnt seem to run this way on p9p


    true. i thought you were using plan 9, sorry.

    if you are using linux and have fuse installed,
    you can still run something like the script,
    as long as you are willing to run just one at a time.

    vacfs -m /vac file.vac
    cp -R /vac/* .
    unmount /vac

    would work, assuming you create a /vac that you
    have write permission to. you can only run one at a time
    since there is only one /vac.

    russ



  10. Re: [9fans] command line tool for storing / reading files on venti

    * Russ Cox wrote:

    Hi,

    > no, it doesn't. on plan 9, when the shell script exits,
    > nothing will have vacfs mounted anymore, so vacfs
    > will get an eof on the 9p connection and exit.
    > that's why i put an rfork n in the script.


    aha, that's probably what the rfork call does ?

    > > Well, this doesnt seem to run this way on p9p

    >
    > true. i thought you were using plan 9, sorry.


    is there any way to get the same behaviour on p9p ?
    maybe the shell could keep track of its childs and
    terminate them on exit.

    > if you are using linux and have fuse installed,
    > you can still run something like the script,
    > as long as you are willing to run just one at a time.


    Uh, I'm not sure whether I can use fuse on the target system.
    I'd prefer doing all in userland.

    Meanwhile I've hacked up vacfs to give it an "autodie" mode.
    It means: automatically terminate if the last file has been
    closed. Seems to work, but now idea how stable it would be
    in production ;-o

    The whole gets a bit more complex:

    I'm using venti as storage backend for an media library.
    People can upload media from one server (via an Joomla-based
    frontend) and later fetch them via another download server.
    That download server is just an minimal httpd which fetches
    by a given key from venti and sends back the result.

    So I could some more things to do:

    * store some additional metadata (at least mime-type)
    * maybe also add some permission information
    * when using vac, I have to use always the same filename
    within vac (ls'ing first would be event oo complex ;-o)


    Maybe it's better to code a new archiver for this.


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  11. Re: [9fans] command line tool for storing / reading files on venti

    >> no, it doesn't. on plan 9, when the shell script exits,
    >> nothing will have vacfs mounted anymore, so vacfs
    >> will get an eof on the 9p connection and exit.
    >> that's why i put an rfork n in the script.

    >
    > aha, that's probably what the rfork call does ?


    rfork n forks the name space, so that any changes
    made by the script don't propagate out to the
    parent process. This means that (1) vacfs can
    mount on /n/vac without any fear of bothering
    some other instance of the script, and (2) when the
    script exits, there will be no more references to
    the mounted /n/vac and the kernel will unmount it.

    If you want simple, you should run Plan 9.

    > Meanwhile I've hacked up vacfs to give it an "autodie" mode.
    > It means: automatically terminate if the last file has been
    > closed. Seems to work, but now idea how stable it would be
    > in production ;-o


    In the time it took you to do that you could probably
    have written unvac instead.

    Russ



  12. Re: [9fans] command line tool for storing / reading files on venti

    * Russ Cox wrote:

    > rfork n forks the name space, so that any changes
    > made by the script don't propagate out to the
    > parent process. This means that (1) vacfs can
    > mount on /n/vac without any fear of bothering
    > some other instance of the script, and (2) when the
    > script exits, there will be no more references to
    > the mounted /n/vac and the kernel will unmount it.


    Aha. But this still does not terminate the vacfs, right ?

    > > Meanwhile I've hacked up vacfs to give it an "autodie" mode.
    > > It means: automatically terminate if the last file has been
    > > closed. Seems to work, but now idea how stable it would be
    > > in production ;-o

    >
    > In the time it took you to do that you could probably
    > have written unvac instead.


    hmm, maybe ;-o

    Meanwhile I've written an new tool which stores single files
    (no directory stuff at all). Not perfect yet, but already
    works quite fine (IMHO):

    svn://nibiru.metux.de/public/plan9port/apps/vtstore/

    In the next step, I'll add http-alike metadata (mimetype, etc).


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  13. Re: [9fans] command line tool for storing / reading files on venti

    On Jun 15, 2008, at 12:02 PM, Enrico Weigelt wrote:

    > Aha. But this still does not terminate the vacfs, right ?



    Once you kill the script, you also kill the processes it created -
    that's what process groups are, remember?

    Please read "The Use of Name Spaces in Plan 9" and the rfork man page
    and proc file system descriptions and all pages that are referenced
    there BEFORE continuing.



  14. Re: [9fans] command line tool for storing / reading files on venti

    * Pietro Gagliardi wrote:
    > On Jun 15, 2008, at 12:02 PM, Enrico Weigelt wrote:
    >
    > >Aha. But this still does not terminate the vacfs, right ?

    >
    >
    > Once you kill the script, you also kill the processes it created -
    > that's what process groups are, remember?


    That doesnt seem to work on p9p


    cu
    --
    ----------------------------------------------------------------------
    Enrico Weigelt, metux IT service -- http://www.metux.de/

    cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    ----------------------------------------------------------------------
    Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    ----------------------------------------------------------------------


  15. Re: [9fans] command line tool for storing / reading files on venti

    Doesn't matter. Process groups are process groups on any Unix clone.
    If it's daemons you're dealing with, then it leaves the scope of this
    room. If it's Windows, you're out of luck.

    On Jun 15, 2008, at 1:13 PM, Enrico Weigelt wrote:

    > * Pietro Gagliardi wrote:
    >> On Jun 15, 2008, at 12:02 PM, Enrico Weigelt wrote:
    >>
    >>> Aha. But this still does not terminate the vacfs, right ?

    >>
    >>
    >> Once you kill the script, you also kill the processes it created -
    >> that's what process groups are, remember?

    >
    > That doesnt seem to work on p9p
    >
    >
    > cu
    > --
    > ----------------------------------------------------------------------
    > Enrico Weigelt, metux IT service -- http://www.metux.de/
    >
    > cellphone: +49 174 7066481 email: info@metux.de skype: nekrad666
    > ----------------------------------------------------------------------
    > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
    > ----------------------------------------------------------------------
    >




  16. Re: [9fans] command line tool for storing / reading files on venti

    >> Aha. But this still does not terminate the vacfs, right ?
    >
    >
    > Once you kill the script, you also kill the processes it created -
    > that's what process groups are, remember?


    incorrect. the script exits, it is not killed. for example

    ; cat script
    #!/bin/rc
    rfork n
    sleep 1492&
    exit ''
    ; script
    ; ps -a|grep sleep
    quanstro 1519 0:00 0:00 8K Sleep sleep 1492
    quanstro 1521 0:00 0:00 156K Pread grep sleep

    when the script exits, the channel is closed, since all
    the writers are gone. this means that when vacfs
    read(2)'s from its 9p connection, it gets -1. it is
    conventional to exit at this point since no further
    work may be done.

    ; cat script
    #!/bin/rc
    rfork n
    ramfs
    exit ''
    ; script
    ; ps -a | grep ramfs
    quanstro 1860 0:00 0:00 156K Pread grep ramfs

    - erik



  17. Re: [9fans] command line tool for storing / reading files on venti

    > hmm, maybe ;-o
    >
    > Meanwhile I've written an new tool which stores single files
    > (no directory stuff at all). Not perfect yet, but already
    > works quite fine (IMHO):
    >
    > svn://nibiru.metux.de/public/plan9port/apps/vtstore/
    >
    > In the next step, I'll add http-alike metadata (mimetype, etc).


    I wasn't kidding. Update your plan9port
    and you will find an unvac command.
    You'll get more leverage if you stick to vac files.
    On the other hand, it won't be as complicated
    as rolling your own format.

    Russ



  18. Re: [9fans] command line tool for storing / reading files on venti

    * Pietro Gagliardi wrote:
    > Doesn't matter. Process groups are process groups on any Unix clone.
    > If it's daemons you're dealing with, then it leaves the scope of this
    > room. If it's Windows, you're out of luck.


    I'm running p9p on Linux. No idea where the actual problem sits,
    but your example left the vacfs-processes running.


    cu
    --
    ---------------------------------------------------------------------
    Enrico Weigelt == metux IT service - http://www.metux.de/
    ---------------------------------------------------------------------
    Please visit the OpenSource QM Taskforce:
    http://wiki.metux.de/public/OpenSource_QM_Taskforce
    Patches / Fixes for a lot dozens of packages in dozens of versions:
    http://patches.metux.de/
    ---------------------------------------------------------------------


  19. Re: [9fans] command line tool for storing / reading files on venti

    * Russ Cox wrote:

    Hi,

    > I wasn't kidding. Update your plan9port
    > and you will find an unvac command.


    my last checkout is a about 5 days old, and I couldn't find
    some working unvac command. But I'll have a re-checkout.

    > You'll get more leverage if you stick to vac files.


    Maybe right, if'd use an vac dir per file to archive, I could
    easily add more metadata later. But I'm very sure I don't
    need that for my current project.

    > On the other hand, it won't be as complicated
    > as rolling your own format.


    It wasn't that complicated, now all done ;-P


    cu
    --
    ---------------------------------------------------------------------
    Enrico Weigelt == metux IT service - http://www.metux.de/
    ---------------------------------------------------------------------
    Please visit the OpenSource QM Taskforce:
    http://wiki.metux.de/public/OpenSource_QM_Taskforce
    Patches / Fixes for a lot dozens of packages in dozens of versions:
    http://patches.metux.de/
    ---------------------------------------------------------------------


  20. Re: [9fans] command line tool for storing / reading files on venti

    Hi,

    Inferno's vacfs lets you cd into a score and read files from there
    without a mount per-score.

    --vs


+ Reply to Thread
Page 1 of 2 1 2 LastLast