[9fans] About 9P ... - Plan9

This is a discussion on [9fans] About 9P ... - Plan9 ; Hello 9fans, Regarding 9P design, I wonder why Fids are chosen by the client : I tried to find an answer in the docs, manuals or in 9fans archive but I have not found an answer (please tell me where ...

+ Reply to Thread
Page 1 of 3 1 2 3 LastLast
Results 1 to 20 of 46

Thread: [9fans] About 9P ...

  1. [9fans] About 9P ...


    Hello 9fans,

    Regarding 9P design, I wonder why Fids are chosen by the client :
    I tried to find an answer in the docs, manuals or in 9fans archive
    but I have not found an answer (please tell me where I can find
    an answer if I missed something).
    Can someone explain me the reason why the Fids are not returned
    by the server (along with the Qids by example) ?
    Because Fids are chosen by the client, the server must search in
    a table (hash table in lib9p) the associated file structure, Whereas
    if Fids are chosen by the server, a simple array is enough. I don't
    know however, if the performance improvement is really important.
    In fact I'm quite sure it is not.
    I also think this is not because of a security issue even if there
    may be multiple clients sharing the same connection. Indeed, because
    the transactions are only between the server and the kernel or an
    remote (tcp/il) connection, it seems a process cannot steal the fid
    of another process (or another namespace). Regarding the transactions
    with the kernel, a process has no access to fids. And regarding
    the transaction with a remote connection, the connection point can
    be used as a kind of fid space.
    Maybe I'm wrong about the security issue, I only want to know why
    9P has been designed with such policy.

    Thank you,
    Phil;


  2. Re: [9fans] About 9P ...

    > Because Fids are chosen by the client, the server must search in
    > a table (hash table in lib9p) the associated file structure, Whereas
    > if Fids are chosen by the server, a simple array is enough. I don't
    > know however, if the performance improvement is really important.
    > In fact I'm quite sure it is not.


    If Fids were chosen by the server you would have written
    the opposite. One side can have a simple array lookup
    and one side can't. It doesn't matter either way.
    You just pick one.

    Or you could do what SSH did and pick two: there the
    equivalent of a "fid" is actually a pair of integers,
    one chosen by the client and one chosen by the server,
    so that both can use an array lookup. Because after
    all the encryption and public key operations, it's really
    looking up the fids that is eating all the cpu.

    Russ


  3. Re: [9fans] About 9P ...

    On Tue, 2007-06-19 at 20:37 -0400, Russ Cox wrote:
    > Because after
    > all the encryption and public key operations, it's really
    > looking up the fids that is eating all the cpu.


    Now this is an interesting point, because it sort of argues
    in favor of doing the lookups on the client and conserving
    CPU cycles of the server where the cycles can be more precious.

    Thanks,
    Roman.

    P.S. But then again, in Plan 9: the server is a client is a
    server ;-)


  4. Re: [9fans] About 9P ...

    perhaps my sense of humor's tweaked, but i understood russ' comment to
    be sarcastic, since the added complexity of making fid lookup slightly
    faster on both ends seems hard to justify given how vastly outweighed
    fid lookup is bound to be by the encryption-related operations.

  5. Re: [9fans] About 9P ...

    On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote:
    >
    > Can someone explain me the reason why the Fids are not returned
    > by the server (along with the Qids by example) ?


    I always assumed it was to enable pipelined operations.

    DF

  6. Re: [9fans] About 9P ...

    What do you mean ?

    Derek Fawcus a écrit :
    > On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote:
    >
    >> Can someone explain me the reason why the Fids are not returned
    >> by the server (along with the Qids by example) ?
    >>

    >
    > I always assumed it was to enable pipelined operations.
    >
    > DF
    >
    >
    >



  7. Re: [9fans] About 9P ...

    On Fri, Jun 22, 2007 at 08:58:04AM +0200, Philippe Anel wrote:
    > What do you mean ?


    Well, I've not read the protocol details for a while.
    But from memory I thought it allowed this type of operation:

    send: open,fid,file
    send: read,fid,args
    send: read,fid,args

    (wait one rtt)

    recv: open success/fail
    recv: read result / read error due to unknown fid
    recv: read result / read error due to unknown fid

    DF

    >
    > Derek Fawcus a écrit :
    > > On Tue, Jun 19, 2007 at 10:11:31PM +0200, Philippe Anel wrote:
    > >
    > >> Can someone explain me the reason why the Fids are not returned
    > >> by the server (along with the Qids by example) ?
    > >>

    > >
    > > I always assumed it was to enable pipelined operations.
    > >
    > > DF
    > >
    > >
    > >


  8. Re: [9fans] About 9P ...

    Well even if it can work, I don't think 9P was designed to allow such
    type of
    operation. Indeed, 9P basis is the transaction : ie a Request followed
    by a Reply.
    So I think you should not pipeline the requests.
    Can a 9P specialist can confirm this ? If so ... I now understand why
    Fids are
    (or must be) choosen by the client.

    Phil;
    > Well, I've not read the protocol details for a while.
    > But from memory I thought it allowed this type of operation:
    >
    > send: open,fid,file
    > send: read,fid,args
    > send: read,fid,args
    >
    > (wait one rtt)
    >
    > recv: open success/fail
    > recv: read result / read error due to unknown fid
    > recv: read result / read error due to unknown fid
    >
    > DF
    >
    >



  9. Re: [9fans] About 9P ...

    Transactions are tagged right? So you can, in fact, have many in flight at
    once.
    Perhaps I'm missing what you meant by pipelined.


    On 6/22/07, Philippe Anel wrote:
    >
    > Well even if it can work, I don't think 9P was designed to allow such
    > type of
    > operation. Indeed, 9P basis is the transaction : ie a Request followed
    > by a Reply.
    > So I think you should not pipeline the requests.
    > Can a 9P specialist can confirm this ? If so ... I now understand why
    > Fids are
    > (or must be) choosen by the client.
    >
    > Phil;
    > > Well, I've not read the protocol details for a while.
    > > But from memory I thought it allowed this type of operation:
    > >
    > > send: open,fid,file
    > > send: read,fid,args
    > > send: read,fid,args
    > >
    > > (wait one rtt)
    > >
    > > recv: open success/fail
    > > recv: read result / read error due to unknown fid
    > > recv: read result / read error due to unknown fid
    > >
    > > DF
    > >
    > >

    >
    >



    --
    - Passage Matthew 5:37:
    But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
    than these cometh of evil.


  10. Re: [9fans] About 9P ...

    > So I think you should not pipeline the requests.
    > Can a 9P specialist can confirm this ?


    not true.


  11. Re: [9fans] About 9P ...

    David Leimbach a écrit :
    > Transactions are tagged right? So you can, in fact, have many in
    > flight at once.
    >
    > Perhaps I'm missing what you meant by pipelined.

    I meant sending a couple of requests on one fid before receiving
    replies.

    Yes transaction are tagged, this doesn't mean that requests are
    serialized (processed in order).

    See the example sent by Derek :

    1 send: open,fid,file
    2 send: read,fid,args
    3 send: read,fid,args

    (wait one rtt)

    4 recv: open success/fail
    5 recv: read result / read error due to unknown fid
    7 recv: read result / read error due to unknown fid

    I see no reason for the server to not reply the 2nd request
    before the 1st one : by example the server can receive all
    the requests and distpach those reqs to different tasks.

    Because all transactions are tagged ... this wouldn't break
    9P. However, this doesn't work as expected.

    That's why I think 9P was not designed to allow this. But
    maybe I'm wrong.


    Phil;



  12. Re: [9fans] About 9P ...

    My bad, I'm not a good english writter ... I meant replies to request are
    not supposed to be returned in request order.

    Skip Tavakkolian a écrit :
    >> So I think you should not pipeline the requests.
    >> Can a 9P specialist can confirm this ?
    >>

    >
    > not true.
    >
    >
    >
    >



  13. Re: [9fans] About 9P ...

    On 6/22/07, Philippe Anel wrote:
    >
    > David Leimbach a écrit :
    > > Transactions are tagged right? So you can, in fact, have many in
    > > flight at once.
    > >
    > > Perhaps I'm missing what you meant by pipelined.

    > I meant sending a couple of requests on one fid before receiving
    > replies.




    That I believe is correct, there's no guarantee that a server has to process
    your requests on a single fid in the order the requests are sent/received.

    It will eventually reply to your requests in some order, and you'll be able
    to tell what order they occurred since I believe 9p requires a transport
    guaranteeing the sequence sent will be the sequence received. (I'm a 9p
    amateur though)


    Yes transaction are tagged, this doesn't mean that requests are
    > serialized (processed in order).
    >
    > See the example sent by Derek :
    >
    > 1 send: open,fid,file
    > 2 send: read,fid,args
    > 3 send: read,fid,args
    >
    > (wait one rtt)
    >
    > 4 recv: open success/fail
    > 5 recv: read result / read error due to unknown fid
    > 7 recv: read result / read error due to unknown fid
    >
    > I see no reason for the server to not reply the 2nd request
    > before the 1st one : by example the server can receive all
    > the requests and distpach those reqs to different tasks.



    Well you may have a point, but why not

    1. send: open, fid, file
    2. wait for reply to 1.
    3. send read, fid, args
    4. send read, fid, args
    5. wait for either...
    6. wait for remaining?

    At that point you're still pipelining, and since you're reading presumably
    into separate buffers, or different locations in the same buffer, who cares
    about the order?



    > Because all transactions are tagged ... this wouldn't break
    > 9P. However, this doesn't work as expected.




    Depends on what's expected :-)


    That's why I think 9P was not designed to allow this. But
    > maybe I'm wrong.




    It's probably designed to allow what I just said, you can sequence some
    operations, but then things that don't need to be sequenced could be
    pipelined.

    Different servers may behave differently, 9P makes no guarantee AFAIK.

    Dave


    Phil;
    >
    >
    >



    --
    - Passage Matthew 5:37:
    But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
    than these cometh of evil.


  14. Re: [9fans] About 9P ...

    Ok I agree ... as long as you don't expect the replies to be returned
    in the same order as the requests, requests can be pipelined.

    Therefore, now it makes sense to have FIDs chosen by the client.

    Thank you.

    >
    > Well you may have a point, but why not
    >
    > 1. send: open, fid, file
    > 2. wait for reply to 1.
    > 3. send read, fid, args
    > 4. send read, fid, args
    > 5. wait for either...
    > 6. wait for remaining?
    >
    > At that point you're still pipelining, and since you're reading
    > presumably into separate buffers, or different locations in the same
    > buffer, who cares about the order?
    >
    >
    >
    > Because all transactions are tagged ... this wouldn't break
    > 9P. However, this doesn't work as expected.
    >
    >
    >
    > Depends on what's expected :-)
    >
    >
    > That's why I think 9P was not designed to allow this. But
    > maybe I'm wrong.
    >
    >
    >
    > It's probably designed to allow what I just said, you can sequence
    > some operations, but then things that don't need to be sequenced could
    > be pipelined.
    >
    > Different servers may behave differently, 9P makes no guarantee AFAIK.
    >
    > Dave
    >
    >
    > Phil;
    >
    >
    >
    >
    >
    > --
    > - Passage Matthew 5:37:
    > But let your communication be, Yea, yea; Nay, nay: for whatsoever
    > is more than these cometh of evil.



  15. Re: [9fans] About 9P ...


    Sometimes, I whish I could think faster.

    The fact request can be pipelined like this -I mean, by taking care of the
    fact replies are not necessarly returned in requests order- doesn't not
    mean that clients have to choose the FIDs. This would also work if FIDs
    are choosen by the server (and even better because the client would not
    have to take care of the order).

    So unless Russ is right and that the fact FIDS are choosen by the client has
    been decided arbitrarily ... I still have no answer to my question.

    Phil;

    Philippe Anel a écrit :
    > Ok I agree ... as long as you don't expect the replies to be returned
    > in the same order as the requests, requests can be pipelined.
    >
    > Therefore, now it makes sense to have FIDs chosen by the client.
    >
    > Thank you.
    >
    >>
    >> Well you may have a point, but why not
    >>
    >> 1. send: open, fid, file
    >> 2. wait for reply to 1.
    >> 3. send read, fid, args
    >> 4. send read, fid, args
    >> 5. wait for either...
    >> 6. wait for remaining?
    >>
    >> At that point you're still pipelining, and since you're reading
    >> presumably into separate buffers, or different locations in the same
    >> buffer, who cares about the order?
    >>
    >>
    >>
    >> Because all transactions are tagged ... this wouldn't break
    >> 9P. However, this doesn't work as expected.
    >>
    >>
    >> Depends on what's expected :-)
    >>
    >>
    >> That's why I think 9P was not designed to allow this. But
    >> maybe I'm wrong.
    >>
    >>
    >>
    >> It's probably designed to allow what I just said, you can sequence
    >> some operations, but then things that don't need to be sequenced
    >> could be pipelined.
    >>
    >> Different servers may behave differently, 9P makes no guarantee AFAIK.
    >>
    >> Dave
    >>
    >>
    >> Phil;
    >>
    >>
    >>
    >>
    >>
    >> --
    >> - Passage Matthew 5:37:
    >> But let your communication be, Yea, yea; Nay, nay: for whatsoever
    >> is more than these cometh of evil.

    >
    >
    >



  16. Re: [9fans] About 9P ...

    On 6/22/07, Philippe Anel wrote:
    >
    >
    > Sometimes, I whish I could think faster.
    >
    > The fact request can be pipelined like this -I mean, by taking care of the
    > fact replies are not necessarly returned in requests order- doesn't not
    > mean that clients have to choose the FIDs. This would also work if FIDs
    > are choosen by the server (and even better because the client would not
    > have to take care of the order).
    >
    > So unless Russ is right and that the fact FIDS are choosen by the client
    > has
    > been decided arbitrarily ... I still have no answer to my question.




    I think servers often serve multiple clients, and therefore it's probably
    best to leave some things client side.

    Perhaps that's why google's not doing full SOAP anymore and just returning
    XML that you have to parse yourself on the client side.

    It may not really be arbitrary after all :-)


    Phil;
    >
    > Philippe Anel a écrit :
    > > Ok I agree ... as long as you don't expect the replies to be returned
    > > in the same order as the requests, requests can be pipelined.
    > >
    > > Therefore, now it makes sense to have FIDs chosen by the client.
    > >
    > > Thank you.
    > >
    > >>
    > >> Well you may have a point, but why not
    > >>
    > >> 1. send: open, fid, file
    > >> 2. wait for reply to 1.
    > >> 3. send read, fid, args
    > >> 4. send read, fid, args
    > >> 5. wait for either...
    > >> 6. wait for remaining?
    > >>
    > >> At that point you're still pipelining, and since you're reading
    > >> presumably into separate buffers, or different locations in the same
    > >> buffer, who cares about the order?
    > >>
    > >>
    > >>
    > >> Because all transactions are tagged ... this wouldn't break
    > >> 9P. However, this doesn't work as expected.
    > >>
    > >>
    > >> Depends on what's expected :-)
    > >>
    > >>
    > >> That's why I think 9P was not designed to allow this. But
    > >> maybe I'm wrong.
    > >>
    > >>
    > >>
    > >> It's probably designed to allow what I just said, you can sequence
    > >> some operations, but then things that don't need to be sequenced
    > >> could be pipelined.
    > >>
    > >> Different servers may behave differently, 9P makes no guarantee AFAIK.
    > >>
    > >> Dave
    > >>
    > >>
    > >> Phil;
    > >>
    > >>
    > >>
    > >>
    > >>
    > >> --
    > >> - Passage Matthew 5:37:
    > >> But let your communication be, Yea, yea; Nay, nay: for whatsoever
    > >> is more than these cometh of evil.

    > >
    > >
    > >

    >
    >



    --
    - Passage Matthew 5:37:
    But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
    than these cometh of evil.


  17. Re: [9fans] About 9P ...

    David Leimbach wrote:

    > Perhaps that's why google's not doing full SOAP anymore and just
    > returning XML that you have to parse yourself on the client side.


    If they wanted to "leave" things to the client side, they wouldn't use
    XML at all.

    --
    Anant

  18. Re: [9fans] About 9P ...

    there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a
    good example of why they're not needed. streaming and long delay
    networks can be handled this way too, as was pointed out (by Russ i
    think) at iwp9.


  19. Re: [9fans] About 9P ...

    Skip wrote:
    :there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a
    :good example of why they're not needed. streaming and long delay
    :networks can be handled this way too, as was pointed out (by Russ i
    :think) at iwp9.

    But there is one problem. Consider "lc".

    Usually you see
    walk
    clunk
    walk
    clunk
    walk
    open
    write
    clunk

    and also
    walk
    stat clunk
    walk
    open
    read
    clunk

    The problem is, how to know which RPCs to pack? I mean, to send without waiting
    for reply. Note that several different fids might be used in these
    dialogs, even though they
    come in a small time window from the same process.
    I tried to imagine how to bundle them, for using 9P over high latency links,
    and couldn't find a good way of doing it (after iwp9).

    In the end, I had to implement a couple of processes to bridge islands with good
    internal latency, connected by poor links. They try to refrain from
    sending requests by
    using cached directory entries (and data) to try to make such runs in
    one or two RPCs.
    The hard part is making it work well both for regular files and for
    synthesized files.
    We have a version that works reasonable well for both, (only
    connections multiplexed
    over the same file, eg. clone opens, are not handled properly).

    Our current version of the protocol has "file descriptors" that are fids in that
    they "mean" a path for a file, and are not fids in that more than one
    fid may map to
    the same descriptor, and also in that paths are sent along with the
    descriptor to make
    the server stateless (or to make clients think so).

    9P is just great for use when latency is reasonable (or not too bad,
    with cfs), but to go further
    away and still be comfortable using remote files, I'd say we need
    another protocol.
    I'd love to be proved wrong

  20. Re: [9fans] About 9P ...

    On Sat, 2007-06-23 at 01:48 +0200, Francisco J Ballesteros wrote:
    > Skip wrote:
    > :there were discussions about aysnc syscalls. /sys/src/cmd/fcp.c is a
    > :good example of why they're not needed. streaming and long delay
    > :networks can be handled this way too, as was pointed out (by Russ i
    > :think) at iwp9.
    >
    > But there is one problem. Consider "lc".
    >
    > Usually you see
    > walk
    > clunk
    > walk
    > clunk
    > walk
    > open
    > write
    > clunk
    >
    > and also
    > walk
    > stat clunk
    > walk
    > open
    > read
    > clunk
    >
    > The problem is, how to know which RPCs to pack?


    Isn't it the very same problem that compilers have with
    instruction pipelining?
    http://en.wikipedia.org/wiki/Instruction_pipelining

    With an additional complication that you can't actually
    look ahead very often?

    Thanks,
    Roman.


+ Reply to Thread
Page 1 of 3 1 2 3 LastLast