Prevent from reading part of information - Unix

This is a discussion on Prevent from reading part of information - Unix ; Hi List, The OS is Linux. I am writting some program which communicated via pipe and unix domain socket. These processes exchange information by reading/writting on pipes and unix domain sockets. In the information reading end, I usually use a ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Prevent from reading part of information

  1. Prevent from reading part of information

    Hi List,

    The OS is Linux. I am writting some program which communicated via
    pipe and unix domain socket. These processes exchange information by
    reading/writting on pipes and unix domain sockets. In the information
    reading end, I usually use a select(2) to find out if some a
    information is comming followed by a read(2) operation on the same
    file descritor. The information is something like below, and let's
    called it `msg':

    struct msg {
    unsigned int len;
    unsigned char data[];
    }

    My question is, in the msg reading end, after select(2) returned and
    indicated there were something to read, is there any chance that the
    reading end only got part of the `msg' via read(2)? I understand,
    for internet socket, it is usual, but since I am now working on Unix
    domain sockets and pipes, so I guess things may be different.

    -
    narke

  2. Re: Prevent from reading part of information

    Steven Woody wrote:
    > Hi List,


    > The OS is Linux. I am writting some program which communicated via
    > pipe and unix domain socket. These processes exchange information by
    > reading/writting on pipes and unix domain sockets. In the information
    > reading end, I usually use a select(2) to find out if some a
    > information is comming followed by a read(2) operation on the same
    > file descritor. The information is something like below, and let's
    > called it `msg':


    > struct msg {
    > unsigned int len;
    > unsigned char data[];
    > }


    > My question is, in the msg reading end, after select(2) returned and
    > indicated there were something to read, is there any chance that the
    > reading end only got part of the `msg' via read(2)?


    I guess you're trying not to have to send the length of the
    struct in advance and instead ask read() to read in a large
    number of bytes and trust that it returns when it has read
    the whole structure. That probably will work in most cases,
    but I don't think it's something you can trust on 100%. One
    scenario were it could fail is when the struct is longer
    than fits into a pipe. In that case you would only get a
    pipe lengths worth of data (the writer will be sleeping
    until you have read that and only then become allowed to
    write the rest of it).
    Regards, Jens
    --
    \ Jens Thoms Toerring ___ jt@toerring.de
    \__________________________ http://toerring.de

  3. Re: Prevent from reading part of information

    Steven Woody writes:
    > The OS is Linux. I am writting some program which communicated via
    > pipe and unix domain socket. These processes exchange information by
    > reading/writting on pipes and unix domain sockets. In the information
    > reading end, I usually use a select(2) to find out if some a
    > information is comming followed by a read(2) operation on the same
    > file descritor. The information is something like below, and let's
    > called it `msg':
    >
    > struct msg {
    > unsigned int len;
    > unsigned char data[];
    > }
    >
    > My question is, in the msg reading end, after select(2) returned and
    > indicated there were something to read, is there any chance that the
    > reading end only got part of the `msg' via read(2)? I understand,
    > for internet socket, it is usual, but since I am now working on Unix
    > domain sockets and pipes, so I guess things may be different.


    An protocol family is only a partial specification of a particular
    socket. If you can, use (PF_UNIX) datagram sockets. They are
    documented as being reliable for Linux and will preserve message
    boundaries, ie if you send one structure with each sending-call, each
    receiving call will return exactly one structure. Regarding pipes: A
    write to a pipe will be atomic when the size written is <=
    PIPE_BUF. This means that as long as each individual write writes less
    than or equal to PIPE_BUF bytes, the pipe buffer in the kernel should
    always contain an integral number of messages and each read call
    should return one message, assuming the size to read is already known
    at the time of the read. Otherwise, partial read can occur. But that
    is something easy to deal with: Call read with a 'large' buffer (eg of
    size PIPE_BUF), process all complete messages and copy trailing
    partial message to the beginning of the buffer before the next read
    call (and adjust the size and initial offset accordingly, of course).
    The same method could be used for any other type of 'byte
    stream'-channel.




  4. Re: Prevent from reading part of information

    On Jun 1, 10:43 pm, Rainer Weikusat wrote:
    > Steven Woody writes:
    > > The OS is Linux. I am writting some program which communicated via
    > > pipe and unix domain socket. These processes exchange information by
    > > reading/writting on pipes and unix domain sockets. In the information
    > > reading end, I usually use a select(2) to find out if some a
    > > information is comming followed by a read(2) operation on the same
    > > file descritor. The information is something like below, and let's
    > > called it `msg':

    >
    > > struct msg {
    > > unsigned int len;
    > > unsigned char data[];
    > > }

    >
    > > My question is, in the msg reading end, after select(2) returned and
    > > indicated there were something to read, is there any chance that the
    > > reading end only got part of the `msg' via read(2)? I understand,
    > > for internet socket, it is usual, but since I am now working on Unix
    > > domain sockets and pipes, so I guess things may be different.

    >
    > An protocol family is only a partial specification of a particular
    > socket. If you can, use (PF_UNIX) datagram sockets. They are
    > documented as being reliable for Linux and will preserve message
    > boundaries, ie if you send one structure with each sending-call, each
    > receiving call will return exactly one structure. Regarding pipes: A
    > write to a pipe will be atomic when the size written is <=
    > PIPE_BUF. This means that as long as each individual write writes less
    > than or equal to PIPE_BUF bytes, the pipe buffer in the kernel should
    > always contain an integral number of messages and each read call
    > should return one message, assuming the size to read is already known
    > at the time of the read. Otherwise, partial read can occur. But that
    > is something easy to deal with: Call read with a 'large' buffer (eg of
    > size PIPE_BUF), process all complete messages and copy trailing
    > partial message to the beginning of the buffer before the next read
    > call (and adjust the size and initial offset accordingly, of course).
    > The same method could be used for any other type of 'byte
    > stream'-channel.


    thanks for all your answer. let me clear the reason why I asked the
    original question:

    I want to write a single routine read_msg(...) for other part of the
    whole program, which will be used by many other programmers. so I
    want the routine as simple as possible, hence the read_msg( ... ) is a
    state-less routien, that is, if one call to the routine read only part
    of the whole message, it is hard to other programmers to use it with
    success by the next call. So, I hope every call to the read_msg( ... )
    will return a single whole message.

    and, some information I need to let you know,

    1, no message will exceed the size limit of PIPE_BUF.
    2, as you have already seen, there is a length field in the message
    struct will can be used to determind the message length
    3, for some reason, I currently prefer to stream sockt instead of
    dgram.

    any other suggestion? thanks in advance.

    -
    narke

  5. Re: Prevent from reading part of information

    Steven Woody writes:

    [...]

    > I want to write a single routine read_msg(...) for other part of the
    > whole program, which will be used by many other programmers. so I
    > want the routine as simple as possible, hence the read_msg( ... ) is a
    > state-less routien, that is, if one call to the routine read only part
    > of the whole message, it is hard to other programmers to use it with
    > success by the next call. So, I hope every call to the read_msg( ... )
    > will return a single whole message.
    >
    > and, some information I need to let you know,
    >
    > 1, no message will exceed the size limit of PIPE_BUF.
    > 2, as you have already seen, there is a length field in the message
    > struct will can be used to determind the message length
    > 3, for some reason, I currently prefer to stream sockt instead of
    > dgram.
    >
    > any other suggestion?


    Taking all the contraints into account, this is impossible. At the
    expense of some efficiency, you could get by by first calling read
    (repeatedly) until the length has been read and then calling read
    (repeatedly) until exactly this number of additional bytes has been
    read, to.

  6. Re: Prevent from reading part of information

    On Jun 3, 2:30 am, Rainer Weikusat wrote:
    > Steven Woody writes:
    >
    > [...]
    >
    >
    >
    > > I want to write a single routine read_msg(...) for other part of the
    > > whole program, which will be used by many other programmers. so I
    > > want the routine as simple as possible, hence the read_msg( ... ) is a
    > > state-less routien, that is, if one call to the routine read only part
    > > of the whole message, it is hard to other programmers to use it with
    > > success by the next call. So, I hope every call to the read_msg( ... )
    > > will return a single whole message.

    >
    > > and, some information I need to let you know,

    >
    > > 1, no message will exceed the size limit of PIPE_BUF.
    > > 2, as you have already seen, there is a length field in the message
    > > struct will can be used to determind the message length
    > > 3, for some reason, I currently prefer to stream sockt instead of
    > > dgram.

    >
    > > any other suggestion?

    >
    > Taking all the contraints into account, this is impossible. At the
    > expense of some efficiency, you could get by by first calling read
    > (repeatedly) until the length has been read and then calling read
    > (repeatedly) until exactly this number of additional bytes has been
    > read, to.


    Thank you!

  7. Re: Prevent from reading part of information

    On Jun 1, 10:08*am, Steven Woody wrote:

    > I want to write a single routine read_msg(...) *for other part of the
    > whole program, which will be used by many other programmers. *so I
    > want the routine as simple as possible, hence the read_msg( ... ) is a
    > state-less routien, that is, if one call to the routine read only part
    > of the whole message, it is hard to other programmers to use it with
    > success by the next call. So, I hope every call to the read_msg( ... )
    > will return a single whole message.


    I believe this is quite literally impossible. You cannot determine the
    number of bytes in the message without reading the number of bytes.
    Once you've read the number of bytes, you have some state you need to
    retain. I'm presuming the intent is that this be non-blocking, since
    otherwise I don't know why you'd be using 'select'.

    MSG_PEEK doesn't work because it can deadlock. If you say, "I won't
    read the size of the message until the whole message is available"
    what happens if the other end says "I won't send the end of the
    message until he reads the size"? (TCP allows writers to wait for
    readers, so it cannot also allow readers to wait for writers.)

    So I don't see a solution. You have to read the size of the message,
    and if the whole message isn't ready, you either have to block or keep
    some state.

    DS

+ Reply to Thread