aio_read/write versus O_NONBLOCK - Unix

This is a discussion on aio_read/write versus O_NONBLOCK - Unix ; Barry Margolin writes: > In article , > Rainer Weikusat wrote: > >> phil-news-nospam@ipal.net writes: >> > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat >> > wrote: >> >> [...] >> >> > | The original statement was ...

+ Reply to Thread
Page 2 of 5 FirstFirst 1 2 3 4 ... LastLast
Results 21 to 40 of 97

Thread: aio_read/write versus O_NONBLOCK

  1. Re: aio_read/write versus O_NONBLOCK

    Barry Margolin writes:
    > In article <87k5hoos5t.fsf@fever.mssgmbh.com>,
    > Rainer Weikusat wrote:
    >
    >> phil-news-nospam@ipal.net writes:
    >> > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat
    >> > wrote:

    >>
    >> [...]
    >>
    >> > | The original statement was that O_NONBLOCK wouldn't make any sense for
    >> > | file descriptors which can't block on read. You then asked why the
    >> > | person would believe 'that to be the case' and I replied that it is
    >> > | 'the case' by definition, hence, not a matter of believing anything.
    >> >
    >> > "sense" has nothing to do with definitions.

    >>
    >> It does not make sense because O_NONBLOCK is defined to be a no-op for
    >> this particular situation.

    >
    > But that's circular reasoning. WHY did they define it to be a no-op,


    Is this really so complicated? They did, consequently, using it
    nevertheless makes no sense because it doesn't do anything.

    > since it makes sense to allow it.


    And that's an entirely different topic.

    > File I/O is NOT really immediate, especially when networks are
    > involved.


    By the time this mechanism was designed, there were no network file
    systems. But this doesn't even really matter: Any 'device access', be
    it disk, network or whatever (eg flash) is done asynchronously be the
    kernel and the 'disk caching subsystem' is supposed to try as hard as
    possible to maintain the illusion of 'immediate operation'.

    > As a result, when they decided it was useful to have a way to operate on
    > files asynchronously, they had to design a new API, the aoi_* functions,
    > rather than reuse the existing select()/poll() API.


    These were two completely different groups of people. And aio is
    something completely different than 'non-blocking I/O': Asynchronous
    I/O is supposed to be performed in parallell with other operations
    done by the program, while a program using non-blocking I/O performs
    its I/O operations synchronously at times when trying to do so will
    not cause the process to be blocked in the kernel waiting for some
    condition to become true before the actual I/O processing can start.

  2. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 14:05:18 +0200 Rainer Weikusat wrote:
    | phil-news-nospam@ipal.net writes:
    |
    | [...]
    |
    |> | Just like poll(2) and select(2) always rval a disk-based file descriptor
    |> | as readable and writeable, if they support disk-based files at all.
    |>
    |> It would be useful if one could do poll(2) or select(2) on descriptors open
    |> to 2 or more disk devices or files when doing multiple I/O.
    |> A typical example is copying a file from one disk to another.
    |
    | UNIX(*) application do not 'copy files from one disk to another', but
    | copy the contents of one file to some other file. Anything beyond
    | that is supposed to be handled transparently by the kernel and not by
    | an application.

    When the files happen to be on different disks, I will refer to it as copying
    a file frm one disk to another, especially when that aspect of it is part of
    the issues. And it is in this case because there can be separate I/O operations
    overlapping between the disks even if the program specified syncronous.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  3. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 12:41:50 +0200 Rainer Weikusat wrote:
    | phil-news-nospam@ipal.net writes:
    |> On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat wrote:
    |
    | [...]
    |
    |> | The original statement was that O_NONBLOCK wouldn't make any sense for
    |> | file descriptors which can't block on read. You then asked why the
    |> | person would believe 'that to be the case' and I replied that it is
    |> | 'the case' by definition, hence, not a matter of believing anything.
    |>
    |> "sense" has nothing to do with definitions.
    |
    | It does not make sense because O_NONBLOCK is defined to be a no-op for
    | this particular situation.

    You really do have trouble understanding that I an suggesting that the
    definition be changed.

    You really do seem to be someone that would argue against the changing of a
    law that prohibits some action by stating that the law prohibits it.


    |> | The text above does not state anything regarding if it would be
    |> | possible to define useful semantics for 'non-blocking disk I/O' and/
    |> | or if defining such semantics would be an important
    |> | improvement. Discussing this topic would again lead to 'how the UNIX
    |> | disk cache actually works' really fast and that's a topic I am not
    |> | going to get into with you again.
    |>
    |> It doesn't need to go there. I'm suggesting that O_NONBLOCK does make
    |> sense for disk files and devices, despite any caching.
    |
    | You are suggesting that it would be possible to define other semantics
    | for O_NONBLOCK on disk files then those which are presently
    | defined. Until this has happened, O_NOBLOCK makes no sense in
    | association with disk files.

    I am stating that letting O_NONBLOCK work for disk files does make sense.
    The reason it doesn't work is because of an arbitrary decision to not let
    it work.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  4. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 17:14:41 +0200 Rainer Weikusat wrote:
    | Barry Margolin writes:
    |> In article <87k5hoos5t.fsf@fever.mssgmbh.com>,
    |> Rainer Weikusat wrote:
    |>
    |>> phil-news-nospam@ipal.net writes:
    |>> > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat
    |>> > wrote:
    |>>
    |>> [...]
    |>>
    |>> > | The original statement was that O_NONBLOCK wouldn't make any sense for
    |>> > | file descriptors which can't block on read. You then asked why the
    |>> > | person would believe 'that to be the case' and I replied that it is
    |>> > | 'the case' by definition, hence, not a matter of believing anything.
    |>> >
    |>> > "sense" has nothing to do with definitions.
    |>>
    |>> It does not make sense because O_NONBLOCK is defined to be a no-op for
    |>> this particular situation.
    |>
    |> But that's circular reasoning. WHY did they define it to be a no-op,
    |
    | Is this really so complicated? They did, consequently, using it
    | nevertheless makes no sense because it doesn't do anything.

    They could have simply changed the standard regarding O_NONBLOCK and disk
    files and allowed it to work. The only reasons I can see for them not to
    have done that are either 1: someone (you?) stood up and used circular
    reasoning, or 2: someone (you?) claimed that his programs depend on the
    fact that O_NONBLOCK is defined as a no-op and would fail if the standard
    is changed as such.


    |> since it makes sense to allow it.
    |
    | And that's an entirely different topic.

    When I said it makes sense, I did not specifically narrow the topic to using
    it strictly within the confines of the existing standard. It is a broader
    statement that absolutely does include going outside the scope of standards.

    But apparently you and I have a different understanding of semantics for the
    term "sense".


    |> File I/O is NOT really immediate, especially when networks are
    |> involved.
    |
    | By the time this mechanism was designed, there were no network file
    | systems. But this doesn't even really matter: Any 'device access', be
    | it disk, network or whatever (eg flash) is done asynchronously be the
    | kernel and the 'disk caching subsystem' is supposed to try as hard as
    | possible to maintain the illusion of 'immediate operation'.

    I copied 6 GB of files from one disk to another a couple hours ago. There
    was absolutely no illusion whatsoever of any 'immediate operation'.

    OTOH, back when that standard was established, no one would even think of
    trying to copy 6 GB of files from one disk to anther :-)


    |> As a result, when they decided it was useful to have a way to operate on
    |> files asynchronously, they had to design a new API, the aoi_* functions,
    |> rather than reuse the existing select()/poll() API.
    |
    | These were two completely different groups of people. And aio is
    | something completely different than 'non-blocking I/O': Asynchronous
    | I/O is supposed to be performed in parallell with other operations
    | done by the program, while a program using non-blocking I/O performs
    | its I/O operations synchronously at times when trying to do so will
    | not cause the process to be blocked in the kernel waiting for some
    | condition to become true before the actual I/O processing can start.

    The two don't work well together, either. A program needs to use one or the
    other, not both. Have you tried using AIO on a socket?

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  5. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    >On Wed, 21 May 2008 12:41:50 +0200 Rainer Weikusat wrote:
    >| phil-news-nospam@ipal.net writes:
    >|> On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat wrote:
    >|
    >| [...]
    >|
    >|> | The original statement was that O_NONBLOCK wouldn't make any sense for
    >|> | file descriptors which can't block on read. You then asked why the
    >|> | person would believe 'that to be the case' and I replied that it is
    >|> | 'the case' by definition, hence, not a matter of believing anything.
    >|>
    >|> "sense" has nothing to do with definitions.
    >|
    >| It does not make sense because O_NONBLOCK is defined to be a no-op for
    >| this particular situation.
    >
    >You really do have trouble understanding that I an suggesting that the
    >definition be changed.


    You seem to have trouble understanding that we collectively don't believe
    such a change would be useful.

    scott

  6. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    >On Wed, 21 May 2008 17:14:41 +0200 Rainer Weikusat wrote:
    >| Barry Margolin writes:
    >|> In article <87k5hoos5t.fsf@fever.mssgmbh.com>,
    >|> Rainer Weikusat wrote:
    >|>
    >|>> phil-news-nospam@ipal.net writes:
    >|>> > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat
    >|>> > wrote:
    >|>>
    >|>> [...]
    >|>>
    >|>> > | The original statement was that O_NONBLOCK wouldn't make any sense for
    >|>> > | file descriptors which can't block on read. You then asked why the
    >|>> > | person would believe 'that to be the case' and I replied that it is
    >|>> > | 'the case' by definition, hence, not a matter of believing anything.
    >|>> >
    >|>> > "sense" has nothing to do with definitions.
    >|>>
    >|>> It does not make sense because O_NONBLOCK is defined to be a no-op for
    >|>> this particular situation.
    >|>
    >|> But that's circular reasoning. WHY did they define it to be a no-op,
    >|
    >| Is this really so complicated? They did, consequently, using it
    >| nevertheless makes no sense because it doesn't do anything.
    >
    >They could have simply changed the standard regarding O_NONBLOCK and disk
    >files and allowed it to work. The only reasons I can see for them not to
    >have done that are either 1: someone (you?) stood up and used circular
    >reasoning, or 2: someone (you?) claimed that his programs depend on the
    >fact that O_NONBLOCK is defined as a no-op and would fail if the standard
    >is changed as such.


    I was part of the standards process for most of the 90's. The reason
    we never allowed O_NONBLOCK on regular files had nothing to do with
    "circular reasoning". It just didn't make sense. It still doesn't.

    scott

  7. Re: aio_read/write versus O_NONBLOCK

    On May 21, 3:47*pm, sc...@slp53.sl.home (Scott Lurndal) wrote:

    > You seem to have trouble understanding that we collectively don't believe
    > such a change would be useful.


    Certainly the typical non-blocking socket operations would be useless
    on a file. A way to block until you should call 'read' would be
    useless because the kernel would have no idea what data you wanted to
    read or how much. A way to block until you should call 'write' would
    similarly be useless because there is usually no reason to delay the
    'write' call.

    Now, imagine if we had a way to fix this problem. For example, instead
    of 'block for read', you'd want a function that could block but would
    know where you wanted to read and how many bytes. And instead of
    'block for write', you'd want a function that could let you know when
    a write finished. What you wind up with when you "fix non-blocking for
    files" is AIO.

    DS

  8. Re: aio_read/write versus O_NONBLOCK

    David Schwartz wrote:
    [snip]
    > Certainly the typical non-blocking socket operations would be useless
    > on a file. A way to block until you should call 'read' would be
    > useless because the kernel would have no idea what data you wanted to
    > read or how much.


    The analagous condition for "regular file ready for reading" is "at
    least one byte in memory from the current position." I see no conceptual
    problem with that - a non-blocking read() could return a short count as
    is the case for pipes/sockets.

    > A way to block until you should call 'write' would
    > similarly be useless because there is usually no reason to delay the
    > 'write' call.


    Do you mean there is usually no reason to postpone the call, or that the
    call will usually not take long to return? In any case, there are
    situations where write() will take a long time to return, and that is
    the reason to postpone the call.

    What am I missing?

    Alex

  9. Re: aio_read/write versus O_NONBLOCK

    On May 22, 10:38*am, Alex Fraser wrote:

    > The analagous condition for "regular file ready for reading" is "at
    > least one byte in memory from the current position." I see no conceptual
    > problem with that - a non-blocking read() could return a short count as
    > is the case for pipes/sockets.


    It would mean that the whole time you were waiting for readability,
    you could not move the file pointer. It would also mean that you
    couldn't use pread/pwrite because they don't move the file pointer.
    While I agree there's no conceptual problem per se, it would be so
    limited as to be basically useless.

    > > A way to block until you should call 'write' would
    > > similarly be useless because there is usually no reason to delay the
    > > 'write' call.


    > Do you mean there is usually no reason to postpone the call, or that the
    > call will usually not take long to return? In any case, there are
    > situations where write() will take a long time to return, and that is
    > the reason to postpone the call.
    >
    > What am I missing?


    How many bytes are you going to write? Where are you going to write
    them? It is quite common to want to wait for a write to complete, but
    almost senseless to wait before starting a write.

    DS

  10. Re: aio_read/write versus O_NONBLOCK

    David Schwartz wrote:
    > On May 22, 10:38 am, Alex Fraser wrote:
    >> The analagous condition for "regular file ready for reading" is "at
    >> least one byte in memory from the current position." I see no conceptual
    >> problem with that - a non-blocking read() could return a short count as
    >> is the case for pipes/sockets.

    >
    > It would mean that the whole time you were waiting for readability,
    > you could not move the file pointer. It would also mean that you
    > couldn't use pread/pwrite because they don't move the file pointer.
    > While I agree there's no conceptual problem per se, it would be so
    > limited as to be basically useless.


    I assume the third sentence is intended to follow from the first two. I
    don't see why the restrictions make it "so limited as to be basically
    useless." The basic restriction, avoided by an interface like AIO, seems
    to be the possibility of multiple oustanding I/O requests on a given
    file descriptor.

    >>> A way to block until you should call 'write' would
    >>> similarly be useless because there is usually no reason to delay the
    >>> 'write' call.

    >
    >> Do you mean there is usually no reason to postpone the call, or that the
    >> call will usually not take long to return? In any case, there are
    >> situations where write() will take a long time to return, and that is
    >> the reason to postpone the call.
    >>
    >> What am I missing?

    >
    > How many bytes are you going to write? Where are you going to write
    > them?


    Well, the answer to these should be obvious from what I previously wrote.

    > It is quite common to want to wait for a write to complete, but
    > almost senseless to wait before starting a write.


    I still don't get it . If write() may take an arbitrary time to return
    on a regular file just as it may for a pipe or socket, why is it
    "senseless to wait before starting a write" on a regular file but not a
    pipe or socket?

    Alex

  11. Re: aio_read/write versus O_NONBLOCK

    Alex Fraser writes:
    > David Schwartz wrote:
    >> On May 22, 10:38 am, Alex Fraser wrote:
    >>> The analagous condition for "regular file ready for reading" is "at
    >>> least one byte in memory from the current position." I see no conceptual
    >>> problem with that - a non-blocking read() could return a short count as
    >>> is the case for pipes/sockets.

    >> It would mean that the whole time you were waiting for readability,
    >> you could not move the file pointer. It would also mean that you
    >> couldn't use pread/pwrite because they don't move the file pointer.
    >> While I agree there's no conceptual problem per se, it would be so
    >> limited as to be basically useless.

    >
    > I assume the third sentence is intended to follow from the first
    > two. I don't see why the restrictions make it "so limited as to be
    > basically useless."


    >>>> A way to block until you should call 'write' would
    >>>> similarly be useless because there is usually no reason to delay the
    >>>> 'write' call.

    >>
    >>> Do you mean there is usually no reason to postpone the call, or that the
    >>> call will usually not take long to return? In any case, there are
    >>> situations where write() will take a long time to return, and that is
    >>> the reason to postpone the call.
    >>>
    >>> What am I missing?

    >> How many bytes are you going to write? Where are you going to write
    >> them?

    >
    > Well, the answer to these should be obvious from what I previously wrote.
    >
    >> It is quite common to want to wait for a write to complete, but
    >> almost senseless to wait before starting a write.

    >
    > I still don't get it . If write() may take an arbitrary time to
    > return on a regular file just as it may for a pipe or socket, why is
    > it "senseless to wait before starting a write" on a regular file but
    > not a pipe or socket?


    The first thing to remark is would be that a read which cannot make
    progress immediatly is the 'rare special case the application has no
    way of really knowing about' and that the same is true for a write
    which cannot immediatly make progess, ie copy the data into 'buffer
    memory' and then return.

    Already at this point, such functionality can be expected to be only
    rarely useful because 'I/O-cache' (page cache, block buffer cache)
    should really satisfy most I/O requests going to block devices,
    because if it doesn't, the system is going to perform really
    bad. There is ample empirical evidence that this caching actually
    works as intended, going back more than twenty-five years.

    The next problem would be that the existing sychronous I/O
    multiplexing primitives are not designed for random access files but
    for (implicitly) time-ordered streams, ie it is completely ok to
    create a socket and then call poll to wait until data to read is
    available. This cannot possibly work on a file. It would be possible
    to extend the semantics to work with files, but this would add lots of
    ugly special cases and 'weird' interface restrictions for (see above)
    very little benefit. Creating a new interface designed to support
    'asychronous file I/O' is a much more straight forward solution.

  12. Re: aio_read/write versus O_NONBLOCK

    On May 22, 10:38*pm, Alex Fraser wrote:

    > I still don't get it . If write() may take an arbitrary time to return
    > on a regular file just as it may for a pipe or socket, why is it
    > "senseless to wait before starting a write" on a regular file but not a
    > pipe or socket?


    Because a pipe or socket has its own buffer, and it makes sense to
    wait for space in that buffer. You can't see on a pipe or socket, so
    the write is always going to occur where the space is. There's also no
    ordering issue between writes to different pipes or sockets.

    On a file, there is no per-file write buffer. There's space in the
    whole buffer cache system, and there's no way to wait for space in
    that buffer. Even if you could wait for it, another process might
    steal it from you before you issue the 'write', leading to possible
    livelock. Waiting to issue the write robs the system of the ability to
    order the writes efficiently.

    DS

  13. Re: aio_read/write versus O_NONBLOCK

    David Schwartz wrote:
    > On May 22, 10:38 pm, Alex Fraser wrote:
    >
    >> I still don't get it . If write() may take an arbitrary time to return
    >> on a regular file just as it may for a pipe or socket, why is it
    >> "senseless to wait before starting a write" on a regular file but not a
    >> pipe or socket?

    >
    > Because a pipe or socket has its own buffer, and it makes sense to
    > wait for space in that buffer. You can't see on a pipe or socket, so
    > the write is always going to occur where the space is. There's also no
    > ordering issue between writes to different pipes or sockets.
    >
    > On a file, there is no per-file write buffer. There's space in the
    > whole buffer cache system, and there's no way to wait for space in
    > that buffer. Even if you could wait for it, another process might
    > steal it from you before you issue the 'write', leading to possible
    > livelock. Waiting to issue the write robs the system of the ability to
    > order the writes efficiently.


    So would you say that offering non-blocking semantics for files would
    impose an implementation (eg one with a per-file write buffer) that is,
    in other respects, less desirable? That would be a valid point, but is
    quite different to saying such semantics would be "useless".

    Alex

  14. Re: aio_read/write versus O_NONBLOCK

    Alex Fraser writes:
    > David Schwartz wrote:
    >> On May 22, 10:38 pm, Alex Fraser wrote:
    >>
    >>> I still don't get it . If write() may take an arbitrary time to return
    >>> on a regular file just as it may for a pipe or socket, why is it
    >>> "senseless to wait before starting a write" on a regular file but not a
    >>> pipe or socket?

    >> Because a pipe or socket has its own buffer, and it makes sense to
    >> wait for space in that buffer. You can't see on a pipe or socket, so
    >> the write is always going to occur where the space is. There's also no
    >> ordering issue between writes to different pipes or sockets.
    >> On a file, there is no per-file write buffer. There's space in the
    >> whole buffer cache system, and there's no way to wait for space in
    >> that buffer. Even if you could wait for it, another process might
    >> steal it from you before you issue the 'write', leading to possible
    >> livelock. Waiting to issue the write robs the system of the ability to
    >> order the writes efficiently.

    >
    > So would you say that offering non-blocking semantics for files would
    > impose an implementation (eg one with a per-file write buffer) that
    > is, in other respects, less desirable?


    The 'livelock' statement is kind-of a red herring because anything
    accessible via file descriptor can be shared among multiple processes
    (the filesystem interface can be regarded as design for shared
    access, though).

    > That would be a valid point, but is quite different to saying such
    > semantics would be "useless".


    The simple fact that actual I/O to such devices is already performed
    asynchronously would make them 'rarely useful', with 'rarely' being a
    situation which should be prevented from ever occuring. Adding any
    significant complication to an existing interface in order to benefit
    a 'should never happen' special-case is IMO quite useless. And the
    existing interfaces for non-blocking I/O cannot be used with random
    access files without adding such complications: 'Ready to read' (or
    write) is a condition which is not defined for such a file. It could
    only be derived as 'not being not ready for read (or write) to a
    specific area of such a file'.

    In the end, a suggestion may be 'download one of the kernels with
    source code available and implement whatever you consider to be useful
    as part of it'. A lot of people (me included) apparently believe this
    would not be worth the effort. They could be wrong. Other people
    believe something different could be wrong, too.


  15. Re: aio_read/write versus O_NONBLOCK

    Rainer Weikusat wrote:
    > Alex Fraser writes:

    [snip]
    >> So would you say that offering non-blocking semantics for files would
    >> impose an implementation (eg one with a per-file write buffer) that
    >> is, in other respects, less desirable?

    >
    > The 'livelock' statement is kind-of a red herring because anything
    > accessible via file descriptor can be shared among multiple processes
    > (the filesystem interface can be regarded as design for shared
    > access, though).


    Sockets and pipes have the same livelock problem (actually, it's more
    like "spurious wakeup" since some progress will usually be made, just
    very inefficiently); the difference is that it's restricted to the
    particular socket/pipe because that's what the buffer is attached to. A
    "per-file" buffer would be no different.

    >> That would be a valid point, but is quite different to saying such
    >> semantics would be "useless".

    >
    > The simple fact that actual I/O to such devices is already performed
    > asynchronously would make them 'rarely useful', with 'rarely' being a
    > situation which should be prevented from ever occuring.


    So AIO is "rarely useful" too?

    [snip]
    > In the end, a suggestion may be 'download one of the kernels with
    > source code available and implement whatever you consider to be useful
    > as part of it'. A lot of people (me included) apparently believe this
    > would not be worth the effort. They could be wrong. Other people
    > believe something different could be wrong, too.


    I think you misunderstand my position - I am not trying to say
    non-blocking I/O on files should be implemented, I am trying to
    understand why it hasn't been, ignoring the existence of AIO (or to put
    it another way, why AIO was created instead). So far, the only sound
    (IMO) reasons I can see are that it would be complicated to implement
    (the AIO interface seems to "fit in" better, based on my rudimentary
    understanding of typical block I/O caching/buffering implementations),
    and perhaps detrimental in other respects too. Do you know of any other
    reasons (which haven't already been mentioned)?

    Alex

  16. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 23:10:18 GMT Scott Lurndal wrote:
    | phil-news-nospam@ipal.net writes:
    |>On Wed, 21 May 2008 17:14:41 +0200 Rainer Weikusat wrote:
    |>| Barry Margolin writes:
    |>|> In article <87k5hoos5t.fsf@fever.mssgmbh.com>,
    |>|> Rainer Weikusat wrote:
    |>|>
    |>|>> phil-news-nospam@ipal.net writes:
    |>|>> > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat
    |>|>> > wrote:
    |>|>>
    |>|>> [...]
    |>|>>
    |>|>> > | The original statement was that O_NONBLOCK wouldn't make any sense for
    |>|>> > | file descriptors which can't block on read. You then asked why the
    |>|>> > | person would believe 'that to be the case' and I replied that it is
    |>|>> > | 'the case' by definition, hence, not a matter of believing anything.
    |>|>> >
    |>|>> > "sense" has nothing to do with definitions.
    |>|>>
    |>|>> It does not make sense because O_NONBLOCK is defined to be a no-op for
    |>|>> this particular situation.
    |>|>
    |>|> But that's circular reasoning. WHY did they define it to be a no-op,
    |>|
    |>| Is this really so complicated? They did, consequently, using it
    |>| nevertheless makes no sense because it doesn't do anything.
    |>
    |>They could have simply changed the standard regarding O_NONBLOCK and disk
    |>files and allowed it to work. The only reasons I can see for them not to
    |>have done that are either 1: someone (you?) stood up and used circular
    |>reasoning, or 2: someone (you?) claimed that his programs depend on the
    |>fact that O_NONBLOCK is defined as a no-op and would fail if the standard
    |>is changed as such.
    |
    | I was part of the standards process for most of the 90's. The reason
    | we never allowed O_NONBLOCK on regular files had nothing to do with
    | "circular reasoning". It just didn't make sense. It still doesn't.

    Then explain why it doesn't make sense to you (of course, without using any
    of the circular reasoning). Also, don't use false assertions like disk I/O
    is too fast to matter, and that no one needs to use O_NONBLOCK to overlap
    I/O cleanly.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  17. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 22:47:13 GMT Scott Lurndal wrote:
    | phil-news-nospam@ipal.net writes:
    |>On Wed, 21 May 2008 12:41:50 +0200 Rainer Weikusat wrote:
    |>| phil-news-nospam@ipal.net writes:
    |>|> On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat wrote:
    |>|
    |>| [...]
    |>|
    |>|> | The original statement was that O_NONBLOCK wouldn't make any sense for
    |>|> | file descriptors which can't block on read. You then asked why the
    |>|> | person would believe 'that to be the case' and I replied that it is
    |>|> | 'the case' by definition, hence, not a matter of believing anything.
    |>|>
    |>|> "sense" has nothing to do with definitions.
    |>|
    |>| It does not make sense because O_NONBLOCK is defined to be a no-op for
    |>| this particular situation.
    |>
    |>You really do have trouble understanding that I an suggesting that the
    |>definition be changed.
    |
    | You seem to have trouble understanding that we collectively don't believe
    | such a change would be useful.

    You seem to have trouble understanding TWO things:
    1. I really would find it useful.
    2. It wouldn't break anything to have allowed it.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  18. Re: aio_read/write versus O_NONBLOCK

    On Wed, 21 May 2008 20:19:47 -0700 (PDT) David Schwartz wrote:
    | On May 21, 3:47?pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
    |
    |> You seem to have trouble understanding that we collectively don't believe
    |> such a change would be useful.
    |
    | Certainly the typical non-blocking socket operations would be useless
    | on a file. A way to block until you should call 'read' would be
    | useless because the kernel would have no idea what data you wanted to
    | read or how much. A way to block until you should call 'write' would
    | similarly be useless because there is usually no reason to delay the
    | 'write' call.

    If you previously called read(2) then the kernel certainly can have an
    idea of what you wanted to read, both position and length. Note that
    it doesn't even have to ensure the full length is available because
    read(2) could also have been defined to provide whatever is available
    at that time (the classic "short read").

    Also, write(2) can be legitimately delayed because a previous write(2)
    was started and the descriptor or the filesystem has the O_SYNC property.


    | Now, imagine if we had a way to fix this problem. For example, instead
    | of 'block for read', you'd want a function that could block but would
    | know where you wanted to read and how many bytes. And instead of
    | 'block for write', you'd want a function that could let you know when
    | a write finished. What you wind up with when you "fix non-blocking for
    | files" is AIO.

    So explain exactly how one does poll(2) on a mix of descriptors that include
    files using AIO and sockets.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  19. Re: aio_read/write versus O_NONBLOCK

    On Fri, 23 May 2008 16:29:15 +0200 Rainer Weikusat wrote:

    | The first thing to remark is would be that a read which cannot make
    | progress immediatly is the 'rare special case the application has no
    | way of really knowing about' and that the same is true for a write
    | which cannot immediatly make progess, ie copy the data into 'buffer
    | memory' and then return.

    You do realize that if O_NONBLOCK is not used, nothing is different about
    read/write. A program that chooses to use O_NONBLOCK on a file would be
    expected to handle how it all works, just as it is expected to handle it
    for other things like serial ports and sockets.


    | Already at this point, such functionality can be expected to be only
    | rarely useful because 'I/O-cache' (page cache, block buffer cache)
    | should really satisfy most I/O requests going to block devices,
    | because if it doesn't, the system is going to perform really
    | bad. There is ample empirical evidence that this caching actually
    | works as intended, going back more than twenty-five years.

    If what is intended does not include being able to overlap I/O in a consistent
    manner (e.g. works the same for files as for all other things), then maybe so.


    | The next problem would be that the existing sychronous I/O
    | multiplexing primitives are not designed for random access files but
    | for (implicitly) time-ordered streams, ie it is completely ok to
    | create a socket and then call poll to wait until data to read is
    | available. This cannot possibly work on a file. It would be possible
    | to extend the semantics to work with files, but this would add lots of
    | ugly special cases and 'weird' interface restrictions for (see above)
    | very little benefit. Creating a new interface designed to support
    | 'asychronous file I/O' is a much more straight forward solution.

    Why do you say it cannot work on a file?

    Creating a new interface _just_ for files is what is really goofy. I/O
    interfaces should be completely uniform for everything I/O ever needs to
    be done for. If they had made AIO work on everything uniformly, then it
    would not be the outcast interface it is today.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  20. Re: aio_read/write versus O_NONBLOCK

    On Sat, 24 May 2008 19:23:28 +0100 Alex Fraser wrote:

    | So AIO is "rarely useful" too?

    It can be useful in some cases. It won't work in others.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

+ Reply to Thread
Page 2 of 5 FirstFirst 1 2 3 4 ... LastLast