aio_read/write versus O_NONBLOCK - Unix

This is a discussion on aio_read/write versus O_NONBLOCK - Unix ; Hi! Could someone point me to some articles or give me some hints on the advantages/disadvantages of asynchronous operations on files (aio_read/ aio_read) versus normal operations (read/write) used with O_NONBLOCK when opening a file. aio_read/aio_write are indeed more flexible. But, ...

+ Reply to Thread
Page 1 of 5 1 2 3 ... LastLast
Results 1 to 20 of 97

Thread: aio_read/write versus O_NONBLOCK

  1. aio_read/write versus O_NONBLOCK

    Hi!

    Could someone point me to some articles or give me some hints on the
    advantages/disadvantages of asynchronous operations on files (aio_read/
    aio_read) versus normal operations (read/write) used with O_NONBLOCK
    when opening a file.

    aio_read/aio_write are indeed more flexible. But, at least on Linux/
    glibc, they are implemented using POSIX threads. Wouldn't a carefully
    designed program using O_NONBLOCK for files best a program using
    aio_read/aio_write? I think my question should be: are asynchronous I/
    O operations only more flexible or are they also faster?

    So, if I were to design a raw speed-priority application, should I
    consider using aio_read/aio_write? (assuming I would be a very clever
    programmer and all the negative aspects related to using O_NONBLOCK
    would vanish).

    Razvan

  2. Re: aio_read/write versus O_NONBLOCK

    RazvanD wrote:

    > Could someone point me to some articles or give me some hints on the
    > advantages/disadvantages of asynchronous operations on files (aio_read/
    > aio_read) versus normal operations (read/write) used with O_NONBLOCK
    > when opening a file.


    Have you read the Wikipedia article?
    http://en.wikipedia.org/wiki/Asynchronous_I/O

    > So, if I were to design a raw speed-priority application [...]


    Raw speed and I/O in the same sentence sounds strange.
    Are you working with multiple 10-GigE interfaces? :-)
    Your problem statement smells like premature optimization.

  3. Re: aio_read/write versus O_NONBLOCK

    In article
    ,
    RazvanD wrote:

    > Hi!
    >
    > Could someone point me to some articles or give me some hints on the
    > advantages/disadvantages of asynchronous operations on files (aio_read/
    > aio_read) versus normal operations (read/write) used with O_NONBLOCK
    > when opening a file.
    >
    > aio_read/aio_write are indeed more flexible. But, at least on Linux/
    > glibc, they are implemented using POSIX threads. Wouldn't a carefully
    > designed program using O_NONBLOCK for files best a program using
    > aio_read/aio_write? I think my question should be: are asynchronous I/
    > O operations only more flexible or are they also faster?


    I'm not certain about Linux, but on many systems O_NONBLOCK has no
    effect on ordinary file streams.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  4. Re: aio_read/write versus O_NONBLOCK

    On May 13, 11:44 pm, Noob wrote:
    > RazvanD wrote:
    > > Could someone point me to some articles or give me some hints on the
    > > advantages/disadvantages of asynchronous operations on files (aio_read/
    > > aio_read) versus normal operations (read/write) used with O_NONBLOCK
    > > when opening a file.

    >
    > Have you read the Wikipedia article?http://en.wikipedia.org/wiki/Asynchronous_I/O
    >
    > > So, if I were to design a raw speed-priority application [...]

    >
    > Raw speed and I/O in the same sentence sounds strange.
    > Are you working with multiple 10-GigE interfaces? :-)
    > Your problem statement smells like premature optimization.


    I'm considering the C10K problem[1].

    Anyway, I have managed to find out that a new KAIO interface has been
    developed for Linux but the aio_read/aio_write interface isn't up to
    date. A non-pthread interface would be a better choice that using
    O_NONBLOCK.

    Razvan

    [1] http://www.kegel.com/c10k.html

  5. Re: aio_read/write versus O_NONBLOCK

    On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    | In article
    | ,
    | RazvanD wrote:
    |
    |> Hi!
    |>
    |> Could someone point me to some articles or give me some hints on the
    |> advantages/disadvantages of asynchronous operations on files (aio_read/
    |> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    |> when opening a file.
    |>
    |> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    |> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    |> designed program using O_NONBLOCK for files best a program using
    |> aio_read/aio_write? I think my question should be: are asynchronous I/
    |> O operations only more flexible or are they also faster?
    |
    | I'm not certain about Linux, but on many systems O_NONBLOCK has no
    | effect on ordinary file streams.

    It has not had any effect on ordinary file streams in Linux in the programs
    I have written that tried it (a couple of them).

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  6. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    >On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    >| In article
    >| ,
    >| RazvanD wrote:
    >|
    >|> Hi!
    >|>
    >|> Could someone point me to some articles or give me some hints on the
    >|> advantages/disadvantages of asynchronous operations on files (aio_read/
    >|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    >|> when opening a file.
    >|>
    >|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    >|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    >|> designed program using O_NONBLOCK for files best a program using
    >|> aio_read/aio_write? I think my question should be: are asynchronous I/
    >|> O operations only more flexible or are they also faster?
    >|
    >| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    >| effect on ordinary file streams.
    >
    >It has not had any effect on ordinary file streams in Linux in the programs
    >I have written that tried it (a couple of them).
    >


    It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    for file descriptors that can't block on a read (i.e. disk-based files).

    scott

  7. Re: aio_read/write versus O_NONBLOCK

    On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    | phil-news-nospam@ipal.net writes:
    |>On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    |>| In article
    |>| ,
    |>| RazvanD wrote:
    |>|
    |>|> Hi!
    |>|>
    |>|> Could someone point me to some articles or give me some hints on the
    |>|> advantages/disadvantages of asynchronous operations on files (aio_read/
    |>|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    |>|> when opening a file.
    |>|>
    |>|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    |>|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    |>|> designed program using O_NONBLOCK for files best a program using
    |>|> aio_read/aio_write? I think my question should be: are asynchronous I/
    |>|> O operations only more flexible or are they also faster?
    |>|
    |>| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    |>| effect on ordinary file streams.
    |>
    |>It has not had any effect on ordinary file streams in Linux in the programs
    |>I have written that tried it (a couple of them).
    |>
    |
    | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    | for file descriptors that can't block on a read (i.e. disk-based files).

    Could you explain why you believe that to be the case?

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  8. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    >On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    >| phil-news-nospam@ipal.net writes:
    >|>On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    >|>| In article
    >|>| ,
    >|>| RazvanD wrote:
    >|>|
    >|>|> Hi!
    >|>|>
    >|>|> Could someone point me to some articles or give me some hints on the
    >|>|> advantages/disadvantages of asynchronous operations on files (aio_read/
    >|>|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    >|>|> when opening a file.
    >|>|>
    >|>|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    >|>|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    >|>|> designed program using O_NONBLOCK for files best a program using
    >|>|> aio_read/aio_write? I think my question should be: are asynchronous I/
    >|>|> O operations only more flexible or are they also faster?
    >|>|
    >|>| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    >|>| effect on ordinary file streams.
    >|>
    >|>It has not had any effect on ordinary file streams in Linux in the programs
    >|>I have written that tried it (a couple of them).
    >|>
    >|
    >| It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    >| for file descriptors that can't block on a read (i.e. disk-based files).
    >
    >Could you explain why you believe that to be the case?
    >


    It's simple. The disk is always there, so a read must always complete
    in a bounded time. O_NONBLOCK was added to Unix to handle cases where
    a blocking read can be unbounded (serial ports, parallel ports, network
    ports).

    Just like poll(2) and select(2) always rval a disk-based file descriptor
    as readable and writeable, if they support disk-based files at all.

    scott

  9. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    > On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:


    [...]

    > | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    > | for file descriptors that can't block on a read (i.e. disk-based files).
    >
    > Could you explain why you believe that to be the case?


    This is not a matter of 'belief': Non-blocking is defined as:

    A property of an open file description that causes function
    calls involving it to return without delay when it is detected
    that the requested action associated with the function call
    cannot be completed without unknown delay.
    (SUS 3.240)

    Having to wait for disk I/O is not considered to be 'an unknown delay'
    because the I/O will always complete within a relatively short amount
    of time, cf

    Regular files shall always poll TRUE for reading and writing.
    (SUS, poll)

    and

    File descriptors associated with regular files shall always
    select true for ready to read, ready to write,
    (SUS, select)

    while 'blocked waiting for I/O' usually means 'suspended until some
    external event occurs', ie user types input at a terminal or a packet
    arrives from the network, which could just never happen. On Linux (and
    presumably other systems), the difference manifests itself as the
    process being in state 'S' (sleeping) while it is considered to be
    'blocked' and in state 'D' (uninterruptible sleep) while waiting for
    disk I/O to complete (which can backfire big time when 'disk I/O'
    isn't really 'disk I/O' but 'communication with some network file
    server', specifcally, NFS).

  10. Re: aio_read/write versus O_NONBLOCK

    On Thu, 15 May 2008 21:56:27 GMT Scott Lurndal wrote:
    | phil-news-nospam@ipal.net writes:
    |>On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    |>| phil-news-nospam@ipal.net writes:
    |>|>On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    |>|>| In article
    |>|>| ,
    |>|>| RazvanD wrote:
    |>|>|
    |>|>|> Hi!
    |>|>|>
    |>|>|> Could someone point me to some articles or give me some hints on the
    |>|>|> advantages/disadvantages of asynchronous operations on files (aio_read/
    |>|>|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    |>|>|> when opening a file.
    |>|>|>
    |>|>|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    |>|>|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    |>|>|> designed program using O_NONBLOCK for files best a program using
    |>|>|> aio_read/aio_write? I think my question should be: are asynchronous I/
    |>|>|> O operations only more flexible or are they also faster?
    |>|>|
    |>|>| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    |>|>| effect on ordinary file streams.
    |>|>
    |>|>It has not had any effect on ordinary file streams in Linux in the programs
    |>|>I have written that tried it (a couple of them).
    |>|>
    |>|
    |>| It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    |>| for file descriptors that can't block on a read (i.e. disk-based files).
    |>
    |>Could you explain why you believe that to be the case?
    |>
    |
    | It's simple. The disk is always there, so a read must always complete
    | in a bounded time. O_NONBLOCK was added to Unix to handle cases where
    | a blocking read can be unbounded (serial ports, parallel ports, network
    | ports).

    But a bounded time is not zero time. There are things that can be done
    while a disk is being read.


    | Just like poll(2) and select(2) always rval a disk-based file descriptor
    | as readable and writeable, if they support disk-based files at all.

    It would be useful if one could do poll(2) or select(2) on descriptors open
    to 2 or more disk devices or files when doing multiple I/O. A typical example
    is copying a file from one disk to another. Using poll(2) would allow which
    descriptor really becomes ready next to be handled right then. Both I/O
    operations could be easily overlapped. Of course it is the case that the OS
    does buffer the writes. In some cases (Linux) it still does this badly so
    it is still useful to do things like O_SYNC or O_DIRECT. Otherwise I have
    to do things like multiprocessing with pipes or multithreading. It just gets
    messy to do what would otherwise be simple if O_NONBLOCK worked on files and
    disk devices.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  11. Re: aio_read/write versus O_NONBLOCK

    On Fri, 16 May 2008 12:48:27 +0200 Rainer Weikusat wrote:
    | phil-news-nospam@ipal.net writes:
    |> On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    |
    | [...]
    |
    |> | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    |> | for file descriptors that can't block on a read (i.e. disk-based files).
    |>
    |> Could you explain why you believe that to be the case?
    |
    | This is not a matter of 'belief': Non-blocking is defined as:
    |
    | A property of an open file description that causes function
    | calls involving it to return without delay when it is detected
    | that the requested action associated with the function call
    | cannot be completed without unknown delay.
    | (SUS 3.240)

    I was asking him why he thought it did not make sense. I was not asking
    about whether the standard specifies it, or not.

    Go back into my many posts many years ago and you can see I proposed that
    these standards be changed to allow using O_NONBLOCK on files and disks.


    | Having to wait for disk I/O is not considered to be 'an unknown delay'
    | because the I/O will always complete within a relatively short amount
    | of time, cf
    |
    | Regular files shall always poll TRUE for reading and writing.
    | (SUS, poll)
    |
    | and
    |
    | File descriptors associated with regular files shall always
    | select true for ready to read, ready to write,
    | (SUS, select)
    |
    | while 'blocked waiting for I/O' usually means 'suspended until some
    | external event occurs', ie user types input at a terminal or a packet
    | arrives from the network, which could just never happen. On Linux (and
    | presumably other systems), the difference manifests itself as the
    | process being in state 'S' (sleeping) while it is considered to be
    | 'blocked' and in state 'D' (uninterruptible sleep) while waiting for
    | disk I/O to complete (which can backfire big time when 'disk I/O'
    | isn't really 'disk I/O' but 'communication with some network file
    | server', specifcally, NFS).

    NFS issues aside, I still say O_NONBLOCK would be useful even with local disk
    files and devices. It would allow, for example, convenient I/O overlapping
    without having to resort to more complex means with multi{process,thread}ing.

    It's more a case of allowing it to be done for those that want to do it. If
    it is allowed, it would not change anything that doesn't use O_NONBLOCK for
    such descriptors.

    How we dug a hole now where there are programs that _depend_ on O_NONBLOCK
    _not_ working with disk files and devices?

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  12. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    > On Fri, 16 May 2008 12:48:27 +0200 Rainer Weikusat wrote:
    > | phil-news-nospam@ipal.net writes:
    > |> On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    > |
    > | [...]
    > |
    > |> | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    > |> | for file descriptors that can't block on a read (i.e. disk-based files).
    > |>
    > |> Could you explain why you believe that to be the case?
    > |
    > | This is not a matter of 'belief': Non-blocking is defined as:
    > |
    > | A property of an open file description that causes function
    > | calls involving it to return without delay when it is detected
    > | that the requested action associated with the function call
    > | cannot be completed without unknown delay.
    > | (SUS 3.240)
    >
    > I was asking him why he thought it did not make sense. I was not asking
    > about whether the standard specifies it, or not.


    It does not make sense, because using O_NONBLOCK with disk files is
    defined to be a no op.

    > Go back into my many posts many years ago and you can see I proposed that
    > these standards be changed to allow using O_NONBLOCK on files and
    > disks.


    That does not change the fact that it is a no op.

  13. Re: aio_read/write versus O_NONBLOCK

    On Mon, 19 May 2008 11:10:30 +0200 Rainer Weikusat wrote:
    | phil-news-nospam@ipal.net writes:
    |> On Fri, 16 May 2008 12:48:27 +0200 Rainer Weikusat wrote:
    |> | phil-news-nospam@ipal.net writes:
    |> |> On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    |> |
    |> | [...]
    |> |
    |> |> | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    |> |> | for file descriptors that can't block on a read (i.e. disk-based files).
    |> |>
    |> |> Could you explain why you believe that to be the case?
    |> |
    |> | This is not a matter of 'belief': Non-blocking is defined as:
    |> |
    |> | A property of an open file description that causes function
    |> | calls involving it to return without delay when it is detected
    |> | that the requested action associated with the function call
    |> | cannot be completed without unknown delay.
    |> | (SUS 3.240)
    |>
    |> I was asking him why he thought it did not make sense. I was not asking
    |> about whether the standard specifies it, or not.
    |
    | It does not make sense, because using O_NONBLOCK with disk files is
    | defined to be a no op.

    You sound like one of those people that, when someone suggested that action
    FOO be legalized, replies that it cannot be legalized because it is against
    the law to do FOO.

    When I ask this of the OP, I ask why it needs to be a no-op, not whether the
    standard has defined this.

    But at the same time, I am curious how many programs _depend_ on this aspect
    of the standard being correctly implemented. How many programs will break
    if O_NONBLOCK is not a no-op for disk files and devices?


    |> Go back into my many posts many years ago and you can see I proposed that
    |> these standards be changed to allow using O_NONBLOCK on files and
    |> disks.
    |
    | That does not change the fact that it is a no op.

    I never said it wasn't.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  14. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    > On Mon, 19 May 2008 11:10:30 +0200 Rainer Weikusat wrote:
    > | phil-news-nospam@ipal.net writes:
    > |> On Fri, 16 May 2008 12:48:27 +0200 Rainer Weikusat wrote:
    > |> | phil-news-nospam@ipal.net writes:
    > |> |> On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    > |> |
    > |> | [...]
    > |> |
    > |> |> | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    > |> |> | for file descriptors that can't block on a read (i.e. disk-based files).
    > |> |>
    > |> |> Could you explain why you believe that to be the case?
    > |> |
    > |> | This is not a matter of 'belief': Non-blocking is defined as:
    > |> |
    > |> | A property of an open file description that causes function
    > |> | calls involving it to return without delay when it is detected
    > |> | that the requested action associated with the function call
    > |> | cannot be completed without unknown delay.
    > |> | (SUS 3.240)
    > |>
    > |> I was asking him why he thought it did not make sense. I was not asking
    > |> about whether the standard specifies it, or not.
    > |
    > | It does not make sense, because using O_NONBLOCK with disk files is
    > | defined to be a no op.
    >
    > You sound like one of those people that, when someone suggested that action
    > FOO be legalized, replies that it cannot be legalized because it is against
    > the law to do FOO.
    >
    > When I ask this of the OP, I ask why it needs to be a no-op, not whether the
    > standard has defined this.


    The original statement was that O_NONBLOCK wouldn't make any sense for
    file descriptors which can't block on read. You then asked why the
    person would believe 'that to be the case' and I replied that it is
    'the case' by definition, hence, not a matter of believing anything.

    The text above does not state anything regarding if it would be
    possible to define useful semantics for 'non-blocking disk I/O' and/
    or if defining such semantics would be an important
    improvement. Discussing this topic would again lead to 'how the UNIX
    disk cache actually works' really fast and that's a topic I am not
    going to get into with you again.

  15. Re: aio_read/write versus O_NONBLOCK

    On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat wrote:
    | phil-news-nospam@ipal.net writes:
    |> On Mon, 19 May 2008 11:10:30 +0200 Rainer Weikusat wrote:
    |> | phil-news-nospam@ipal.net writes:
    |> |> On Fri, 16 May 2008 12:48:27 +0200 Rainer Weikusat wrote:
    |> |> | phil-news-nospam@ipal.net writes:
    |> |> |> On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    |> |> |
    |> |> | [...]
    |> |> |
    |> |> |> | It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    |> |> |> | for file descriptors that can't block on a read (i.e. disk-based files).
    |> |> |>
    |> |> |> Could you explain why you believe that to be the case?
    |> |> |
    |> |> | This is not a matter of 'belief': Non-blocking is defined as:
    |> |> |
    |> |> | A property of an open file description that causes function
    |> |> | calls involving it to return without delay when it is detected
    |> |> | that the requested action associated with the function call
    |> |> | cannot be completed without unknown delay.
    |> |> | (SUS 3.240)
    |> |>
    |> |> I was asking him why he thought it did not make sense. I was not asking
    |> |> about whether the standard specifies it, or not.
    |> |
    |> | It does not make sense, because using O_NONBLOCK with disk files is
    |> | defined to be a no op.
    |>
    |> You sound like one of those people that, when someone suggested that action
    |> FOO be legalized, replies that it cannot be legalized because it is against
    |> the law to do FOO.
    |>
    |> When I ask this of the OP, I ask why it needs to be a no-op, not whether the
    |> standard has defined this.
    |
    | The original statement was that O_NONBLOCK wouldn't make any sense for
    | file descriptors which can't block on read. You then asked why the
    | person would believe 'that to be the case' and I replied that it is
    | 'the case' by definition, hence, not a matter of believing anything.

    "sense" has nothing to do with definitions.


    | The text above does not state anything regarding if it would be
    | possible to define useful semantics for 'non-blocking disk I/O' and/
    | or if defining such semantics would be an important
    | improvement. Discussing this topic would again lead to 'how the UNIX
    | disk cache actually works' really fast and that's a topic I am not
    | going to get into with you again.

    It doesn't need to go there. I'm suggesting that O_NONBLOCK does make
    sense for disk files and devices, despite any caching.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  16. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    >On Thu, 15 May 2008 21:56:27 GMT Scott Lurndal wrote:
    >| phil-news-nospam@ipal.net writes:
    >|>On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    >|>| phil-news-nospam@ipal.net writes:
    >|>|>On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    >|>|>| In article
    >|>|>| ,
    >|>|>| RazvanD wrote:
    >|>|>|
    >|>|>|> Hi!
    >|>|>|>
    >|>|>|> Could someone point me to some articles or give me some hints on the
    >|>|>|> advantages/disadvantages of asynchronous operations on files (aio_read/
    >|>|>|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    >|>|>|> when opening a file.
    >|>|>|>
    >|>|>|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    >|>|>|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    >|>|>|> designed program using O_NONBLOCK for files best a program using
    >|>|>|> aio_read/aio_write? I think my question should be: are asynchronous I/
    >|>|>|> O operations only more flexible or are they also faster?
    >|>|>|
    >|>|>| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    >|>|>| effect on ordinary file streams.
    >|>|>
    >|>|>It has not had any effect on ordinary file streams in Linux in the programs
    >|>|>I have written that tried it (a couple of them).
    >|>|>
    >|>|
    >|>| It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    >|>| for file descriptors that can't block on a read (i.e. disk-based files).
    >|>
    >|>Could you explain why you believe that to be the case?
    >|>
    >|
    >| It's simple. The disk is always there, so a read must always complete
    >| in a bounded time. O_NONBLOCK was added to Unix to handle cases where
    >| a blocking read can be unbounded (serial ports, parallel ports, network
    >| ports).
    >
    >But a bounded time is not zero time. There are things that can be done
    >while a disk is being read.


    I don't think you really understand how file-based I/O works in linux.
    In the majority of cases, by the time you issue the read(2)/pread(2) system
    call, it is likely that the data has already been read from the disk.

    In any case, what you want is asychronous I/O, rather than non-blocking I/O.

    See aio_read/aio_write/lio_listio.

    Btw, if you're copying one file to another, use mmap/madvise
    not read/pread/write/pwrite.

    scott

  17. Re: aio_read/write versus O_NONBLOCK

    On Tue, 20 May 2008 23:49:36 GMT Scott Lurndal wrote:
    | phil-news-nospam@ipal.net writes:
    |>On Thu, 15 May 2008 21:56:27 GMT Scott Lurndal wrote:
    |>| phil-news-nospam@ipal.net writes:
    |>|>On Thu, 15 May 2008 18:55:43 GMT Scott Lurndal wrote:
    |>|>| phil-news-nospam@ipal.net writes:
    |>|>|>On Wed, 14 May 2008 00:13:59 -0400 Barry Margolin wrote:
    |>|>|>| In article
    |>|>|>| ,
    |>|>|>| RazvanD wrote:
    |>|>|>|
    |>|>|>|> Hi!
    |>|>|>|>
    |>|>|>|> Could someone point me to some articles or give me some hints on the
    |>|>|>|> advantages/disadvantages of asynchronous operations on files (aio_read/
    |>|>|>|> aio_read) versus normal operations (read/write) used with O_NONBLOCK
    |>|>|>|> when opening a file.
    |>|>|>|>
    |>|>|>|> aio_read/aio_write are indeed more flexible. But, at least on Linux/
    |>|>|>|> glibc, they are implemented using POSIX threads. Wouldn't a carefully
    |>|>|>|> designed program using O_NONBLOCK for files best a program using
    |>|>|>|> aio_read/aio_write? I think my question should be: are asynchronous I/
    |>|>|>|> O operations only more flexible or are they also faster?
    |>|>|>|
    |>|>|>| I'm not certain about Linux, but on many systems O_NONBLOCK has no
    |>|>|>| effect on ordinary file streams.
    |>|>|>
    |>|>|>It has not had any effect on ordinary file streams in Linux in the programs
    |>|>|>I have written that tried it (a couple of them).
    |>|>|>
    |>|>|
    |>|>| It's pretty much of a waste of time. O_NONBLOCK doesn't make any sense
    |>|>| for file descriptors that can't block on a read (i.e. disk-based files).
    |>|>
    |>|>Could you explain why you believe that to be the case?
    |>|>
    |>|
    |>| It's simple. The disk is always there, so a read must always complete
    |>| in a bounded time. O_NONBLOCK was added to Unix to handle cases where
    |>| a blocking read can be unbounded (serial ports, parallel ports, network
    |>| ports).
    |>
    |>But a bounded time is not zero time. There are things that can be done
    |>while a disk is being read.
    |
    | I don't think you really understand how file-based I/O works in linux.
    | In the majority of cases, by the time you issue the read(2)/pread(2) system
    | call, it is likely that the data has already been read from the disk.

    Why do you not think I really understand how file-based I/O works in Linux?
    When the data is already read from disk, then read(2) will be expected to
    not return EAGAIN when the descriptor is set to O_NONBLOCK.


    | In any case, what you want is asychronous I/O, rather than non-blocking I/O.
    |
    | See aio_read/aio_write/lio_listio.

    So how does one make a process wait for these asyncronous I/O requests AND AT
    THE SAME TIME ALSO wait for descriptors that represent other things such as
    sockets? Function aio_suspend(3) does not seem to be the way to do this.


    | Btw, if you're copying one file to another, use mmap/madvise
    | not read/pread/write/pwrite.

    That's what I'm doing now. But msync(2) does not provide a non-blocking way to
    determine I/O completion. That's why I need to find something better. And the
    O_NONBLOCK and poll(2) way would be just right if it were allowed for disk files
    and devices.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  18. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:
    > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat wrote:


    [...]

    > | The original statement was that O_NONBLOCK wouldn't make any sense for
    > | file descriptors which can't block on read. You then asked why the
    > | person would believe 'that to be the case' and I replied that it is
    > | 'the case' by definition, hence, not a matter of believing anything.
    >
    > "sense" has nothing to do with definitions.


    It does not make sense because O_NONBLOCK is defined to be a no-op for
    this particular situation.

    [...]

    > | The text above does not state anything regarding if it would be
    > | possible to define useful semantics for 'non-blocking disk I/O' and/
    > | or if defining such semantics would be an important
    > | improvement. Discussing this topic would again lead to 'how the UNIX
    > | disk cache actually works' really fast and that's a topic I am not
    > | going to get into with you again.
    >
    > It doesn't need to go there. I'm suggesting that O_NONBLOCK does make
    > sense for disk files and devices, despite any caching.


    You are suggesting that it would be possible to define other semantics
    for O_NONBLOCK on disk files then those which are presently
    defined. Until this has happened, O_NOBLOCK makes no sense in
    association with disk files.

  19. Re: aio_read/write versus O_NONBLOCK

    phil-news-nospam@ipal.net writes:

    [...]

    > | Just like poll(2) and select(2) always rval a disk-based file descriptor
    > | as readable and writeable, if they support disk-based files at all.
    >
    > It would be useful if one could do poll(2) or select(2) on descriptors open
    > to 2 or more disk devices or files when doing multiple I/O.
    > A typical example is copying a file from one disk to another.


    UNIX(*) application do not 'copy files from one disk to another', but
    copy the contents of one file to some other file. Anything beyond
    that is supposed to be handled transparently by the kernel and not by
    an application.


  20. Re: aio_read/write versus O_NONBLOCK

    In article <87k5hoos5t.fsf@fever.mssgmbh.com>,
    Rainer Weikusat wrote:

    > phil-news-nospam@ipal.net writes:
    > > On Tue, 20 May 2008 12:34:51 +0200 Rainer Weikusat
    > > wrote:

    >
    > [...]
    >
    > > | The original statement was that O_NONBLOCK wouldn't make any sense for
    > > | file descriptors which can't block on read. You then asked why the
    > > | person would believe 'that to be the case' and I replied that it is
    > > | 'the case' by definition, hence, not a matter of believing anything.
    > >
    > > "sense" has nothing to do with definitions.

    >
    > It does not make sense because O_NONBLOCK is defined to be a no-op for
    > this particular situation.


    But that's circular reasoning. WHY did they define it to be a no-op,
    since it makes sense to allow it. File I/O is NOT really immediate,
    especially when networks are involved.

    As a result, when they decided it was useful to have a way to operate on
    files asynchronously, they had to design a new API, the aoi_* functions,
    rather than reuse the existing select()/poll() API.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

+ Reply to Thread
Page 1 of 5 1 2 3 ... LastLast