how to return from a blocked read() ? - Linux

This is a discussion on how to return from a blocked read() ? - Linux ; Hello, I have a thread blocking on a read() call. Does anyone know how to make a blocking read call return from another thread? It seems like it would not be difficult but I have searched far and wide for ...

+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 20 of 24

Thread: how to return from a blocked read() ?

  1. how to return from a blocked read() ?

    Hello,

    I have a thread blocking on a read() call. Does anyone know how to
    make a blocking read call return from another thread? It seems like it
    would not be difficult but I have searched far and wide for this
    answer to no avail... unless i've been missing something.

    Thanks in advance.

    Bob


  2. Re: how to return from a blocked read() ?

    On Jul 16, 10:53 am, Bob wrote:
    > Hello,
    >
    > I have a thread blocking on a read() call. Does anyone know how to
    > make a blocking read call return from another thread? It seems like it
    > would not be difficult but I have searched far and wide for this
    > answer to no avail... unless i've been missing something.


    If it's a TCP connection, use 'shutdown' on the connection. If it's
    anything else, don't use a blocking read if you don't want to block
    until data is available.

    By the way, your question has an erroneous premise. You do not know
    that the thread is blocked on the 'read' call. It could just as well
    be *about* to block on the 'read' call. This is why 'close' will not
    work.

    DS


  3. Re: how to return from a blocked read() ?

    On Jul 16, 2:49 pm, David Schwartz wrote:
    > On Jul 16, 10:53 am, Bob wrote:
    >
    > > Hello,

    >
    > > I have a thread blocking on a read() call. Does anyone know how to
    > > make a blocking read call return from another thread? It seems like it
    > > would not be difficult but I have searched far and wide for this
    > > answer to no avail... unless i've been missing something.

    >
    > If it's a TCP connection, use 'shutdown' on the connection. If it's
    > anything else, don't use a blocking read if you don't want to block
    > until data is available.
    >
    > By the way, your question has an erroneous premise. You do not know
    > that the thread is blocked on the 'read' call. It could just as well
    > be *about* to block on the 'read' call. This is why 'close' will not
    > work.
    >
    > DS


    David,

    Thanks for the reply. I am reading an open file descriptor (Gadget
    Serial driver). I don't understand what you mean by "about to block".
    Could you clarify? The whole reason for this question is that I am
    running Linux on an embedded ARM. I am using gadget serial so the
    embedded app can be a USB "device". I read() waiting for data to come
    in. If the USB plug is disconnected, the read() is not returning and
    my thread is blocked. I'm monitoring the Gadget driver in another
    thread by cat /proc/kmsg. I can clearly see when the USB is
    disconnected and connected, I just can't get a darn return from that
    read call to clean things up.

    Bob


  4. Re: how to return from a blocked read() ?

    On Jul 16, 12:01 pm, Bob wrote:

    > Thanks for the reply. I am reading an open file descriptor (Gadget
    > Serial driver). I don't understand what you mean by "about to block".
    > Could you clarify?


    How do you know the other thread is blocked in 'read'? The thread
    looks like this:

    A: do_some_stuff();
    B: fd=something();
    C: read(...

    Now, you say the thread is blocked in read. But how do you know this?
    It could have completed line B, the scheduler switched threads, and
    hasn't gotten a chance to get back to that thread yet.

    You do not know it is actually blocked in 'read'. You know that is
    either blocked in 'read' or going to do so at some point in the
    future.

    There is no atomic 'unlock and read' function, so you cannot be sure
    the thread has blocked in 'read' yet. This is why 'close' will not
    work. Consider:

    1) The thread that's going to call 'read' is about to call 'read'. It
    has made the decision to do so.

    2) Another thread runs, calls 'close' to try to unblock thread 1,
    sadly, the thread was not actually blocked (but how could it know?)

    3) Another thread runs, it opens a new connection and gets the same
    descriptor that was just closed. It writes some data and is about to
    call 'read' to look for a reply. (Note that this thread could be in a
    library or something, you may have no knowledge of its existence.)

    4) The first thread runs again and reads the data the thread in step 3
    was about to read, totally screwing up the thread that ran in step 3.

    > The whole reason for this question is that I am
    > running Linux on an embedded ARM. I am using gadget serial so the
    > embedded app can be a USB "device". I read() waiting for data to come
    > in. If the USB plug is disconnected, the read() is not returning and
    > my thread is blocked. I'm monitoring the Gadget driver in another
    > thread by cat /proc/kmsg. I can clearly see when the USB is
    > disconnected and connected, I just can't get a darn return from that
    > read call to clean things up.


    One way might be to send a signal to that thread. But why not just
    leave the thread blocked in 'read'? Are you trying to clean up the
    whole process? Or do you plan to keep running? If so, the thread will
    eventually return, right?

    DS


  5. Re: how to return from a blocked read() ?

    On Jul 16, 3:10 pm, David Schwartz wrote:
    > On Jul 16, 12:01 pm, Bob wrote:
    >
    > > Thanks for the reply. I am reading an open file descriptor (Gadget
    > > Serial driver). I don't understand what you mean by "about to block".
    > > Could you clarify?

    >
    > How do you know the other thread is blocked in 'read'? The thread
    > looks like this:
    >
    > A: do_some_stuff();
    > B: fd=something();
    > C: read(...
    >
    > Now, you say the thread is blocked in read. But how do you know this?
    > It could have completed line B, the scheduler switched threads, and
    > hasn't gotten a chance to get back to that thread yet.
    >
    > You do not know it is actually blocked in 'read'. You know that is
    > either blocked in 'read' or going to do so at some point in the
    > future.
    >
    > There is no atomic 'unlock and read' function, so you cannot be sure
    > the thread has blocked in 'read' yet. This is why 'close' will not
    > work. Consider:
    >
    > 1) The thread that's going to call 'read' is about to call 'read'. It
    > has made the decision to do so.
    >
    > 2) Another thread runs, calls 'close' to try to unblock thread 1,
    > sadly, the thread was not actually blocked (but how could it know?)
    >
    > 3) Another thread runs, it opens a new connection and gets the same
    > descriptor that was just closed. It writes some data and is about to
    > call 'read' to look for a reply. (Note that this thread could be in a
    > library or something, you may have no knowledge of its existence.)
    >
    > 4) The first thread runs again and reads the data the thread in step 3
    > was about to read, totally screwing up the thread that ran in step 3.
    >
    > > The whole reason for this question is that I am
    > > running Linux on an embedded ARM. I am using gadget serial so the
    > > embedded app can be a USB "device". I read() waiting for data to come
    > > in. If the USB plug is disconnected, the read() is not returning and
    > > my thread is blocked. I'm monitoring the Gadget driver in another
    > > thread by cat /proc/kmsg. I can clearly see when the USB is
    > > disconnected and connected, I just can't get a darn return from that
    > > read call to clean things up.

    >
    > One way might be to send a signal to that thread. But why not just
    > leave the thread blocked in 'read'? Are you trying to clean up the
    > whole process? Or do you plan to keep running? If so, the thread will
    > eventually return, right?
    >
    > DS


    David,

    Thanks again. I'm quite sure that I'm in the read() call since the
    only thing that I'm doing is calling read() and monitoring the USB
    driver. I've been trying different things for a couple of days now and
    in the time that we have been posting, I might have found the answer
    although I'm not quite sure why.

    I find that if I:

    read() in thread "A"
    Monitor USB in thread "B"
    detect USB disconnect using monitor thread "B"
    In thread "B", send signal SIGTSTP to thread "A" to stop it
    In thread "B", close thread "A's" file descriptor
    In thread "B", send signal SIGCONT to continue thread "A"
    VOILA, the read() in thread "A" returns

    Lots of trial and error to figure this out.

    Bob


  6. Re: how to return from a blocked read() ?

    On Jul 16, 12:44 pm, Bob wrote:

    > Thanks again. I'm quite sure that I'm in the read() call since the
    > only thing that I'm doing is calling read() and monitoring the USB
    > driver.


    No, you are not quite sure. You could be *about* to call 'read'. There
    is no way you can be sure the thread is in 'read' because there is no
    distinction between "blocked in read" and "about to call read" that is
    visible from another thread.

    > I've been trying different things for a couple of days now and
    > in the time that we have been posting, I might have found the answer
    > although I'm not quite sure why.
    >
    > I find that if I:
    >
    > read() in thread "A"
    > Monitor USB in thread "B"
    > detect USB disconnect using monitor thread "B"
    > In thread "B", send signal SIGTSTP to thread "A" to stop it
    > In thread "B", close thread "A's" file descriptor
    > In thread "B", send signal SIGCONT to continue thread "A"
    > VOILA, the read() in thread "A" returns
    >
    > Lots of trial and error to figure this out.


    Sorry, it's broken. After thread B calls 'close', another thread could
    open a new connection getting the same descriptor, causing thread A to
    call 'read' on the wrong thing.

    DS


  7. Re: how to return from a blocked read() ?

    On Jul 16, 5:53 pm, David Schwartz wrote:
    > On Jul 16, 12:44 pm, Bob wrote:
    >
    > No, you are not quite sure. You could be *about* to call 'read'. There
    > is no way you can be sure the thread is in 'read' because there is no
    > distinction between "blocked in read" and "about to call read" that is
    > visible from another thread.
    >
    >

    Sure I can. I wrote the thread procedure. When I get the process ID of
    the thread in question in my other thread I know what state it's in..
    because I KNOW that nothing else would be blocking it. Therefore, if
    it's blocking, it's in the read. There is no doubt about it.
    Absitivelyposilutely. What it sounds like you are suggesting is that a
    thread could NEVER discern that another thread was in a blocking read
    call, and that is just not the case, as evidenced by this example.
    >
    > Sorry, it's broken. After thread B calls 'close', another thread could
    > open a new connection getting the same descriptor, causing thread A to
    > call 'read' on the wrong thing.


    It doesn't matter. That thread can use that descriptor... I don't need
    it anymore. When I clean up, I'll be opening another file descriptor
    which can be anything... and thread B will know what that descriptor
    is, and start monitoring the USB all over again until the next
    disconnect.



  8. Re: how to return from a blocked read() ?

    On Jul 16, 3:51 pm, Bob wrote:

    > On Jul 16, 5:53 pm, David Schwartz wrote:> On Jul 16, 12:44 pm, Bob wrote:


    > > No, you are not quite sure. You could be *about* to call 'read'. There
    > > is no way you can be sure the thread is in 'read' because there is no
    > > distinction between "blocked in read" and "about to call read" that is
    > > visible from another thread.


    > Sure I can. I wrote the thread procedure.


    It doesn't matter. At some point in the code, the following line must
    appear:

    read(something);

    You cannot possibly know whether the thread is block in read or
    *about* to call read. It is literally impossible.

    > When I get the process ID of
    > the thread in question in my other thread I know what state it's in..


    How? What function can tell you whether the thread is blocked in read
    or about to call read?

    > because I KNOW that nothing else would be blocking it. Therefore, if
    > it's blocking, it's in the read. There is no doubt about it.


    How do you know it's blocked? There is no function 'IsThreadBlocked'.
    The thread could simply be waiting for the scheduler to schedule it.

    > Absitivelyposilutely. What it sounds like you are suggesting is that a
    > thread could NEVER discern that another thread was in a blocking read
    > call, and that is just not the case, as evidenced by this example.


    Correct. A thread can never tell that another thread is in a blocking
    read call.

    Any code that relies on another thread being in a blocking read call
    is broken since you can never, ever be sure this is the case. The
    'read' could just have returned. The thread could be about to call
    'read'. Or it could be blocked in 'read'. There is literally no way
    you can tell.

    > > Sorry, it's broken. After thread B calls 'close', another thread could
    > > open a new connection getting the same descriptor, causing thread A to
    > > call 'read' on the wrong thing.


    > It doesn't matter. That thread can use that descriptor... I don't need
    > it anymore. When I clean up, I'll be opening another file descriptor
    > which can be anything... and thread B will know what that descriptor
    > is, and start monitoring the USB all over again until the next
    > disconnect.


    It does matter. If the thread was about to call 'read', it will still
    call 'read'. This could cause it to consume data that was intended for
    another thread entirely. This can not only cause huge problems for
    that unrelated connection but it can also cause the original thread to
    process unrelated data and never learn that its origin descriptor was
    closed.

    DS


  9. Re: how to return from a blocked read() ?

    On Jul 16, 6:53 pm, Bob wrote:
    > Hello,
    >
    > I have a thread blocking on a read() call. Does anyone know how to
    > make a blocking read call return from another thread? It seems like it
    > would not be difficult but I have searched far and wide for this
    > answer to no avail... unless i've been missing something.


    You could write data into the other end of the pipe. I've
    done this from the shell when debugging:

    1) connect debugger to blocked thread
    2) from a terminal, write data to /proc/pid/fd/n
    3) watch the read call return.

    You could do something similar and have the writing
    thread indicate to the reading thread that it has
    inserted bogus data (eg have the reader check
    a flag when the read returns).

    I suspect that this is not really a good idea, but
    it's doable.


  10. Re: how to return from a blocked read() ?

    Bob wrote:
    [...]
    > I read() waiting for data to come
    > in. If the USB plug is disconnected, the read() is not returning and
    > my thread is blocked.


    If the underlying hardware goes away, I would have thought that you oughtto
    get an EIO on any pending IO using that hardware --- this looks a bit like a
    device driver bug to me.

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "There does not now, nor will there ever, exist a programming language in
    │ which it is the least bit hard to write bad programs." --- Flon's Axiom

  11. Re: how to return from a blocked read() ?

    On Jul 17, 6:54 am, David Given wrote:
    > Bob wrote:
    >
    > [...]
    >
    > > I read() waiting for data to come
    > > in. If the USB plug is disconnected, the read() is not returning and
    > > my thread is blocked.

    >
    > If the underlying hardware goes away, I would have thought that you ought to
    > get an EIO on any pending IO using that hardware --- this looks a bit like a
    > device driver bug to me.
    >
    > --
    > http://www.cowlark.com
    >
    > "There does not now, nor will there ever, exist a programming language in
    > which it is the least bit hard to write bad programs." --- Flon's Axiom


    Yes that is exactly right, it is a device driver bug, that's been one
    of the most frustrating parts of this. Thanks for your constructive
    input.


  12. Re: how to return from a blocked read() ?

    On Jul 17, 12:19 am, David Schwartz wrote:
    > On Jul 16, 3:51 pm, Bob wrote:
    >
    > > On Jul 16, 5:53 pm, David Schwartz wrote:> On Jul 16, 12:44 pm, Bob wrote:
    > > > No, you are not quite sure. You could be *about* to call 'read'. There
    > > > is no way you can be sure the thread is in 'read' because there is no
    > > > distinction between "blocked in read" and "about to call read" that is
    > > > visible from another thread.

    > > Sure I can. I wrote the thread procedure.

    >
    > It doesn't matter. At some point in the code, the following line must
    > appear:
    >
    > read(something);
    >
    > You cannot possibly know whether the thread is block in read or
    > *about* to call read. It is literally impossible.
    >
    > > When I get the process ID of
    > > the thread in question in my other thread I know what state it's in..

    >
    > How? What function can tell you whether the thread is blocked in read
    > or about to call read?
    >
    > > because I KNOW that nothing else would be blocking it. Therefore, if
    > > it's blocking, it's in the read. There is no doubt about it.

    >
    > How do you know it's blocked? There is no function 'IsThreadBlocked'.
    > The thread could simply be waiting for the scheduler to schedule it.
    >
    > > Absitivelyposilutely. What it sounds like you are suggesting is that a
    > > thread could NEVER discern that another thread was in a blocking read
    > > call, and that is just not the case, as evidenced by this example.

    >
    > Correct. A thread can never tell that another thread is in a blocking
    > read call.
    >
    > Any code that relies on another thread being in a blocking read call
    > is broken since you can never, ever be sure this is the case. The
    > 'read' could just have returned. The thread could be about to call
    > 'read'. Or it could be blocked in 'read'. There is literally no way
    > you can tell.
    >
    > > > Sorry, it's broken. After thread B calls 'close', another thread could
    > > > open a new connection getting the same descriptor, causing thread A to
    > > > call 'read' on the wrong thing.

    > > It doesn't matter. That thread can use that descriptor... I don't need
    > > it anymore. When I clean up, I'll be opening another file descriptor
    > > which can be anything... and thread B will know what that descriptor
    > > is, and start monitoring the USB all over again until the next
    > > disconnect.

    >
    > It does matter. If the thread was about to call 'read', it will still
    > call 'read'. This could cause it to consume data that was intended for
    > another thread entirely. This can not only cause huge problems for
    > that unrelated connection but it can also cause the original thread to
    > process unrelated data and never learn that its origin descriptor was
    > closed.
    >
    > DS


    Then you do not understand what I'm doing, and that doesn't surprise
    me, because you now have made a whole bunch of assumptions without
    even knowing what is going on in my code. When I originally posted, I
    purposely kept the post about the essence of my question ( How can you
    return from a read call?) The reason that I kept the question brief is
    because inevitably someone like you will pounce to show everyone how
    smart you are, all the while forgetting to address the problem,
    thereby showing how un-smart you are. Then, I end up having to explain
    things that I didn't need to get in to, and that others who are trying
    to learn don't need to get in to. After your original dismissive
    response which I patiently and politely responded to, I checked back
    on some of your posts. Yes, you are one of them. You get off on
    arguing and showing people your "brilliance" Perhaps you should step
    back, take a breath, and remember what these forums are for; to help
    people. They shouldn't be used as a place to get back at all the kids
    who picked on you when you were back in school. Please don't respond
    to any more posts that I may put up in this forum, you are absolutely
    no help; a hindrance in fact.


  13. Re: how to return from a blocked read() ?

    On Jul 17, 4:57 am, Bob wrote:

    > Then you do not understand what I'm doing, and that doesn't surprise
    > me, because you now have made a whole bunch of assumptions without
    > even knowing what is going on in my code. When I originally posted, I
    > purposely kept the post about the essence of my question ( How can you
    > return from a read call?) The reason that I kept the question brief is
    > because inevitably someone like you will pounce to show everyone how
    > smart you are, all the while forgetting to address the problem,
    > thereby showing how un-smart you are. Then, I end up having to explain
    > things that I didn't need to get in to, and that others who are trying
    > to learn don't need to get in to. After your original dismissive
    > response which I patiently and politely responded to, I checked back
    > on some of your posts. Yes, you are one of them. You get off on
    > arguing and showing people your "brilliance" Perhaps you should step
    > back, take a breath, and remember what these forums are for; to help
    > people. They shouldn't be used as a place to get back at all the kids
    > who picked on you when you were back in school. Please don't respond
    > to any more posts that I may put up in this forum, you are absolutely
    > no help; a hindrance in fact.


    I am trying to help you, and at the same time I'm trying to prevent
    you from spreading mistakes to others. I can't help it if you
    militantly refuse to learn.

    What you are doing is broken. It has caused other people pain and will
    cause you pain.

    You have made a literally impossible claim, that you know that one of
    your threads is in fact blocked in 'read' as opposed to about to call
    'read'.

    I am not trying to show you how smart I am, but rather how mistaken
    you are.

    The first step to understanding how to solve your problem is
    understanding your problem. You do not understand your problem, so it
    is almost impossible to help you solve it.

    DS


  14. Re: how to return from a blocked read() ?

    On Jul 17, 4:57 am, Bob wrote:
    > On Jul 17, 12:19 am, David Schwartz wrote:
    >
    >
    >
    > > On Jul 16, 3:51 pm, Bob wrote:

    >
    > > > On Jul 16, 5:53 pm, David Schwartz wrote:> On Jul 16, 12:44 pm, Bob wrote:
    > > > > No, you are not quite sure. You could be *about* to call 'read'. There
    > > > > is no way you can be sure the thread is in 'read' because there is no
    > > > > distinction between "blocked in read" and "about to call read" that is
    > > > > visible from another thread.
    > > > Sure I can. I wrote the thread procedure.

    >
    > > It doesn't matter. At some point in the code, the following line must
    > > appear:

    >
    > > read(something);

    >
    > > You cannot possibly know whether the thread is block in read or
    > > *about* to call read. It is literally impossible.

    >
    > > > When I get the process ID of
    > > > the thread in question in my other thread I know what state it's in..

    >
    > > How? What function can tell you whether the thread is blocked in read
    > > or about to call read?

    >
    > > > because I KNOW that nothing else would be blocking it. Therefore, if
    > > > it's blocking, it's in the read. There is no doubt about it.

    >
    > > How do you know it's blocked? There is no function 'IsThreadBlocked'.
    > > The thread could simply be waiting for the scheduler to schedule it.

    >
    > > > Absitivelyposilutely. What it sounds like you are suggesting is that a
    > > > thread could NEVER discern that another thread was in a blocking read
    > > > call, and that is just not the case, as evidenced by this example.

    >
    > > Correct. A thread can never tell that another thread is in a blocking
    > > read call.

    >
    > > Any code that relies on another thread being in a blocking read call
    > > is broken since you can never, ever be sure this is the case. The
    > > 'read' could just have returned. The thread could be about to call
    > > 'read'. Or it could be blocked in 'read'. There is literally no way
    > > you can tell.

    >
    > > > > Sorry, it's broken. After thread B calls 'close', another thread could
    > > > > open a new connection getting the same descriptor, causing thread A to
    > > > > call 'read' on the wrong thing.
    > > > It doesn't matter. That thread can use that descriptor... I don't need
    > > > it anymore. When I clean up, I'll be opening another file descriptor
    > > > which can be anything... and thread B will know what that descriptor
    > > > is, and start monitoring the USB all over again until the next
    > > > disconnect.

    >
    > > It does matter. If the thread was about to call 'read', it will still
    > > call 'read'. This could cause it to consume data that was intended for
    > > another thread entirely. This can not only cause huge problems for
    > > that unrelated connection but it can also cause the original thread to
    > > process unrelated data and never learn that its origin descriptor was
    > > closed.

    >
    > > DS

    >
    > Then you do not understand what I'm doing, and that doesn't surprise
    > me, because you now have made a whole bunch of assumptions without
    > even knowing what is going on in my code. When I originally posted, I
    > purposely kept the post about the essence of my question ( How can you
    > return from a read call?) The reason that I kept the question brief is
    > because inevitably someone like you will pounce to show everyone how
    > smart you are, all the while forgetting to address the problem,
    > thereby showing how un-smart you are.


    The fact is, if I presented you with the correct solution, you would
    ignore it because you don't understand the question. The correct
    solution is:

    1) Set the descriptor non-blocking.

    2) Use 'select' on both the descriptor and the output end of a 'pipe'.

    3) You can get the thread out of 'select' by writing a byte to the
    input end of that pipe, the thread will never block in 'read'.

    However, I know from experience that you will ignore this solution
    because you think what you have now is correct. It is not.

    > Then, I end up having to explain
    > things that I didn't need to get in to, and that others who are trying
    > to learn don't need to get in to.


    What?! If you are in some strange one-in-a-million situation, you had
    damn well better explain it if you care at all about others who are
    trying to learn. Otherwise, they will assume the situation applies to
    them too, when it does not.

    > After your original dismissive
    > response which I patiently and politely responded to, I checked back
    > on some of your posts. Yes, you are one of them. You get off on
    > arguing and showing people your "brilliance" Perhaps you should step
    > back, take a breath, and remember what these forums are for; to help
    > people.


    Giving you a broken solution doesn't help you or anyone else reading
    this.

    If you have some miraculous way to know the thread is blocked in
    'read', please, share it with everyone else. Otherwise, at a minimum,
    all the other people who don't know that their thread is blocked in
    'read' will find the solution of no use to them.

    Real code has broken horribly due to precisely this kind of mistake.

    > They shouldn't be used as a place to get back at all the kids
    > who picked on you when you were back in school. Please don't respond
    > to any more posts that I may put up in this forum, you are absolutely
    > no help; a hindrance in fact.


    You are welcome to go on making mistakes and being immune to
    education, but I will not let you pass your recklessness on to others.
    It is damn near certain that you do not know the thread is blocked in
    read and that you are doing the wrong thing for precisely the reasons
    I explained. If this were not the case, it would take you only a few
    words to explain how you know this, much less than you have wasted
    being upset about the fact that I seem smarter than you.

    DS


  15. Re: how to return from a blocked read() ?

    On Jul 17, 4:48 am, Bob wrote:

    > Yes that is exactly right, it is a device driver bug, that's been one
    > of the most frustrating parts of this. Thanks for your constructive
    > input.


    You can argue that it's a feature, as it allows you to wait for the
    device to reconnect should it be about to do so. But I agree with you
    that the problems this creates outweigh the advantages. For one thing,
    even if the device does 'come right back', odds are it will come back
    in a reset state. So it's unlikely that continuing where you left off
    is going to be good and could possibly be quite bad.

    What happens if the device comes back?

    DS


  16. Re: how to return from a blocked read() ?

    David Schwartz writes:
    > On Jul 17, 4:57 am, Bob wrote:


    [...]

    >> Then you do not understand what I'm doing, and that doesn't surprise
    >> me, because you now have made a whole bunch of assumptions without
    >> even knowing what is going on in my code. When I originally posted, I
    >> purposely kept the post about the essence of my question ( How can you
    >> return from a read call?) The reason that I kept the question brief is
    >> because inevitably someone like you will pounce to show everyone how
    >> smart you are, all the while forgetting to address the problem,
    >> thereby showing how un-smart you are.

    >
    > The fact is, if I presented you with the correct solution, you would
    > ignore it because you don't understand the question. The correct
    > solution is:
    >
    > 1) Set the descriptor non-blocking.
    >
    > 2) Use 'select' on both the descriptor and the output end of a 'pipe'.
    >
    > 3) You can get the thread out of 'select' by writing a byte to the
    > input end of that pipe, the thread will never block in 'read'.


    That is at best one of serveral possibilities to accomplish a
    userspace wakeup, and it does not address the original problem except
    'also'. It is, for instance, conceivable that their is additional
    locking around the read, meaning, the intention is to wake up the
    thread if it happens to blocked in read, instead of having been caught
    somewhere else. I have a program which does just that and which uses
    pthread_kill to get all threads out of a blocking recv that are
    actually blocked in the recv, while all threads which are not are
    handled differently. During peak usages times, it presently handles in
    excess of 2500 requests coming in over the net per minute and the
    wakeup is needed once per hour, so doing two syscalls per request,
    instead of one would be a bad tradeoff.

  17. Re: how to return from a blocked read() ?

    Bob wrote:
    [...]
    > Then you do not understand what I'm doing, and that doesn't surprise
    > me, because you now have made a whole bunch of assumptions without
    > even knowing what is going on in my code.


    Imagine the following code.

    int myread(int fd, char* buffer, size_t bsize)
    {
    /* 1 */
    return read(fd, buffer, bsize); /* <-- 2 is inside read() */
    /* 3 */
    }

    Thread A calls myread(). While that's executing, thread B notices the USB
    disconnect and sends the SIGTSTP to suspend thread A, as per your original
    algorithm.

    Now we know that thread A has been suspended. But we don't know *where* it has
    been suspended. It could be (simplified) at any of points 1, 2 or 3.

    If it's at point 2, then thread A is blocked inside the kernel. Closing the
    file descriptor will cause the thread to become unblocked; read() will return;
    myread() will return.

    If it's at point 3, then thread A has completed the read and has been
    preempted. Thread A resumes and no problem occurs.

    The issue is at point 1. If thread A gets suspended at point 1, then it hasn't
    started reading yet. Thread B has closed the file descriptor, which meansthat
    fd is no longer valid. Therefore the read() should fail with a EBADFD, right?
    Well... no. The problem is thread C. If thread C happens to run between the
    point where thread B has closed the file descriptor and thread A calls read(),
    it *might* call open(), allocating a new file descriptor, *on the same number*
    as fd. Which means that when thread A calls read(fd), it'll read from thenew
    file descriptor instead of the original one. This will cause hideous, hideous
    bugs that will be really hard to diagnose. It's a classic race condition of
    the worst kind.

    Here are the three rules of multithreaded programming:

    1. Multiple threads that access shared data must always synchronise when
    necessary.

    2. It is always necessary.

    3. (For experts only) Sometimes you get lucky.

    Depending on your situation your code here may fall under (3) --- if you only
    have two threads, for example, there is no thread C and therefore you canbe
    (reasonably) sure that the above race condition won't happen. As you say,we
    don't know your initial assumptions. But the algorithm you posted is one of
    the perfect examples of an algorithm that will *mostly* work --- but not always.

    --
    ┌── dg*cowlark.com ─── http://www.cowlark.com ──────────────── ──

    │ "There does not now, nor will there ever, exist a programming language in
    │ which it is the least bit hard to write bad programs." --- Flon's Axiom

  18. Re: how to return from a blocked read() ?

    David Given writes:
    > Bob wrote:
    > [...]
    >> Then you do not understand what I'm doing, and that doesn't surprise
    >> me, because you now have made a whole bunch of assumptions without
    >> even knowing what is going on in my code.

    >
    > Imagine the following code.
    >
    > int myread(int fd, char* buffer, size_t bsize)
    > {
    > /* 1 */
    > return read(fd, buffer, bsize); /* <-- 2 is inside read() */
    > /* 3 */
    > }
    >
    > Thread A calls myread(). While that's executing, thread B notices the USB
    > disconnect and sends the SIGTSTP to suspend thread A, as per your original
    > algorithm.
    >
    > Now we know that thread A has been suspended. But we don't know *where* it has
    > been suspended. It could be (simplified) at any of points 1, 2 or 3.
    >
    > If it's at point 2, then thread A is blocked inside the kernel. Closing the
    > file descriptor will cause the thread to become unblocked; read() will return;
    > myread() will return.
    >
    > If it's at point 3, then thread A has completed the read and has been
    > preempted. Thread A resumes and no problem occurs.
    >
    > The issue is at point 1. If thread A gets suspended at point 1, then it hasn't
    > started reading yet. Thread B has closed the file descriptor, which means that
    > fd is no longer valid. Therefore the read() should fail with a EBADFD, right?
    > Well... no. The problem is thread C.


    [...]

    > But the algorithm you posted is one of the perfect examples of an
    > algorithm that will *mostly* work --- but not always.


    There was no 'thread C' in the posted example. This means that your
    claim about 'mostly working algorithms' has a hidden premise, namely
    'assuming there was a thread C doing unsychronized open calls, it
    could happen that ...'. Which reduces your statement regarding the
    'mostly working algorithm' to 'this algorithm cannot solve any problem
    I (meaning you) can imagine'.

    And that is a pretty useless statement.



  19. Re: how to return from a blocked read() ?

    My response to you got lost in the ether, but the gist of it is this:
    It is not possible to help you until you realize what the actual
    problem is. Your statement of the problem is almost certainly mistake
    because you almost certainly don't know for sure that the thread is
    blocked in 'read'. There is no standard way to acquire this
    information.

    It's remotely possible that you have this information some extremely
    strange way. You could theoretically confirm that the thread is
    blocked in 'read' by looking in the kernel after you know that the
    device is gone. You would then know that there is no way for the
    thread to exit 'read'. You would also need to know that re-attaching
    the device does not unblock a read. I think the odds that this is your
    actual situation are extremely remote.

    In any event, if you are in this kind of one-in-a-million situation,
    it's very important that one of us make this perfectly clear to
    everyone else. Otherwise, they may think the solution applies to them.
    A lot of real-world code has broken with painful consequences because
    of race conditions just like this one.

    The correct solution to your problem, assuming you're not in the one-
    in-a-million situation where you somehow interrogate the kernel and
    confirm the thread is in 'read' and somehow also know that it can't
    leave 'read' before you can complete your operation is as follows:

    1) Set the descriptor non-blocking.
    2) Create a pipe.
    3) Block in 'select' rather than 'read', adding the read end of the
    pipe to the readability set.
    4) To unblock the thread from 'select', write a byte to the write end
    of the pipe.
    5) Now you can know the thread is *not* blocked in 'read' because the
    descriptor is non-blocking.

    However, until you appreciate that you don't know that the thread is
    blocked in 'read', it's very unlikely that you would be willing and
    able to code this correctly.

    As for helping people, I have seen precisely the type of mistake you
    seem militantly insistent on making causing *real* damage. In one
    case, a program sent sensitive data to the wrong person because a
    'close' in one thread occurred just before a 'write' in another
    thread. The writing thread, when it finally got back to 'write', wrote
    the sensitive data to another client.

    You have said several times that you *know* the thread is blocked in
    'read' as opposed to being about to call 'read'. So, I give up, how do
    you know this?

    DS


  20. Re: how to return from a blocked read() ?

    On Jul 18, 2:25 am, Rainer Weikusat wrote:

    > That is at best one of serveral possibilities to accomplish a
    > userspace wakeup, and it does not address the original problem except
    > 'also'.


    The original problem, as stated, cannot be solved. That's why I didn't
    offer an immediate solution.

    > It is, for instance, conceivable that their is additional
    > locking around the read, meaning, the intention is to wake up the
    > thread if it happens to blocked in read, instead of having been caught
    > somewhere else.


    It can't be done. You would need an atomic 'unlock and read' function,
    which does not exist.

    Theoretically, you could ask the kernel or threading library if the
    thread is blocked in a kernel function, but there would still be the
    risk that the thread might exit 'read' before you got a chance to call
    'close'. One can imagine situations where this might not be the case,
    but as far as I know, no actual person has ever encountered one.

    > I have a program which does just that and which uses
    > pthread_kill to get all threads out of a blocking recv that are
    > actually blocked in the recv, while all threads which are not are
    > handled differently.


    That you can do. Because in this case, it makes no difference whether
    the thread is killed before, during, or after the 'recv' because you
    aren't releasing a resource it might or might not be using. You can
    also set cancellation so that the 'recv' is your only cancellation
    point.

    I generally don't recommend thread cancellation to people because of
    the complexity. It's also kind of a personal bias, since the thread
    libraries I first worked on didn't support cancellation properly and I
    got personally burned a bit.

    But it's a very good suggestion I think.

    > During peak usages times, it presently handles in
    > excess of 2500 requests coming in over the net per minute and the
    > wakeup is needed once per hour, so doing two syscalls per request,
    > instead of one would be a bad tradeoff.


    I presume you're talking about a program that handles a very small
    number of connections. I would think, unless that number is one, the
    extra context switches wouldn't be worth it. Every time you call a
    blocking 'read' while there are other things your process could be
    doing, you force a context switch if the 'read' blocks.

    DS


+ Reply to Thread
Page 1 of 2 1 2 LastLast