Last portion SSL_read only after a SSL_write . Please Help. - Openssl

This is a discussion on Last portion SSL_read only after a SSL_write . Please Help. - Openssl ; Hallo I made a simple SSL Client on win32 The request and response works perfectly when the amount of data is small enough I made a request with SSL_write Servers answers with "..200 OK ..." etc SSL_read gives 7159 bytes. ...

+ Reply to Thread
Results 1 to 17 of 17

Thread: Last portion SSL_read only after a SSL_write . Please Help.

  1. Last portion SSL_read only after a SSL_write . Please Help.


    Hallo

    I made a simple SSL Client on win32
    The request and response works perfectly
    when the amount of data is small enough


    I made a request with SSL_write
    Servers answers with "..200 OK ..." etc

    SSL_read gives 7159 bytes.
    SSL_get_error gives allways SSL_ERROR_NONE
    and therefor the loop stops

    but there are 975 bytes missing.

    and when i call SSL_read again it blocks.


    My progamm then want to make another request.
    again invoking SSL_write and SSL_read.

    Then this SSL_read give the missing 975 bytes.

    The request is ignored, i think

    What happens here ?



    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18717834.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  2. RE: Last portion SSL_read only after a SSL_write . Please Help.


    > Hallo
    >
    > I made a simple SSL Client on win32
    > The request and response works perfectly
    > when the amount of data is small enough
    >
    > I made a request with SSL_write
    > Servers answers with "..200 OK ..." etc
    >
    > SSL_read gives 7159 bytes.
    > SSL_get_error gives allways SSL_ERROR_NONE
    > and therefor the loop stops


    This sounds like broken logic. The loop should stop when all the data has
    been received. Why are you calling SSL_get_error when there was no error
    indication?

    > but there are 975 bytes missing.


    So why did your loop stop? You need to keep looping until you receive all
    the data.

    > and when i call SSL_read again it blocks.


    I'm not sure exactly what you mean by this. If your loop stopped, why would
    you call SSL_read again? Are you combining the results from two different
    test runs? Or do you mean something other than the obvious by "the loop
    stops".

    > My progamm then want to make another request.
    > again invoking SSL_write and SSL_read.
    >
    > Then this SSL_read give the missing 975 bytes.


    Because you didn't read them before, leaving them in the buffer. By why
    would your program send a request before it receives the entire reply from
    the previous request anyway? Are you intentionally trying to pipeline? If
    so, then that's what happens when you pipeline, you sometimes get part of
    the reply to the first request after you send the second. If you aren't
    pipelining intentionally, why are you sending the second query before you
    get the response to the first?

    > The request is ignored, i think
    >
    > What happens here ?


    This sounds like a case of a common type of "debugging gone wrong" scenario.
    You have a problem, and you change a whole bunch of things to try to fix it.
    You add an extra SSL_read call after you think you're done reading. You
    don't get the whole reply, so you send the next request anyway, just to keep
    going. As a result, after a few iterations you have your original problem
    plus all the broken logic to added to try to "work around" it.

    Take out all the nonsense and get back to your original problem. Send your
    request. Keep calling SSL_read until you either get an error or the entire
    reply. Then move onto the next request.

    Tell us what goes wrong with that simple case. And don't bastardize it
    trying to fix it.

    DS


    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  3. Re: Last portion SSL_read only after a SSL_write . Please Help.



    Thank you for the fast answer. :-)

    On
    ftp://erbse:erbse@77.190.173.220/
    there is the request/response string and the function SSL_WR_RD().C

    but SSL_read says "there is no more to read" stops the loop
    because SSL_get_error give SSL_ERROR_NONE and return from function.
    The request/response data must end with "..."

    All the source examples I found use SSL_get_error to find out
    "there is no more to read".

    But this obviously doesn't work. Perhabs doesnt work on w32 machines.

    You are right I have to make the next request when all is read.
    The question is, how can I detect the read has finished ?


    thanks Peter






    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18725602.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  4. Re: Last portion SSL_read only after a SSL_write . Please Help.

    Sounds to me like a case of a pending renegotiation or other alarm
    signal happening and application level code not allowing this due to
    your particular implementation of read/write loops, but that is a long
    shot, as it may be due to several other issues as well.


    To quickly get that Win32 item out of the way: I've been using OpenSSL
    on Windows for almost a decade now: no issues that differ from UNIX
    (unless you count a few very subtle TCP/IP protocol stack buggers
    which are most definitely irrelevant here).


    Sorry, I did/do not have time to have a close look at your code (and
    FTP is disabled in the corporate firewall). You did not mention in
    email if you are using blocking or non-blocking I/O, so here's the
    general idea that applies to both cases:


    Check your code; compare to the implementation of s_client and the
    other samples provided with OpenSSL (yeah, yeah, I know, it sometimes
    looks a bit cluttered in there, but heck, it's sample code, not
    production quality ready to go) and particularly look at the way the
    OpenSSL sample code checks for pending incoming data and permission to
    send additional data. Also check how your code compares to s_client et
    al when it comes to handling WANT_READ and other particulars
    (renegotiation, handshaking, ...); I've met a few issues like yours
    when code did not care about those error codes when they appeared,
    barging on into a lockup a while later in the process.

    Best advice I can give you this morning (when time to answer email is
    almost over for me): get the O'Reilly OpenSSL book; pay particular
    attention to the chapters discussing blocking and non-blocking I/O.
    Keep in mind that you should best code your SSL I/O using a single
    loop handling both read & write operations at the same time,
    especially when using non-blocking I/O.


    On the subject of 'end of transmission':
    sounds like you're using a HTTP Keep-Alive 'persistent' connection
    here on top of SSL/TCP: in that case the plaintext should contain a
    header which informs you how many bytes will follow in the answer from
    the server. When using custom 'persistent' protocols, the same sort of
    thing applies as well: if an 'exchange' is not terminated by an 'end
    of connection', you (a) have to inform the receiver upfront how much
    will follow, so you can cut the incoming data into proper 'pieces'
    (messages), and (b) you may run into additional 'fun' with TCP as it
    is a stream-based and /not/ a message-based protocol, so there's a big
    chance the last few raw bytes are lingering in the buffers at the
    server side, waiting to be bundled with additional data to follow.
    Which will only happen when you request more, or when a (rather long)
    timeout expires.
    When subsequent requests (client-side transmit) are not dependent on
    each other, you're safe, but should remind yourself at all times, in
    both blocking and non-blocking I/O cases, that response data comes in
    asynchronously: do NOT expect to receive all data for request N-1
    before sending request N when you are still sitting on top of the very
    same TCP connection. That's a TCP thing, not SSL, BTW. Even though
    OpenSSL does some additional buffering on it's own as well.
    Heck, you don't have any guarantee whatsoever the data for request N-2
    (N-3, ...) will have arrived by the time you transmit request N: if
    the reponses are small enough, they'll be bundled in the server-side
    TCP stack buffers before transmission (to reduce network load due to
    otherwise flooding the pipe with small TCP packets and consequently
    extra, useless, message and header overhead), meaning you're going to
    wait a bloody long time for them if you don't ask the server to send
    more data still (or close the connection). Lovely beasts, persistent
    connections. :-)


    (And just to warn you up front if you persue the 'TCP is stream, not
    messages' thing mentioned above: I can guarantee you the 'Nagle'
    option is /not/ a proper solution, though it may look like one. Just a
    friendly warning. Stay clear of Nagle et al, unless you know /exactly/
    what you are doing, protocol-wise. )


    HTH

    Ger


    On Wed, Jul 30, 2008 at 5:38 AM, petekolle123 wrote:
    >
    >
    > Thank you for the fast answer. :-)
    >
    > On
    > ftp://erbse:erbse@77.190.173.220/
    > there is the request/response string and the function SSL_WR_RD().C
    >
    > but SSL_read says "there is no more to read" stops the loop
    > because SSL_get_error give SSL_ERROR_NONE and return from function.
    > The request/response data must end with "..."
    >
    > All the source examples I found use SSL_get_error to find out
    > "there is no more to read".
    >
    > But this obviously doesn't work. Perhabs doesnt work on w32 machines.
    >
    > You are right I have to make the next request when all is read.
    > The question is, how can I detect the read has finished ?
    >
    >
    > thanks Peter
    >
    >
    >
    >
    >
    >
    > --
    > View this message in context: http://www.nabble.com/Last-portion-S...p18725602.html
    > Sent from the OpenSSL - Dev mailing list archive at Nabble.com.
    >
    > __________________________________________________ ____________________
    > OpenSSL Project http://www.openssl.org
    > Development Mailing List openssl-dev@openssl.org
    > Automated List Manager majordomo@openssl.org
    >




    --
    Met vriendelijke groeten / Best regards,

    Ger Hobbelt

    --------------------------------------------------
    web: http://www.hobbelt.com/
    http://www.hebbut.net/
    mail: ger@hobbelt.com
    mobile: +31-6-11 120 978
    --------------------------------------------------
    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  5. Re: Last portion SSL_read ....




    ftp://erbse:erbse@77.190.173.220/

    Please copy this link above in your browser address line
    your browser will be able to follow this link

    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18729136.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  6. Re: Last portion SSL_read ....

    If the firewall blocks FTP, and there is no proxy that responds to
    requests for FTP URLs, the browser will not be able to follow the
    link. Do you have a webserver (http or https) upon which you can
    place the file?

    -Kyle H

    On Wed, Jul 30, 2008 at 2:11 AM, petekolle123 wrote:
    >
    >
    >
    > ftp://erbse:erbse@77.190.173.220/
    >
    > Please copy this link above in your browser address line
    > your browser will be able to follow this link
    >
    > --
    > View this message in context: http://www.nabble.com/Last-portion-S...p18729136.html
    > Sent from the OpenSSL - Dev mailing list archive at Nabble.com.
    >
    > __________________________________________________ ____________________
    > OpenSSL Project http://www.openssl.org
    > Development Mailing List openssl-dev@openssl.org
    > Automated List Manager majordomo@openssl.org
    >

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  7. Re: Last portion SSL_read ...



    Ok,plz try this:
    http://www.student.uni-oldenburg.de/...valentin/test/


    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18731385.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  8. RE: Last portion SSL_read only after a SSL_write . Please Help.


    > Hallo


    Okay, you have two choices.

    1) You can pipeline, where you send the next request before you receive the
    entire reply to the previous request. In this case, you may receive the rest
    of the reply to the first request after you send the second.

    2) You can not pipeline, where you don't send the next request until you
    receive the entire reply to the previous request. In this case, you must
    keep calling SSL_read until you either get the entire reply, or an error.

    > I make a request with SSL_write
    > Servers answers properly.


    Good.

    > SSL_read gives 7159 bytes.
    > SSL_get_error gives allways SSL_ERROR_NONE
    > and therefor the loop stops


    > but there are 975 bytes missing.


    Okay, so you are pipelining. You have allowed the loop to stop before you
    received the entire reply.

    > My progamm send next request and then read the response.
    >
    > This SSL_read give the missing 975 bytes.
    > This next request is ignored, i think


    If you pipeline, and send the second request before you receive the entire
    reply to the first request, then you will receive the rest of the first
    reply, possibly combined with all or parts of the second reply, after you
    send the second request. If you don't want that, don't pipeline. That is, do
    not send the second request until you have received the entire first reply.

    > What happens here ?


    It's simple.

    1) You send the first request.

    2) You get part of the first reply. There has been no error. But you end
    your loop anyway.

    3) You send the second request.

    4) You get the rest of the first reply, possibly combined with all or part
    of the second reply.

    If you don't want this to happen, don't pipeline. Do not break out of the
    SSL_read loop until you've either read the entire reply or gotten an error.

    Your error may be a classic TCP error. TCP will not tell you when you've
    gotten an entire reply. The protocol you are implementing on top of TCP does
    that. If you are expecting SSL or TCP to somehow tell you "that's the end of
    the first reply", then it won't do that. It has no way to know.

    It sounds like you claim HTTP 1.1 compliance when you didn't actually
    implement HTTP 1.1. HTTP 1.1 requires support for chunked encoding. It
    requires that either chunked encoding or a content length header be used and
    understood by the client to mark the end of the reply. It also specifies
    that where neither of these things happen, the end of a reply is marked by
    the connection closing.

    Perhaps you should set your sights lower and implement HTTP 1.0 which does
    not require this level of support. Otherwise, use an HTTP library that
    properly supports HTTP 1.1.

    DS


    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  9. Re: Last portion SSL_read only after a SSL_write . Please Help.

    On Wed, Jul 30, 2008 at 07:32:26AM -0700, David Schwartz wrote:
    >
    > > Hallo

    >
    > Okay, you have two choices.
    >
    > 1) You can pipeline, where you send the next request before you receive the
    > entire reply to the previous request. In this case, you may receive the rest
    > of the reply to the first request after you send the second.


    One should note that this is not a required feature in HTTP version 1.1,
    and that you are likely to encounter clients or servers which misbehave
    when confronted with it.

    Thor
    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  10. Re: Last portion SSL_read ...



    dear adviser

    on
    http://www.student.uni-oldenburg.de/...valentin/test/
    I add some little c source pieces .
    Is there anything wrong with initSSL or tcp_connect(w32) functions ?

    I guess SSL_read is in blocking mode
    and therefor SSL_pending(mySSL->ssl) and
    getting "SSL_ERROR_WANT_READ" doesn`t work


    > not send the second request until you have received the entire first
    > reply.


    yes you are right but the indication for "nothing else to read" is invalid.

    SSL_pending(...) is zero after the first portion
    therefor nothing else to read and then loop stops

    And when I try something like that
    r = SSL_read(......)
    if ( r > 0 ) { save data , stay in loop }
    else
    stop loop

    this cause r = SSL_read(......) to block for ever.

    I also can parse response and find "Content-Length: 7967"
    I would get 7195 bytes and try one more r = SSL_read(......)
    and it block for ever. I tried it.


    I do not get "SSL_ERROR_WANT_READ" error
    that is the indication for "there is more data read once more".


    I only use the HTTP header string
    GET /global/v3/BFGlobalService HTTP/1.1
    Host: api.betfair.com:443
    Content-Type: text/xml; charset=UTF-8
    Content-Length: 609
    SOAPAction: "getActiveEventTypes"
    empty line
    609 bytes

    Perhabs I have to add something like
    "Allow SEND ALL DATA AT ONCE"






    thank you

    Peter


    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18737753.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  11. RE: Last portion SSL_read ...


    > > not send the second request until you have received the entire first
    > > reply.


    > yes you are right but the indication for "nothing else to read"
    > is invalid.


    Where are you getting this indication from? If your answer is anything other
    than "my HTTP implementation", then you are confused. Only your HTTP
    implementation can tell if there is anything to read or not.

    > SSL_pending(...) is zero after the first portion
    > therefor nothing else to read and then loop stops


    The loop should not stop until you receive the entire reply. Your loop logic
    is broken.

    You should only exit the loop if three things are true:

    1) You receive the entire reply. This is not the case, as you have
    explained.

    2) You receive an error indication. This is not the case, as you have
    explained.

    3) You are pipelining, and don't mind receiving all or part of the first
    reply after you sent the first request. This seems to be a problem for you,
    so this is not what you want.

    So why are you exiting your loop?!

    > And when I try something like that
    > r = SSL_read(......)
    > if ( r > 0 ) { save data , stay in loop }
    > else
    > stop loop
    >
    > this cause r = SSL_read(......) to block for ever.


    Right. This is broken. You have to exit the loop when you receive the entire
    reply.

    > I also can parse response and find "Content-Length: 7967"
    > I would get 7195 bytes and try one more r = SSL_read(......)
    > and it block for ever. I tried it.


    Then troubleshoot *THAT* code. As I explained earlier, you are making dozens
    of changes to try to fix a problem and as a result you are adding more and
    more problems.

    > I do not get "SSL_ERROR_WANT_READ" error
    > that is the indication for "there is more data read once more".


    Right, you won't get that.

    > I only use the HTTP header string
    > GET /global/v3/BFGlobalService HTTP/1.1
    > Host: api.betfair.com:443
    > Content-Type: text/xml; charset=UTF-8
    > Content-Length: 609
    > SOAPAction: "getActiveEventTypes"
    > empty line
    > 609 bytes


    > Perhabs I have to add something like
    > "Allow SEND ALL DATA AT ONCE"


    No. You need to follow the HTTP 1.1 specification and keep calling SSL_read
    until you either get an error or the entire reply. You said when you try to
    do this, SSL_read blocks forever -- so that's the case you need to
    troubleshoot.

    Most likely, you are incorrectly computing the number of bytes of reply you
    have gotten and so are calling SSL_read when you should not. But don't make
    it worse by exiting the loop before you have a complete reply -- that will
    definitely not work.

    DS


    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  12. Re: Last portion SSL_read only after a SSL_write . Please Help.


    Hurrey
    I have the solution!
    :clap::-D

    If SSL_get_error give SSL_ERROR_NONE
    I test with strcmp ( at the end, "") .
    because all data from the server must end with "".
    If my data is not complete I send CR LF and the last 975 Bytes appeared
    SSL_write(mySSL->ssl,"\r\n", 2);

    Thanks Peter



    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18767511.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  13. Re: Last portion SSL_read only after a SSL_write . Please Help.

    I'm glad this works for you, but be warned: the fact that that cute
    hack works in your environment now will not guarantee it will work
    elsewhere. The fact that you need to transmit another few ('faked')
    bytes to get the data you want is a clear sign there's a TCP stream
    (or SSL BIO) issue underlying it all.

    As I mentioned before, TCP is a stream protocol, not a message
    protocol, so when you expect to receive all data before sending
    another request over the same TCP socket, that is plain *wrong*,
    though some people here appear to think this is possible. (It is only
    possible when you actively 'tweak' your TCP stack into forcing it NOT
    to buffer any data. Which is not part of any standard and thus highly
    os dependent, but I digress.)

    If you really NEED to receive *complete* answers before you can
    transmit the next request (and watch these words: 'on the same SSL/TCP
    socket'!) the only proper way of doing so is by using single
    transactions (no pipelining). In HTTP terms that means you should
    transmit the header

    Connection:close

    along with your request, thus requiring the HTTP server to close the
    connection - which will result in its buffers being flushed for sure
    and you receiving all your data. (Another way which can accomplish the
    same thing is a TCP write-side half-close, but I am not sure that'll
    work with SSL sitting in between, though it has SSL_shutdown(). Never
    tried and tested it extensively in my own code so I can't say.)

    If you don't want /that/ (one connection per request), the only
    alternative is pipelining multiple requests on a single SSL/TCP
    connection, which is harder to accomplish and as you found out will
    never guarantee you will receive all data for request N-1 before you
    send request N at application layer of the network stack. With or
    without SSL sitting in that network stack.
    The statement 'TCP is a stream-based protocol, /not/ a message-based
    protocol' is not just words to fill a page. It has some very
    far-reaching implications that you should be aware of at all times.


    What I'm saying above may sound odd or condescending to some. If so,
    sorry for that. But try running this kind of code on different network
    infrastructures (switch from LAN over Gigabit Ethernet to, say, TCP
    over PPP on a dial-up on TCP over ADSL and you will find out that on
    some of those networks the code acts 'weird' or not at all. That's
    because you didn't treat TCP as the stream protocol it is and when the
    network has a different MTU you may well end up back at square one.
    Adding SSL on top does not change any of this.
    And note that when you are talking HTTP, generally that includes
    InTERnet usage of said protocol, where you do not have /full/ control
    over the MTU and other network factors which /will/ impact your code
    when you treat SSL/TCP as a message-based protocol. Which, I can't
    repeat this often enough, it isn't.

    (And if you consider improving the quality of your solution, check out
    the books written by W. Richard Stevens. They don't cover the hottest
    stuff (IPv6), but sure as hell do cover TCP and all the other
    protocols in a very good way.)


    Sorry to stick a stick in the works like this, but it's rather better
    to spend time on it while in development than having someone (or
    yourself) attempting to 'service' a production server with such code
    while people are breathing down your neck.

    Test your code with different TCP and SSL buffer sizes (see
    setsockopt(SO_SNDBUF / SO_RCVBUF) or ioctl/fcntl equivalents on your
    system) and test again: the code should perform correctly with all
    buffer sizes and settings,


    Take care,

    Ger




    On Fri, Aug 1, 2008 at 5:11 AM, petekolle123 wrote:
    >
    > Hurrey
    > I have the solution!
    > :clap::-D
    >
    > If SSL_get_error give SSL_ERROR_NONE
    > I test with strcmp ( at the end, "") .
    > because all data from the server must end with "".
    > If my data is not complete I send CR LF and the last 975 Bytes appeared
    > SSL_write(mySSL->ssl,"\r\n", 2);
    >
    > Thanks Peter
    >
    >
    >
    > --
    > View this message in context: http://www.nabble.com/Last-portion-S...p18767511.html
    > Sent from the OpenSSL - Dev mailing list archive at Nabble.com.
    >
    > __________________________________________________ ____________________
    > OpenSSL Project http://www.openssl.org
    > Development Mailing List openssl-dev@openssl.org
    > Automated List Manager majordomo@openssl.org
    >
    >




    --
    Met vriendelijke groeten / Best regards,

    Ger Hobbelt

    --------------------------------------------------
    web: http://www.hobbelt.com/
    http://www.hebbut.net/
    mail: ger@hobbelt.com
    mobile: +31-6-11 120 978
    --------------------------------------------------
    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  14. RE: Last portion SSL_read only after a SSL_write . Please Help.


    > Hurrey
    > I have the solution!
    > :clap::-D
    >
    > If SSL_get_error give SSL_ERROR_NONE
    > I test with strcmp ( at the end, "") .
    > because all data from the server must end with "".
    > If my data is not complete I send CR LF and the last 975 Bytes appeared
    > SSL_write(mySSL->ssl,"\r\n", 2);


    So what happens if the server uses chunked encoding and the '/soap' and the
    ':Envelope' wind up in different chunks? This is not a solution. It's a
    "happens to work when I tried it".

    Your code still has a fundamental and serious error. It tells the server
    that it is compliant with HTTP 1.1 when it is not.

    DS


    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  15. Re: Last portion SSL_read only after a SSL_write . Please Help.

    >>>>> On Fri, 1 Aug 2008 08:26:55 +0200, Ger Hobbelt said:
    >
    > I'm glad this works for you, but be warned: the fact that that cute
    > hack works in your environment now will not guarantee it will work
    > elsewhere. The fact that you need to transmit another few ('faked')
    > bytes to get the data you want is a clear sign there's a TCP stream
    > (or SSL BIO) issue underlying it all.
    >
    > As I mentioned before, TCP is a stream protocol, not a message
    > protocol, so when you expect to receive all data before sending
    > another request over the same TCP socket, that is plain *wrong*,
    > though some people here appear to think this is possible.


    How would any interactive TCP protocol such as ssh be usable if what you say
    were true for most people's setup? Even the good old ESMTP and POP3 protocols
    depend on the command-response messages being received within a reasonable
    time.

    __Martin
    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  16. Re: Last portion SSL_read only after a SSL_write . Please Help.

    >> As I mentioned before, TCP is a stream protocol, not a message
    >> protocol, so when you expect to receive all data before sending
    >> another request over the same TCP socket, that is plain *wrong*,
    >> though some people here appear to think this is possible.

    >
    > How would any interactive TCP protocol such as ssh be usable if what you say
    > were true for most people's setup? Even the good old ESMTP and POP3 protocols
    > depend on the command-response messages being received within a reasonable
    > time.


    Ah, yeah, correct. Granted, I exaggerated. To answer your question:
    there's two ways: one is by disabling Nagle for your TCP connection
    (setsockopt(TCP_NODELAY)) so that your write()/send() data is passed
    through the stack and onto the hardware line immediately. That's
    standard for telnet and other interactive protocols which ride on top
    of TCP. No delays whatsoever; drawback is more network packets which
    may contain just a few application data bytes.

    Default TCP stack behaviour is to wait for more data to be passed down
    by the application through socket write()/send() when the previous
    amount was 'too small' (which depends on your stack's configuration
    parameters). That 'waiting for more to be sent' is disabled by the
    TCP_NODELAY option.
    Which leaves us 'solution' number 2 if we don't want to disable Nagle:
    wait for the timeout to happen as it will 'flush' the remaining send
    data onto the line. IIRC it's somewhere in the neighbourhood of 200 ..
    500 ms, so for a single request-response it's certainly within
    'reasonable time' limits. SMTP/NNTP/... don't usually disable Nagle
    and there you see that timeout happen ever so often when a request was
    sent and the client is waiting for another 2xx (or other) response.
    (... checked the books here to make sure I didn't completely screw up:
    See W. Richard Stevens TCP/IP Illustrated Vol. 1, section 19.4, Nagle
    algorithm, pp. 267-273; WRS mentions 200ms there, BTW. See also same
    book (Vol.1), chapter 28: SMTP, pp. 441 a.o., where the network
    message trace at page 444 shows the ~200 ms timeouts for each
    exchange. Further info in TCP/IP Illustrated Vol.2: TCP_NODELAY, pp858
    and Appendix C, RFC1122 compliance, pp1120, where it says the delay
    before data to sent is flushed must be < 0.5 seconds.

    All in all, perfectly reasonable timing. What I forgot to mention is
    that when you don't know this and you are transferring 'web pages'
    across persistent HTTP (TCP) connections, those timeouts are going to
    hit you relatively bad when you have pages which a constructed from
    several URLs (say 1 html page, 1 CSS file, 1 JavaScript and maybe 10
    images: grand total of 13 requests. Worst case time _loss_ due to TCP
    send timeouts: (13-1)*(<0.5) (let's assume it's 200 ms everywhere) =
    12*0.2 = 2.4 seconds delay: no data is transfered during that period
    as the HTTP server TCP stack is 'waiting for more to come'. Of course,
    that's worse case, so in practice the delay due to this particular
    reason alone will be lower for such pages, but when you have graphic
    intensive (= lots of tiny icons and stuff) pages which are loaded
    using only one or two 'persistent' sockets (most browsers limit the
    number of parallel connections to site X), user experience is going to
    hurt. There, 'reasonable' is not 'reasonable' anymore. While no-one is
    going to sit and wait out an SMTP exchange, so those .2 seconds per
    request are perfectly okay there.
    (All of this does not apply to telnet & co. as those 'interactive
    protocols' have TCP_NODELAY turned on on each transmit side, so zero
    time lost twiddling thumbs waiting for the user/application to
    send()/write() any more data.)

    All the above does not consider the added (possibly also buffering)
    SSL layer; I have not taken the time to verify that code again to see
    if it may or will add additional delays in either reception or
    transmission of small amounts of data due to clustering of transmitted
    data (similar to what Nagle does for TCP). The only thing I am certain
    about there is that OpenSSL offers a specific API for turning off
    Nagle on socket BIOs: BIO_set_tcp_ndelay() in bio/b_sock.c -- which I
    have used in the past to successfully create 'interactive
    communication' applications over SSL/TCP.


    If I screwed up in there or am otherwise saying wrong stuff, please
    correct me. It is appreciated. Your remark made me verify my own 'off
    the top of my head' data, which is always a good thing.


    --
    Met vriendelijke groeten / Best regards,

    Ger Hobbelt

    --------------------------------------------------
    web: http://www.hobbelt.com/
    http://www.hebbut.net/
    mail: ger@hobbelt.com
    mobile: +31-6-11 120 978
    --------------------------------------------------
    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


  17. RE: Last portion SSL_read only after a SSL_write . Please Help.





    >Your code still has a fundamental and serious error. It tells the server
    >that it is compliant with HTTP 1.1 when it is not.


    yes, thats not clean solution.
    I plan to take nanohttp.
    There is a reference but unfortunately there is no manual.
    I have to trial and error. :-(

    Peter

    -----------joke mode ON------------
    in C:

    printf("Hello\n");

    in JAVA:

    printf("H");
    t=10000000; while(t--);
    printf("e");
    t=10000000; while(t--);
    printf("l");
    t=10000000; while(t--);
    printf("l");
    t=10000000; while(t--);
    printf("o");
    t=10000000; while(t--);
    printf("\n");
    t=10000000; while(t--);

    -----------joke mode OFF------------


    --
    View this message in context: http://www.nabble.com/Last-portion-S...p18789461.html
    Sent from the OpenSSL - Dev mailing list archive at Nabble.com.

    __________________________________________________ ____________________
    OpenSSL Project http://www.openssl.org
    Development Mailing List openssl-dev@openssl.org
    Automated List Manager majordomo@openssl.org


+ Reply to Thread