c-kermit8 and slow http downloads - Protocols

This is a discussion on c-kermit8 and slow http downloads - Protocols ; Hi all-- I know we're not supposed to say kermit is slow, but for my http downloads, it appears to be! I wrote to kermit-support but on second thought I probably shouldn't have bothered them without asking here first. so ...

+ Reply to Thread
Results 1 to 6 of 6

Thread: c-kermit8 and slow http downloads

  1. c-kermit8 and slow http downloads

    Hi all--

    I know we're not supposed to say kermit is slow, but for my http
    downloads, it appears to be!

    I wrote to kermit-support but on second thought I probably shouldn't
    have bothered them without asking here first. so here goes:

    I have been using c-kermit on Solaris 8 recently with some scripts for
    file push and retrieval. For push, I'm doing ftp scripting. For
    retrievals, I'm using the http command set. They both work well in a
    scripting environment.

    One thing I notice though, doing an 'http get' on a file only gives me a
    download speed of about 16kB/s. Using 'wget' on the same file and same
    webserver, it comes down at over 200kB/s.

    I've tried my binary, a pre-compiled binary, using 'fast' on the kermit
    command line....I just can't figure out why it's running at such a slow
    download speed.

    My version is 8.0.211, Solaris 8, gcc, default compile options.


    Anyone have some ideas on how I can improve the speed? Thank you!

    -Matt

  2. Re: c-kermit8 and slow http downloads

    Matt SF wrote:
    > Hi all--
    >
    > I know we're not supposed to say kermit is slow, but for my http
    > downloads, it appears to be!


    I added the HTTP interface in Kermit to deal with a need to script
    operations with a web site for the purpose of extracting information
    such as the Public IP address from a NAT box. The HTTP implementation
    is not particularly sophisticated and is only a subset of the full
    specification.

    From a performance perspective I expect it will be slow as it does
    not implement any block read buffering. Instead it is written using
    byte at a time reads. A significant performance gain would be expected
    simply by re-write http_inc() to performed buffered reads.

    Jeffrey Altman
    Secure Endpoints Inc.

  3. Re: c-kermit8 and slow http downloads

    On 2005-12-14, Matt SF wrote:
    : I know we're not supposed to say kermit is slow, but for my http
    : downloads, it appears to be!
    :
    : I wrote to kermit-support but on second thought I probably shouldn't
    : have bothered them without asking here first. so here goes:
    :
    : I have been using c-kermit on Solaris 8 recently with some scripts for
    : file push and retrieval. For push, I'm doing ftp scripting. For
    : retrievals, I'm using the http command set. They both work well in a
    : scripting environment.
    :
    : One thing I notice though, doing an 'http get' on a file only gives me a
    : download speed of about 16kB/s. Using 'wget' on the same file and same
    : webserver, it comes down at over 200kB/s.
    :
    : I've tried my binary, a pre-compiled binary, using 'fast' on the kermit
    : command line....I just can't figure out why it's running at such a slow
    : download speed.
    :
    : My version is 8.0.211, Solaris 8, gcc, default compile options.
    :
    : Anyone have some ideas on how I can improve the speed? Thank you!
    :
    I checked this locally just now. There does indeed seem to be some slowness
    in Kermit's HTTP GET. Downloading a 2MB file with Kermit HTTP I get 50K cps,
    versus 1002K cps with Kermit FTP from/to the same hosts. Slower by a factor
    of 20. On the other hand, I have had similar reports about Kermit FTP, which
    seem to happen only to certain people and not to anybody else. So maybe
    it's a universal problem with the HTTP code, or maybe it's particular to
    certain circumstances.

    I invite anybody who feels like an afternoon of fun to take a look at the
    routine http_get() in ckcnet.c. It's best if you look at the current working
    version of the code:

    http://www.columbia.edu/kermit/ckdaily.html

    A quick glance at the code shows there are two scenarios: chunked and
    non-chunked. The server announces which method to use. The non-chunked
    method looks like it might be really slow. Suggestion: add a debug() clause
    here:

    } else if (!ckstrcmp(buf,"Transfer-Encoding:",18,0)) {
    if ( ckindex("chunked",buf,18,0,0) != 0 )
    chunked = 1;
    }

    to see which method was used. If non-chunked, probably what's needed is
    a whole new layer of buffering.

    - Frank

  4. Re: c-kermit8 and slow http downloads

    Frank da Cruz wrote:
    > I invite anybody who feels like an afternoon of fun to take a look at the
    > routine http_get() in ckcnet.c. It's best if you look at the current working
    > version of the code:
    >
    > http://www.columbia.edu/kermit/ckdaily.html
    >
    > A quick glance at the code shows there are two scenarios: chunked and
    > non-chunked. The server announces which method to use. The non-chunked
    > method looks like it might be really slow. Suggestion: add a debug() clause
    > here:
    >
    > } else if (!ckstrcmp(buf,"Transfer-Encoding:",18,0)) {
    > if ( ckindex("chunked",buf,18,0,0) != 0 )
    > chunked = 1;
    > }
    >
    > to see which method was used. If non-chunked, probably what's needed is
    > a whole new layer of buffering.
    >
    > - Frank


    Frank and Jeffrey, thanks for replying! I thought it might be something
    along those lines like a change to the buffering code. Alas, this is
    way beyond my ability.

    I'll just keep an eye on the c-kermit development and see what happens.
    Meanwhile, it works great for the small automated ftp transfers I have
    to do a few days a week. Thanks for all your work on the project!

    -Matt

  5. Re: c-kermit8 and slow http downloads

    On 2005-12-14, Matt SF wrote:
    : [About slow HTTP downloads...]
    :
    : Frank and Jeffrey, thanks for replying! I thought it might be something
    : along those lines like a change to the buffering code. Alas, this is
    : way beyond my ability.
    :
    : I'll just keep an eye on the c-kermit development and see what happens.
    : Meanwhile, it works great for the small automated ftp transfers I have
    : to do a few days a week. Thanks for all your work on the project!
    :
    Try it now:

    http://www.columbia.edu/kermit/ckdaily.html

    - Frank

  6. Re: c-kermit8 and slow http downloads

    Frank da Cruz wrote:
    > Try it now:
    >
    > http://www.columbia.edu/kermit/ckdaily.html
    >
    > - Frank


    wow, very nice! I grabbed the code and did a quick compile for Solaris
    8 with gcc. I ran a few tests and found that its average http download
    speed for me now is around 350kB/s !

    the comparison speed is about 1.1Mb/s via wget, same file, same
    webserver (this is across a switched WAN link).

    So the kermit may still be a bit CPU-bound in comparison, but it's
    plenty fast for me now. Plus, the client box I'm using is not quite the
    fastest hunk of junk in the galaxy..Sun Ultra 10 with an ultraSparc IIi
    @ 300Mhz ... but at least it's solid

    Thank you, Frank!
    Happy Holidays!

    -Matt

+ Reply to Thread