Maximize Throughput for FTP - SCO

This is a discussion on Maximize Throughput for FTP - SCO ; I have a 6 gig database I need to pull off a SCO Openserver 5.0.5 box. I tried using an FTP freeware product and internet explorer. Both took hours and hours to port these files. Can anyone point me to ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: Maximize Throughput for FTP

  1. Maximize Throughput for FTP

    I have a 6 gig database I need to pull off a SCO Openserver 5.0.5 box.
    I tried using an FTP freeware product and internet explorer. Both took
    hours and hours to port these files. Can anyone point me to kernal
    parameters to check or a better FTP utility to get this data off the
    system in less then 24 hours??

    Thanks


  2. Re: Maximize Throughput for FTP


    ----- Original Message -----
    From:
    Newsgroups: comp.unix.sco.misc
    To:
    Sent: Tuesday, October 18, 2005 9:02 PM
    Subject: Maximize Throughput for FTP


    >I have a 6 gig database I need to pull off a SCO Openserver 5.0.5 box.
    > I tried using an FTP freeware product and internet explorer. Both took
    > hours and hours to port these files. Can anyone point me to kernal
    > parameters to check or a better FTP utility to get this data off the
    > system in less then 24 hours??
    >
    > Thanks


    Log in, start a tape backup.
    While the tape is running, drive over there.
    wait for tape to finish, take tape, drive back.

    The differences in ftp clients and in kernel/tcp-stack tunables are
    insignificant to the upload speed of the net connection the sco box is
    using.
    What is that net connection, and what is the advertised upload speed?

    The best answer is rsync.

    Install rsync on the remote sco box and on your local box.
    Here is a sco binary
    http://www.aljex.com/bkw/sco#rsync
    this requires installing oss646c first

    If you don't have a unix/linux/freebsd box at your end, then you need to
    install cygwin on windows, and in the cygwin installer select a basic
    minimal install file set, and add rsync, from the list of many other things.


    Suppose the data is all in a top level directory /u/data

    On the remote sco box that has the data, create these two files:
    /etc/rsyncd.conf :
    -------------------------------------------
    uid = root
    gid = sys
    secrets file = /etc/rsyncd.secrets
    list = false

    [data]
    path = /u/data
    auth users = data
    -------------------------------------------

    /etc/rsyncd.secrets :
    -------------------------------------------
    data:somepassword
    -------------------------------------------

    Then run:
    /etc/init.d/rsyncd enable
    /etc/init.d/rsyncd start

    At this point, the sco box is set from now on to run an rsync daemon just
    like it runs a telnet daemon.
    You don't have to do anything more on the sco box after this, even after a
    reboot, etc...

    Then, on your end run:
    mkdir -p /u/data (just one time)
    rsync -avz --delete data@remotescobox::data/* /u/data

    You run the rsync command on your end repeatedly as often as you want.
    whenever you want your copy of the data updated to match the sco box.

    In the above commands "/u/data" is the directory on your machine where the
    files will be copied to.
    If the local box is windows then I don't know what the exact syntax will
    need to be.
    Perhaps since you would be running this inside the cygwin bash shell, maybe
    the above syntax would work just like it is.
    Possibly in that case the files would really turn up in c:\cygwin\u\data
    Possibly there is some other cygwin-specific syntax you can use to specify a
    path outside of the cygwin home dir.

    The first time it will take a long time.
    After the first time it will take a tiny fraction of the time it took the
    first time.
    If you have already copied the files once the hard way, put whatever you
    have, however old or incomplete, in the local target directory on your end
    before doing the first rsync, that will drastically reduce the time it takes
    to do the initial rsync.

    rsync only transfers the difference between the source and the destination.
    even an old and/or incomplete copy of the data will still be mostly the same
    as the source live data, and so rsync will have a lot less than 100% of
    every file to send over the wire. After you do the first rsync, then it only
    needs to transfer what changed since the last time. So if you run again 5
    minutes later, it has almost nothing to do and so it finishes almost
    immediately. With very large file sets, it does spend a certain, unchanging,
    amount of time at the beginning of each job, gathering a list of files that
    are in the job. That might take a solid several minutes and never decreases.

    You will need tcp port 873 to be able to reach the sco box from the
    internet.

    There are other ways to do this but this is, in my opinion, the simplest,
    given that the sco box is old enough that it probably does not have ssh or
    the other packages it requires, and given that a box that old will be
    noticeably slow doing the ssh encryption, and given that the other simple
    way is to use rsh/rcmd/rlogin and it's unwise to allow the r-services tcp
    ports to reach the sco box from the internet while those services are
    enabled on the sco box, which they are by default on 5.0.5

    Brian K. White -- brian@aljex.com -- http://www.aljex.com/bkw/
    +++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
    filePro BBx Linux SCO FreeBSD #callahans Satriani Filk!


+ Reply to Thread