Maximize Throughput for FTP
I have a 6 gig database I need to pull off a SCO Openserver 5.0.5 box.
I tried using an FTP freeware product and internet explorer. Both took
hours and hours to port these files. Can anyone point me to kernal
parameters to check or a better FTP utility to get this data off the
system in less then 24 hours??
Re: Maximize Throughput for FTP
----- Original Message -----
Sent: Tuesday, October 18, 2005 9:02 PM
Subject: Maximize Throughput for FTP
>I have a 6 gig database I need to pull off a SCO Openserver 5.0.5 box.
> I tried using an FTP freeware product and internet explorer. Both took
> hours and hours to port these files. Can anyone point me to kernal
> parameters to check or a better FTP utility to get this data off the
> system in less then 24 hours??
Log in, start a tape backup.
While the tape is running, drive over there.
wait for tape to finish, take tape, drive back.
The differences in ftp clients and in kernel/tcp-stack tunables are
insignificant to the upload speed of the net connection the sco box is
What is that net connection, and what is the advertised upload speed?
The best answer is rsync.
Install rsync on the remote sco box and on your local box.
Here is a sco binary
this requires installing oss646c first
If you don't have a unix/linux/freebsd box at your end, then you need to
install cygwin on windows, and in the cygwin installer select a basic
minimal install file set, and add rsync, from the list of many other things.
Suppose the data is all in a top level directory /u/data
On the remote sco box that has the data, create these two files:
uid = root
gid = sys
secrets file = /etc/rsyncd.secrets
list = false
path = /u/data
auth users = data
At this point, the sco box is set from now on to run an rsync daemon just
like it runs a telnet daemon.
You don't have to do anything more on the sco box after this, even after a
Then, on your end run:
mkdir -p /u/data (just one time)
rsync -avz --delete data@remotescobox::data/* /u/data
You run the rsync command on your end repeatedly as often as you want.
whenever you want your copy of the data updated to match the sco box.
In the above commands "/u/data" is the directory on your machine where the
files will be copied to.
If the local box is windows then I don't know what the exact syntax will
need to be.
Perhaps since you would be running this inside the cygwin bash shell, maybe
the above syntax would work just like it is.
Possibly in that case the files would really turn up in c:\cygwin\u\data
Possibly there is some other cygwin-specific syntax you can use to specify a
path outside of the cygwin home dir.
The first time it will take a long time.
After the first time it will take a tiny fraction of the time it took the
If you have already copied the files once the hard way, put whatever you
have, however old or incomplete, in the local target directory on your end
before doing the first rsync, that will drastically reduce the time it takes
to do the initial rsync.
rsync only transfers the difference between the source and the destination.
even an old and/or incomplete copy of the data will still be mostly the same
as the source live data, and so rsync will have a lot less than 100% of
every file to send over the wire. After you do the first rsync, then it only
needs to transfer what changed since the last time. So if you run again 5
minutes later, it has almost nothing to do and so it finishes almost
immediately. With very large file sets, it does spend a certain, unchanging,
amount of time at the beginning of each job, gathering a list of files that
are in the job. That might take a solid several minutes and never decreases.
You will need tcp port 873 to be able to reach the sco box from the
There are other ways to do this but this is, in my opinion, the simplest,
given that the sco box is old enough that it probably does not have ssh or
the other packages it requires, and given that a box that old will be
noticeably slow doing the ssh encryption, and given that the other simple
way is to use rsh/rcmd/rlogin and it's unwise to allow the r-services tcp
ports to reach the sco box from the internet while those services are
enabled on the sco box, which they are by default on 5.0.5
Brian K. White -- [email]email@example.com[/email] -- [url]http://www.aljex.com/bkw/[/url]
filePro BBx Linux SCO FreeBSD #callahans Satriani Filk!