FTP skipping commands in AIX - Aix

This is a discussion on FTP skipping commands in AIX - Aix ; I have an FTP script that operates on a VPN tunnel between AIX and a remote server. It goes out to the remote host and tries to pick up a data file that provides updates for incoming material (essentially a ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: FTP skipping commands in AIX

  1. FTP skipping commands in AIX

    I have an FTP script that operates on a VPN tunnel between AIX and a
    remote server. It goes out to the remote host and tries to pick up a
    data file that provides updates for incoming material (essentially a
    form of EDI). Our server is using the native FTP in AIX 5.1 and the
    remote server is running something called "vsFTPd 2.0.5".

    The script is relatively simple...We set the local and remote paths,
    delete any old file that exist, pick up the new file, then rename the
    file we just picked up to the old name. Here is the basic logic:

    ftp -i ${GET_HOST} << END >> ${LOG_FILE}
    verbose
    prompt
    lcd ${WORK_DIR}
    cd ${GET_DIR}
    mdelete ${GET_FILE}.old
    mget ${GET_FILE}
    rename ${GET_FILE} ${GET_FILE}.hold
    quit
    END

    Our problem that is arising is that one of the four servers we
    installed has reported missing transactions. After investigation, we
    determined that some of the commands are being "skipped". For example,
    every once in a while, it will do the mdelete command, skip the mget,
    then execute the rename and the quit.

    This is a problem as it means we never picked up the file, but renamed
    it, and the next sequence through if it was not caught will simply
    delete that old file. To do some verifications, I added debug logic in
    there, and to see if it was a timing issue, I added some extra pwd/dir
    commands to "buy time". This seems to work even though sometimes
    things are still missed. Here is the new logic:

    ftp -i ${GET_HOST} << END >> ${LOG_FILE}
    verbose
    prompt
    debug
    lcd ${WORK_DIR}
    cd ${GET_DIR}
    pwd
    dir
    mdelete ${GET_FILE}.hold
    mget ${GET_FILE}
    pwd
    rename ${GET_FILE} ${GET_FILE}.hold
    quit
    END


    I know that there are many different and probably more secure methods
    than FTP out there and people will be asking why we still use this
    method. Suffice it to say it is an older legacy system that cannot
    currently be changed.

    So my questions are:

    1) Does anyone know why an FTP script (which I always thought uses
    sequential processing similar to a DOS batch script) would skip
    certain steps? Could there be an incompatibility issue or perhaps this
    is a known issue in AIX 5.1 (other servers are AIX 5.2 or higher and
    don't seem to have this issue)

    2) Has anyone seen this behavior before and have suggestions on how to
    improve the logic? (One idea I have is to get the file here, confirm
    it exists, then open a new FTP session to go back and delete the file
    at the remote server...)

    Thx in advance

    Steve

  2. Re: FTP skipping commands in AIX

    On Oct 16, 4:56 pm, "steven_nospam at Yahoo! Canada"
    wrote:
    > I have an FTP script that operates on a VPN tunnel between AIX and a
    > remote server. It goes out to the remote host and tries to pick up a
    > data file that provides updates for incoming material (essentially a
    > form of EDI). Our server is using the native FTP in AIX 5.1 and the
    > remote server is running something called "vsFTPd 2.0.5".
    >
    > The script is relatively simple...We set the local and remote paths,
    > delete any old file that exist, pick up the new file, then rename the
    > file we just picked up to the old name. Here is the basic logic:
    >
    > ftp -i ${GET_HOST} << END >> ${LOG_FILE}
    > verbose
    > prompt
    > lcd ${WORK_DIR}
    > cd ${GET_DIR}
    > mdelete ${GET_FILE}.old
    > mget ${GET_FILE}
    > rename ${GET_FILE} ${GET_FILE}.hold
    > quit
    > END
    >
    > Our problem that is arising is that one of the four servers we
    > installed has reported missing transactions. After investigation, we
    > determined that some of the commands are being "skipped". For example,
    > every once in a while, it will do the mdelete command, skip the mget,
    > then execute the rename and the quit.
    >
    > This is a problem as it means we never picked up the file, but renamed
    > it, and the next sequence through if it was not caught will simply
    > delete that old file. To do some verifications, I added debug logic in
    > there, and to see if it was a timing issue, I added some extra pwd/dir
    > commands to "buy time". This seems to work even though sometimes
    > things are still missed. Here is the new logic:
    >
    > ftp -i ${GET_HOST} << END >> ${LOG_FILE}
    > verbose
    > prompt
    > debug
    > lcd ${WORK_DIR}
    > cd ${GET_DIR}
    > pwd
    > dir
    > mdelete ${GET_FILE}.hold
    > mget ${GET_FILE}
    > pwd
    > rename ${GET_FILE} ${GET_FILE}.hold
    > quit
    > END
    >
    > I know that there are many different and probably more secure methods
    > than FTP out there and people will be asking why we still use this
    > method. Suffice it to say it is an older legacy system that cannot
    > currently be changed.
    >
    > So my questions are:
    >
    > 1) Does anyone know why an FTP script (which I always thought uses
    > sequential processing similar to a DOS batch script) would skip
    > certain steps? Could there be an incompatibility issue or perhaps this
    > is a known issue in AIX 5.1 (other servers are AIX 5.2 or higher and
    > don't seem to have this issue)
    >
    > 2) Has anyone seen this behavior before and have suggestions on how to
    > improve the logic? (One idea I have is to get the file here, confirm
    > it exists, then open a new FTP session to go back and delete the file
    > at the remote server...)
    >
    > Thx in advance
    >
    > Steve



    Possible workarounds:
    - use a .netrc file with macro definition ( Might be even created
    dynamicly )
    - use expect
    - use kermit ( ckermit) with macros.

    From IBM about ftp macros:

    Writing ftp macros
    These steps describe how to create an ftp macro.
    You must have created the $HOME/.netrc file.
    To write an ftp macro:

    1. Edit the $HOME/.netrc file to include the following
    instructions:

    macdef init
    put schedule

    Be sure to insert a blank line at the end of your ftp macro. The
    blank line terminates the ftp macro. In the above example, the macdef
    subcommand defines the subcommand macro init. The line following is
    the command the macro specifies, in this case put schedule, where
    schedule is the name of a file.
    2. After you create the ftp macro, at the command line prompt,
    type:

    ftp hostname

    Where hostname is the name of the host to which you are
    connecting.
    ftp scans the $HOME/.netrc file for a login definition matching
    your host name and uses that login definition to log you in.
    3. After you log in, at the command line prompt, type:

    ftp init

    In this example, ftp scans for the macro named init and executes
    the command or commands the macro specifies.
    An ftp macro is associated with the login entry immediately
    preceding it. ftp macros are not global to the $HOME/.netrc file. The
    macro init is executed automatically upon login. Other macros can be
    executed from the ftp prompt (ftp>) by typing the following:

    $getit

    In this example, the $ executes the ftp macro getit.


  3. Re: FTP skipping commands in AIX

    steven_nospam at Yahoo! Canada wrote:

    > Our problem that is arising is that one of the four servers we
    > installed has reported missing transactions. After investigation, we
    > determined that some of the commands are being "skipped". For example,
    > every once in a while, it will do the mdelete command, skip the mget,
    > then execute the rename and the quit.


    Try using expect.

    Mike


  4. Re: FTP skipping commands in AIX

    Hajo Ehlers wrote:
    > Possible workarounds:
    > - use a .netrc file with macro definition ( Might be even created
    > dynamicly )
    > - use expect
    > - use kermit ( ckermit) with macros.


    Use ssh and scp in a shell script that verifies the results of each
    command.

+ Reply to Thread