Have a shell script check for a file to exist before processing another file - Aix

This is a discussion on Have a shell script check for a file to exist before processing another file - Aix ; I have a shell script that runs all the time looking for a certain type of file and then it processes the file through a series of other scripts. The script is watching a directory that has files uploaded to ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: Have a shell script check for a file to exist before processing another file

  1. Have a shell script check for a file to exist before processing another file

    I have a shell script that runs all the time looking for a certain type
    of file and then it processes the file through a series of other
    scripts. The script is watching a directory that has files uploaded to
    it via SFTP. It already checks the size of the file to make sure that
    it is not still uploading before it starts processing. I would like to
    place another check in the script that looks for the existance of
    another file before processing begins. The script looks like:

    Code:
    #!/bin/ksh
    PATH=/gers/nurev/menu/pub/sbin:/gers/nurev/menu/pub/bin:/gers/nurev/menu/pub/mac
    :/gers/nurev/menu/adm/sbin:/gers/nurev/menu/adm/bin:/gers/nurev/menu/adm/mac:/ge
    rs/nurev/custom:/gers/nurev/fix:/gers/nurev/src_rev/fix:/gers/nurev/opt/path:/ge
    rs/nurev/bin:/g/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/sbin:.
    ORACLE_HOME=/gers/nurev
    ORACLE_SID=nurev
    export PATH
    export ORACLE_HOME
    export ORACLE_SID
    
    #
    # Function : is_file_arrived file
    # Arg(s)   : file = file to verify
    # Output   : None
    # Status   : 0 = yes file arrived, 1 = no
    # Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
    #
    
    is_file_arrived() {
    [ -z "$1" ] && return 1
    local file=$1
    local arrived=1
    local size1 size2
    if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
    size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
    sleep ${IFA_WAIT:-15}
    size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
    [ ${size1:-1} -eq ${size2:-2} ] && arrived=0
    fi
    return $arrived
    }
    
    
    processFile ()
    {
    local fileName=$1
    local fileExtension=$2
    local fileNewName="/gers/nurev/datafiles/str${fileExtension}.asc"
    local filePrintPath="/gers/nurev/print"
    local fileTmpPath="/gers/nurev/tmp"
    local fileODIName="str${fileExtension}.pos"
    mv -Eignore $fileName $fileNewName
    prepup $fileNewName $fileExtension
    mv -Eignore  $filePrintPath/$fileODIName $fileTmpPath/$fileODIName
    save2tmp $fileExtension
    call_siu $fileExtension
    }
    
    # Main Processing
    
    nsec=1
    while [[ "$(date +%H%M)" -lt 2329 ]]
    do
    for fileName in
    /gers/nurev/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9
    ][0-9]
    do
    fileExtension=${fileName#*.}
    is_file_arrived "$fileName" && nsec=1 && processFile $fileName
    $fileExtension
    done
    sleep $nsec
    case $nsec in
    1)   nsec=15;;
    15)  nsec=45;;
    45)  nsec=90;;
    90)  nsec=300;;
    300) nsec=600;;
    600) nsec=900;;
    *)   nsec=1800;;
    esac
    done
    ....I'd like to check for a file in /gers/genret/tmp/poll_####.txt
    (where "####" is the $fileExtension variable. I think the file check
    should be in the "Main Processing" section before it sends to the file
    to the "is_file_arrived" function? I'f there is a "poll_####.txt" file
    present I would just like the script to move onto the next file and
    look at it again later. This would prevent two files of the same type
    processing at the same time. I'm thinking something like a "if-then"?


  2. Re: Have a shell script check for a file to exist before processing another file


    heprox wrote:
    > I have a shell script that runs all the time looking for a certain type
    > of file and then it processes the file through a series of other
    > scripts. The script is watching a directory that has files uploaded to
    > it via SFTP. It already checks the size of the file to make sure that
    > it is not still uploading before it starts processing. I would like to
    > place another check in the script that looks for the existance of
    > another file before processing begins. The script looks like:
    >
    >
    Code:
    > #!/bin/ksh
    > PATH=/gers/nurev/menu/pub/sbin:/gers/nurev/menu/pub/bin:/gers/nurev/menu/pub/mac
    > :/gers/nurev/menu/adm/sbin:/gers/nurev/menu/adm/bin:/gers/nurev/menu/adm/mac:/ge
    > rs/nurev/custom:/gers/nurev/fix:/gers/nurev/src_rev/fix:/gers/nurev/opt/path:/ge
    > rs/nurev/bin:/g/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/sbin:.
    > ORACLE_HOME=/gers/nurev
    > ORACLE_SID=nurev
    > export PATH
    > export ORACLE_HOME
    > export ORACLE_SID
    >
    > #
    > # Function : is_file_arrived file
    > # Arg(s)   : file = file to verify
    > # Output   : None
    > # Status   : 0 = yes file arrived, 1 = no
    > # Env.     : IFA_WAIT : interval (secs) for file size check (def=5)
    > #
    >
    > is_file_arrived() {
    >    [ -z "$1" ] && return 1
    >    local file=$1
    >    local arrived=1
    >    local size1 size2
    >    if [ -f "$file" -a -z "$(fuser $file 2> /dev/null)" ] ; then
    >       size1=$(ls -l $file 2>/dev/null | awk '{print $5}')
    >       sleep ${IFA_WAIT:-15}
    >       size2=$(ls -l $file 2>/dev/null | awk '{print $5}')
    >       [ ${size1:-1} -eq ${size2:-2} ] && arrived=0
    >    fi
    >    return $arrived
    > }
    >
    >
    > processFile ()
    > {
    >    local fileName=$1
    >    local fileExtension=$2
    >    local fileNewName="/gers/nurev/datafiles/str${fileExtension}.asc"
    >    local filePrintPath="/gers/nurev/print"
    >    local fileTmpPath="/gers/nurev/tmp"
    >    local fileODIName="str${fileExtension}.pos"
    >    mv -Eignore $fileName $fileNewName
    >    prepup $fileNewName $fileExtension
    >    mv -Eignore  $filePrintPath/$fileODIName $fileTmpPath/$fileODIName
    >    save2tmp $fileExtension
    >    call_siu $fileExtension
    > }
    >
    > # Main Processing
    >
    > nsec=1
    > while [[ "$(date +%H%M)" -lt 2329 ]]
    > do
    >    for fileName in
    > /gers/nurev/datafiles/[Uu][Pp][Ll][Oo][Aa][Dd].[0-9][0-9][0-9
    > ][0-9]
    >    do
    >       fileExtension=${fileName#*.}
    >       is_file_arrived "$fileName" && nsec=1 && processFile $fileName
    > $fileExtension
    >    done
    >    sleep $nsec
    >    case $nsec in
    >       1)   nsec=15;;
    >       15)  nsec=45;;
    >       45)  nsec=90;;
    >       90)  nsec=300;;
    >       300) nsec=600;;
    >       600) nsec=900;;
    >       *)   nsec=1800;;
    >    esac
    > done
    >
    >
    > ...I'd like to check for a file in /gers/genret/tmp/poll_####.txt
    > (where "####" is the $fileExtension variable. I think the file check
    > should be in the "Main Processing" section before it sends to the file
    > to the "is_file_arrived" function? I'f there is a "poll_####.txt" file
    > present I would just like the script to move onto the next file and
    > look at it again later. This would prevent two files of the same type
    > processing at the same time. I'm thinking something like a "if-then"?


    Hey,
    We have a similar situation. Here is how we do it.
    the program that creates the file should create it with an extension of
    let's say tmp.
    In your case: /gers/genret/tmp/poll_####.txt.tmp.
    when the file is totally created the same program would rename the file
    to: /gers/genret/tmp/poll_####.txt

    Your script would always check for such
    files:/gers/genret/tmp/poll_####.txt NOT the *.tmp


+ Reply to Thread