unattended file transfer with ssh - SSH

This is a discussion on unattended file transfer with ssh - SSH ; These are notes I created for company docs. They've been modified to remove all mention of company apps, file systems, directories, etc. They're sorta specific to my application, but give you an idea of how to get it to work. ...

+ Reply to Thread
Results 1 to 3 of 3

Thread: unattended file transfer with ssh

  1. unattended file transfer with ssh

    These are notes I created for company docs. They've been modified to
    remove all mention of company apps, file systems, directories, etc.
    They're sorta specific to my application, but give you an idea of how
    to get it to work. In my case I'm getting a listing of files that meet
    a certain criteria. Then I pipe them to cpio, then to ssh. Then, on the
    remote end, cpio takes over again and deposits the files, if all goes
    well. Easiset thing to do is cut and paste the commands only changing
    the machine and directory and ID names. In my case, the file transfer
    was from Solaris to Linux
    ================================================== ======================

    INTRODUCTION:

    transfer files use unattended ssh or batch mode. The shell script's
    code looks something like this:

    SSHOPTS="-i ~/.ssh/batch_remote -q -o 'BatchMode yes'"
    RPTROOT=/mnt/datadir/stuff
    cd $RPTROOT
    find . -follow -name '*_weekly' -newer $lasttime -print -type f | cpio
    -oc | ssh $SSHOPTS $REMOTESRVER "cd $REMOTESRVEAPPROOT; cpio -icdmuv"

    Note: the file "batch_remote" referred to in the first line is the
    private key on the local host. The public key file resides in
    $HOME/.ssh/authorized_keys on the remote host.

    INSTRUCTIONS:

    1. Create private key/public key
    * On the local (sending) machine invoke this command:
    ssh-keygen -t dsa
    * When prompted for a file name accept the default
    ($HOME/.ssh/id_dsa)
    * When prompted for a pass phrase press the Enter key
    * Two files will be created: $HOME/.ssh/id_dsa AND
    $HOME/.ssh/id_dsa.pub
    2. Setup files using the same UNIX/Linux ID on both servers. Example
    ID: transfer_id
    * Local server: mv $HOME/.ssh/id_dsa $HOME/.ssh/batch_remote
    * The file name must match the file name used in your script
    for the "identity" file (see above code)
    * Local server: scp $HOME/.ssh/id_dsa.pub
    transfer_id@remote_host:/homedir/transfer_id/.ssh
    * Remote server: cat $HOME/.ssh/id_dsa.pub >>
    $HOME/.ssh/authorized_keys
    * Remote server: chmod 755 $HOME/.ssh; chmod 644
    $HOME/.ssh/authorized_keys
    This is critical!!
    3. Manually connect to the remote host
    * Local server: ssh transfer_id@remote_host
    * This will insure proper hash value added to
    $HOME/.ssh/known_hosts

    TEST:

    Example code:

    #!/usr/bin/bash
    set -x
    ssh -v -v -v -i /users/transfer_id/.ssh/batch_remote -o 'BatchMode yes'
    10.5.110.22 "cd /tmp;ls -l"

    Use the -v -v -v options to get as much diagnostic output as possible.
    In this example, we go to the temp directory and list the files to test
    connectivity and the ability to execute a command.


  2. Re: unattended file transfer with ssh

    "phynkel" writes:

    >These are notes I created for company docs. They've been modified to
    >remove all mention of company apps, file systems, directories, etc.
    >They're sorta specific to my application, but give you an idea of how
    >to get it to work. In my case I'm getting a listing of files that meet
    >a certain criteria. Then I pipe them to cpio, then to ssh. Then, on the
    >remote end, cpio takes over again and deposits the files, if all goes
    >well. Easiset thing to do is cut and paste the commands only changing
    >the machine and directory and ID names. In my case, the file transfer
    >was from Solaris to Linux
    >================================================== ======================


    Use rsync. It uses ssh ( if that is what you want).


    rsync -av remotemachine:/the/directory/ /local/directory

    It also has teh advantage that it only transfers the differences in the
    files between the remote an local.
    No need for cpio.

    To set up ssh, the stuff following is fine.




    >INTRODUCTION:


    >transfer files use unattended ssh or batch mode. The shell script's
    >code looks something like this:


    >SSHOPTS="-i ~/.ssh/batch_remote -q -o 'BatchMode yes'"
    >RPTROOT=/mnt/datadir/stuff
    >cd $RPTROOT
    >find . -follow -name '*_weekly' -newer $lasttime -print -type f | cpio
    >-oc | ssh $SSHOPTS $REMOTESRVER "cd $REMOTESRVEAPPROOT; cpio -icdmuv"


    >Note: the file "batch_remote" referred to in the first line is the
    >private key on the local host. The public key file resides in
    >$HOME/.ssh/authorized_keys on the remote host.


    >INSTRUCTIONS:


    > 1. Create private key/public key
    > * On the local (sending) machine invoke this command:
    >ssh-keygen -t dsa
    > * When prompted for a file name accept the default
    >($HOME/.ssh/id_dsa)
    > * When prompted for a pass phrase press the Enter key
    > * Two files will be created: $HOME/.ssh/id_dsa AND
    >$HOME/.ssh/id_dsa.pub
    > 2. Setup files using the same UNIX/Linux ID on both servers. Example
    >ID: transfer_id
    > * Local server: mv $HOME/.ssh/id_dsa $HOME/.ssh/batch_remote
    > * The file name must match the file name used in your script
    >for the "identity" file (see above code)
    > * Local server: scp $HOME/.ssh/id_dsa.pub
    >transfer_id@remote_host:/homedir/transfer_id/.ssh
    > * Remote server: cat $HOME/.ssh/id_dsa.pub >>
    >$HOME/.ssh/authorized_keys
    > * Remote server: chmod 755 $HOME/.ssh; chmod 644
    >$HOME/.ssh/authorized_keys
    > This is critical!!
    > 3. Manually connect to the remote host
    > * Local server: ssh transfer_id@remote_host
    > * This will insure proper hash value added to
    >$HOME/.ssh/known_hosts


    >TEST:


    >Example code:


    >#!/usr/bin/bash
    >set -x
    >ssh -v -v -v -i /users/transfer_id/.ssh/batch_remote -o 'BatchMode yes'
    >10.5.110.22 "cd /tmp;ls -l"


    >Use the -v -v -v options to get as much diagnostic output as possible.
    >In this example, we go to the temp directory and list the files to test
    >connectivity and the ability to execute a command.



  3. Re: unattended file transfer with ssh

    You're right, of course and I've used rsync just as you describe, but
    in this particular application, my method was used because ..
    because... because it was already in use in a similar shell script and
    made development and testing easier. I'm trying to think of a
    disadvantage to using rsync and can't come up with one : )

    On Dec 7, 12:02 am, Unruh wrote:
    > "phynkel" writes:
    > >These are notes I created for company docs. They've been modified to
    > >remove all mention of company apps, file systems, directories, etc.
    > >They're sorta specific to my application, but give you an idea of how
    > >to get it to work. In my case I'm getting a listing of files that meet
    > >a certain criteria. Then I pipe them to cpio, then to ssh. Then, on the
    > >remote end, cpio takes over again and deposits the files, if all goes
    > >well. Easiset thing to do is cut and paste the commands only changing
    > >the machine and directory and ID names. In my case, the file transfer
    > >was from Solaris to Linux
    > >================================================== ======================Use rsync. It uses ssh ( if that is what you want).

    >
    > rsync -av remotemachine:/the/directory/ /local/directory
    >
    > It also has teh advantage that it only transfers the differences in the
    > files between the remote an local.
    > No need for cpio.
    >
    > To set up ssh, the stuff following is fine.
    >
    > >INTRODUCTION:
    > >transfer files use unattended ssh or batch mode. The shell script's
    > >code looks something like this:
    > >SSHOPTS="-i ~/.ssh/batch_remote -q -o 'BatchMode yes'"
    > >RPTROOT=/mnt/datadir/stuff
    > >cd $RPTROOT
    > >find . -follow -name '*_weekly' -newer $lasttime -print -type f | cpio
    > >-oc | ssh $SSHOPTS $REMOTESRVER "cd $REMOTESRVEAPPROOT; cpio -icdmuv"
    > >Note: the file "batch_remote" referred to in the first line is the
    > >private key on the local host. The public key file resides in
    > >$HOME/.ssh/authorized_keys on the remote host.
    > >INSTRUCTIONS:
    > > 1. Create private key/public key
    > > * On the local (sending) machine invoke this command:
    > >ssh-keygen -t dsa
    > > * When prompted for a file name accept the default
    > >($HOME/.ssh/id_dsa)
    > > * When prompted for a pass phrase press the Enter key
    > > * Two files will be created: $HOME/.ssh/id_dsa AND
    > >$HOME/.ssh/id_dsa.pub
    > > 2. Setup files using the same UNIX/Linux ID on both servers. Example
    > >ID: transfer_id
    > > * Local server: mv $HOME/.ssh/id_dsa $HOME/.ssh/batch_remote
    > > * The file name must match the file name used in your script
    > >for the "identity" file (see above code)
    > > * Local server: scp $HOME/.ssh/id_dsa.pub
    > >transfer_id@remote_host:/homedir/transfer_id/.ssh
    > > * Remote server: cat $HOME/.ssh/id_dsa.pub >>
    > >$HOME/.ssh/authorized_keys
    > > * Remote server: chmod 755 $HOME/.ssh; chmod 644
    > >$HOME/.ssh/authorized_keys
    > > This is critical!!
    > > 3. Manually connect to the remote host
    > > * Local server: ssh transfer_id@remote_host
    > > * This will insure proper hash value added to
    > >$HOME/.ssh/known_hosts
    > >TEST:
    > >Example code:
    > >#!/usr/bin/bash
    > >set -x
    > >ssh -v -v -v -i /users/transfer_id/.ssh/batch_remote -o 'BatchMode yes'
    > >10.5.110.22 "cd /tmp;ls -l"
    > >Use the -v -v -v options to get as much diagnostic output as possible.
    > >In this example, we go to the temp directory and list the files to test
    > >connectivity and the ability to execute a command.



+ Reply to Thread