cygwin scp -r fails - SSH

This is a discussion on cygwin scp -r fails - SSH ; Hi. I just installed the latest cygwin on my winXP laptop, and I am having an issue using scp -r. I have tried it to multiple Linux and Solaris boxes, so I am quite sure it's not on the receiving ...

+ Reply to Thread
Results 1 to 18 of 18

Thread: cygwin scp -r fails

  1. cygwin scp -r fails

    Hi.

    I just installed the latest cygwin on my winXP laptop, and I am having
    an issue using scp -r. I have tried it to multiple Linux and Solaris
    boxes, so I am quite sure it's not on the receiving server side.
    I am not sure if this is a bug or my problem; here's what I do:
    scp -rv 2006* user@unix:.

    Normally this would (and did on the older version) copy all of my 2006*
    folders into the home directory of the "user" on the box "unix" - what
    it does now, it creates empty folders.
    When the command is ran with multiple v for verbosity (and I am using
    folder windows for testing here) it still does not work, and here's the
    debug info.

    Has anyone seen it? Any ideas? I have found 2 similar postings on the
    web, but no resolutions so far.

    My cygwin version info:
    OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
    Cygwin version I am not sure how to tell, but the cygwin dll is
    1.5.19-4

    Thx in advance for any help/ideas.

    GregT

    ------ debug info -----

    bash-3.00$ scp -r -vvvv windows root@server:.
    Executing: program /usr/bin/ssh host server, user root, command scp -v
    -r -t
    OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
    debug2: ssh_connect: needpriv 0
    debug1: Connecting to server [10.10.10.10] port 22.
    debug1: Connection established.
    debug1: identity file /home/myuser/.ssh/identity type -1
    debug1: identity file /home/myuser/.ssh/id_rsa type -1
    debug1: identity file /home/myuser/.ssh/id_dsa type -1
    debug1: Remote protocol version 2.0, remote software version
    Sun_SSH_1.0.1
    debug1: match: Sun_SSH_1.0.1 pat Sun_SSH_1.0*
    debug1: Enabling compatibility mode for protocol 2.0
    debug1: Local version string SSH-2.0-OpenSSH_4.3
    debug2: fd 3 setting O_NONBLOCK
    debug1: SSH2_MSG_KEXINIT sent
    debug1: SSH2_MSG_KEXINIT received
    debug2: kex_parse_kexinit:
    diffie-hellman-group-exchange-sha1,diffie-hellman-g
    up14-sha1,diffie-hellman-group1-sha1
    debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
    debug2: kex_parse_kexinit:
    aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfou
    28,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128
    tr,aes192-ctr,aes256-ctr
    debug2: kex_parse_kexinit:
    aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfou
    28,arcfour256,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128
    tr,aes192-ctr,aes256-ctr
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@op
    ssh.com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit:
    hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@op
    ssh.com,hmac-sha1-96,hmac-md5-96
    debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
    debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit:
    debug2: kex_parse_kexinit: first_kex_follows 0
    debug2: kex_parse_kexinit: reserved 0
    debug2: kex_parse_kexinit: diffie-hellman-group1-sha1
    debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
    debug2: kex_parse_kexinit: aes128-cbc,blowfish-cbc,3des-cbc
    debug2: kex_parse_kexinit: aes128-cbc,blowfish-cbc,3des-cbc
    debug2: kex_parse_kexinit: hmac-sha1,hmac-md5
    debug2: kex_parse_kexinit: hmac-sha1,hmac-md5
    debug2: kex_parse_kexinit: none,zlib
    debug2: kex_parse_kexinit: none,zlib
    debug2: kex_parse_kexinit:
    \377\377\377\377,geo,lcttab,iso_8859_1,iso_8859_15 ,
    _CA,en_CA.ISO8859-1,en_US,en_US.ISO8859-1,en_US.ISO8859-15,en_US.ISO8859-15@eu
    ,es,es_MX,es_MX.ISO8859-1,fr,fr_CA,fr_CA.ISO8859-1,en
    debug2: kex_parse_kexinit:
    \377\377\377\377,geo,lcttab,iso_8859_1,iso_8859_15 ,
    _CA,en_CA.ISO8859-1,en_US,en_US.ISO8859-1,en_US.ISO8859-15,en_US.ISO8859-15@eu
    ,es,es_MX,es_MX.ISO8859-1,fr,fr_CA,fr_CA.ISO8859-1,en
    debug2: kex_parse_kexinit: first_kex_follows 0
    debug2: kex_parse_kexinit: reserved 0
    debug2: mac_init: found hmac-md5
    debug1: kex: server->client aes128-cbc hmac-md5 none
    debug2: mac_init: found hmac-md5
    debug1: kex: client->server aes128-cbc hmac-md5 none
    debug2: dh_gen_key: priv key bits set: 111/256
    debug2: bits set: 522/1024
    debug1: sending SSH2_MSG_KEXDH_INIT
    debug1: expecting SSH2_MSG_KEXDH_REPLY
    debug3: check_host_in_hostfile: filename /home/myuser/.ssh/known_hosts
    debug3: check_host_in_hostfile: match line 4
    debug3: check_host_in_hostfile: filename /home/myuser/.ssh/known_hosts
    debug3: check_host_in_hostfile: match line 4
    debug1: Host 'server' is known and matches the RSA host key.
    debug1: Found key in /home/myuser/.ssh/known_hosts:4
    debug2: bits set: 537/1024
    debug1: ssh_rsa_verify: signature correct
    debug2: kex_derive_keys
    debug2: set_newkeys: mode 1
    debug1: SSH2_MSG_NEWKEYS sent
    debug1: expecting SSH2_MSG_NEWKEYS
    debug2: set_newkeys: mode 0
    debug1: SSH2_MSG_NEWKEYS received
    debug1: SSH2_MSG_SERVICE_REQUEST sent
    debug2: service_accept: ssh-userauth
    debug1: SSH2_MSG_SERVICE_ACCEPT received
    debug2: key: /home/myuser/.ssh/identity (0x0)
    debug2: key: /home/myuser/.ssh/id_rsa (0x0)
    debug2: key: /home/myuser/.ssh/id_dsa (0x0)
    debug1: Authentications that can continue: publickey,password
    debug3: start over, passed a different list publickey,password
    debug3: preferred publickey,keyboard-interactive,password
    debug3: authmethod_lookup publickey
    debug3: remaining preferred: keyboard-interactive,password
    debug3: authmethod_is_enabled publickey
    debug1: Next authentication method: publickey
    debug1: Trying private key: /home/myuser/.ssh/identity
    debug3: no such identity: /home/myuser/.ssh/identity
    debug1: Trying private key: /home/myuser/.ssh/id_rsa
    debug3: no such identity: /home/myuser/.ssh/id_rsa
    debug1: Trying private key: /home/myuser/.ssh/id_dsa
    debug3: no such identity: /home/myuser/.ssh/id_dsa
    debug2: we did not send a packet, disable method
    debug3: authmethod_lookup password
    debug3: remaining preferred: ,password
    debug3: authmethod_is_enabled password
    debug1: Next authentication method: password
    root@server's password:
    debug3: packet_send2: adding 64 (len 57 padlen 7 extra_pad 64)
    debug2: we sent a password packet, wait for reply
    debug1: Authentication succeeded (password).
    debug2: fd 4 setting O_NONBLOCK
    debug2: fd 5 setting O_NONBLOCK
    debug1: channel 0: new [client-session]
    debug3: ssh_session2_open: channel_new: 0
    debug2: channel 0: send open
    debug1: Entering interactive session.
    debug2: callback start
    debug2: client_session2_setup: id 0
    debug1: Sending command: scp -v -r -t .
    debug2: channel 0: request exec confirm 0
    debug2: callback done
    debug2: channel 0: open confirm rwindow 0 rmax 16384
    debug2: channel 0: rcvd adjust 32768
    Entering directory: D0770 0 windows
    debug2: channel 0: read<=0 rfd 4 len 0
    debug2: channel 0: read failed
    debug2: channel 0: close_read
    debug2: channel 0: input open -> drain
    debug2: channel 0: ibuf empty
    debug2: channel 0: send eof
    debug2: channel 0: input drain -> closed
    debug2: channel 0: rcvd eof
    debug2: channel 0: output open -> drain
    debug2: channel 0: obuf empty
    debug2: channel 0: close_write
    debug2: channel 0: output drain -> closed
    debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
    debug2: channel 0: rcvd close
    debug3: channel 0: will not send data after close
    debug2: channel 0: almost dead
    debug2: channel 0: gc: notify user
    debug2: channel 0: gc: user detached
    debug2: channel 0: send close
    debug2: channel 0: is dead
    debug2: channel 0: garbage collecting
    debug1: channel 0: free: client-session, nchannels 1
    debug3: channel 0: status: The following connections are open:
    #0 client-session (t4 r0 i3/0 o3/0 fd -1/-1 cfd -1)

    debug3: channel 0: close_fds r -1 w -1 e 6 c -1
    debug3: fd 0 is not O_NONBLOCK
    debug3: fd 1 is not O_NONBLOCK
    debug1: Transferred: stdin 0, stdout 0, stderr 0 bytes in 0.7 seconds
    debug1: Bytes per second: stdin 0.0, stdout 0.0, stderr 0.0
    debug1: Exit status 0
    bash-3.00$


  2. Re: cygwin scp -r fails

    On 2006-02-15, gregt@post.pl wrote:
    > I just installed the latest cygwin on my winXP laptop, and I am having
    > an issue using scp -r. I have tried it to multiple Linux and Solaris
    > boxes, so I am quite sure it's not on the receiving server side.
    > I am not sure if this is a bug or my problem; here's what I do:
    > scp -rv 2006* user@unix:.
    >
    > Normally this would (and did on the older version) copy all of my 2006*
    > folders into the home directory of the "user" on the box "unix" - what
    > it does now, it creates empty folders.
    > When the command is ran with multiple v for verbosity (and I am using
    > folder windows for testing here) it still does not work, and here's the
    > debug info.
    >
    > Has anyone seen it? Any ideas? I have found 2 similar postings on the
    > web, but no resolutions so far.


    You're running it from the Cygwin shell or a Windows command prompt?
    I suspect the latter.

    On Unix, the shell expands wildcards but on Windows it's left up to
    individual apps.

    Up until 4.3p1, scp invoked ssh via the system(), which has the side
    effect of expanding wildcards (since it uses the shell). 4.3p1 and
    newer don't use system(), instead they construct an arglist and invoke
    execl directly.

    So, given a directory containing the files "foo" and "bar", on Unix
    the command "scp * host:" will be expanded to "scp foo bar host:", scp
    will open "foo" and "bar" in turn and send them to "host". On the
    Windows CLI, the "*" won't be expanded and scp will try to open a
    file literally called "*" which doesn't exist.

    A workaround is to have the Cygwin shell expand the wildcards, eg:
    c:\> sh -c "scp -r 2006* server:"

    --
    Darren Tucker (dtucker at zip.com.au)
    GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
    usually comes from bad judgement.

  3. Re: cygwin scp -r fails

    I tried, and runnish it from sh or bash or even cmd prompt makes no
    difference. Single (or multiple) files are just fine, the problem is
    only occuring when I try to do a recursive copy of a directory.
    Just an empty directory gets created.
    I never get any error that the file was not found.
    Thx for your suggestion though; I am sure it will save me a headache
    some time

    GregT


  4. Re: cygwin scp -r fails

    gregt@post.pl wrote:

    > I tried, and runnish it from sh or bash or even cmd prompt makes no
    > difference. Single (or multiple) files are just fine, the problem is
    > only occuring when I try to do a recursive copy of a directory.
    > Just an empty directory gets created.
    > I never get any error that the file was not found.
    > Thx for your suggestion though; I am sure it will save me a headache
    > some time


    Have you verified if on the remote filesystem you have enough space?

    >
    > GregT


    --
    kind regards,
    Claudiu Costin

  5. Re: cygwin scp -r fails

    Oh yeas, multiple times on multiple systems.. There's just no way space
    is an issue. Permissions, I do not see how either, as I am scping as
    root.
    The only other suspicion would be the windows rights... but in cygwin I
    can get into these directories and cat files out..... or cp locally,
    including cp -r, so... I am really at a loss.
    I wonder if I should look for another version of opensshm but I was
    hoping not to bother with this so much.. Last time I installed about a
    year ago everything just worked flawlessly, so I did not anticipate
    this becoming such a project.
    BTW, on the windows system I also have enough disk space - even if some
    huge temp files were needed....

    I think I will fail over to do ssh and tar with a pipe inside, it's
    just that scp used to be so convenient.

    Well; if anyone else has any ideas I am all eyes

    I take it the debug info did not help much... If there's a command I
    can run to get more info captured I'd be happy to do so.

    Thx.

    GregT


  6. Re: cygwin scp -r fails

    I have the same issue. Recursive scp under cygwin isn't working for me
    anymore...

    test:

    open an xterm...
    $ which ssh
    /usr/bin/ssh
    $ mkdir test
    $ echo "test" > test/test.dat
    $ scp -r test remotehost:~/
    $ ssh remotehost ls -l test
    total: 0

    The directory gets created on remotehost, but no files are copied!
    Yay!!

    pls pls let's get this fixed...


  7. Re: cygwin scp -r fails

    I did play with doing it in different shells, no luck; on my solaris
    boxes it's working fine; so certainly the problem is with cygwin or
    certain people's setup.
    In the process I tried to use a workaround that I would use (and it
    works) on solaris. Well, it didn;t work at first, but eventually did
    once I went into ksh in cygwin (seems like sh or bash, whatever the
    default is does not like something there)
    anyhow, this worked for me:

    ksh

    /cygdrive/c/Documents and Settings/admin/My Documents

    tar -cf - 2006* | ssh user@server "cd public_html; tar -xvf - "

    So.. just for the time being, this does work, and does the job...

    I am sort of happy to see someone else has the issue, but I'd rather
    see the resolution...

    GregT


  8. Re: cygwin scp -r fails

    On 2006-02-16, gregt@post.pl wrote:
    > I tried, and runnish it from sh or bash or even cmd prompt makes no
    > difference. Single (or multiple) files are just fine, the problem is
    > only occuring when I try to do a recursive copy of a directory.
    > Just an empty directory gets created.


    I wasn't able to reproduce it from your description. Could somone please
    email me a tarball of a directory that exhibits the problem? Or better
    yet, open a bug at bugzilla.mindrot.org and attach the tarball?

    --
    Darren Tucker (dtucker at zip.com.au)
    GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
    usually comes from bad judgement.

  9. Re: cygwin scp -r fails

    This is ANY directory.
    Simply scp with the -r does not work any which way I've tried it.
    I take it that Darren can copy recursively no problem?
    I just reinstalled cygwin, no change.
    I'll go on and submit a bug, will see what will happen.


  10. Re: cygwin scp -r fails


  11. Re: cygwin scp -r fails

    gregt@post.pl wrote:
    > Hi.


    > I just installed the latest cygwin on my winXP laptop, and I am having


    I just upgraded mine.
    $ ssh -V
    OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005


    > ------ debug info -----

    [snip]

    > Entering directory: D0770 0 windows
    > debug2: channel 0: read<=0 rfd 4 len 0
    > debug2: channel 0: read failed


    That part seems odd..

    I wonder if you can run it under strace and see what's happening when it
    goes into the directory.

    On my machine between the "Entering directory" and the "read<=0" I
    actually copy the file... Mine is just "file" under "dir".

    [snip...]
    Entering directory: D0755 0 dir
    debug2: channel 0: rcvd ext data 18
    Sink: D0755 0 dir
    debug2: channel 0: written 18 to efd 6
    Sending file modes: C0644 0 file
    debug2: channel 0: rcvd ext data 19
    Sink: C0644 0 file
    debug2: channel 0: written 19 to efd 6
    file 100% 0 0.0KB/s 00:00
    debug2: channel 0: rcvd ext data 8
    Sink: E
    debug2: channel 0: written 8 to efd 6
    debug2: channel 0: read<=0 rfd 4 len 0
    [snip]

    --
    Darren Dunham ddunham@taos.com
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >

  12. Re: cygwin scp -r fails

    On 2006-02-22, gregt@post.pl wrote:
    > This is ANY directory.
    > Simply scp with the -r does not work any which way I've tried it.
    > I take it that Darren can copy recursively no problem?


    Yep. 4.3p2 on both ends and scp -r works fine for me.

    I think Darren Dunham is on the right track (with the "read failed" thing)
    but I have no idea why it affects some people and not others. Maybe a
    cygwin thing binary/ascii thing? What's the CYGWIN env var on the problem
    machines?

    --
    Darren Tucker (dtucker at zip.com.au)
    GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
    usually comes from bad judgement.

  13. Re: cygwin scp -r fails

    Thx!

    Good idea.

    Comming from solaris i got so used to truss, that I did not even think
    of strace. Not to mention, I really did not expect CygWin to be that
    unixlike!

    Anyhow, I have generated a trace and added it to the bug. I cannot make
    enough sense out of it at the moment as I cannot compare it to a
    working one, but I agree, this is the right track to take.

    Thx for the idea!

    GregT


  14. Re: cygwin scp -r fails

    In article <1140749620.580265.280010@u72g2000cwu.googlegroups. com>
    gregt@post.pl writes:
    >Thx!
    >
    >Good idea.


    http://cfaj.freeshell.org/google/

    --Per Hedeland
    per@hedeland.org

  15. Re: cygwin scp -r fails

    I'm also having this problem with the latest scp on Cygwin: scripts
    which were working find are now failing to copy files within
    subdirectories for -r. I'd say its a nice, hard failure... Anything I
    can contribute to the resolution?

    les at ivsds dot com


    Darren Dunham wrote:
    > gregt@post.pl wrote:
    > > Hi.

    >
    > > I just installed the latest cygwin on my winXP laptop, and I am having

    >
    > I just upgraded mine.
    > $ ssh -V
    > OpenSSH_4.3p1, OpenSSL 0.9.8a 11 Oct 2005
    >
    >
    > > ------ debug info -----

    > [snip]
    >
    > > Entering directory: D0770 0 windows
    > > debug2: channel 0: read<=0 rfd 4 len 0
    > > debug2: channel 0: read failed

    >
    > That part seems odd..
    >
    > I wonder if you can run it under strace and see what's happening when it
    > goes into the directory.
    >
    > On my machine between the "Entering directory" and the "read<=0" I
    > actually copy the file... Mine is just "file" under "dir".
    >
    > [snip...]
    > Entering directory: D0755 0 dir
    > debug2: channel 0: rcvd ext data 18
    > Sink: D0755 0 dir
    > debug2: channel 0: written 18 to efd 6
    > Sending file modes: C0644 0 file
    > debug2: channel 0: rcvd ext data 19
    > Sink: C0644 0 file
    > debug2: channel 0: written 19 to efd 6
    > file 100% 0 0.0KB/s 00:00
    > debug2: channel 0: rcvd ext data 8
    > Sink: E
    > debug2: channel 0: written 8 to efd 6
    > debug2: channel 0: read<=0 rfd 4 len 0
    > [snip]
    >
    > --
    > Darren Dunham ddunham@taos.com
    > Senior Technical Consultant TAOS http://www.taos.com/
    > Got some Dr Pepper? San Francisco, CA bay area
    > < This line left intentionally blank to confuse you. >



  16. Re: cygwin scp -r fails

    On 2006-02-25, les.matheson@gmail.com wrote:
    > I'm also having this problem with the latest scp on Cygwin: scripts
    > which were working find are now failing to copy files within
    > subdirectories for -r. I'd say its a nice, hard failure... Anything I
    > can contribute to the resolution?


    I think we've figured it out: Cygwin's struct dirent changed (the d_ino
    member doesn't exist any more) and I think the bundled scp is built
    against the old headers.

    There's a patch attached to the bug, you could try that.
    http://bugzilla.mindrot.org/show_bug.cgi?id=1161

    --
    Darren Tucker (dtucker at zip.com.au)
    GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
    usually comes from bad judgement.

  17. Re: cygwin scp -r fails

    Thx Darren.

    I see tha patch at the bugzilla.

    I assume that I would have to get the source of openssh and recompile
    it after applying the patch?

    I am not sure how this process works on CygWin.

    Thx for figuring it out.

    GT


  18. Re: cygwin scp -r fails

    On 2006-02-27, gregt@post.pl wrote:
    > I assume that I would have to get the source of openssh and recompile
    > it after applying the patch?


    Yes. If you apply the second one you will also need to run "autoconf"
    to rebuild configure.

    > I am not sure how this process works on CygWin.


    Install the prereq packages (they're listed in README.platform and
    contrib/cygwin/README), apply the patch to the unpacked source and
    the "./configure && make".

    > Thx for figuring it out.


    You're welcome. I'll mention it to the cygwin folks.

    --
    Darren Tucker (dtucker at zip.com.au)
    GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
    Good judgement comes with experience. Unfortunately, the experience
    usually comes from bad judgement.

+ Reply to Thread