ssh with -t outputs additional ^m's - SSH

This is a discussion on ssh with -t outputs additional ^m's - SSH ; Why does the following happen: $ ssh -t localhost ls -l /tmp | cat -vet total 20^M$ -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M$ -rw-rw-r-- 1 root root 0 Sep 16 21:12 file2^M$ -rw-rw-r-- 1 root root 0 ...

+ Reply to Thread
Results 1 to 9 of 9

Thread: ssh with -t outputs additional ^m's

  1. ssh with -t outputs additional ^m's

    Why does the following happen:

    $ ssh -t localhost ls -l /tmp | cat -vet
    total 20^M$
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M$
    -rw-rw-r-- 1
    root root 0 Sep 16 21:12 file2^M$
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file3^M$

    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file4^M$
    -rw-rw-r-- 1 root root 0 Sep 16
    21:12 file5^M$
    Connection to localhost closed.

    IOW when -t is specified to ssh the output contains additional ^M's and
    the is "stairstepped". We need to use -t in order to see the output from
    programs run on the remote system who apparently write to /dev/tty.
    However having the extra carriage returns messes things up. For example
    doing something like:

    $ processFiles `ssh -t remotesystem ls /tmp`

    gives "processFiles" filenames ending in ^M! Now if you have the source
    code for "processFiles" or the script it could account for the extra
    ^M's but what if you don't (as is my case)? Why does ssh output extra
    ^M's (and in some cases ^I's, etc.)? Is there a way to turn that off?
    --
    Andrew DeFaria
    That's a hell of an ambition, to be mellow. It's like wanting to be
    senile. - Randy Newman


  2. Re: ssh with -t outputs additional ^m's

    Andrew DeFaria wrote:
    > Why does the following happen:
    >
    > $ ssh -t localhost ls -l /tmp | cat -vet
    > total 20^M$
    > -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M$
    > -rw-rw-r-- 1
    > root root 0 Sep 16 21:12 file2^M$
    > -rw-rw-r-- 1 root root 0 Sep 16 21:12 file3^M$
    >
    > -rw-rw-r-- 1 root root 0 Sep 16 21:12 file4^M$
    > -rw-rw-r-- 1 root root 0 Sep 16
    > 21:12 file5^M$
    > Connection to localhost closed.
    >
    > IOW when -t is specified to ssh the output contains additional ^M's and
    > the is "stairstepped". We need to use -t in order to see the output from
    > programs run on the remote system who apparently write to /dev/tty.
    > However having the extra carriage returns messes things up. For example
    > doing something like:
    >
    > $ processFiles `ssh -t remotesystem ls /tmp`
    >
    > gives "processFiles" filenames ending in ^M! Now if you have the source
    > code for "processFiles" or the script it could account for the extra
    > ^M's but what if you don't (as is my case)? Why does ssh output extra
    > ^M's (and in some cases ^I's, etc.)? Is there a way to turn that off?
    > --
    > Andrew DeFaria
    > That's a hell of an ambition, to be mellow. It's like wanting to be
    > senile. - Randy Newman


    First question - why are you ssh'ing to localhost? If that's a type-o is
    the remote host a windows server? What is the outout of "ssh remotehost
    uname -a"?

  3. Re: ssh with -t outputs additional ^m's

    Chuck wrote:
    > First question - why are you ssh'ing to localhost?

    Sorry. I used it as an example. The problem we are having is at work but
    I don't have access to comp.security.ssh from work. At home here I have
    two machines: an XP box running Cygwin and a Fedora Linux box. I did the
    example on the Linux box because I didn't want to introduce Cygwin as a
    part of the equation. Even using localhost the example nicely
    demonstrated itself.

    At work we are using RHEL 3 and RHEL 4 as well as Solaris 9 (but I don't
    think there are any Solaris 9's in the group I'm having the problem with).
    > If that's a type-o is the remote host a windows server?

    No, I kept Cygwin out of the picture. In the example localhost was my
    Linux server. However, if I ssh from my Cygwin box to my Linux box I get
    the same ^M's at the end of each line. The key appears to be -t, if not
    specified then no ^M's; if specified then ^M's. The question is why.
    > What is the outout of "ssh remotehost uname -a"?

    $ uname -a
    Linux jupiter 2.6.12-1.1381_FC3 #1 Fri Oct 21 03:46:55 EDT 2005 i686
    athlon i386 GNU/Linux

    and the Cygwin box, in case you are interested:

    $ uname -a
    CYGWIN_NT-5.1 voyager 1.5.25(0.156/4/2) 2008-04-17 12:11 i686 Cygwin
    --
    Andrew DeFaria
    A friend of mine is into Voodoo Acupuncture. You don't have to go.
    You'll just be walking down the street, and...........ooooohhhhhh,
    that's much better...


  4. Re: ssh with -t outputs additional ^m's

    In article <48d0856c$0$89871$815e3792@news.qwest.net> Andrew DeFaria
    writes:
    >
    >Why does the following happen:
    >
    >$ ssh -t localhost ls -l /tmp | cat -vet
    >total 20^M$
    > -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M$
    > -rw-rw-r-- 1
    >root root 0 Sep 16 21:12 file2^M$
    > -rw-rw-r-- 1 root root 0 Sep 16 21:12 file3^M$
    >
    >-rw-rw-r-- 1 root root 0 Sep 16 21:12 file4^M$
    > -rw-rw-r-- 1 root root 0 Sep 16
    >21:12 file5^M$
    > Connection to localhost closed.
    >
    >IOW when -t is specified to ssh the output contains additional ^M's and
    >the is "stairstepped".


    That's the expected result - the ^M's are sent by the tty driver for the
    remote tty, this is standard behaviour in "cooked mode", and the
    stairstepping happens because you're using the -v option to cat, which
    turns the ^M's into '^' + 'M' instead of a carrriage return. None of it
    has anything to do with ssh, other than the fact that it's probably the
    only program that allows you to easily run a remote program behind a tty
    like this.

    > We need to use -t in order to see the output from
    >programs run on the remote system who apparently write to /dev/tty.


    That would be an extremely weird thing for a program to do. Some
    programs can only run when their stdin/out is a tty though.

    >However having the extra carriage returns messes things up. For example
    >doing something like:
    >
    >$ processFiles `ssh -t remotesystem ls /tmp`
    >
    >gives "processFiles" filenames ending in ^M! Now if you have the source
    >code for "processFiles" or the script it could account for the extra
    >^M's but what if you don't (as is my case)? Why does ssh output extra
    >^M's (and in some cases ^I's, etc.)? Is there a way to turn that off?


    SSH outputs neither ^M's nor ^I's - the latter is probably the result of
    'ls' trying to do a nicely formatted output, you could use the -1 option
    to get rid of that. And you can perhaps get rid of the ^M's by doing

    ssh -t remotesystem 'stty -onlcr; ls /tmp'

    (assumes Unix on the remote, of course).

    --Per Hedeland
    per@hedeland.org


  5. Re: ssh with -t outputs additional ^m's

    Per Hedeland wrote:
    > In article <48d0856c$0$89871$815e3792@news.qwest.net> Andrew DeFaria
    > writes:
    >> Why does the following happen:
    >>
    >> $ ssh -t localhost ls -l /tmp | cat -vet
    >> total 20^M$
    >> -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M$
    >> -rw-rw-r-- 1
    >> root root 0 Sep 16 21:12 file2^M$
    >> -rw-rw-r-- 1 root root 0 Sep 16 21:12 file3^M$
    >>
    >> -rw-rw-r-- 1 root root 0 Sep 16 21:12 file4^M$
    >> -rw-rw-r-- 1 root root 0 Sep 16
    >> 21:12 file5^M$
    >> Connection to localhost closed.
    >>
    >> IOW when -t is specified to ssh the output contains additional ^M's and
    >> the is "stairstepped".

    > That's the expected result - the ^M's are sent by the tty driver for
    > the remote tty, this is standard behaviour in "cooked mode", and the
    > stairstepping happens because you're using the -v option to cat, which
    > turns the ^M's into '^' + 'M' instead of a carrriage return. None of
    > it has anything to do with ssh, other than the fact that it's probably
    > the only program that allows you to easily run a remote program behind
    > a tty like this.

    I know that the ^+M's are coming out from cat's -v. I'm using the -v to
    show you the problem. Without the cat the actual ^M are still there:

    $ ssh -t localhost ls -l /tmp > output
    $ cat -v output
    total 20^M
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file1^M
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file2^M
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file3^M
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file4^M
    -rw-rw-r-- 1 root root 0 Sep 16 21:12 file5^M

    The problem arises because we are trying to provide Rational Clearcase
    access to a machine that doesn't have Clearcase installed (for various
    reasons). So we've wrapped cleartool (i.e. ct) with ssh to a machine
    that does. But people can have things like:

    $ alias ciwork='ct ci -nc $(ct lsco -s -me -avobs)'

    This says "list all checkouts by me in all vobs" (ct lsco -s -me -avobs)
    which produces a list of filenames that are checked out. This list is
    then used by ct ci to check in the files. The problem is that the file
    names have a ^M tacked on to them so the checkin fails. This is just one
    example.
    >> We need to use -t in order to see the output from programs run on the
    >> remote system who apparently write to /dev/tty.

    > That would be an extremely weird thing for a program to do. Some
    > programs can only run when their stdin/out is a tty though.

    Not really. For example, there is the ct man command, which will give
    you a man page for a cleartool subcommand. It's piping things to $PAGER
    or more and more's prompt doesn't display. You can see this with Unix's
    man(1) too. Do ssh -t man man and it pages as expected. But
    take off the -t and it doesn't page at all!
    >> However having the extra carriage returns messes things up. For
    >> example doing something like:
    >>
    >> $ processFiles `ssh -t remotesystem ls /tmp`
    >>
    >> gives "processFiles" filenames ending in ^M! Now if you have the
    >> source code for "processFiles" or the script it could account for the
    >> extra ^M's but what if you don't (as is my case)? Why does ssh output
    >> extra ^M's (and in some cases ^I's, etc.)? Is there a way to turn
    >> that off?

    > SSH outputs neither ^M's nor ^I's - the latter is probably the result
    > of 'ls' trying to do a nicely formatted output,

    I just demonstrated above that it does.
    > you could use the -1 option to get rid of that. And you can perhaps
    > get rid of the ^M's by doing
    >
    > ssh -t remotesystem 'stty -onlcr; ls /tmp'
    >
    > (assumes Unix on the remote, of course).

    Hmmm... This stty -onlcr shows promise! I'll have to more fully test
    this at work. I don't have Clearcase nor the proper environment at home.
    Thanks for the tip!

    Note I still mentioned all of the above for completeness sake. I'll try
    to remember to report back if this works. Once again, thanks for the
    pointer...
    --
    Andrew DeFaria
    Oops. My brain just hit a bad sector.


  6. Re: ssh with -t outputs additional ^m's

    In article <48D31BE7.9050105@DeFaria.com> Andrew DeFaria
    writes:

    >Per Hedeland wrote:
    >> That's the expected result - the ^M's are sent by the tty driver for
    >> the remote tty, this is standard behaviour in "cooked mode", and the
    >> stairstepping happens because you're using the -v option to cat, which
    >> turns the ^M's into '^' + 'M' instead of a carrriage return. None of
    >> it has anything to do with ssh, other than the fact that it's probably
    >> the only program that allows you to easily run a remote program behind
    >> a tty like this.

    >I know that the ^+M's are coming out from cat's -v. I'm using the -v to
    >show you the problem. Without the cat the actual ^M are still there:


    Yes of course - I only said that the stairstepping happened because you
    turned the ^M's into ^+M's by using cat -v.

    >>> We need to use -t in order to see the output from programs run on the
    >>> remote system who apparently write to /dev/tty.

    >> That would be an extremely weird thing for a program to do. Some
    >> programs can only run when their stdin/out is a tty though.

    >Not really. For example, there is the ct man command, which will give
    >you a man page for a cleartool subcommand. It's piping things to $PAGER
    >or more and more's prompt doesn't display. You can see this with Unix's
    >man(1) too. Do ssh -t man man and it pages as expected. But
    >take off the -t and it doesn't page at all!


    This doesn't mean that it's writing to /dev/tty (i.e. the process'
    controlling terminal), only (as I said) that it needs its std(in/)out to
    be a tty. man(1) most certainly doesn't write to /dev/tty, but
    $PAGER/more can't work if its stdout isn't a tty.

    >> SSH outputs neither ^M's nor ^I's - the latter is probably the result
    >> of 'ls' trying to do a nicely formatted output,

    >I just demonstrated above that it does.


    No, you demonstrated that you *saw* ^M's and ^I's - as I explained, SSH
    is not "outputting" them, it only passes on what comes from the program
    + tty at the other end. Doing anything else would be a bad bug.

    --Per Hedeland
    per@hedeland.org

  7. Re: ssh with -t outputs additional ^m's

    > Hmmm... This stty -onlcr shows promise! I'll have to more fully test
    > this at work. I don't have Clearcase nor the proper environment at
    > home. Thanks for the tip!

    This doesn't quite work. When using stty -onlcr it's true that the ^M's
    disappear when redirected to a file. However when not redirected the
    output is still stair stepped.

    To state this differently:

    1. Using ssh without -t I get no ^M's in any redirected output nor do
    I get stair stepping of output to stdout.
    2. Using ssh with -t I get ^M's in redirected output but no stair
    stepping of output to stdout
    3. Using ssh with -t and adding stty -onlcr I get no ^M's in
    redirected output but I do get stair stepping of output.

    Is there a way I can get #1 (No ^M's in redirected output nor stair
    stepping to stdout) with -t specified to ssh?

    Again I need to specify -t because some programs, like man, will not
    work properly or output stuff properly over ssh without -t. Another
    example would be something like xauth. Run through ssh without -t you
    don't see it's prompt.
    --
    Andrew DeFaria
    I'm not into working out. My philosophy is no pain, no pain.


  8. Re: ssh with -t outputs additional ^m's

    In article <48d87d1f$0$48224$815e3792@news.qwest.net> Andrew DeFaria
    writes:
    >
    >This doesn't quite work. When using stty -onlcr it's true that the ^M's
    >disappear when redirected to a file. However when not redirected the
    >output is still stair stepped.


    The data is the same in both cases, you have to decide what you want.

    It seems to me that you are unaware of certain fundamental things that
    have absolutely nothing to do with SSH (but quite a bit to do with
    Unix):

    1. ^M is a carriage return, a.k.a. CR. Its meaning when sent to a
    terminal is "move the cursor to the left margin".

    2. What you call "stairstepping" is the result of another basic control
    character, newline a.k.a. NL. Its meaning when sent to a terminal is
    "move the cursor one line down" (no horizontal movement).

    3. On Unix, lines are delimited by NL *only*.

    4. On Unix, when data is output through a tty device in standard a.k.a.
    "cooked" mode, NL is translated to CR+NL (this is what the onlcr setting
    does).

    When you give ssh the -t option, it turns off I/O processing like onlcr
    for the local tty - otherwise a) that processing would be done twice
    (with erroneous results) if the remote program runs its tty in "cooked"
    mode, or b) a remote program that wants to run its tty in "raw" mode
    can't work, since the local tty is in "cooked" mode.

    Hence what you get from ssh is *exactly* what comes from the remote tty
    device, and fit for sending to a terminal without further processing -
    which is exactly what happens when you run ssh -t interactively on a
    tty.

    Now you want to (or you said that you wanted to) change that output to
    make it fit for a different use - direct processing by a program. Using
    'stty -onlcr' on the remote tty is one way to do get rid of the
    (generated) CRs, but as you found out (without understanding it),
    various programs may make various assumptions when they find that their
    stdout is a tty, e.g. 'ls' formats its output differently.

    And if you send the modified output to the local tty that ssh has set up
    to work right with the unmodified output, you will *of course* not get
    the "right" result - the local tty doesn't generate CR+LF from the bare
    LFs in the output, so you get "stairstepping". Whereas if you save the
    output to a file and just 'cat' it afterwards, you *will* get the
    "right" result, since the tty is then again doing the onlcr processing.

    --Per Hedeland
    per@hedeland.org

  9. Re: ssh with -t outputs additional ^m's

    Per Hedeland wrote:
    > In article <48d87d1f$0$48224$815e3792@news.qwest.net> Andrew DeFaria
    > writes:
    >> This doesn't quite work. When using stty -onlcr it's true that the
    >> ^M's disappear when redirected to a file. However when not redirected
    >> the output is still stair stepped.

    > The data is the same in both cases, you have to decide what you want.
    >
    > It seems to me that you are unaware of certain fundamental things that
    > have absolutely nothing to do with SSH (but quite a bit to do with Unix):
    >
    > 1. ^M is a carriage return, a.k.a. CR. Its meaning when sent to a
    > terminal is "move the cursor to the left margin".
    >
    > 2. What you call "stairstepping" is the result of another basic
    > control character, newline a.k.a. NL. Its meaning when sent to a
    > terminal is "move the cursor one line down" (no horizontal movement).
    >
    > 3. On Unix, lines are delimited by NL *only*.
    >
    > 4. On Unix, when data is output through a tty device in standard
    > a.k.a. "cooked" mode, NL is translated to CR+NL (this is what the
    > onlcr setting does).

    Actually I'm aware of all of that already.
    > When you give ssh the -t option, it turns off I/O processing like
    > onlcr for the local tty - otherwise a) that processing would be done
    > twice (with erroneous results) if the remote program runs its tty in
    > "cooked" mode, or b) a remote program that wants to run its tty in
    > "raw" mode can't work, since the local tty is in "cooked" mode.
    >
    > Hence what you get from ssh is *exactly* what comes from the remote
    > tty device, and fit for sending to a terminal without further
    > processing - which is exactly what happens when you run ssh -t
    > interactively on a tty.
    >
    > Now you want to (or you said that you wanted to) change that output to
    > make it fit for a different use - direct processing by a program.
    > Using 'stty -onlcr' on the remote tty is one way to do get rid of the
    > (generated) CRs, but as you found out (without understanding it),
    > various programs may make various assumptions when they find that
    > their stdout is a tty, e.g. 'ls' formats its output differently.

    Actually I'm not wanting to deal with cooked or raw modes at all. The
    only thing I'm trying to deal with is programs that assume they can
    write to the tty directly (e.g. xauth or cleartool). I just want them to
    write through the ssh so that they show up on the other side. But ssh
    gives me only one level to control that, -t, which gets into cooked vs.
    raw output. Now it may be that there is no way to deal with one issue
    without dealing with the other. I don't know. I don't normally do low
    level ioctrl stuff.

    And yes I know that there are some programs that will fail or not work
    right unless they can write to the tty directly but I'm not trying to
    address or accommodate for them at this time. Stated differently, there
    is no problem and no reason why xauth or cleartool needs to write to the
    tty directly. There may be reasons for other programs to do so but not
    these (AFAICT).
    > And if you send the modified output to the local tty that ssh has set
    > up to work right with the unmodified output, you will *of course* not
    > get the "right" result - the local tty doesn't generate CR+LF from the
    > bare LFs in the output, so you get "stairstepping". Whereas if you
    > save the output to a file and just 'cat' it afterwards, you *will* get
    > the "right" result, since the tty is then again doing the onlcr
    > processing.

    Would it be possible to write a small filter that reads from stdin,
    set's onlcr and writes out what it read in? Would that solve this problem?

    I wrote a small Perl script to re-add ^M's iff writing to stdout. This
    seems to be working better...
    --
    Andrew DeFaria
    Very funny Scotty - now beam down my clothes.


+ Reply to Thread