compress non-ssh X11 - BSD

This is a discussion on compress non-ssh X11 - BSD ; I'm having problems w/ XDMX response times on a 100Mbit network. All seems to run well, I can run TWM, Fluxbox, GNOME w/ acceptable response times, but window focus swithes between applications are painfully slow. I've posted on XDMX, but ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: compress non-ssh X11

  1. compress non-ssh X11

    I'm having problems w/ XDMX response times on a 100Mbit network. All
    seems to run well, I can run TWM, Fluxbox, GNOME w/ acceptable response
    times, but window focus swithes between applications are painfully slow.
    I've posted on XDMX, but the last post entry was in 2005, so I'm not
    holding my breath for a response.

    I was wondering if I could somehow compress the X11 communication
    between machines to hopefully increase reponse time for window repaints
    (I'm presuming that what's causing my problem switching windows). I've
    tried SSH tunning w/ compression for port 6000, but when I setenv
    DISPLAY to server, I get a displayed window but it's not using port
    6000. I can't seem to tell in advance what port X will use for steady
    state communication between the machines. TIA.

    Regards,

    Monty

  2. Re: compress non-ssh X11

    Monty Hall wrote:
    +---------------
    | I was wondering if I could somehow compress the X11 communication
    | between machines to hopefully increase reponse time for window repaints
    | (I'm presuming that what's causing my problem switching windows). I've
    | tried SSH tunning w/ compression for port 6000, but when I setenv
    | DISPLAY to server, I get a displayed window but it's not using port
    | 6000. I can't seem to tell in advance what port X will use for steady
    | state communication between the machines. TIA.
    +---------------

    You're confusing the ports X uses on a local display [usually 6000,
    but may include 6001, 6002, etc., if you have more than one physical
    display] with the ports SSH uses on the SSH-server side to tunnel the
    X connection back to the SSH client machine, which is typically 6010+n
    [n = 0, 1, ... for each distinct SSH connection with X forwarding].

    For example, at the moment I'm away from home, and currently have
    three SSH sessions into my home server/desktop, so there are *four*
    available X "displays" [counting the physical one on the server/desktop].
    The active ports on the server/desktop:

    % sockstat -4 | grep '60[0-9][0-9]'
    rpw3 xmag 22510 3 tcp4 127.0.0.1:51728 127.0.0.1:6011
    rpw3 xbiff 16583 3 tcp4 127.0.0.1:50994 127.0.0.1:6010
    rpw3 sshd 12087 9 tcp4 127.0.0.1:6012 *:*
    rpw3 sshd 12067 9 tcp4 127.0.0.1:6011 *:*
    rpw3 sshd 12067 11 tcp4 127.0.0.1:6011 127.0.0.1:51728
    rpw3 sshd 12041 9 tcp4 127.0.0.1:6010 *:*
    rpw3 sshd 12041 13 tcp4 127.0.0.1:6010 127.0.0.1:50994
    root Xorg 876 3 tcp4 *:6000 *:*
    %

    And, yes, all three forwarded X connections (ports 6010, -11, -12)
    on the server/desktop get forwarded to port 6000 [a real display]
    on my remote client [a FreeBSD laptop] -- oops, actually *not* TCP
    port 6000 but the Unix-domain socket "/tmp/.X11-unix/X0".

    Oh, and, yes, the main SSH connection (that the other two SSH sessions
    are tunneled through -- long story) is using compression, so all of
    those X connections are automatically getting compressed as well.

    Does that help?


    -Rob

    -----
    Rob Warnock
    627 26th Avenue
    San Mateo, CA 94403 (650)572-2607


  3. Re: compress non-ssh X11

    On Sat, 05 Jul 2008 15:03:23 -0400, Monty Hall wrote:

    > I'm having problems w/ XDMX response times on a 100Mbit network. All
    > seems to run well, I can run TWM, Fluxbox, GNOME w/ acceptable response
    > times, but window focus swithes between applications are painfully slow.
    > I've posted on XDMX, but the last post entry was in 2005, so I'm not
    > holding my breath for a response.


    Early warning: I know nothing about xdmx. Something I'll have to
    investigate. When I run remote X on a secure local network, I just open
    port 6000 on the x server (usually laptop) machine and make sure that X
    is started without the -notcp switch (which is almost universally the
    default, these days). Then xhost +remote-machine is enough to allow an
    initial xterm -display laptop:0.0, and everything is hunky-dory.

    Clearly there are security issues with this approach, but it has
    substantially the best performance, for my (occasional) use.

    > I was wondering if I could somehow compress the X11 communication
    > between machines to hopefully increase reponse time for window repaints


    Compressing the protocol, particularly in a non-X-specific way, like ssh,
    is definitely a good way to *increase* the response *time*.
    Unfortunately I doubt that's what you really want. The trouble with
    compression schemes, particularly where you have access to a reasonable
    amount of raw bandwidth, is that they necessarily buffer the packets (to
    find enough redundancy to produce compression), and that means that the
    first, short packets take longer to get across the link than they would
    if uncompressed. X11 is critically latency-limited. Everything that you
    can do to minimize protocol latency will make it seem faster.


    > (I'm presuming that what's causing my problem switching windows). I've
    > tried SSH tunning w/ compression for port 6000, but when I setenv
    > DISPLAY to server, I get a displayed window but it's not using port
    > 6000. I can't seem to tell in advance what port X will use for steady
    > state communication between the machines. TIA.


    I think that Rob has answered the port number question nicely, so I won't
    duplicate that.

    I will add, though, that you'll get a lot of benefit if you can make sure
    that your X server has "backing store" turned on, and uses as much of it
    as you can arrange. Backing-store is where the server keeps a copy of
    the contents of obscured windows, rather than throwing that away, on the
    assumption that the application can redraw when exposed. Keeping backing
    store turned on doesn't help with some applications (those that are
    busily changing their contents even when not on top) but it helps most of
    the more static, terminal-like ones, and it provides a very good
    qualitative "feel" difference, IMO.

    I have to admit, though, that most of the time these days, I get the best
    performance "feel" by changing the "network cleavage" point from the user
    interface to the file system: running the application locally against a
    cached or remote file system is usually a nicer experience than running
    remote X, IMO. (Fuse+sshfs and rsync can both help a lot, here.)

    Cheers,

    --
    Andrew

  4. Re: compress non-ssh X11

    Andrew Reilly wrote:
    +---------------
    | Monty Hall wrote:
    | > I was wondering if I could somehow compress the X11 communication
    | > between machines to hopefully increase reponse time for window repaints
    |
    | Compressing the protocol, particularly in a non-X-specific way,
    | like ssh, is definitely a good way to *increase* the response *time*.
    +---------------

    Actually, in the specific case of SSH, it *doesn't* [unless you
    have a *really* slow CPU at either end]. You really do win, for
    two main reasons:

    1. SSH turns off the Nagle algorithm on its sockets, see routine
    "set_nodelay()" in "/usr/src/crypto/openssh/misc.c". This allows
    SSH to provide reasonable [well, as good as you're going to get!]
    latency, which you need with X11 or when typing/editing with
    remote echo, etc. In particular, it *is* disabled for X11 forwarding
    sockets as well as "shell" channels, see "channel_post_x11_listener()"
    in file "/usr/src/crypto/openssh/channels.c".

    2. The compression used in SSH carefully does not ever wait for
    new input data before compressing what it has and sending it to
    the output. See the comments before routine "buffer_compress()"
    in "/usr/src/crypto/openssh/compress.c". Yes, this may produce
    less than optimal compression, but still does a very good job
    while not compromising latency.

    +---------------
    | The trouble with compression schemes, particularly where you have
    | access to a reasonable amount of raw bandwidth, is that they
    | necessarily buffer the packets (to find enough redundancy to
    | produce compression), and that means that the first, short packets
    | take longer to get across the link than they would if uncompressed.
    +---------------

    Fortunately, that's *not* true for SSH, see #2 above.

    +---------------
    | X11 is critically latency-limited. Everything that you
    | can do to minimize protocol latency will make it seem faster.
    +---------------

    This is *very* true, and that's why compression of SSH-forwarded
    X11 streams wins... well, somewhat. [You simply can't win very much
    over a really slow link!] In particular, any X11 op that would have
    needed two packets uncompressed but can do with one when compressed
    will be faster.

    Still, the fact remains that almost any X11 network connection except
    a local GbE is going to feel sluggish compared to a same-machine
    display. E.g., when I'm using EDGE2 (ping latency ~250-500 ms!!)
    or even HSDPA (~150-200 ms) "broadband" wireless from my laptop I
    almost *never* open a reverse X11 connection [just a bunch of local
    xterms SSH'd to the server]. But when I'm borrowing somebody's
    low-latency DSL or cable-modem line (~30-35 ms), then using the
    occasional X widget isn't too painful when tunneled over "ssh -C -X".
    Echo from typing is acceptable. Menus are usable.

    +---------------
    | I will add, though, that you'll get a lot of benefit if you can make
    | sure that your X server has "backing store" turned on, and uses as
    | much of it as you can arrange.
    +---------------

    True, though note that the backing store in a lot of X server drivers
    is broken and/or disabled unless used with *precisely* the right hardware.
    E.g., it doesn't work on my laptop. From "/var/log/XFree86.0.log":

    (--) PCI:*(1:5:0) ATI Technologies Inc Radeon Mobility U1 rev 0,
    Mem @ 0xe0000000/28, 0xd0100000/16, I/O @ 0x9000/8
    ...
    (II) VESA: driver for VESA chipsets: vesa
    (II) Primary Device is: PCI 01:05:0
    (--) Chipset vesa found
    ...
    (II) VESA(0): VESA VBE OEM Vendor: ATI Technologies Inc.
    (II) VESA(0): VESA VBE OEM Product: U1
    (II) VESA(0): VESA VBE OEM Product Rev: 01.00
    (WW) VESA(0): Failed to set write-combining range (0xe0000000,0x3ff0000)
    (II) VESA(0): virtual address = 0x28400000,
    physical address = 0xe0000000, size = 67043328
    (==) VESA(0): Default visual is TrueColor
    (==) VESA(0): Backing store disabled
    ...

    (*sigh*)

    +---------------
    | I have to admit, though, that most of the time these days, I get the best
    | performance "feel" by changing the "network cleavage" point from the user
    | interface to the file system: running the application locally against a
    | cached or remote file system is usually a nicer experience than running
    | remote X, IMO. (Fuse+sshfs and rsync can both help a lot, here.)
    +---------------

    Yup. Though since I was trained to tolerate "rubber-band" typing on
    an ASR-33 on TOPS-10 (keypress-to-print latency 200 ms *min*!), I can
    tolerate editing with remote-echo on suprisingly slow connections! ;-}


    -Rob

    -----
    Rob Warnock
    627 26th Avenue
    San Mateo, CA 94403 (650)572-2607


  5. Re: compress non-ssh X11

    On Sun, 06 Jul 2008 21:55:23 -0500, Rob Warnock wrote:

    > Actually, in the specific case of SSH, it *doesn't* [unless you have a
    > *really* slow CPU at either end]. You really do win, for two main
    > reasons:
    >
    > 1. SSH turns off the Nagle algorithm on its sockets, see routine
    > "set_nodelay()" in "/usr/src/crypto/openssh/misc.c". This allows SSH
    > to provide reasonable [well, as good as you're going to get!]
    > latency, which you need with X11 or when typing/editing with remote
    > echo, etc. In particular, it *is* disabled for X11 forwarding sockets
    > as well as "shell" channels, see "channel_post_x11_listener()" in
    > file "/usr/src/crypto/openssh/channels.c".
    >
    > 2. The compression used in SSH carefully does not ever wait for
    > new input data before compressing what it has and sending it to the
    > output. See the comments before routine "buffer_compress()" in
    > "/usr/src/crypto/openssh/compress.c". Yes, this may produce less than
    > optimal compression, but still does a very good job while not
    > compromising latency.


    Well, that's all good to hear. Makes sense, too, since at least SSH
    knows explicitly that it's forwarding X11 sessions.

    It just hasn't been my experience. When I'm using a laptop at home, with
    an 802.11n link between me and the host, there is a clear performance
    advantage to using a direct remote TCP connection to the laptop X server,
    compared to forwarding over ssh. I wonder if it's a completely unrelated
    issue, like accessing font information on one side of the link or the
    other? Maybe it varies, depending on the widget set that the application
    is using? I tend to use mostly GNOME things.

    Cheers,

    --
    Andrew

  6. Re: compress non-ssh X11

    In article <6ddj83F21emoU1@mid.individual.net>,
    Andrew Reilly wrote:
    >
    >
    >On Sun, 06 Jul 2008 21:55:23 -0500, Rob Warnock wrote:
    >
    >> Actually, in the specific case of SSH, it *doesn't* [unless you have a
    >> *really* slow CPU at either end]. You really do win, for two main
    >> reasons:
    >>
    >> 1. SSH turns off the Nagle algorithm on its sockets, see routine
    >> "set_nodelay()" in "/usr/src/crypto/openssh/misc.c". This allows SSH
    >> to provide reasonable [well, as good as you're going to get!]
    >> latency, which you need with X11 or when typing/editing with remote
    >> echo, etc. In particular, it *is* disabled for X11 forwarding sockets
    >> as well as "shell" channels, see "channel_post_x11_listener()" in
    >> file "/usr/src/crypto/openssh/channels.c".
    >>
    >> 2. The compression used in SSH carefully does not ever wait for
    >> new input data before compressing what it has and sending it to the
    >> output. See the comments before routine "buffer_compress()" in
    >> "/usr/src/crypto/openssh/compress.c". Yes, this may produce less than
    >> optimal compression, but still does a very good job while not
    >> compromising latency.

    >
    >Well, that's all good to hear. Makes sense, too, since at least SSH
    >knows explicitly that it's forwarding X11 sessions.
    >
    >It just hasn't been my experience. When I'm using a laptop at home, with
    >an 802.11n link between me and the host, there is a clear performance
    >advantage to using a direct remote TCP connection to the laptop X server,
    >compared to forwarding over ssh. I wonder if it's a completely unrelated
    >issue, like accessing font information on one side of the link or the
    >other? Maybe it varies, depending on the widget set that the application
    >is using? I tend to use mostly GNOME things.
    >
    >Cheers,
    >
    >--
    >Andrew


    Have you tried running the remote session on VNC and viewing with tightvnc
    rather than forwarding X itself?

    Ted
    --
    ------
    columbiaclosings.com
    What's not in Columbia anymore..

  7. Re: compress non-ssh X11

    Rob Warnock wrote:
    > Monty Hall wrote:
    > +---------------
    > | I was wondering if I could somehow compress the X11 communication
    > | between machines to hopefully increase reponse time for window repaints
    > | (I'm presuming that what's causing my problem switching windows). I've
    > | tried SSH tunning w/ compression for port 6000, but when I setenv
    > | DISPLAY to server, I get a displayed window but it's not using port
    > | 6000. I can't seem to tell in advance what port X will use for steady
    > | state communication between the machines. TIA.
    > +---------------
    >
    > You're confusing the ports X uses on a local display [usually 6000,
    > but may include 6001, 6002, etc., if you have more than one physical
    > display] with the ports SSH uses on the SSH-server side to tunnel the
    > X connection back to the SSH client machine, which is typically 6010+n
    > [n = 0, 1, ... for each distinct SSH connection with X forwarding].
    >
    > For example, at the moment I'm away from home, and currently have
    > three SSH sessions into my home server/desktop, so there are *four*
    > available X "displays" [counting the physical one on the server/desktop].
    > The active ports on the server/desktop:
    >
    > % sockstat -4 | grep '60[0-9][0-9]'
    > rpw3 xmag 22510 3 tcp4 127.0.0.1:51728 127.0.0.1:6011
    > rpw3 xbiff 16583 3 tcp4 127.0.0.1:50994 127.0.0.1:6010
    > rpw3 sshd 12087 9 tcp4 127.0.0.1:6012 *:*
    > rpw3 sshd 12067 9 tcp4 127.0.0.1:6011 *:*
    > rpw3 sshd 12067 11 tcp4 127.0.0.1:6011 127.0.0.1:51728
    > rpw3 sshd 12041 9 tcp4 127.0.0.1:6010 *:*
    > rpw3 sshd 12041 13 tcp4 127.0.0.1:6010 127.0.0.1:50994
    > root Xorg 876 3 tcp4 *:6000 *:*
    > %
    >
    > And, yes, all three forwarded X connections (ports 6010, -11, -12)
    > on the server/desktop get forwarded to port 6000 [a real display]
    > on my remote client [a FreeBSD laptop] -- oops, actually *not* TCP
    > port 6000 but the Unix-domain socket "/tmp/.X11-unix/X0".
    >
    > Oh, and, yes, the main SSH connection (that the other two SSH sessions
    > are tunneled through -- long story) is using compression, so all of
    > those X connections are automatically getting compressed as well.
    >
    > Does that help?
    >
    >
    > -Rob
    >
    > -----
    > Rob Warnock
    > 627 26th Avenue
    > San Mateo, CA 94403 (650)572-2607
    >

    Somewhat, Xdmx doesn't run under SSH. It looks like remote X may choose
    the first free port, ie:the above's 51728, 50994, for a remote X
    session. My problem now is that I don't know in advance what port X
    will use for it's remote session in order to tunnel a compressed stream.

    For now, I found turning on the backing store possibly helpful -per
    Andrew's note- but dropping the synchronization rate most helpful. It
    no longer has ~5 second response time for application focus switches -
    though its a a little more rubberbandy.

    I'm using Xdmx to get an additional 2 display heads - via connecting to
    a dual headed machine over a 100M bit network - because my laptop lacks
    a decent port replicator. I've already damaged my VGA cable from
    constant plugging and unplugging. Would you know if there is an
    alternative to Xdmx?

    Monty


+ Reply to Thread