Re: [9fans] mmap and shared libraries - Plan9

This is a discussion on Re: [9fans] mmap and shared libraries - Plan9 ; On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia wrote: >> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is >> about 1/2 M. Vim on DOS is 3x larger? impressive. > ...

+ Reply to Thread
Results 1 to 4 of 4

Thread: Re: [9fans] mmap and shared libraries

  1. Re: [9fans] mmap and shared libraries

    On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
    wrote:
    >> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
    >> about 1/2 M. Vim on DOS is 3x larger? impressive.

    >
    > My intent was, of course, to show your comparison is baseless. It seems you
    > still haven't realized that. You think Plan 9 is great?


    Actually, to keep it simple, I was trying to make a simple point, with
    a bit of humor intended, and not actually even directed at you,: that
    code built with shared libraries is not
    necessarily, or always, smaller than code built otherwise, for
    programs of similar capability. I don't have an emacs
    linked statically, however, so I took the nearest materials at hand.
    It depends on a lot of circumstances.

    There is the example of xclock which is small with X11 .so, and quite
    large otherwise. And there, for the case of a "clock", shared
    libraries clearly reduce disk footprint:
    xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..

    Sadly, the picture changes at run time: clock on plan 9 is 128k in
    memory, xclock is 4.2M RSS and 10M VSZ.
    Sic transit gloria .so. Of course, then we hear that "well, all that
    is shared". Hmm. Prove it.

    FWIW, the whole issue of goodness/badness of shared libraries has
    never been systematically measured as far as I know -- in terms of
    performance cost, overhead, throughput, the usual suspects -- or at
    least
    I've never seen it done. It would be a lot of work to do it correctly.
    Might be interesting.

    > The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes


    So the compressed version is only 50% larger than acme. The point being?

    > because the bigger part of the uncompressed 1,585,152 bytes is redundant
    > filler required by the binary format specification.


    Does the redundant litter cross the network when I have it mounted via
    a share and execute the program or not? That was the original
    discussion.

    > The same happens for the
    > vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
    > binary format and ELF both require the executable image to contain all
    > initialized but essentially redundant (i.e. zeroed) parts of the data
    > segement. Don't pretend you didn't know that.


    Wrong for ELF.

    Simple example from /bin/cat:
    Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
    LOAD 0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW 0x10000

    note that filesize is < memory size. I'll let you figure out how it works.

    This is the problem with your posts. You sound very authoritative, and
    I'm sure you figure you know all these bits,
    but you're wrong so often. It's confusing. It would be unfortunate
    were people to believe you are as right as you think you are.

    >
    > Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
    > vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So you
    > have been unfair. No kidding.
    >


    Ah! Now we're into feature comparisons! I'm game. How did you get to 1
    in 5 and not 1 in 4.8, or 1 in 6?
    Of course, you have to take into account that acme's extensions live
    in /bin or your number is bogus.

    ron


  2. Re: [9fans] mmap and shared libraries

    Please forgive the repeated messages. It didn't appear in my mail client's
    Sent view after I hit send. Thought it might have been lost so I re-wrote
    it.


  3. Re: [9fans] mmap and shared libraries

    > Sadly, the picture changes at run time: clock on plan 9 is 128k in
    > memory, xclock is 4.2M RSS and 10M VSZ.
    > Sic transit gloria .so. Of course, then we hear that "well, all that
    > is shared". Hmm. Prove it.


    I know one thing: every major operating system I have ever heard of
    leverages shared libraries. Can all those people be wrong? I don't think
    so.

    > FWIW, the whole issue of goodness/badness of shared libraries has
    > never been systematically measured as far as I know -- in terms of
    > performance cost, overhead, throughput, the usual suspects -- or at
    > least
    > I've never seen it done. It would be a lot of work to do it correctly.
    > Might be interesting.


    Title: Dynamic Linking of Software Components
    Author: Michael Franz
    YoP: 1997

    > ABSTRACT
    >
    > The concepts underlying dynamic linking reached maturity through modular
    > operating systems and are being reintroduced in some of the most popular
    > workstation and PC operating systems.




    The abstract says this is an overall survey of dynamic linking. I can't
    access the full text and I wouldn't understand it even if I could. You are
    the CS/CE person. Be the judge, too.

    > So the compressed version is only 50% larger than acme. The point being?


    That there's a lot of redundancy in the executable image principally
    imposed by the binary format.

    $ readelf -e /usr/local/bin/vim

    Reveals over 170 KB of data only in .data, .rodata, and .bss sections.

    $ objdump -s /usr/local/bin/vim

    Displays tons of zeroes in all sections except .text. Even RLE could have
    considerably reduced the image size.

    > Does the redundant litter cross the network when I have it mounted via
    > a share and execute the program or not? That was the original
    > discussion.


    I don't know. The size of Emacs executable has as much connection to that
    question as I do, but it was _you_ who made the bad Emacs joke.

    > Wrong for ELF.


    About ELF:

    > Defining data elements in the bss section is somewhat different from
    > defining them in the data section. Instead of declaring specific data
    > types, you just declare raw segments of memory that are reserved for
    > whatever purpose you need them for.
    >
    > [...]
    >
    > One benefit to declaring data in the bss section is that the data is not
    > included in the executable program. When data is defined in the data
    > section, it must be included in the executable program, since it must be
    > initialized with a specific value. Because the data areas declared in the
    > bss section are not initialized with program data, the memory areas are
    > reserved at runtime, and do not have to be included in the final program.


    -- Professional Assembly Programming, Richard Blum, in the context of using
    (g)as and ELF

    Exactly what I said. I don't want to sound authoritative but when something
    is right it is right and there's nothing you or I can do about it.

    Executable image size can be less than memory size because there can be
    _uninitialized_ data reserved in data segments, e.g. when you reserve a
    buffer but don't care what it initially contains. However, _initialized_
    data always and invariably gets written to disk--think of a whole 64K of
    zeroes.

    Example from a simple program for Flat Assembler with Win32 target:

    section '.data' data readable writeable

    buffer_ptr dd 00h <---- a pointer reserved, and initialized

    buffer db 00FFFFh dup (?) <---- a buffer reserved, but not initialized


    > Ah! Now we're into feature comparisons! I'm game. How did you get to 1
    > in 5 and not 1 in 4.8, or 1 in 6?


    You know very well that the ratio is an inaccurate measure as the context
    suggests. 1 to 5 is like your pinkie to all your fingers. Trying to
    "context-switch" and turn a guesstimate into an exact number is only a
    diversionary technique you have employed to evade an easily demonstrable
    fact. Namely that Acme is a minimal text editor claiming to be an IDE while
    Emacs is a behemoth with more features than you could count in a day, and
    the following night--not that I believe Emacs is any better than Acme. Vim
    leans more towards Emacs.

    --On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich
    wrote:

    > On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
    > wrote:
    >>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
    >>> about 1/2 M. Vim on DOS is 3x larger? impressive.

    >>
    >> My intent was, of course, to show your comparison is baseless. It seems
    >> you still haven't realized that. You think Plan 9 is great?

    >
    > Actually, to keep it simple, I was trying to make a simple point, with
    > a bit of humor intended, and not actually even directed at you,: that
    > code built with shared libraries is not
    > necessarily, or always, smaller than code built otherwise, for
    > programs of similar capability. I don't have an emacs
    > linked statically, however, so I took the nearest materials at hand.
    > It depends on a lot of circumstances.
    >
    > There is the example of xclock which is small with X11 .so, and quite
    > large otherwise. And there, for the case of a "clock", shared
    > libraries clearly reduce disk footprint:
    > xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..
    >
    > Sadly, the picture changes at run time: clock on plan 9 is 128k in
    > memory, xclock is 4.2M RSS and 10M VSZ.
    > Sic transit gloria .so. Of course, then we hear that "well, all that
    > is shared". Hmm. Prove it.
    >
    > FWIW, the whole issue of goodness/badness of shared libraries has
    > never been systematically measured as far as I know -- in terms of
    > performance cost, overhead, throughput, the usual suspects -- or at
    > least
    > I've never seen it done. It would be a lot of work to do it correctly.
    > Might be interesting.
    >
    >> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes

    >
    > So the compressed version is only 50% larger than acme. The point being?
    >
    >> because the bigger part of the uncompressed 1,585,152 bytes is redundant
    >> filler required by the binary format specification.

    >
    > Does the redundant litter cross the network when I have it mounted via
    > a share and execute the program or not? That was the original
    > discussion.
    >
    >> The same happens for the
    >> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
    >> binary format and ELF both require the executable image to contain all
    >> initialized but essentially redundant (i.e. zeroed) parts of the data
    >> segement. Don't pretend you didn't know that.

    >
    > Wrong for ELF.
    >
    > Simple example from /bin/cat:
    > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
    > LOAD 0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW
    > 0x10000
    >
    > note that filesize is < memory size. I'll let you figure out how it works.
    >
    > This is the problem with your posts. You sound very authoritative, and
    > I'm sure you figure you know all these bits,
    > but you're wrong so often. It's confusing. It would be unfortunate
    > were people to believe you are as right as you think you are.
    >
    >>
    >> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
    >> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
    >> you have been unfair. No kidding.
    >>

    >
    > Ah! Now we're into feature comparisons! I'm game. How did you get to 1
    > in 5 and not 1 in 4.8, or 1 in 6?
    > Of course, you have to take into account that acme's extensions live
    > in /bin or your number is bogus.
    >
    > ron
    >



  4. Re: [9fans] mmap and shared libraries

    On Nov 5, 2008, at 2:13 PM, ron minnich wrote:
    > On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
    > wrote:
    >>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan
    >>> 9 is
    >>> about 1/2 M. Vim on DOS is 3x larger? impressive.

    >>
    >> My intent was, of course, to show your comparison is baseless. It
    >> seems you
    >> still haven't realized that. You think Plan 9 is great?

    >
    > Sadly, the picture changes at run time: clock on plan 9 is 128k in
    > memory, xclock is 4.2M RSS and 10M VSZ.
    > Sic transit gloria .so. Of course, then we hear that "well, all that
    > is shared". Hmm. Prove it.


    Exactly! On Linux with ld.so accepting relocations (think non-PIC code)
    in .text the appearance of some .so's being shared is highly
    deceiving. Solaris is better in that regard, but the pressure from
    userland community expecting their project to "just work" is mounting.

    Thanks,
    Roman.


+ Reply to Thread