Re: [9fans] mmap and shared libraries - Plan9

This is a discussion on Re: [9fans] mmap and shared libraries - Plan9 ; > Sadly, the picture changes at run time: clock on plan 9 is 128k in > memory, xclock is 4.2M RSS and 10M VSZ. > Sic transit gloria .so. Of course, then we hear that "well, all that > is ...

+ Reply to Thread
Results 1 to 2 of 2

Thread: Re: [9fans] mmap and shared libraries

  1. Re: [9fans] mmap and shared libraries

    > Sadly, the picture changes at run time: clock on plan 9 is 128k in
    > memory, xclock is 4.2M RSS and 10M VSZ.
    > Sic transit gloria .so. Of course, then we hear that "well, all that
    > is shared". Hmm. Prove it.


    I know one thing: shared libraries are employed on every major operating
    system I have ever heard of. Can all these people be wrong? I don't think
    so.

    > FWIW, the whole issue of goodness/badness of shared libraries has
    > never been systematically measured as far as I know -- in terms of
    > performance cost, overhead, throughput, the usual suspects -- or at
    > least
    > I've never seen it done. It would be a lot of work to do it correctly.
    > Might be interesting.


    Title: Dynamic Linking of Software Components
    Author: Michael Franz
    YoP: 1997

    > ABSTRACT
    >
    > The concepts underlying dynamic linking reached maturity through modular
    > operating systems and are being reintroduced in some of the most popular
    > workstation and PC operating systems. [...]




    The abstract says it is an overall survey of dynamic linking. I don't have
    access to the full text and even if I had I wouldn't understand a word of
    it. You are the CS/CE person. Be the judge, too.

    > So the compressed version is only 50% larger than acme. The point being?


    That there's a lot of redundancy in the executable principally imposed by
    the binary format.

    In case of the FreeBSD vim executable:

    $ readelf -e /usr/local/bin/vim

    Reveals 171,365 bytes of redundant data only in .data, .rodata, and .bss
    sections.

    $ objdump -s /usr/local/bin/vim

    Will display tons of zeroes in all other sections except .text. Even RLE
    could significantly reduce the image size.

    > note that filesize is < memory size. I'll let you figure out how it works.


    I'm not that intelligent but didn't I say it was _initialized_ data that
    had to be written to executable image?

    Example from a simple program I wrote to be assembled by Flat Assembler for
    Win32:

    --------------------------------------------------------------------------------
    section '.data' data readable writeable
    ..
    ..
    ..
    buffer_ptr dd 00h <----- pointer reserved, and initialized
    ..
    ..
    ..
    buffer db 00FFFFh dup (?) <----- buffer reserved, not initialized
    ..
    ..
    ..
    --------------------------------------------------------------------------------

    File size can be less than memory size when you have data reserved but not
    initialized. That happens in many cases, e.g. when you reserve a buffer.

    As for ELF:

    > Defining data elements in the bss section is somewhat different from
    > defining them in the data section. Instead of declaring specific data
    > types, you just declare raw segments of memory that are reserved for
    > whatever purpose you need them for.
    >
    > [...]
    >
    > One benefit to declaring data in the bss section is that the data is not
    > included in the executable program. When data is defined in the data
    > section, it must be included in the executable program, since it must be
    > initialized with a specific value. Because the data areas declared in the
    > bss section are not initialized with program data, the memory areas are
    > reserved at runtime, and do not have to be included in the final program.


    -- Richard Blum, Professional Assembly Language, in the context of using
    (g)as and ELF format

    I don't want to sound authoritative but when something is right it is right
    and there's nothing you or I can do about it.

    > Does the redundant litter cross the network when I have it mounted via
    > a share and execute the program or not? That was the original
    > discussion.


    I don't know. The size of Emacs executable has as much connection to that
    question as I do, but it was _you_ who made the bad Emacs joke.

    > Ah! Now we're into feature comparisons! I'm game. How did you get to 1
    > in 5 and not 1 in 4.8, or 1 in 6?


    You know very well that the ratio is just an inaccurate measure as the
    context suggests. 1 to 5 means like your pinkie compared to your entire
    hand. Trying to "context-switch" and move from an inaccurate measure to an
    exact number is just a diversionary technique you have employed for
    countering an easily demonstrable fact. Namely that Acme is a minimal text
    editor claiming to be an IDE while Emacs is a behemoth with more features
    than you could count through a day, and the following night--not that I
    believe Emacs is any better than Acme.

    --On Wednesday, November 05, 2008 2:13 PM -0800 ron minnich
    wrote:

    > On Wed, Nov 5, 2008 at 12:54 PM, Eris Discordia
    > wrote:
    >>> yes, I agree, I was being terribly unfair to plan 9. Acme on plan 9 is
    >>> about 1/2 M. Vim on DOS is 3x larger? impressive.

    >>
    >> My intent was, of course, to show your comparison is baseless. It seems
    >> you still haven't realized that. You think Plan 9 is great?

    >
    > Actually, to keep it simple, I was trying to make a simple point, with
    > a bit of humor intended, and not actually even directed at you,: that
    > code built with shared libraries is not
    > necessarily, or always, smaller than code built otherwise, for
    > programs of similar capability. I don't have an emacs
    > linked statically, however, so I took the nearest materials at hand.
    > It depends on a lot of circumstances.
    >
    > There is the example of xclock which is small with X11 .so, and quite
    > large otherwise. And there, for the case of a "clock", shared
    > libraries clearly reduce disk footprint:
    > xclock binary is about 1/10 the size of plan 9 clock (19K vs. 158K)..
    >
    > Sadly, the picture changes at run time: clock on plan 9 is 128k in
    > memory, xclock is 4.2M RSS and 10M VSZ.
    > Sic transit gloria .so. Of course, then we hear that "well, all that
    > is shared". Hmm. Prove it.
    >
    > FWIW, the whole issue of goodness/badness of shared libraries has
    > never been systematically measured as far as I know -- in terms of
    > performance cost, overhead, throughput, the usual suspects -- or at
    > least
    > I've never seen it done. It would be a lot of work to do it correctly.
    > Might be interesting.
    >
    >> The GUI version of Vim on Windows (gvim.exe) compresses to 734,713 bytes

    >
    > So the compressed version is only 50% larger than acme. The point being?
    >
    >> because the bigger part of the uncompressed 1,585,152 bytes is redundant
    >> filler required by the binary format specification.

    >
    > Does the redundant litter cross the network when I have it mounted via
    > a share and execute the program or not? That was the original
    > discussion.
    >
    >> The same happens for the
    >> vim executable on FreeBSD (1,221,212 bzip2'ed to 616,236). The Windows PE
    >> binary format and ELF both require the executable image to contain all
    >> initialized but essentially redundant (i.e. zeroed) parts of the data
    >> segement. Don't pretend you didn't know that.

    >
    > Wrong for ELF.
    >
    > Simple example from /bin/cat:
    > Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
    > LOAD 0x0049e8 0x100149e8 0x100149e8 0x00284 0x003cc RW
    > 0x10000
    >
    > note that filesize is < memory size. I'll let you figure out how it works.
    >
    > This is the problem with your posts. You sound very authoritative, and
    > I'm sure you figure you know all these bits,
    > but you're wrong so often. It's confusing. It would be unfortunate
    > were people to believe you are as right as you think you are.
    >
    >>
    >> Also, Acme in p9p or on Plan 9 performs less than 1 in 5 of the functions
    >> vi/vim does. That ratio is even smaller when comparing Acme to Emacs. So
    >> you have been unfair. No kidding.
    >>

    >
    > Ah! Now we're into feature comparisons! I'm game. How did you get to 1
    > in 5 and not 1 in 4.8, or 1 in 6?
    > Of course, you have to take into account that acme's extensions live
    > in /bin or your number is bogus.
    >
    > ron
    >



  2. Re: [9fans] mmap and shared libraries

    > File size can be less than memory size when you have data reserved but not
    > initialized. That happens in many cases, e.g. when you reserve a buffer.


    >> One benefit to declaring data in the bss section is that the data is not
    >> included in the executable program. When data is defined in the data
    >> section, it must be included in the executable program, since it must be
    >> initialized with a specific value. Because the data areas declared in the
    >> bss section are not initialized with program data, the memory areas are
    >> reserved at runtime, and do not have to be included in the final program.


    i don't think uninitialized means what you think it does. for the
    benefit of those who might be confused by this largely purposeful
    misdirection, the bss is initialized to zero. the c standard
    specifies that external declarations without explicit initialization
    are initialized to zero. this is intialized data that doesn't appear
    in the executable.

    because i'm pedantic, i'll point out that zero has always been a
    specific value.

    - erik



+ Reply to Thread