What's the substitution of brk system call in POSIX standard? - Unix

This is a discussion on What's the substitution of brk system call in POSIX standard? - Unix ; I read from the brk manual page that brk() and sbrk() "are deliberately excluded from the POSIX.1 standard". Why is that and what's the substitution? Thanks!...

+ Reply to Thread
Results 1 to 20 of 20

Thread: What's the substitution of brk system call in POSIX standard?

  1. What's the substitution of brk system call in POSIX standard?

    I read from the brk manual page that brk() and sbrk() "are deliberately
    excluded from the POSIX.1 standard". Why is that and what's the
    substitution?
    Thanks!

  2. Re: What's the substitution of brk system call in POSIX standard?

    In article , Terry wrote:

    > I read from the brk manual page that brk() and sbrk() "are deliberately
    > excluded from the POSIX.1 standard". Why is that and what's the
    > substitution?
    > Thanks!


    malloc() and mmap(), I guess. brk() depends on a process memory layout
    that's particular to traditional Unix implementations, but may not be
    used in other environments and would be difficult to emulate. Programs
    that need to call these directly are extremely rare, and it's hard to
    imagine them being portable to OSes with different memory layouts.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  3. Re: What's the substitution of brk system call in POSIX standard?

    Barry Margolin wrote:
    > In article , Terry wrote:
    >
    >> I read from the brk manual page that brk() and sbrk() "are deliberately
    >> excluded from the POSIX.1 standard". Why is that and what's the
    >> substitution?
    >> Thanks!

    >
    > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > that's particular to traditional Unix implementations, but may not be
    > used in other environments and would be difficult to emulate. Programs
    > that need to call these directly are extremely rare, and it's hard to
    > imagine them being portable to OSes with different memory layouts.
    >


    malloc() is a library routine and is originally implemented with brk()
    system call. K&R 's book contains a sample implementation of it. Now
    that brk() is removed, if its substitution is mmap(), then, all memory
    allocations have to have something to do with virtual memory since this
    is natural implementation of mmap(). This seems to have totally changed
    the semantic of memory allocation by malloc() or brk() in that the
    allocated memory is not instantly physically ready unless it's accessed.
    Although mlock() can be used to reserve the space in RAM after mmap(),
    it another system call. Then, brk()=mmap()+mlock()?

  4. Re: What's the substitution of brk system call in POSIX standard?

    Terry wrote:
    > Barry Margolin wrote:
    >> In article , Terry wrote:
    >>
    >>> I read from the brk manual page that brk() and sbrk() "are
    >>> deliberately excluded from the POSIX.1 standard". Why is that and
    >>> what's the substitution?
    >>> Thanks!

    >>
    >> malloc() and mmap(), I guess. brk() depends on a process memory
    >> layout that's particular to traditional Unix implementations, but may
    >> not be used in other environments and would be difficult to emulate.
    >> Programs that need to call these directly are extremely rare, and it's
    >> hard to imagine them being portable to OSes with different memory
    >> layouts.
    >>

    >
    > malloc() is a library routine and is originally implemented with brk()
    > system call. K&R 's book contains a sample implementation of it. Now
    > that brk() is removed, if its substitution is mmap(), then, all memory
    > allocations have to have something to do with virtual memory since this
    > is natural implementation of mmap(). This seems to have totally changed
    > the semantic of memory allocation by malloc() or brk() in that the
    > allocated memory is not instantly physically ready unless it's accessed.


    How so? From the program's point of view, it is. The mechanism used to
    make the memory available is an implementation detail.

    All (user space) memory mappings on Unix systems use virtual memory.

    --
    Ian Collins.

  5. Re: What's the substitution of brk system call in POSIX standard?

    In article , Terry wrote:

    > Barry Margolin wrote:
    > > In article , Terry wrote:
    > >
    > >> I read from the brk manual page that brk() and sbrk() "are deliberately
    > >> excluded from the POSIX.1 standard". Why is that and what's the
    > >> substitution?
    > >> Thanks!

    > >
    > > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > > that's particular to traditional Unix implementations, but may not be
    > > used in other environments and would be difficult to emulate. Programs
    > > that need to call these directly are extremely rare, and it's hard to
    > > imagine them being portable to OSes with different memory layouts.
    > >

    >
    > malloc() is a library routine and is originally implemented with brk()
    > system call. K&R 's book contains a sample implementation of it. Now


    But POSIX assumes you have a C library already, so malloc() is
    available. I don't think there's any expectation that everything in the
    C library can be written portably using only POSIX system calls.

    > that brk() is removed, if its substitution is mmap(), then, all memory
    > allocations have to have something to do with virtual memory since this
    > is natural implementation of mmap(). This seems to have totally changed
    > the semantic of memory allocation by malloc() or brk() in that the
    > allocated memory is not instantly physically ready unless it's accessed.
    > Although mlock() can be used to reserve the space in RAM after mmap(),
    > it another system call. Then, brk()=mmap()+mlock()?


    brk() uses virtual memory, too. All it does is change the size of the
    data segment, it doesn't make any changes to physical memory layout.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  6. Re: What's the substitution of brk system call in POSIX standard?

    On Wed, 21 May 2008 00:04:51 -0400 Barry Margolin wrote:
    | In article , Terry wrote:
    |
    |> I read from the brk manual page that brk() and sbrk() "are deliberately
    |> excluded from the POSIX.1 standard". Why is that and what's the
    |> substitution?
    |> Thanks!
    |
    | malloc() and mmap(), I guess. brk() depends on a process memory layout
    | that's particular to traditional Unix implementations, but may not be
    | used in other environments and would be difficult to emulate. Programs
    | that need to call these directly are extremely rare, and it's hard to
    | imagine them being portable to OSes with different memory layouts.

    IOW, if you want to call brk() or sbrk() you need to know some things about
    the particular system that may be different enough to require different code
    on some other system.

    I've used mmap() for low level allocation of a few pages of memory in lieu
    of using brk(). It seems to work fine.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  7. Re: What's the substitution of brk system call in POSIX standard?

    Barry Margolin writes:
    > In article , Terry wrote:
    >> I read from the brk manual page that brk() and sbrk() "are deliberately
    >> excluded from the POSIX.1 standard". Why is that and what's the
    >> substitution?
    >> Thanks!

    >
    > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > that's particular to traditional Unix implementations, but may not be
    > used in other environments and would be difficult to emulate. Programs
    > that need to call these directly are extremely rare,


    All malloc implementations for Linux-based systems I have used so far
    are based on brk/ sbrk. Hence, programs not calling either of the two
    are extremly rare, not the other way round. Wrapping some amount of
    code intended to solve the 'lost' research problem of general purpose
    memory allocation 'good enough' around this does not change it.

    I have quite a bit of code without any calls to malloc, except insofar
    standard library calls do so on their own. When I really need
    'dynamic' memory allocation, ie the ability to repurpose an already
    allocated chunk of memory, I usually use slab-style allocation based
    on mmap'ing pages, otherwise, simple object caching based on a
    shared, mmap'ed heap. What is only temporarily needed can usually be
    allocated on the stack and if a program needed n different objects of
    type x at a certain time, it will need all n of them again in future,
    and there is no reason to 'free' them in between. Where applicable,
    the code uses sbrk directly, eg if a certain amount of memory needs to
    be allocated during initialization and persists during the entire
    lifetime of the process, but is not know at compile time.

  8. Re: What's the substitution of brk system call in POSIX standard?

    In article <87wsloot50.fsf@fever.mssgmbh.com>,
    Rainer Weikusat wrote:

    > Barry Margolin writes:
    > > In article , Terry wrote:
    > >> I read from the brk manual page that brk() and sbrk() "are deliberately
    > >> excluded from the POSIX.1 standard". Why is that and what's the
    > >> substitution?
    > >> Thanks!

    > >
    > > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > > that's particular to traditional Unix implementations, but may not be
    > > used in other environments and would be difficult to emulate. Programs
    > > that need to call these directly are extremely rare,

    >
    > All malloc implementations for Linux-based systems I have used so far
    > are based on brk/ sbrk. Hence, programs not calling either of the two
    > are extremly rare, not the other way round.


    I said "call these DIRECTLY". That qualifier is crucial to the point I
    was making.

    The point is that malloc() usually has to be written in an
    implementation-dependent way, and brk/sbrk are the
    implementation-specific calls that it uses. But portable applications
    can just use malloc() and not have to worry about the low-level system
    calls that are used. POSIX is about portability.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  9. Re: What's the substitution of brk system call in POSIX standard?

    On May 20, 9:35*pm, Terry wrote:

    > malloc() is a library routine and is originally implemented with brk()
    > system call.


    Yes, but it can be implemented any way at all, POSIX doesn't care.

    > K&R 's book contains a sample implementation of it. Now
    > that brk() is removed, if its substitution is mmap(), then, all memory
    > allocations have to have something to do with virtual memory since this
    > is natural implementation of mmap().



    Umm, no, because 'malloc' is still there, and 'malloc' can allocate
    memory any way it wants to. It can use 'brk' because there is no law
    that says that standard library implementation has to be strictly
    POSIX compliant. (And typically it won't be, since the standard
    library is more like the POSIX implementation than it is like
    application code.)

    > This seems to have totally changed
    > the semantic of memory allocation by malloc() or brk() in that the
    > allocated memory is not instantly physically ready unless it's accessed.


    I have no idea what you think this means. The implementation is free
    to allocate physical memory, allocate virtual memory, or manipulate
    tea leaves. So long as a strictly-POSIX-compliant application finds
    that the memory works when 'mmap' or 'malloc' returns, POSIX is happy.

    > Although mlock() can be used to reserve the space in RAM after mmap(),
    > it another system call. Then, brk()=mmap()+mlock()?


    What makes you think 'brk' reserves space in RAM? That would defeat
    the whole point of virtual memory with memory allocated using
    'malloc', if 'malloc' were implemented using 'brk'. With the exception
    of 'mlock', these are all virtual memory operations.

    DS

  10. Re: What's the substitution of brk system call in POSIX standard?

    Barry Margolin writes:
    > In article <87wsloot50.fsf@fever.mssgmbh.com>,
    > Rainer Weikusat wrote:
    >> Barry Margolin writes:
    >> > In article , Terry wrote:
    >> >> I read from the brk manual page that brk() and sbrk() "are deliberately
    >> >> excluded from the POSIX.1 standard". Why is that and what's the
    >> >> substitution?
    >> >> Thanks!
    >> >
    >> > malloc() and mmap(), I guess. brk() depends on a process memory layout
    >> > that's particular to traditional Unix implementations, but may not be
    >> > used in other environments and would be difficult to emulate. Programs
    >> > that need to call these directly are extremely rare,

    >>
    >> All malloc implementations for Linux-based systems I have used so far
    >> are based on brk/ sbrk. Hence, programs not calling either of the two
    >> are extremly rare, not the other way round.

    >
    > I said "call these DIRECTLY". That qualifier is crucial to the point I
    > was making.


    And omitting it was crucial to highlighting the point you are trying
    to hide: All applications which call malloc end up calling sbrk/ brk,
    except for 'unusual' circumstances. This means that using sbrk/ brk is
    actually pretty portable, because every mainstream UNIX(*) and
    UNIX(*)-like system (I know of) has always supported it and will most likely
    always support it.

    > The point is that malloc() usually has to be written in an
    > implementation-dependent way, and brk/sbrk are the
    > implementation-specific calls that it uses. But portable applications
    > can just use malloc() and not have to worry about the low-level system
    > calls that are used.


    General purpose dynamic heap allocation is rarely needed and
    better methods than 'malloc' are available where it is. Even better
    'portably standardized' methods, namely, mmap.

  11. Re: What's the substitution of brk system call in POSIX standard?

    Terry writes:
    >Barry Margolin wrote:
    >> In article , Terry wrote:
    >>
    >>> I read from the brk manual page that brk() and sbrk() "are deliberately
    >>> excluded from the POSIX.1 standard". Why is that and what's the
    >>> substitution?
    >>> Thanks!

    >>
    >> malloc() and mmap(), I guess. brk() depends on a process memory layout
    >> that's particular to traditional Unix implementations, but may not be
    >> used in other environments and would be difficult to emulate. Programs
    >> that need to call these directly are extremely rare, and it's hard to
    >> imagine them being portable to OSes with different memory layouts.
    >>

    >
    >malloc() is a library routine and is originally implemented with brk()


    The idea of posix is to describe the interfaces an application can use,
    not the implementation. An implementation is perfectly free to provide
    sbrk and use it in the malloc implementation. Portable applications are not
    allowed to use sbrk/brk, but must use malloc instead.

    scott

  12. Re: What's the substitution of brk system call in POSIX standard?

    On Wed, 21 May 2008 12:35:22 +0800 Terry wrote:
    | Barry Margolin wrote:
    |> In article , Terry wrote:
    |>
    |>> I read from the brk manual page that brk() and sbrk() "are deliberately
    |>> excluded from the POSIX.1 standard". Why is that and what's the
    |>> substitution?
    |>> Thanks!
    |>
    |> malloc() and mmap(), I guess. brk() depends on a process memory layout
    |> that's particular to traditional Unix implementations, but may not be
    |> used in other environments and would be difficult to emulate. Programs
    |> that need to call these directly are extremely rare, and it's hard to
    |> imagine them being portable to OSes with different memory layouts.
    |>
    |
    | malloc() is a library routine and is originally implemented with brk()
    | system call. K&R 's book contains a sample implementation of it. Now
    | that brk() is removed, if its substitution is mmap(), then, all memory
    | allocations have to have something to do with virtual memory since this
    | is natural implementation of mmap(). This seems to have totally changed
    | the semantic of memory allocation by malloc() or brk() in that the
    | allocated memory is not instantly physically ready unless it's accessed.
    | Although mlock() can be used to reserve the space in RAM after mmap(),
    | it another system call. Then, brk()=mmap()+mlock()?

    A malloc() implementation for some operating system FOO-OS could make use of
    FOO-OS specific system calls which might not involve anything that even looks
    like brk() or sbrk() or mmap() or shmat().

    FYI, I have used shmat() before as a means to get low level memory access
    with certain semantics that I needed. See lines 310 and 311 of this code:
    http://vrb.slashusr.org/source/vrb/s...vrb/vrb_init.c

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  13. Re: What's the substitution of brk system call in POSIX standard?

    On Wed, 21 May 2008 19:23:32 +0200 Rainer Weikusat wrote:
    | Barry Margolin writes:
    |> In article <87wsloot50.fsf@fever.mssgmbh.com>,
    |> Rainer Weikusat wrote:
    |>> Barry Margolin writes:
    |>> > In article , Terry wrote:
    |>> >> I read from the brk manual page that brk() and sbrk() "are deliberately
    |>> >> excluded from the POSIX.1 standard". Why is that and what's the
    |>> >> substitution?
    |>> >> Thanks!
    |>> >
    |>> > malloc() and mmap(), I guess. brk() depends on a process memory layout
    |>> > that's particular to traditional Unix implementations, but may not be
    |>> > used in other environments and would be difficult to emulate. Programs
    |>> > that need to call these directly are extremely rare,
    |>>
    |>> All malloc implementations for Linux-based systems I have used so far
    |>> are based on brk/ sbrk. Hence, programs not calling either of the two
    |>> are extremly rare, not the other way round.
    |>
    |> I said "call these DIRECTLY". That qualifier is crucial to the point I
    |> was making.
    |
    | And omitting it was crucial to highlighting the point you are trying
    | to hide: All applications which call malloc end up calling sbrk/ brk,
    | except for 'unusual' circumstances. This means that using sbrk/ brk is
    | actually pretty portable, because every mainstream UNIX(*) and
    | UNIX(*)-like system (I know of) has always supported it and will most likely
    | always support it.

    No. All applications which call malloc end up calling whatever system calls
    that malloc implementation happens to use. Apparently the default malloc in
    Glibc for Linux does call sbrk/brk (I won't bother verifying this because I
    don't care nor should I need to care). Alternate malloc implementations that
    work on Linux may well use mmap(). I can't even rule out them using shmat().
    And malloc implementations on other POSIX compliant systems can use anything
    the underlying kernel provides. POSIX is about what interfaces are exposed
    to the application program, not about how things like libraries are implemented.

    The real point is that whatever ends up being called by the implementation of
    malloc is not what is considered by the application itself. What the application
    calls is only what it is coded to call. The whole VM process that ends up doing
    some ABI trap into the kernel to do kernel system calls is more than just the
    application; it is the application and linked in support library code as well.
    It is not the application that ends up calling sbrk/brk in the referenced case
    involving Linux; it is the whole of the process, application plus library.

    This is all terminology semantics, of course.

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  14. Re: What's the substitution of brk system call in POSIX standard?

    In comp.unix.programmer, phil-news-nospam@ipal.net wrote:

    > On Wed, 21 May 2008 12:35:22 +0800 Terry wrote:
    > | Barry Margolin wrote:
    > |> In article , Terry wrote:
    > |>
    > |>> I read from the brk manual page that brk() and sbrk() "are
    > |>> deliberately excluded from the POSIX.1 standard". Why is that and
    > |>> what's the substitution?

    [snip]
    > A malloc() implementation for some operating system FOO-OS could make use
    > of FOO-OS specific system calls which might not involve anything that even
    > looks like brk() or sbrk() or mmap() or shmat().


    Like on OS/390 Unix System Services (the IBM MVS OS subsystem that was
    certified as Unix95 by The Open Systems Group), the system-level
    memory-acquisition syscall is "GETMAIN" (and the release syscall
    is "FREEMAIN"). There is no brk() or sbrk() there, but it is still a
    branded Unix system supporting the POSIX/SUS APIs.

    --
    Lew Pitcher

    Master Codewright & JOAT-in-training | Registered Linux User #112576
    http://pitcher.digitalfreehold.ca/ | GPG public key available by request
    ---------- Slackware - Because I know what I'm doing. ------



  15. Re: What's the substitution of brk system call in POSIX standard?

    In article <871w3vpo4r.fsf@fever.mssgmbh.com>,
    Rainer Weikusat wrote:

    > Barry Margolin writes:
    > > In article <87wsloot50.fsf@fever.mssgmbh.com>,
    > > Rainer Weikusat wrote:
    > >> Barry Margolin writes:
    > >> > In article , Terry wrote:
    > >> >> I read from the brk manual page that brk() and sbrk() "are deliberately
    > >> >> excluded from the POSIX.1 standard". Why is that and what's the
    > >> >> substitution?
    > >> >> Thanks!
    > >> >
    > >> > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > >> > that's particular to traditional Unix implementations, but may not be
    > >> > used in other environments and would be difficult to emulate. Programs
    > >> > that need to call these directly are extremely rare,
    > >>
    > >> All malloc implementations for Linux-based systems I have used so far
    > >> are based on brk/ sbrk. Hence, programs not calling either of the two
    > >> are extremly rare, not the other way round.

    > >
    > > I said "call these DIRECTLY". That qualifier is crucial to the point I
    > > was making.

    >
    > And omitting it was crucial to highlighting the point you are trying
    > to hide: All applications which call malloc end up calling sbrk/ brk,
    > except for 'unusual' circumstances. This means that using sbrk/ brk is
    > actually pretty portable, because every mainstream UNIX(*) and
    > UNIX(*)-like system (I know of) has always supported it and will most likely
    > always support it.


    Right. But we're talking about POSIX, not Unix. POSIX is a Unix-like
    API that can be provided on many different OSes. brk() would be pretty
    difficult to shoehorn into an OS that doesn't use Unix-like memory
    layout.

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  16. Re: What's the substitution of brk system call in POSIX standard?

    scott@slp53.sl.home (Scott Lurndal) writes:
    > Terry writes:
    >>Barry Margolin wrote:
    >>> In article , Terry wrote:
    >>>
    >>>> I read from the brk manual page that brk() and sbrk() "are deliberately
    >>>> excluded from the POSIX.1 standard". Why is that and what's the
    >>>> substitution?
    >>>> Thanks!
    >>>
    >>> malloc() and mmap(), I guess. brk() depends on a process memory layout
    >>> that's particular to traditional Unix implementations, but may not be
    >>> used in other environments and would be difficult to emulate. Programs
    >>> that need to call these directly are extremely rare, and it's hard to
    >>> imagine them being portable to OSes with different memory layouts.
    >>>

    >>
    >>malloc() is a library routine and is originally implemented with brk()

    >
    > The idea of posix is to describe the interfaces an application can use,
    > not the implementation. An implementation is perfectly free to provide
    > sbrk and use it in the malloc implementation. Portable
    > applications are not allowed to use sbrk/brk, but must use malloc
    > instead.


    Not necessarily. Provided that it is supported, they could use mmap.
    It would even be easy to emulate (s)brk based on mmap (or malloc, of
    course) sufficiently complete to be useful (by allocating a large
    area, leaving it to the VM subsystem to deal with actually providing
    RAM pages as needed, and implement the mark/ release-style interface
    on top of that).

    An example why this is useful: One of the applications running
    on the appliances I mostly work on is a simple job scheduler. On
    startup, it reads a file with lines as shown below:

    : <"pass to execlp">

    and sets up the necessary data structures to start any jobs defined
    this way every 'interval' seconds. There is no way to tell it to
    reread this file, because that would just complicate the program
    without any real gain. Changes to the job definition file are rare and
    the time needed to parse it is negligient (probably misspelled), so,
    just killing the process (will be restarted by init) after the file
    has been changed is sufficient. Because the number of jobs isn't
    known until after the file has been parsed and neither is the
    total size of the execlp-strings, the memory used to store those
    strings and the associated job management structures must be allocated
    at run time. The C-library used is an old version of uClibc using a
    brk-based power-of-two-freelist-allocator as malloc. When this
    allocator gets a 'small' request, it allocates 4K and divides into
    equally sized blocks with the size being the requested size rounded to
    the next power of two. In order to reduce its external fragmentation,
    I have already modified it to use a minimum block size of 32
    bytes. But it would still tie up a complete page for the lifetime of
    the process to store (presently) less than 128 bytes of string data
    and three management structures. The unmodified allocator would have
    used three pages to accomplish the same. There is no possibility
    for paging anything on this system, which means that once a process
    has accessed a particular page, it will permanently sit on
    these particular 4K (total RAM is 64M). By allocating from a continous
    area, there is no external fragmentation anymore and internal
    fragementation is reduced to the overhead necessary for proper
    alignment (4). 'brk' is the ideal primitive for this type of dynamic
    memory usage.

    BTW, the Linux brk is already an emulation: (Unfortunately) calling
    it does not cause the process data segment to grow but allocates
    (on 2.4) address space in an area just above the static data area.

  17. Re: What's the substitution of brk system call in POSIX standard?

    phil-news-nospam@ipal.net writes:
    > On Wed, 21 May 2008 19:23:32 +0200 Rainer Weikusat wrote:
    > | Barry Margolin writes:
    > |> In article <87wsloot50.fsf@fever.mssgmbh.com>,
    > |> Rainer Weikusat wrote:
    > |>> Barry Margolin writes:
    > |>> > In article , Terry wrote:
    > |>> >> I read from the brk manual page that brk() and sbrk() "are deliberately
    > |>> >> excluded from the POSIX.1 standard". Why is that and what's the
    > |>> >> substitution?
    > |>> >> Thanks!
    > |>> >
    > |>> > malloc() and mmap(), I guess. brk() depends on a process memory layout
    > |>> > that's particular to traditional Unix implementations, but may not be
    > |>> > used in other environments and would be difficult to emulate. Programs
    > |>> > that need to call these directly are extremely rare,
    > |>>
    > |>> All malloc implementations for Linux-based systems I have used so far
    > |>> are based on brk/ sbrk. Hence, programs not calling either of the two
    > |>> are extremly rare, not the other way round.
    > |>
    > |> I said "call these DIRECTLY". That qualifier is crucial to the point I
    > |> was making.
    > |
    > | And omitting it was crucial to highlighting the point you are trying
    > | to hide: All applications which call malloc end up calling sbrk/ brk,
    > | except for 'unusual' circumstances. This means that using sbrk/ brk is
    > | actually pretty portable, because every mainstream UNIX(*) and
    > | UNIX(*)-like system (I know of) has always supported it and will most likely
    > | always support it.
    >
    > No. All applications which call malloc end up calling whatever system calls
    > that malloc implementation happens to use.


    That's not a contradiction to my text above, just a 'vaguefication'.

    > Apparently the default malloc in
    > Glibc for Linux does call sbrk/brk (I won't bother verifying this because I
    > don't care nor should I need to care).


    If you honestly believe that you should not care how memory allocation
    works for an environment you develop for, you should

    a) refrain from discussing it

    b) presumably, use Perl/ Phyton/ Java/ Ruby etc instead
    of any compiled language, because they enable you to avoid
    more work you shouldn't be caring about either

    NB: That's not something I would consider to be an inferior choice. I
    am always happy when I can get away without bothering with such
    details. But I often cannot.

    > Alternate malloc implementations that work on Linux may well use
    > mmap().


    Typically, they use both. But I wasn't writing about hypothetical
    malloc-implementations for Linux-based systems and not even about
    actual malloc-implementations for such systems, but about the subset
    of the latter which I have used so far.

    [...]

    > The real point is that whatever ends up being called by the implementation of
    > malloc is not what is considered by the application itself.
    > What the application calls is only what it is coded to call. The
    > whole VM process that ends up doing some ABI trap into the kernel to
    > do kernel system calls is more than just the application; it is the
    > application and linked in support library code as well. It is not
    > the application that ends up calling sbrk/brk in the referenced case
    > involving Linux; it is the whole of the process, application plus library.
    >
    > This is all terminology semantics, of course.


    I would rather call it 'this is all the result of operating a battery
    of smoke mortars' and suggest a simpler distinction: The application
    is written to use a specific set of interfaces and uses any code which
    is executed on behalf of it at runtime.

  18. Re: What's the substitution of brk system call in POSIX standard?

    On Wed, 21 May 2008 18:43:40 -0400 Lew Pitcher wrote:
    | In comp.unix.programmer, phil-news-nospam@ipal.net wrote:
    |
    |> On Wed, 21 May 2008 12:35:22 +0800 Terry wrote:
    |> | Barry Margolin wrote:
    |> |> In article , Terry wrote:
    |> |>
    |> |>> I read from the brk manual page that brk() and sbrk() "are
    |> |>> deliberately excluded from the POSIX.1 standard". Why is that and
    |> |>> what's the substitution?
    | [snip]
    |> A malloc() implementation for some operating system FOO-OS could make use
    |> of FOO-OS specific system calls which might not involve anything that even
    |> looks like brk() or sbrk() or mmap() or shmat().
    |
    | Like on OS/390 Unix System Services (the IBM MVS OS subsystem that was
    | certified as Unix95 by The Open Systems Group), the system-level
    | memory-acquisition syscall is "GETMAIN" (and the release syscall
    | is "FREEMAIN"). There is no brk() or sbrk() there, but it is still a
    | branded Unix system supporting the POSIX/SUS APIs.

    :-) BTDT (in ASM/370 a lot, w/o the POSIX APIs).

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  19. Re: What's the substitution of brk system call in POSIX standard?

    On Thu, 22 May 2008 17:03:54 +0200 Rainer Weikusat wrote:

    | If you honestly believe that you should not care how memory allocation
    | works for an environment you develop for, you should
    |
    | a) refrain from discussing it
    |
    | b) presumably, use Perl/ Phyton/ Java/ Ruby etc instead
    | of any compiled language, because they enable you to avoid
    | more work you shouldn't be caring about either
    |
    | NB: That's not something I would consider to be an inferior choice. I
    | am always happy when I can get away without bothering with such
    | details. But I often cannot.

    I do care how it is done. But when considering the standard malloc()
    interface, I don't need to care when what it does serves my needs.
    My "do not care" assertion is in the context of making use of malloc()
    which I do often in my programming ... along with alloc() in cases where
    that non-POSIX interface is more useful (needing a variable chunk of
    memory to last only for as long as a function is in control and the
    performance of just grabbing it from the stack and not tracking its
    release through many possible function return paths).


    | I would rather call it 'this is all the result of operating a battery
    | of smoke mortars' and suggest a simpler distinction: The application
    | is written to use a specific set of interfaces and uses any code which
    | is executed on behalf of it at runtime.

    The way I look at it is the application that calls malloc() is only calling
    malloc() and nothing more. If sbrk/brk/mmap/shmat/GETMAIN/DMSFREE/foo get
    called by some implementation of malloc(), I do not think of it as that
    application calling that system function. The process would (but that is
    a mix of application and linked library).

    --
    |WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
    | by the abuse department, bellsouth.net is blocked. If you post to |
    | Usenet from these places, find another Usenet provider ASAP. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  20. Re: What's the substitution of brk system call in POSIX standard?

    phil-news-nospam@ipal.net writes:
    > On Thu, 22 May 2008 17:03:54 +0200 Rainer Weikusat wrote:
    > | If you honestly believe that you should not care how memory allocation
    > | works for an environment you develop for, you should
    > |
    > | a) refrain from discussing it
    > |
    > | b) presumably, use Perl/ Phyton/ Java/ Ruby etc instead
    > | of any compiled language, because they enable you to avoid
    > | more work you shouldn't be caring about either
    > |
    > | NB: That's not something I would consider to be an inferior choice. I
    > | am always happy when I can get away without bothering with such
    > | details. But I often cannot.
    >
    > I do care how it is done. But when considering the standard malloc()
    > interface, I don't need to care when what it does serves my needs.


    IOW, you do not care how it works, because of the a priori assumption
    that however whatever malloc is implemented, it will 'serve your
    needs'. But you already wrote that and my answer is still 'Fine. Be
    happy. Consider using automatic memory managment.'

    [...]

    > | I would rather call it 'this is all the result of operating a battery
    > | of smoke mortars' and suggest a simpler distinction: The application
    > | is written to use a specific set of interfaces and uses any code which
    > | is executed on behalf of it at runtime.
    >
    > The way I look at it is the application that calls malloc() is only calling
    > malloc() and nothing more. If sbrk/brk/mmap/shmat/GETMAIN/DMSFREE/foo get
    > called by some implementation of malloc(), I do not think of it as that
    > application calling that system function.


    And you already wrote that, too, and my answer would again be the same.

+ Reply to Thread