sharing memory map between processes (same parent) - Unix

This is a discussion on sharing memory map between processes (same parent) - Unix ; On Mon, 28 Apr 2008 23:08:43 -0600 Chris Friesen wrote: | phil-news-nospam@ipal.net wrote: |> On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris wrote: | |> | You want to avoid having the discussion go off track? Then describe ...

+ Reply to Thread
Page 2 of 3 FirstFirst 1 2 3 LastLast
Results 21 to 40 of 43

Thread: sharing memory map between processes (same parent)

  1. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 23:08:43 -0600 Chris Friesen wrote:
    | phil-news-nospam@ipal.net wrote:
    |> On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris wrote:
    |
    |> | You want to avoid having the discussion go off track? Then describe the
    |> | problem as fully as possible, the issues that restrict what you can do,
    |> | what methods you've rejected and why you've rejected them. Otherwise
    |> | people WILL waste time giving you solutions you've already rejected,
    |> | because you've given them no reason to believe it won't work.
    |
    |> That would be too long of a post.
    |
    | It would be shorter than what you've now written in this long thread,
    | and everyone's time would have been saved.

    How do you come to that conclusion?


    | Seriously, the above advice is excellent.

    I'm not yet convinced.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  2. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 23:08:43 -0600 Chris Friesen wrote:
    | phil-news-nospam@ipal.net wrote:
    |> On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris wrote:
    |
    |> | You want to avoid having the discussion go off track? Then describe the
    |> | problem as fully as possible, the issues that restrict what you can do,
    |> | what methods you've rejected and why you've rejected them. Otherwise
    |> | people WILL waste time giving you solutions you've already rejected,
    |> | because you've given them no reason to believe it won't work.
    |
    |> That would be too long of a post.
    |
    | It would be shorter than what you've now written in this long thread,
    | and everyone's time would have been saved.
    |
    | Seriously, the above advice is excellent.

    Oh, and BTW, one thing I have seen happen when people do post "what they want
    to accomplish", as enticing as that is to many readers, is the other extreme
    of people just saying things like "do your own research" and "why should we
    design your program for you".

    You're damned if you do; you're damned if you don't.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  3. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:
    > On Mon, 28 Apr 2008 23:08:54 -0600 Chris Friesen wrote:
    > | phil-news-nospam@ipal.net wrote:
    > |> On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:
    > |
    > |> | Setting up mappings is relatively expensive, so it should only be done
    > |> | in the process that is going to use the mapping.
    > |>
    > |> I wanted to overlap setting up the mapping with the work already taking place
    > |> in process B. Much of the work of setting up a mapping (like reading block
    > |> pointer records for the file) would not need to be repeated in such a transfer,
    > |> so the time delay in process B to merely accept the mapping from process A
    > |> means process B can get on to other things more quickly.
    > |
    > | A large part of the cost of setting up a mapping is in the fact that the
    > | tlb needs to be flushed (to force the cpu to see the new mapping). This
    > | would need to be done in the second process no matter what since it is
    > | in a separate memory map. This overhead (as well as the overhead of
    > | transferring file descriptors via unix sockets) could be minimized by
    > | simply using threads.
    >
    > But isn't this a cost associated with context switching?
    >
    > Yes, I know threads is an option, and is a good one with respect to this
    > aspect. I have other reasons to not use threads. One is that at the time
    > my program will run, no libraries are available.


    How can you fork and map files without libraries?

    > Conceivably, I could still do "threads" via a clone() call.


    Aren't threads part of libC on your system?

    --
    Ian Collins.

  4. Re: sharing memory map between processes (same parent)

    > Aren't threads part of libC on your system?
    >
    > --
    > Ian Collins.


    he's not talking about any libc interface, just kernel syscalls like
    clone and mmap() etc.

    -- paulo

  5. Re: sharing memory map between processes (same parent)

    On 2008-04-29, Wayne C. Morris wrote:
    > In article , phil-news-nospam@ipal.net
    > wrote:
    >
    >> I do purposely try to avoid answering questions that go off track of
    >> what I an seeking answers for. I used to not do that, and that would
    >> almost always result in a pointless drawn out thread with someone
    >> else trying to suggest ways to do things that would not really work
    >> because of many issues in the program, that I didn't post in the
    >> first place because they were not relevant to

    >
    > If those issues prevent you from doing it the usual way, they ARE
    > relevant to the discussion. They're part of the problem you're trying
    > to solve.
    >
    > You want to avoid having the discussion go off track? Then describe the
    > problem as fully as possible, the issues that restrict what you can do,
    > what methods you've rejected and why you've rejected them.




    > Otherwise people WILL waste time giving you solutions you've already
    > rejected, because you've given them no reason to believe it won't work.


    In other words, it will "... result in a pointless drawn out thread with
    someone else trying to suggest ways to do things that would not really
    work because ..." the OP didn't describe the context properly.

    --


  6. Re: sharing memory map between processes (same parent)

    On Wed, 30 Apr 2008 08:57:14 +1200 Ian Collins wrote:
    | phil-news-nospam@ipal.net wrote:
    |> On Mon, 28 Apr 2008 23:08:54 -0600 Chris Friesen wrote:
    |> | phil-news-nospam@ipal.net wrote:
    |> |> On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:
    |> |
    |> |> | Setting up mappings is relatively expensive, so it should only be done
    |> |> | in the process that is going to use the mapping.
    |> |>
    |> |> I wanted to overlap setting up the mapping with the work already taking place
    |> |> in process B. Much of the work of setting up a mapping (like reading block
    |> |> pointer records for the file) would not need to be repeated in such a transfer,
    |> |> so the time delay in process B to merely accept the mapping from process A
    |> |> means process B can get on to other things more quickly.
    |> |
    |> | A large part of the cost of setting up a mapping is in the fact that the
    |> | tlb needs to be flushed (to force the cpu to see the new mapping). This
    |> | would need to be done in the second process no matter what since it is
    |> | in a separate memory map. This overhead (as well as the overhead of
    |> | transferring file descriptors via unix sockets) could be minimized by
    |> | simply using threads.
    |>
    |> But isn't this a cost associated with context switching?
    |>
    |> Yes, I know threads is an option, and is a good one with respect to this
    |> aspect. I have other reasons to not use threads. One is that at the time
    |> my program will run, no libraries are available.
    |
    | How can you fork and map files without libraries?

    With syscalls.


    |> Conceivably, I could still do "threads" via a clone() call.
    |
    | Aren't threads part of libC on your system?

    Yes.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  7. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:
    > On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:

    [snip]
    > | Setting up mappings is relatively expensive, so it should only be done
    > | in the process that is going to use the mapping.
    >
    > I wanted to overlap setting up the mapping with the work already taking place
    > in process B. Much of the work of setting up a mapping (like reading block
    > pointer records for the file) would not need to be repeated in such a transfer,


    I doubt mmap() triggers I/O related to the file mapped beyond the inode
    (for length, and atime update). This seems the best way to efficiently
    support a large mapping that is sparsely accessed - ie the kernel
    shouldn't bother to figure out which bits of backing store to read/write
    (let alone actually read any content) in advance because it might never
    happen.

    POSIX_MADV_WILLNEED is the only thing I can see that has any chance of
    causing I/O other than accessing the pages, and Google suggests that for
    Linux it will cause already-cached pages to be mapped (effectively
    trigger all the minor faults) and decrease the likelihood of the pages
    being reclaimed, but no I/O. (This also fits in with my guess of the
    implementation of mmap() - that it does the least possible work before
    returning - not even establishing which, if any, pages are already in
    memory.)

    Alex

  8. Re: sharing memory map between processes (same parent)

    On Apr 28, 9:56 pm, phil-news-nos...@ipal.net wrote:

    > If they misunderstood what I ask, and how narrow it is, I supposed so.
    > "How do I do X?" ... "To do X, there is method Y and also method Z".
    > Why would someone wanting to do something other than X try to apply
    > method Y or method Z?


    X might be such that the vast majority of people who think they want
    to do X are trying to solve problems for which X is not the best
    solution, but where X seems like the right solution. This is very
    common. A classic example of this kind of question, "how can my
    program tell how much free memory my Linux system has?"

    X might be the kind of thing that causes people to think that it's a
    good idea just from the suggestion or that causes people to think it's
    more generally applicable than it really is. People who never though
    to ever do X might see the question, an answer as you suggest, and
    conclude that perhaps X is the kind of thing they want to do. A
    classic example of this kind of question, "how can I disable forced
    unit access and make my Windows I/O faster?"

    And, of course, the last reason -- the Socratic method. Often people
    don't really have a good understanding of their question itself. For
    example, an answer to "how can my program tell how much free memory my
    Linux system has?" might be "do you mean physical or virtual memory?
    And what about memory pages that contain clean copies of file data
    that can be discarded instantaneously?"

    DS

  9. Re: sharing memory map between processes (same parent)

    On Apr 28, 10:00 pm, phil-news-nos...@ipal.net wrote:

    > I wanted to overlap setting up the mapping with the work already taking place
    > in process B. Much of the work of setting up a mapping (like reading block
    > pointer records for the file) would not need to be repeated in such a transfer,
    > so the time delay in process B to merely accept the mapping from process A
    > means process B can get on to other things more quickly. On a slow machine,
    > it might not be any advantage. On a fast machine, maybe there would be.


    I'm betting you didn't measure, so this whole thing is premature
    optimization. In fact, it's premature pessimization since the cost of
    making the mapping is still paid, but added onto that is the cost of
    giving and receiving the mapping.

    Transferring the mapping is going to be wildly expensive because it
    requires synchronization. I haven't measured either, but my gut tells
    me this would be a pessimization. It *might* make sense to have the
    other process warm the cache by 'stat'ing the file and maybe even
    reading it, but I doubt even that.

    DS

  10. Re: sharing memory map between processes (same parent)

    On Wed, 30 Apr 2008 01:10:49 -0700 (PDT) David Schwartz wrote:
    | On Apr 28, 10:00 pm, phil-news-nos...@ipal.net wrote:
    |
    |> I wanted to overlap setting up the mapping with the work already taking place
    |> in process B. Much of the work of setting up a mapping (like reading block
    |> pointer records for the file) would not need to be repeated in such a transfer,
    |> so the time delay in process B to merely accept the mapping from process A
    |> means process B can get on to other things more quickly. On a slow machine,
    |> it might not be any advantage. On a fast machine, maybe there would be.
    |
    | I'm betting you didn't measure, so this whole thing is premature
    | optimization. In fact, it's premature pessimization since the cost of
    | making the mapping is still paid, but added onto that is the cost of
    | giving and receiving the mapping.

    I plan to measure. But I plan to do that when I have enough alternate ideas
    to measure against (if any).


    | Transferring the mapping is going to be wildly expensive because it
    | requires synchronization. I haven't measured either, but my gut tells
    | me this would be a pessimization. It *might* make sense to have the
    | other process warm the cache by 'stat'ing the file and maybe even
    | reading it, but I doubt even that.

    Well, I have concluded that the feature does not exist. I don't know that it
    should exist. OTOH, I've seen many features that do exist which I cannot
    explain why they would exist (but someone else might have a good explanation).
    I really don't understand why the discussion is going down the path of trying
    to say why such a feature is not practical to implement. Has someone asked
    to have it implemented?

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  11. Re: sharing memory map between processes (same parent)

    On Apr 30, 11:58 am, phil-news-nos...@ipal.net wrote:

    > I plan to measure. But I plan to do that when I have enough alternate ideas
    > to measure against (if any).


    Yeah, it's kind of hard to measure something without something to
    measure it against. Again, I think in practice the best you can do is
    probably nothing at all. It may help slightly to have another process
    'warm the cache' by getting the inode into memory. Sadly, many UNIXes
    don't let you warm the cache properly -- doing an
    'open'/'fadvise'/'close' may well *not* keep the cache warm for a
    subsequent 'mmap'.

    > Well, I have concluded that the feature does not exist. I don't know that it
    > should exist. OTOH, I've seen many features that do exist which I cannot
    > explain why they would exist (but someone else might have a good explanation).


    True that. Some of them have bizarre but rational explanations, some
    of them don't.

    > I really don't understand why the discussion is going down the path of trying
    > to say why such a feature is not practical to implement. Has someone asked
    > to have it implemented?


    The point is that if it's not practical to implement, you wouldn't
    want to use it even if it existed because it wouldn't optimize
    anything. Synchronization is expensive and rarely optimizes anything.
    The main cost of threads is the expense of synchronizing changes to
    the shared memory mappings.

    DS

  12. Re: sharing memory map between processes (same parent)

    On Wed, 30 Apr 2008 16:17:39 -0700 (PDT) David Schwartz wrote:
    | On Apr 30, 11:58 am, phil-news-nos...@ipal.net wrote:
    |
    |> I plan to measure. But I plan to do that when I have enough alternate ideas
    |> to measure against (if any).
    |
    | Yeah, it's kind of hard to measure something without something to
    | measure it against. Again, I think in practice the best you can do is
    | probably nothing at all. It may help slightly to have another process
    | 'warm the cache' by getting the inode into memory. Sadly, many UNIXes
    | don't let you warm the cache properly -- doing an
    | 'open'/'fadvise'/'close' may well *not* keep the cache warm for a
    | subsequent 'mmap'.

    OTOH, in my original question, I never asked anyone to measure anything.
    I only asked if it existed and what it might be. If it did exist and I
    could use it, then I would explore if it were practical to my applications,
    past, present, and future.

    Did you mean to include munmap() in that sequence of cache warning steps?
    It might make a difference if the memory remained mapped to some process
    or not, as it affected warming the cache.


    |> I really don't understand why the discussion is going down the path of trying
    |> to say why such a feature is not practical to implement. Has someone asked
    |> to have it implemented?
    |
    | The point is that if it's not practical to implement, you wouldn't
    | want to use it even if it existed because it wouldn't optimize
    | anything. Synchronization is expensive and rarely optimizes anything.
    | The main cost of threads is the expense of synchronizing changes to
    | the shared memory mappings.

    One risk if trying to warm cache is that too many I/O requests in a wider
    variety of places can result in more head thrashing. If I have just read
    some blocks, and will need to read some blocks immediately following, but
    some other process (in the application) tries to warm cache from some other
    location, then that might hurt more than help.

    I want to overlap I/O _just_ _enough_ to keep the disk working without the
    gaps of turnaround through the process that requested the I/O. Maybe two
    processes will be enough. Or maybe they won't be enough to let the disk do
    enough elevator operation to minimize seeks and maximize data read at each
    seek. These are things I need to explore, and am not yet ready to ask about
    them. OTOH, once I'm within a couple percent of theoretically perfect
    throughput, it is probably impractical to improve upon just because the
    speedup wouldn't really be noticed.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  13. Re: sharing memory map between processes (same parent)

    On Thu, 01 May 2008 03:22:48 +0000, phil-news-nospam wrote:


    > One risk if trying to warm cache is that too many I/O requests in a
    > wider variety of places can result in more head thrashing. If I have
    > just read some blocks, and will need to read some blocks immediately
    > following, but some other process (in the application) tries to warm
    > cache from some other location, then that might hurt more than help.


    The other risk with cache warming is that you might perform reads for
    blocks you actually don't need. That would *waste* I/O bandwith.
    It depends on your access pattern.

    > I want to overlap I/O _just_ _enough_ to keep the disk working without
    > the gaps of turnaround through the process that requested the I/O.
    > Maybe two processes will be enough. Or maybe they won't be enough to
    > let the disk do enough elevator operation to minimize seeks and maximize
    > data read at each seek. These are things I need to explore, and am not
    > yet ready to ask about them. OTOH, once I'm within a couple percent of
    > theoretically perfect throughput, it is probably impractical to improve
    > upon just because the speedup wouldn't really be noticed.


    Again: it depends on your access pattern. (and the size of the mapped
    file / available bufferspace / available PTE's / etc)
    If the access pattern is sparse (and bizar), you'll need a way for the
    "parent" to instruct the I/O slaves which pages to prime.

    [NB : for the matrix-transpose exercise in an older tread, I used a mmap()
    ed file to store the matrix. Since every block is touched *exactly once*,
    and order is unimportant, this could be split between fork()ed children.
    IIRC the best result was 8 processes (for a matrix of 64MB on a 2GB
    machine) ]

    BTW: you mentioned passing filedescriptors/ mmap()ped regions.
    Have you considered using the filesystem as a way to communicate ?
    Does the file have a link to the filesystem ?
    Why not let the slave processes open() before mmap() ?

    HTH,
    AvK

  14. Re: sharing memory map between processes (same parent)

    On Apr 30, 8:22 pm, phil-news-nos...@ipal.net wrote:

    > | Yeah, it's kind of hard to measure something without something to
    > | measure it against. Again, I think in practice the best you can do is


    > OTOH, in my original question, I never asked anyone to measure anything.
    > I only asked if it existed and what it might be. If it did exist and I
    > could use it, then I would explore if it were practical to my applications,


    I think you misunderstood what I was saying. That wasn't a criticism,
    it was a concession. I'm saying that it would have been hard for you
    to measure without knowing what to measure. Your original question is
    something like "is there a better way to do X?" And if I say "well,
    have you measured whether it's better", a rational response can be,
    "well, once I know the better ways, I can measure to see how much
    better there are, but I can't measure a mechanism until I at least
    know whether it exists."

    DS

  15. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net writes:
    >On Wed, 30 Apr 2008 16:17:39 -0700 (PDT) David Schwartz wrote:
    >| On Apr 30, 11:58 am, phil-news-nos...@ipal.net wrote:
    >|
    >|> I plan to measure. But I plan to do that when I have enough alternate ideas
    >|> to measure against (if any).
    >|
    >| Yeah, it's kind of hard to measure something without something to
    >| measure it against. Again, I think in practice the best you can do is
    >| probably nothing at all. It may help slightly to have another process
    >| 'warm the cache' by getting the inode into memory. Sadly, many UNIXes
    >| don't let you warm the cache properly -- doing an
    >| 'open'/'fadvise'/'close' may well *not* keep the cache warm for a
    >| subsequent 'mmap'.
    >
    >OTOH, in my original question, I never asked anyone to measure anything.
    >I only asked if it existed and what it might be. If it did exist and I
    >could use it, then I would explore if it were practical to my applications,
    >past, present, and future.
    >
    >Did you mean to include munmap() in that sequence of cache warning steps?
    >It might make a difference if the memory remained mapped to some process
    >or not, as it affected warming the cache.


    How about we take a look at the approximate steps that mmap needs to
    take (generic unix, but applicable to most for illustrative purposes)

    1) For the size of the mapping, add page table entries (PTE) to the
    page tables for the process virtual address space. Note that the
    page tables are per-process. These PTE's will be marked not-present
    with a reference to the appropriate block in the backing file (or swap
    for anonymous mappings).

    done.

    Note that the 'open' of the file preceeded the call to mmap, and this is
    the step that will load the on-disk inode into the inode cache in memory.
    Note however, that through the completion of the mmap, none of the backing
    file content has been loaded into memory, and it won't be until referenced.

    So, whether the mapping is transferred to a second process, or the second
    process calls mmap, the exact same sequence of underlying operating system
    events needs to occur. Thus, there can be no performance benefit to being
    able to tranfer the mapping (aside from the relatively small 'open' overhead,
    which can be ameliorated by shipping the fd to the new process directly, then
    mmaping it).

    As far as two processes mapping the same file, they'll be sharing the same
    pages of physical memory assuming MAP_SHARED; clearly the processor caches
    will only be warm if the both processes sharing the mapping are executing
    on the same core/socket.

    It's not clear that you (phil) realize that the call to mmap(2) itself doesn't
    cause any data to be read from the backing file/device, but rather the
    data is read when the mapped virtual address is referenced (absent any
    os initiated optimizations like read-ahead).

    >I want to overlap I/O _just_ _enough_ to keep the disk working without the
    >gaps of turnaround through the process that requested the I/O. Maybe two
    >processes will be enough. Or maybe they won't be enough to let the disk do


    If you're using mmap, then you should be using madvise to describe your
    access pattern to the operating system; the operating system will then
    attempt to make intelligent decisions about what parts of the mapping to
    pre-fetch into memory.

    scott

  16. Re: sharing memory map between processes (same parent)

    Scott Lurndal wrote:
    > phil-news-nospam@ipal.net writes:

    [snip]
    >> I want to overlap I/O _just_ _enough_ to keep the disk working without the
    >> gaps of turnaround through the process that requested the I/O. Maybe two
    >> processes will be enough. Or maybe they won't be enough to let the disk do

    >
    > If you're using mmap, then you should be using madvise to describe your
    > access pattern to the operating system; the operating system will then
    > attempt to make intelligent decisions about what parts of the mapping to
    > pre-fetch into memory.


    Phil mentioned madvise() at the top of the thread, so presumably he is
    looking to do better than that. That is difficult in general, but may be
    possible in some circumstances - essentially because an application may
    be able to make more intelligent decisions than the operating system
    about prefetching than can be conveyed through madvise(). For instance,
    under Linux, it seems that a mapped file will only be read in response
    to a page fault (at which point some read-ahead may occur).

    I think Phil has not provided enough information to determine whether
    there is any point going beyond using madvise() in his case.

    Alex

  17. Re: sharing memory map between processes (same parent)

    On Thu, 01 May 2008 12:14:42 +0200 moi wrote:
    | On Thu, 01 May 2008 03:22:48 +0000, phil-news-nospam wrote:
    |
    |
    |> One risk if trying to warm cache is that too many I/O requests in a
    |> wider variety of places can result in more head thrashing. If I have
    |> just read some blocks, and will need to read some blocks immediately
    |> following, but some other process (in the application) tries to warm
    |> cache from some other location, then that might hurt more than help.
    |
    | The other risk with cache warming is that you might perform reads for
    | blocks you actually don't need. That would *waste* I/O bandwith.
    | It depends on your access pattern.
    |
    |> I want to overlap I/O _just_ _enough_ to keep the disk working without
    |> the gaps of turnaround through the process that requested the I/O.
    |> Maybe two processes will be enough. Or maybe they won't be enough to
    |> let the disk do enough elevator operation to minimize seeks and maximize
    |> data read at each seek. These are things I need to explore, and am not
    |> yet ready to ask about them. OTOH, once I'm within a couple percent of
    |> theoretically perfect throughput, it is probably impractical to improve
    |> upon just because the speedup wouldn't really be noticed.
    |
    | Again: it depends on your access pattern. (and the size of the mapped
    | file / available bufferspace / available PTE's / etc)
    | If the access pattern is sparse (and bizar), you'll need a way for the
    | "parent" to instruct the I/O slaves which pages to prime.
    |
    | [NB : for the matrix-transpose exercise in an older tread, I used a mmap()
    | ed file to store the matrix. Since every block is touched *exactly once*,
    | and order is unimportant, this could be split between fork()ed children.
    | IIRC the best result was 8 processes (for a matrix of 64MB on a 2GB
    | machine) ]
    |
    | BTW: you mentioned passing filedescriptors/ mmap()ped regions.
    | Have you considered using the filesystem as a way to communicate ?
    | Does the file have a link to the filesystem ?
    | Why not let the slave processes open() before mmap() ?

    I don't know what you mean by slave processes. Of course mmap() needs an
    opened descriptor.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  18. Re: sharing memory map between processes (same parent)

    On Thu, 1 May 2008 08:08:16 -0700 (PDT) David Schwartz wrote:
    | On Apr 30, 8:22 pm, phil-news-nos...@ipal.net wrote:
    |
    |> | Yeah, it's kind of hard to measure something without something to
    |> | measure it against. Again, I think in practice the best you can do is
    |
    |> OTOH, in my original question, I never asked anyone to measure anything.
    |> I only asked if it existed and what it might be. If it did exist and I
    |> could use it, then I would explore if it were practical to my applications,
    |
    | I think you misunderstood what I was saying. That wasn't a criticism,
    | it was a concession. I'm saying that it would have been hard for you
    | to measure without knowing what to measure. Your original question is
    | something like "is there a better way to do X?" And if I say "well,
    | have you measured whether it's better", a rational response can be,
    | "well, once I know the better ways, I can measure to see how much
    | better there are, but I can't measure a mechanism until I at least
    | know whether it exists."

    Or "I can't know if it is better or not, in one or more situations I have now
    or may have in the future, without knowing if the method exists".

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  19. Re: sharing memory map between processes (same parent)

    On Thu, 01 May 2008 20:45:18 GMT Scott Lurndal wrote:

    | 1) For the size of the mapping, add page table entries (PTE) to the
    | page tables for the process virtual address space. Note that the
    | page tables are per-process. These PTE's will be marked not-present
    | with a reference to the appropriate block in the backing file (or swap
    | for anonymous mappings).

    How does it know where the "appropriate block" is? It can't make that
    reference in terms of an open descriptor because the descriptor can be
    closed after mmap(), and the file (or portion thereof) is still mapped.
    So it would need to figure out what blocks are being mapped during the
    mmap() call, or somehow make the reference in terms of the inode of the
    file.


    | Note that the 'open' of the file preceeded the call to mmap, and this is
    | the step that will load the on-disk inode into the inode cache in memory.
    | Note however, that through the completion of the mmap, none of the backing
    | file content has been loaded into memory, and it won't be until referenced.

    However, I suspect the blocks have been looked up.


    | So, whether the mapping is transferred to a second process, or the second
    | process calls mmap, the exact same sequence of underlying operating system
    | events needs to occur. Thus, there can be no performance benefit to being
    | able to tranfer the mapping (aside from the relatively small 'open' overhead,
    | which can be ameliorated by shipping the fd to the new process directly, then
    | mmaping it).

    The sequence of finding the blocks to map could be overlapped with I/O being
    done by that process that will soon get the mmapped memory passed to it.
    Process B will read the file from the mmapped memory. For the very first
    file, there would be no benefit. For subsequent files, there could be some
    overlapping and maybe a speedup.


    | As far as two processes mapping the same file, they'll be sharing the same
    | pages of physical memory assuming MAP_SHARED; clearly the processor caches
    | will only be warm if the both processes sharing the mapping are executing
    | on the same core/socket.

    In my case, once process A transfers it to process B, process A no longer
    needs it. If the mechanism that might have existed merely make a copy,
    then process A would have to munmap().


    | It's not clear that you (phil) realize that the call to mmap(2) itself doesn't
    | cause any data to be read from the backing file/device, but rather the
    | data is read when the mapped virtual address is referenced (absent any
    | os initiated optimizations like read-ahead).

    I'm still not convinced. The mapping setup has to reference the file pages
    in some way, and it cannot depend on the file remaining open since the file
    descriptor mmap() used can be closed after mmap() is done, and that will not
    affect the pages that were mapped.


    |>I want to overlap I/O _just_ _enough_ to keep the disk working without the
    |>gaps of turnaround through the process that requested the I/O. Maybe two
    |>processes will be enough. Or maybe they won't be enough to let the disk do
    |
    | If you're using mmap, then you should be using madvise to describe your
    | access pattern to the operating system; the operating system will then
    | attempt to make intelligent decisions about what parts of the mapping to
    | pre-fetch into memory.

    I would use madvise(). It will be sequential. I don't know if process A
    should do it before the transfer, or if process B should do it after the
    transfer. That would have been another aspect to explore.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  20. Re: sharing memory map between processes (same parent)

    On Fri, 02 May 2008 08:41:58 +0000, phil-news-nospam wrote:

    > On Thu, 01 May 2008 12:14:42 +0200 moi wrote:
    > | On Thu, 01 May 2008 03:22:48 +0000, phil-news-nospam wrote: |


    > |> I want to overlap I/O _just_ _enough_ to keep the disk working
    > without |> the gaps of turnaround through the process that requested the
    > I/O. |> Maybe two processes will be enough. Or maybe they won't be
    > enough to |> let the disk do enough elevator operation to minimize seeks
    > and maximize |> data read at each seek. These are things I need to
    > explore, and am not |> yet ready to ask about them. OTOH, once I'm
    > within a couple percent of |> theoretically perfect throughput, it is
    > probably impractical to improve |> upon just because the speedup
    > wouldn't really be noticed. |

    |
    > | BTW: you mentioned passing filedescriptors/ mmap()ped regions. | Have
    > you considered using the filesystem as a way to communicate ? | Does the
    > file have a link to the filesystem ? | Why not let the slave processes
    > open() before mmap() ?
    >
    > I don't know what you mean by slave processes. Of course mmap() needs
    > an opened descriptor.


    Slave processes are child processes which do the hard work. In the pre-
    thread era, slave processes were used to perform the disk-I/O (blocking
    on disk reads) and putting the resulting buffers into shared memory. In
    your case you use mmap(), which causes the slaves to "block" on
    pagefaults instead of explicit reads, but the mechanism is basically the
    same.

    Of course mmap() needs an open filedescriptor.
    In the OP you mentioned the method by which it should be passed from
    parent to child.
    As been pointed out, there should hardly be any difference between mmap()
    before fork() or mmap() after fork(). The amount of work for the kernel
    is the same, but you'll need one or two additional system calls to ask
    the kernel to get them done. This overhead is probably neglectable
    compared to the fork().

    If your program *needs* the result of a disk-read, in order to issue the
    _next_ read, it *cannot* benefit from parallel reads and cache filling,
    unless the footprint is small enough to prefetch (almost) everything into
    the page cache. (parallel queries may benefit, though).
    Basically it just depends on the (sizeof footprint / sizeof pagecache).
    Plus some other factors you did not specify.

    HTH,
    AvK

+ Reply to Thread
Page 2 of 3 FirstFirst 1 2 3 LastLast