sharing memory map between processes (same parent) - Unix

This is a discussion on sharing memory map between processes (same parent) - Unix ; I have forked two processes from the same parent. I'd like to have one of them do open() and mmap() to map a file into memory (madvise would also be used). Then I would like the other process to be ...

+ Reply to Thread
Page 1 of 3 1 2 3 LastLast
Results 1 to 20 of 43

Thread: sharing memory map between processes (same parent)

  1. sharing memory map between processes (same parent)

    I have forked two processes from the same parent. I'd like to have one of them
    do open() and mmap() to map a file into memory (madvise would also be used).
    Then I would like the other process to be able to access that mapping. Is
    there a way to get that mapping shared with another process (much like it
    would be had the parent mapped it first then forked the two child processes)
    or at least transfer the mapping from one process to another? Note that it
    will not necessarily be a mapping of the entire file, and the first process
    may be keeping the descriptor open for later mapping of another part of the
    file. Also, it may be possible to make it so the first process never does
    any access of the mapped pages.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  2. Re: sharing memory map between processes (same parent)

    In article , phil-news-nospam@ipal.net
    wrote:

    > I have forked two processes from the same parent. I'd like to have one of
    > them
    > do open() and mmap() to map a file into memory (madvise would also be used).
    > Then I would like the other process to be able to access that mapping. Is
    > there a way to get that mapping shared with another process (much like it
    > would be had the parent mapped it first then forked the two child processes)
    > or at least transfer the mapping from one process to another? Note that it
    > will not necessarily be a mapping of the entire file, and the first process
    > may be keeping the descriptor open for later mapping of another part of the
    > file. Also, it may be possible to make it so the first process never does
    > any access of the mapped pages.


    I don't know of any way to transfer a mapping. Processes don't
    generally have the ability to change anything about other properties,
    other than through low-level manipulation like ptrace().

    Why not just have both processes mmap() the same file?

    --
    Barry Margolin, barmar@alum.mit.edu
    Arlington, MA
    *** PLEASE post questions in newsgroups, not directly to me ***
    *** PLEASE don't copy me on replies, I'll read them in the group ***

  3. Re: sharing memory map between processes (same parent)

    On Apr 27, 4:04 pm, phil-news-nos...@ipal.net wrote:
    > I have forked two processes from the same parent. I'd like to have one of them
    > do open() and mmap() to map a file into memory (madvise would also be used).
    > Then I would like the other process to be able to access that mapping. Is
    > there a way to get that mapping shared with another process (much like it
    > would be had the parent mapped it first then forked the two child processes)
    > or at least transfer the mapping from one process to another? Note that it
    > will not necessarily be a mapping of the entire file, and the first process
    > may be keeping the descriptor open for later mapping of another part of the
    > file. Also, it may be possible to make it so the first process never does
    > any access of the mapped pages.


    not possible with posix/unix apis. unless you advertise to your child
    what to map at runtime, there no way for process A to map shared
    region Y to process B.
    As Barry said the only way is to continue using mmap and maybe provide
    a shared id available to your children processes when they fork and
    signal them when they have something to map: they fetch the shared
    memory name from the shared id and map it to their address space (note
    that name can be a sysv id, or a posix name).
    The mapping of shared regions, at least on most unix systems I know,
    is on demand.

    -- paulo

  4. Re: sharing memory map between processes (same parent)

    On Apr 27, 1:04 pm, phil-news-nos...@ipal.net wrote:

    > I have forked two processes from the same parent. I'd like to have one of them
    > do open() and mmap() to map a file into memory (madvise would also be used).
    > Then I would like the other process to be able to access that mapping. Is
    > there a way to get that mapping shared with another process (much like it
    > would be had the parent mapped it first then forked the two child processes)
    > or at least transfer the mapping from one process to another? Note that it
    > will not necessarily be a mapping of the entire file, and the first process
    > may be keeping the descriptor open for later mapping of another part of the
    > file. Also, it may be possible to make it so the first process never does
    > any access of the mapped pages.


    What's your outer problem? Why not just have the other process create
    the mapping itself? The answer to your question would be very
    different if the reason was that the mapping was private than it would
    be if the problem was that the mapping contains pointers and needs to
    be at the same address for all processes.

    You have a habit of posting suggested solutions rather than problems.

    DS

  5. Re: sharing memory map between processes (same parent)

    On Sun, 27 Apr 2008 21:00:12 -0400 Barry Margolin wrote:
    | In article , phil-news-nospam@ipal.net
    | wrote:
    |
    |> I have forked two processes from the same parent. I'd like to have one of
    |> them
    |> do open() and mmap() to map a file into memory (madvise would also be used).
    |> Then I would like the other process to be able to access that mapping. Is
    |> there a way to get that mapping shared with another process (much like it
    |> would be had the parent mapped it first then forked the two child processes)
    |> or at least transfer the mapping from one process to another? Note that it
    |> will not necessarily be a mapping of the entire file, and the first process
    |> may be keeping the descriptor open for later mapping of another part of the
    |> file. Also, it may be possible to make it so the first process never does
    |> any access of the mapped pages.
    |
    | I don't know of any way to transfer a mapping. Processes don't
    | generally have the ability to change anything about other properties,
    | other than through low-level manipulation like ptrace().
    |
    | Why not just have both processes mmap() the same file?

    The other process will be busy at the time. I was hoping for something that
    was similar to being able to transfer an open file descriptor.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  6. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 07:24:07 -0700 (PDT) ppi wrote:
    | On Apr 27, 4:04 pm, phil-news-nos...@ipal.net wrote:
    |> I have forked two processes from the same parent. I'd like to have one of them
    |> do open() and mmap() to map a file into memory (madvise would also be used).
    |> Then I would like the other process to be able to access that mapping. Is
    |> there a way to get that mapping shared with another process (much like it
    |> would be had the parent mapped it first then forked the two child processes)
    |> or at least transfer the mapping from one process to another? Note that it
    |> will not necessarily be a mapping of the entire file, and the first process
    |> may be keeping the descriptor open for later mapping of another part of the
    |> file. Also, it may be possible to make it so the first process never does
    |> any access of the mapped pages.
    |
    | not possible with posix/unix apis. unless you advertise to your child
    | what to map at runtime, there no way for process A to map shared
    | region Y to process B.
    | As Barry said the only way is to continue using mmap and maybe provide
    | a shared id available to your children processes when they fork and
    | signal them when they have something to map: they fetch the shared
    | memory name from the shared id and map it to their address space (note
    | that name can be a sysv id, or a posix name).
    | The mapping of shared regions, at least on most unix systems I know,
    | is on demand.

    The fallback plan is to transfer the open descriptor from process A to process
    B, then let process B do mmap and read from it.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  7. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 07:37:57 -0700 (PDT) David Schwartz wrote:
    | On Apr 27, 1:04 pm, phil-news-nos...@ipal.net wrote:
    |
    |> I have forked two processes from the same parent. I'd like to have one of them
    |> do open() and mmap() to map a file into memory (madvise would also be used).
    |> Then I would like the other process to be able to access that mapping. Is
    |> there a way to get that mapping shared with another process (much like it
    |> would be had the parent mapped it first then forked the two child processes)
    |> or at least transfer the mapping from one process to another? Note that it
    |> will not necessarily be a mapping of the entire file, and the first process
    |> may be keeping the descriptor open for later mapping of another part of the
    |> file. Also, it may be possible to make it so the first process never does
    |> any access of the mapped pages.
    |
    | What's your outer problem? Why not just have the other process create
    | the mapping itself? The answer to your question would be very
    | different if the reason was that the mapping was private than it would
    | be if the problem was that the mapping contains pointers and needs to
    | be at the same address for all processes.

    The idea is to have some of the work spread out between processes and avoid
    having to use threads (because I don't want to share the whole VM). There
    is a way to transfer (already open) file descriptors between processes. I
    was hoping there might be a similar way to transfer already mapped pages
    between processes. But if there is not, I will just have to do things some
    other way.


    | You have a habit of posting suggested solutions rather than problems.

    You have a habit of being accusatory. But I've only noticed this over the
    past couple years. I remember long ago your postings were more civil.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  8. Re: sharing memory map between processes (same parent)

    On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:

    > | What's your outer problem? Why not just have the other process create
    > | the mapping itself? The answer to your question would be very
    > | different if the reason was that the mapping was private than it would
    > | be if the problem was that the mapping contains pointers and needs to
    > | be at the same address for all processes.


    > The idea is to have some of the work spread out between processes and avoid
    > having to use threads (because I don't want to share the whole VM). There
    > is a way to transfer (already open) file descriptors between processes. I
    > was hoping there might be a similar way to transfer already mapped pages
    > between processes. But if there is not, I will just have to do things some
    > other way.


    You didn't answer my question. I still don't know why you don't just
    have each process open the mapping itself. Is it because you need the
    mappings to be at the same address? Is it because the mappings are
    private? What is the problem that you think this would solve?

    > | You have a habit of posting suggested solutions rather than problems.


    > You have a habit of being accusatory. But I've only noticed this over the
    > past couple years. I remember long ago your postings were more civil.


    I remember you as being a pretty bright guy, but honestly, your posts
    seems to be getting, for lack of a better word, dumber. This is the
    second time I've asked what I thought was a pretty clear question and
    even included examples of what would be a good or possible answer, and
    didn't get anywhere.

    Any time you ask a question, expect the response "why don't you do it
    the usual way?" and give the answer in the question. Why not just have
    each process map it itself? That's more efficient, because it doesn't
    require extra coordination.

    DS

  9. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 14:25:48 -0700 (PDT) David Schwartz wrote:
    | On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:
    |
    |> | What's your outer problem? Why not just have the other process create
    |> | the mapping itself? The answer to your question would be very
    |> | different if the reason was that the mapping was private than it would
    |> | be if the problem was that the mapping contains pointers and needs to
    |> | be at the same address for all processes.
    |
    |> The idea is to have some of the work spread out between processes and avoid
    |> having to use threads (because I don't want to share the whole VM). There
    |> is a way to transfer (already open) file descriptors between processes. I
    |> was hoping there might be a similar way to transfer already mapped pages
    |> between processes. But if there is not, I will just have to do things some
    |> other way.
    |
    | You didn't answer my question. I still don't know why you don't just
    | have each process open the mapping itself. Is it because you need the
    | mappings to be at the same address? Is it because the mappings are
    | private? What is the problem that you think this would solve?

    There's no need to open the mapping twice. It's only needed one. The
    idea is for one process to go ahead and open and map the file and have
    it all ready, maybe even with a few pages referenced, and hand it over
    to the next process. That's what I do now with open descriptors. One
    process opens the descriptor and hands it over to another process to
    read from.


    |> | You have a habit of posting suggested solutions rather than problems.
    |
    |> You have a habit of being accusatory. But I've only noticed this over the
    |> past couple years. I remember long ago your postings were more civil.
    |
    | I remember you as being a pretty bright guy, but honestly, your posts
    | seems to be getting, for lack of a better word, dumber. This is the
    | second time I've asked what I thought was a pretty clear question and
    | even included examples of what would be a good or possible answer, and
    | didn't get anywhere.

    I do purposely try to avoid answering questions that go off track of what I
    an seeking answers for. I used to not do that, and that would almost always
    result in a pointless drawn out thread with someone else trying to suggest
    ways to do things that would not really work because of many issues in the
    program, that I didn't post in the first place because they were not relevant
    to


    | Any time you ask a question, expect the response "why don't you do it
    | the usual way?" and give the answer in the question. Why not just have
    | each process map it itself? That's more efficient, because it doesn't
    | require extra coordination.

    When I ask, expect that I have checked out the "usual" way, and that either
    that way doesn't work for some reason, or that I am already doing it that way
    and am exploring to find another way.

    If I describe some particular way to do something and ask if it is available
    in some system, then I'm not interested in other ways, even though one of the
    other ways might be the usual. If I describe what end result I want and ask
    how to accomplish it, then any method to accomplish it is fair game.

    If what I want to do cannot be done, then just say "no" or don't say. But do
    not make assumptions about what other stuff I may or may not have explored.
    If you are curious what I have explored, you can always ask. I may or may not
    be interested in pursing that subthread.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  10. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:
    > On Mon, 28 Apr 2008 14:25:48 -0700 (PDT) David Schwartz wrote:
    > | On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:
    > |
    > |> | What's your outer problem? Why not just have the other process create
    > |> | the mapping itself? The answer to your question would be very
    > |> | different if the reason was that the mapping was private than it would
    > |> | be if the problem was that the mapping contains pointers and needs to
    > |> | be at the same address for all processes.
    > |
    > |> The idea is to have some of the work spread out between processes and avoid
    > |> having to use threads (because I don't want to share the whole VM). There
    > |> is a way to transfer (already open) file descriptors between processes. I
    > |> was hoping there might be a similar way to transfer already mapped pages
    > |> between processes. But if there is not, I will just have to do things some
    > |> other way.
    > |
    > | You didn't answer my question. I still don't know why you don't just
    > | have each process open the mapping itself. Is it because you need the
    > | mappings to be at the same address? Is it because the mappings are
    > | private? What is the problem that you think this would solve?
    >
    > There's no need to open the mapping twice. It's only needed one. The
    > idea is for one process to go ahead and open and map the file and have
    > it all ready, maybe even with a few pages referenced, and hand it over
    > to the next process. That's what I do now with open descriptors. One
    > process opens the descriptor and hands it over to another process to
    > read from.
    >

    Well that's the normal way of doing things (passing an fd between
    processes), is it causing problems? There shouldn't be much of a
    performance hit from remapping the file.

    --
    Ian Collins.

  11. Re: sharing memory map between processes (same parent)

    On Apr 28, 2:44 pm, phil-news-nos...@ipal.net wrote:

    > I do purposely try to avoid answering questions that go off track of what I
    > an seeking answers for. I used to not do that, and that would almost always
    > result in a pointless drawn out thread with someone else trying to suggest
    > ways to do things that would not really work because of many issues in the
    > program, that I didn't post in the first place because they were not relevant
    > to


    You can't use USENET that way. It's just not fair.

    If you ask a really unusual question without any discussion of why
    it's unusual, and you get (maybe even by luck) exactly the right
    answer, the exchange will help nobody but you. Worse, other people
    looking at the exchange may think the answer is the normal or
    appropriate way, though it's only right because of all your "many
    issues in the program", which they don't even know exists.

    If you want help from USENET, you have to explain your situation such
    that it's, at a minimum, not misleading to people in the future who
    hunt around for useful information.

    > | Any time you ask a question, expect the response "why don't you do it
    > | the usual way?" and give the answer in the question. Why not just have
    > | each process map it itself? That's more efficient, because it doesn't
    > | require extra coordination.


    > When I ask, expect that I have checked out the "usual" way, and that either
    > that way doesn't work for some reason, or that I am already doing it that way
    > and am exploring to find another way.


    Unfortunately, that doesn't work for two reasons. First, the time when
    you haven't checked out the usual way, it will lead you further
    astray. Second, people who need the more typical answer will be lead
    astray by the exchange.

    These are real, common, problems whose consequences have been learned
    by hard and painful experience. You are asking for things that just
    don't work on USENET.

    > If I describe some particular way to do something and ask if it is available
    > in some system, then I'm not interested in other ways, even though one of the
    > other ways might be the usual. If I describe what end result I want and ask
    > how to accomplish it, then any method to accomplish it is fair game.


    But you don't describe what end result you want. You describe how you
    think you could accomplish that end result.

    For example, in this thread, was the end result the mapping at the
    same address in both processes or not?

    The only way to know what result you wanted was to understand what you
    were trying to accomplish. Otherwise, we'd have to tease through every
    tedious permutation of how two processes could wind up with the "same
    mapping".

    > If what I want to do cannot be done, then just say "no" or don't say. But do
    > not make assumptions about what other stuff I may or may not have explored.
    > If you are curious what I have explored, you can always ask. I may or may not
    > be interested in pursing that subthread.


    That's not a fair attitude on USENET. USENET is a give and take, and
    you cannot refuse to give. Spend the extra ten minutes explaining your
    problem. Not only will that help novices to better understand whether
    the answers you get apply to them or not, you will get better answers.
    Plus, you will spend less time explaining why the answers you get
    aren't helpful or saying "I already tried X, that was obvious" because
    you didn't make clear why X wasn't what you wanted.

    And this post is not just for you. It's more for anyone reading this
    thread who might think that your post serves as a good example of how
    to ask a question.

    This is probably the most frequently posted link on USENET:
    http://catb.org/~esr/faqs/smart-questions.html
    See especially the section no describing the ultimate goal and the
    section on being specific.

    DS

  12. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:

    > There's no need to open the mapping twice. It's only needed one. The
    > idea is for one process to go ahead and open and map the file and have
    > it all ready, maybe even with a few pages referenced, and hand it over
    > to the next process. That's what I do now with open descriptors. One
    > process opens the descriptor and hands it over to another process to
    > read from.


    Why set up the mapping in the first process if it isn't going to
    actually use it?

    Setting up mappings is relatively expensive, so it should only be done
    in the process that is going to use the mapping.

    Chris

  13. Re: sharing memory map between processes (same parent)

    In article , phil-news-nospam@ipal.net
    wrote:

    > I do purposely try to avoid answering questions that go off track of
    > what I an seeking answers for. I used to not do that, and that would
    > almost always result in a pointless drawn out thread with someone
    > else trying to suggest ways to do things that would not really work
    > because of many issues in the program, that I didn't post in the
    > first place because they were not relevant to


    If those issues prevent you from doing it the usual way, they ARE
    relevant to the discussion. They're part of the problem you're trying
    to solve.

    You want to avoid having the discussion go off track? Then describe the
    problem as fully as possible, the issues that restrict what you can do,
    what methods you've rejected and why you've rejected them. Otherwise
    people WILL waste time giving you solutions you've already rejected,
    because you've given them no reason to believe it won't work.

  14. Re: sharing memory map between processes (same parent)

    On Tue, 29 Apr 2008 09:54:26 +1200 Ian Collins wrote:

    |> There's no need to open the mapping twice. It's only needed one. The
    |> idea is for one process to go ahead and open and map the file and have
    |> it all ready, maybe even with a few pages referenced, and hand it over
    |> to the next process. That's what I do now with open descriptors. One
    |> process opens the descriptor and hands it over to another process to
    |> read from.
    |>
    | Well that's the normal way of doing things (passing an fd between
    | processes), is it causing problems? There shouldn't be much of a
    | performance hit from remapping the file.

    I'm just exploring to see if there is something better. Threading should be
    better, but I want to explore possible ways to avoid threading.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  15. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 15:16:47 -0700 (PDT) David Schwartz wrote:
    | On Apr 28, 2:44 pm, phil-news-nos...@ipal.net wrote:
    |
    |> I do purposely try to avoid answering questions that go off track of what I
    |> an seeking answers for. I used to not do that, and that would almost always
    |> result in a pointless drawn out thread with someone else trying to suggest
    |> ways to do things that would not really work because of many issues in the
    |> program, that I didn't post in the first place because they were not relevant
    |> to
    |
    | You can't use USENET that way. It's just not fair.
    |
    | If you ask a really unusual question without any discussion of why
    | it's unusual, and you get (maybe even by luck) exactly the right
    | answer, the exchange will help nobody but you. Worse, other people
    | looking at the exchange may think the answer is the normal or
    | appropriate way, though it's only right because of all your "many
    | issues in the program", which they don't even know exists.
    |
    | If you want help from USENET, you have to explain your situation such
    | that it's, at a minimum, not misleading to people in the future who
    | hunt around for useful information.

    If you want the whole can of worms, I'm writing a program to rapidly install
    a system during the time the system is booting up. If you want to discuss
    that whole thing, feel free to ... in another thread that I will not take
    any part in. Until such time as I have completed the exploration of it, I
    do don't plan to participate it what I suspect such a thread will become
    (based on past Usenet experience). But if it interests you and others, go
    right ahead on your own.

    And you might wonder, if it is for Linux, why not ask in a Linux system
    group. I'd rather find portable ways to do it in case I might ever want
    to do the same thing with BSD.


    |> | Any time you ask a question, expect the response "why don't you do it
    |> | the usual way?" and give the answer in the question. Why not just have
    |> | each process map it itself? That's more efficient, because it doesn't
    |> | require extra coordination.
    |
    |> When I ask, expect that I have checked out the "usual" way, and that either
    |> that way doesn't work for some reason, or that I am already doing it that way
    |> and am exploring to find another way.
    |
    | Unfortunately, that doesn't work for two reasons. First, the time when
    | you haven't checked out the usual way, it will lead you further
    | astray. Second, people who need the more typical answer will be lead
    | astray by the exchange.

    If they misunderstood what I ask, and how narrow it is, I supposed so.
    "How do I do X?" ... "To do X, there is method Y and also method Z".
    Why would someone wanting to do something other than X try to apply
    method Y or method Z?


    | These are real, common, problems whose consequences have been learned
    | by hard and painful experience. You are asking for things that just
    | don't work on USENET.
    |
    |> If I describe some particular way to do something and ask if it is available
    |> in some system, then I'm not interested in other ways, even though one of the
    |> other ways might be the usual. If I describe what end result I want and ask
    |> how to accomplish it, then any method to accomplish it is fair game.
    |
    | But you don't describe what end result you want. You describe how you
    | think you could accomplish that end result.

    That depends on the scope of what is being considered. It can be looked
    at in a broad way or a narrow way. When you are considering just the
    application, then what you are asking is what the application needs to
    accomplish. What if no one else is doing that application? But if I am
    asking the "how to" in a more narrow way ...


    | For example, in this thread, was the end result the mapping at the
    | same address in both processes or not?

    No. Just that the mapped pages got transferred. They could have become
    mapped into a different address. I did not confine the goal to include
    keeping the same address. Clearly process B would need to know what the
    addresses are. I can envision a number of ways to transfer mapped pages
    between processes and by that, a number of ways for process B to find out
    the addresses it got them at, or potentially even a way to specify where
    to put them (options it has in the mmap call now).


    | The only way to know what result you wanted was to understand what you
    | were trying to accomplish. Otherwise, we'd have to tease through every
    | tedious permutation of how two processes could wind up with the "same
    | mapping".

    If a mechanism had existed to do this, the specifics of that mechanism
    would have been the answer. There would have been no pondering. If the
    mechanism only supported inserting the mapping into process B at addresses
    not yet mapped to anythingdetermined by the kernel, then whoever would
    know about and suggest such a mechanism would not be pondering other ways
    such as letting process B pick the addresses. If I has narrowed what I
    was asking for to one where process B can pick the addresses, but the only
    solution was one that did not, then that would not fit what I was asking.


    |> If what I want to do cannot be done, then just say "no" or don't say. But do
    |> not make assumptions about what other stuff I may or may not have explored.
    |> If you are curious what I have explored, you can always ask. I may or may not
    |> be interested in pursing that subthread.
    |
    | That's not a fair attitude on USENET. USENET is a give and take, and
    | you cannot refuse to give. Spend the extra ten minutes explaining your
    | problem. Not only will that help novices to better understand whether
    | the answers you get apply to them or not, you will get better answers.
    | Plus, you will spend less time explaining why the answers you get
    | aren't helpful or saying "I already tried X, that was obvious" because
    | you didn't make clear why X wasn't what you wanted.

    It has been my experience that this approach has led to very long discourse,
    often times rather heated; far more than this thread is even approaching.
    I want to avoid that kind of thing.

    OTOH, I do give in other ways. I do post answers to what other people ask.
    Sometimes I do "guess" at what they are trying to do, and think maybe they
    have overlooked a better path if they just back up one and go another way.
    Maybe they didn't. If I am confident of my guess, I will make such a
    suggestion. Most of the time my guesses have been wrong. Occaisionally
    they are right. If I were to post such alternatives all the time, then I
    would be more of a pest than I seem to be now.


    | And this post is not just for you. It's more for anyone reading this
    | thread who might think that your post serves as a good example of how
    | to ask a question.
    |
    | This is probably the most frequently posted link on USENET:
    | http://catb.org/~esr/faqs/smart-questions.html
    | See especially the section no describing the ultimate goal and the
    | section on being specific.

    Now my goal is to find the words to get people to not drag me unto subthreads
    of discussion that get off the subject of what I was asking. And these need
    to be polite enough that I can stick them in every initiated question I ask.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  16. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:
    | phil-news-nospam@ipal.net wrote:
    |
    |> There's no need to open the mapping twice. It's only needed one. The
    |> idea is for one process to go ahead and open and map the file and have
    |> it all ready, maybe even with a few pages referenced, and hand it over
    |> to the next process. That's what I do now with open descriptors. One
    |> process opens the descriptor and hands it over to another process to
    |> read from.
    |
    | Why set up the mapping in the first process if it isn't going to
    | actually use it?

    Well, if it can't transfer it to the second process, then there is no need to
    map the file in the first process.


    | Setting up mappings is relatively expensive, so it should only be done
    | in the process that is going to use the mapping.

    I wanted to overlap setting up the mapping with the work already taking place
    in process B. Much of the work of setting up a mapping (like reading block
    pointer records for the file) would not need to be repeated in such a transfer,
    so the time delay in process B to merely accept the mapping from process A
    means process B can get on to other things more quickly. On a slow machine,
    it might not be any advantage. On a fast machine, maybe there would be.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  17. Re: sharing memory map between processes (same parent)

    On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris wrote:
    | In article , phil-news-nospam@ipal.net
    | wrote:
    |
    |> I do purposely try to avoid answering questions that go off track of
    |> what I an seeking answers for. I used to not do that, and that would
    |> almost always result in a pointless drawn out thread with someone
    |> else trying to suggest ways to do things that would not really work
    |> because of many issues in the program, that I didn't post in the
    |> first place because they were not relevant to
    |
    | If those issues prevent you from doing it the usual way, they ARE
    | relevant to the discussion. They're part of the problem you're trying
    | to solve.
    |
    | You want to avoid having the discussion go off track? Then describe the
    | problem as fully as possible, the issues that restrict what you can do,
    | what methods you've rejected and why you've rejected them. Otherwise
    | people WILL waste time giving you solutions you've already rejected,
    | because you've given them no reason to believe it won't work.

    That would be too long of a post.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

  18. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:
    > On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris wrote:


    > | You want to avoid having the discussion go off track? Then describe the
    > | problem as fully as possible, the issues that restrict what you can do,
    > | what methods you've rejected and why you've rejected them. Otherwise
    > | people WILL waste time giving you solutions you've already rejected,
    > | because you've given them no reason to believe it won't work.


    > That would be too long of a post.


    It would be shorter than what you've now written in this long thread,
    and everyone's time would have been saved.

    Seriously, the above advice is excellent.

    Chris

  19. Re: sharing memory map between processes (same parent)

    phil-news-nospam@ipal.net wrote:
    > On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:


    > | Setting up mappings is relatively expensive, so it should only be done
    > | in the process that is going to use the mapping.
    >
    > I wanted to overlap setting up the mapping with the work already taking place
    > in process B. Much of the work of setting up a mapping (like reading block
    > pointer records for the file) would not need to be repeated in such a transfer,
    > so the time delay in process B to merely accept the mapping from process A
    > means process B can get on to other things more quickly.


    A large part of the cost of setting up a mapping is in the fact that the
    tlb needs to be flushed (to force the cpu to see the new mapping). This
    would need to be done in the second process no matter what since it is
    in a separate memory map. This overhead (as well as the overhead of
    transferring file descriptors via unix sockets) could be minimized by
    simply using threads.

    Chris

  20. Re: sharing memory map between processes (same parent)

    On Mon, 28 Apr 2008 23:08:54 -0600 Chris Friesen wrote:
    | phil-news-nospam@ipal.net wrote:
    |> On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen wrote:
    |
    |> | Setting up mappings is relatively expensive, so it should only be done
    |> | in the process that is going to use the mapping.
    |>
    |> I wanted to overlap setting up the mapping with the work already taking place
    |> in process B. Much of the work of setting up a mapping (like reading block
    |> pointer records for the file) would not need to be repeated in such a transfer,
    |> so the time delay in process B to merely accept the mapping from process A
    |> means process B can get on to other things more quickly.
    |
    | A large part of the cost of setting up a mapping is in the fact that the
    | tlb needs to be flushed (to force the cpu to see the new mapping). This
    | would need to be done in the second process no matter what since it is
    | in a separate memory map. This overhead (as well as the overhead of
    | transferring file descriptors via unix sockets) could be minimized by
    | simply using threads.

    But isn't this a cost associated with context switching?

    Yes, I know threads is an option, and is a good one with respect to this
    aspect. I have other reasons to not use threads. One is that at the time
    my program will run, no libraries are available. Conceivably, I could still
    do "threads" via a clone() call. But I don't want to go into that territory
    right now. So I'm exploring what options, few as they may be, exist for
    processes.

    --
    |WARNING: Due to extreme spam, I no longer see any articles originating from |
    | Google Groups. If you want your postings to be seen by more readers |
    | you will need to find a different place to post on Usenet. |
    | Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

+ Reply to Thread
Page 1 of 3 1 2 3 LastLast