sharing memory map between processes (same parent)

I have forked two processes from the same parent. I'd like to have one of them
do open() and mmap() to map a file into memory (madvise would also be used).
Then I would like the other process to be able to access that mapping. Is
there a way to get that mapping shared with another process (much like it
would be had the parent mapped it first then forked the two child processes)
or at least transfer the mapping from one process to another? Note that it
will not necessarily be a mapping of the entire file, and the first process
may be keeping the descriptor open for later mapping of another part of the
file. Also, it may be possible to make it so the first process never does
any access of the mapped pages.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

In article <fv2m7o01es1@news1.newsguy.com>, [email]phil-news-nospam@ipal.net[/email]
wrote:
[color=blue]
> I have forked two processes from the same parent. I'd like to have one of
> them
> do open() and mmap() to map a file into memory (madvise would also be used).
> Then I would like the other process to be able to access that mapping. Is
> there a way to get that mapping shared with another process (much like it
> would be had the parent mapped it first then forked the two child processes)
> or at least transfer the mapping from one process to another? Note that it
> will not necessarily be a mapping of the entire file, and the first process
> may be keeping the descriptor open for later mapping of another part of the
> file. Also, it may be possible to make it so the first process never does
> any access of the mapped pages.[/color]

I don't know of any way to transfer a mapping. Processes don't
generally have the ability to change anything about other properties,
other than through low-level manipulation like ptrace().

Why not just have both processes mmap() the same file?

--
Barry Margolin, [email]barmar@alum.mit.edu[/email]
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

On Apr 27, 4:04 pm, phil-news-nos...@ipal.net wrote:[color=blue]
> I have forked two processes from the same parent. I'd like to have one of them
> do open() and mmap() to map a file into memory (madvise would also be used).
> Then I would like the other process to be able to access that mapping. Is
> there a way to get that mapping shared with another process (much like it
> would be had the parent mapped it first then forked the two child processes)
> or at least transfer the mapping from one process to another? Note that it
> will not necessarily be a mapping of the entire file, and the first process
> may be keeping the descriptor open for later mapping of another part of the
> file. Also, it may be possible to make it so the first process never does
> any access of the mapped pages.[/color]

not possible with posix/unix apis. unless you advertise to your child
what to map at runtime, there no way for process A to map shared
region Y to process B.
As Barry said the only way is to continue using mmap and maybe provide
a shared id available to your children processes when they fork and
signal them when they have something to map: they fetch the shared
memory name from the shared id and map it to their address space (note
that name can be a sysv id, or a posix name).
The mapping of shared regions, at least on most unix systems I know,
is on demand.

-- paulo

On Apr 27, 1:04 pm, phil-news-nos...@ipal.net wrote:
[color=blue]
> I have forked two processes from the same parent. I'd like to have one of them
> do open() and mmap() to map a file into memory (madvise would also be used).
> Then I would like the other process to be able to access that mapping. Is
> there a way to get that mapping shared with another process (much like it
> would be had the parent mapped it first then forked the two child processes)
> or at least transfer the mapping from one process to another? Note that it
> will not necessarily be a mapping of the entire file, and the first process
> may be keeping the descriptor open for later mapping of another part of the
> file. Also, it may be possible to make it so the first process never does
> any access of the mapped pages.[/color]

What's your outer problem? Why not just have the other process create
the mapping itself? The answer to your question would be very
different if the reason was that the mapping was private than it would
be if the problem was that the mapping contains pointers and needs to
be at the same address for all processes.

You have a habit of posting suggested solutions rather than problems.

DS

On Sun, 27 Apr 2008 21:00:12 -0400 Barry Margolin <barmar@alum.mit.edu> wrote:
| In article <fv2m7o01es1@news1.newsguy.com>, [email]phil-news-nospam@ipal.net[/email]
| wrote:
|
|> I have forked two processes from the same parent. I'd like to have one of
|> them
|> do open() and mmap() to map a file into memory (madvise would also be used).
|> Then I would like the other process to be able to access that mapping. Is
|> there a way to get that mapping shared with another process (much like it
|> would be had the parent mapped it first then forked the two child processes)
|> or at least transfer the mapping from one process to another? Note that it
|> will not necessarily be a mapping of the entire file, and the first process
|> may be keeping the descriptor open for later mapping of another part of the
|> file. Also, it may be possible to make it so the first process never does
|> any access of the mapped pages.
|
| I don't know of any way to transfer a mapping. Processes don't
| generally have the ability to change anything about other properties,
| other than through low-level manipulation like ptrace().
|
| Why not just have both processes mmap() the same file?

The other process will be busy at the time. I was hoping for something that
was similar to being able to transfer an open file descriptor.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Mon, 28 Apr 2008 07:24:07 -0700 (PDT) ppi <vodoom@gmail.com> wrote:
| On Apr 27, 4:04 pm, phil-news-nos...@ipal.net wrote:
|> I have forked two processes from the same parent. I'd like to have one of them
|> do open() and mmap() to map a file into memory (madvise would also be used).
|> Then I would like the other process to be able to access that mapping. Is
|> there a way to get that mapping shared with another process (much like it
|> would be had the parent mapped it first then forked the two child processes)
|> or at least transfer the mapping from one process to another? Note that it
|> will not necessarily be a mapping of the entire file, and the first process
|> may be keeping the descriptor open for later mapping of another part of the
|> file. Also, it may be possible to make it so the first process never does
|> any access of the mapped pages.
|
| not possible with posix/unix apis. unless you advertise to your child
| what to map at runtime, there no way for process A to map shared
| region Y to process B.
| As Barry said the only way is to continue using mmap and maybe provide
| a shared id available to your children processes when they fork and
| signal them when they have something to map: they fetch the shared
| memory name from the shared id and map it to their address space (note
| that name can be a sysv id, or a posix name).
| The mapping of shared regions, at least on most unix systems I know,
| is on demand.

The fallback plan is to transfer the open descriptor from process A to process
B, then let process B do mmap and read from it.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Mon, 28 Apr 2008 07:37:57 -0700 (PDT) David Schwartz <davids@webmaster.com> wrote:
| On Apr 27, 1:04 pm, phil-news-nos...@ipal.net wrote:
|
|> I have forked two processes from the same parent. I'd like to have one of them
|> do open() and mmap() to map a file into memory (madvise would also be used).
|> Then I would like the other process to be able to access that mapping. Is
|> there a way to get that mapping shared with another process (much like it
|> would be had the parent mapped it first then forked the two child processes)
|> or at least transfer the mapping from one process to another? Note that it
|> will not necessarily be a mapping of the entire file, and the first process
|> may be keeping the descriptor open for later mapping of another part of the
|> file. Also, it may be possible to make it so the first process never does
|> any access of the mapped pages.
|
| What's your outer problem? Why not just have the other process create
| the mapping itself? The answer to your question would be very
| different if the reason was that the mapping was private than it would
| be if the problem was that the mapping contains pointers and needs to
| be at the same address for all processes.

The idea is to have some of the work spread out between processes and avoid
having to use threads (because I don't want to share the whole VM). There
is a way to transfer (already open) file descriptors between processes. I
was hoping there might be a similar way to transfer already mapped pages
between processes. But if there is not, I will just have to do things some
other way.


| You have a habit of posting suggested solutions rather than problems.

You have a habit of being accusatory. But I've only noticed this over the
past couple years. I remember long ago your postings were more civil.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:
[color=blue]
> | What's your outer problem? Why not just have the other process create
> | the mapping itself? The answer to your question would be very
> | different if the reason was that the mapping was private than it would
> | be if the problem was that the mapping contains pointers and needs to
> | be at the same address for all processes.[/color]
[color=blue]
> The idea is to have some of the work spread out between processes and avoid
> having to use threads (because I don't want to share the whole VM). There
> is a way to transfer (already open) file descriptors between processes. I
> was hoping there might be a similar way to transfer already mapped pages
> between processes. But if there is not, I will just have to do things some
> other way.[/color]

You didn't answer my question. I still don't know why you don't just
have each process open the mapping itself. Is it because you need the
mappings to be at the same address? Is it because the mappings are
private? What is the problem that you think this would solve?
[color=blue]
> | You have a habit of posting suggested solutions rather than problems.[/color]
[color=blue]
> You have a habit of being accusatory. But I've only noticed this over the
> past couple years. I remember long ago your postings were more civil.[/color]

I remember you as being a pretty bright guy, but honestly, your posts
seems to be getting, for lack of a better word, dumber. This is the
second time I've asked what I thought was a pretty clear question and
even included examples of what would be a good or possible answer, and
didn't get anywhere.

Any time you ask a question, expect the response "why don't you do it
the usual way?" and give the answer in the question. Why not just have
each process map it itself? That's more efficient, because it doesn't
require extra coordination.

DS

On Mon, 28 Apr 2008 14:25:48 -0700 (PDT) David Schwartz <davids@webmaster.com> wrote:
| On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:
|
|> | What's your outer problem? Why not just have the other process create
|> | the mapping itself? The answer to your question would be very
|> | different if the reason was that the mapping was private than it would
|> | be if the problem was that the mapping contains pointers and needs to
|> | be at the same address for all processes.
|
|> The idea is to have some of the work spread out between processes and avoid
|> having to use threads (because I don't want to share the whole VM). There
|> is a way to transfer (already open) file descriptors between processes. I
|> was hoping there might be a similar way to transfer already mapped pages
|> between processes. But if there is not, I will just have to do things some
|> other way.
|
| You didn't answer my question. I still don't know why you don't just
| have each process open the mapping itself. Is it because you need the
| mappings to be at the same address? Is it because the mappings are
| private? What is the problem that you think this would solve?

There's no need to open the mapping twice. It's only needed one. The
idea is for one process to go ahead and open and map the file and have
it all ready, maybe even with a few pages referenced, and hand it over
to the next process. That's what I do now with open descriptors. One
process opens the descriptor and hands it over to another process to
read from.


|> | You have a habit of posting suggested solutions rather than problems.
|
|> You have a habit of being accusatory. But I've only noticed this over the
|> past couple years. I remember long ago your postings were more civil.
|
| I remember you as being a pretty bright guy, but honestly, your posts
| seems to be getting, for lack of a better word, dumber. This is the
| second time I've asked what I thought was a pretty clear question and
| even included examples of what would be a good or possible answer, and
| didn't get anywhere.

I do purposely try to avoid answering questions that go off track of what I
an seeking answers for. I used to not do that, and that would almost always
result in a pointless drawn out thread with someone else trying to suggest
ways to do things that would not really work because of many issues in the
program, that I didn't post in the first place because they were not relevant
to


| Any time you ask a question, expect the response "why don't you do it
| the usual way?" and give the answer in the question. Why not just have
| each process map it itself? That's more efficient, because it doesn't
| require extra coordination.

When I ask, expect that I have checked out the "usual" way, and that either
that way doesn't work for some reason, or that I am already doing it that way
and am exploring to find another way.

If I describe some particular way to do something and ask if it is available
in some system, then I'm not interested in other ways, even though one of the
other ways might be the usual. If I describe what end result I want and ask
how to accomplish it, then any method to accomplish it is fair game.

If what I want to do cannot be done, then just say "no" or don't say. But do
not make assumptions about what other stuff I may or may not have explored.
If you are curious what I have explored, you can always ask. I may or may not
be interested in pursing that subthread.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> On Mon, 28 Apr 2008 14:25:48 -0700 (PDT) David Schwartz <davids@webmaster.com> wrote:
> | On Apr 28, 10:53 am, phil-news-nos...@ipal.net wrote:
> |
> |> | What's your outer problem? Why not just have the other process create
> |> | the mapping itself? The answer to your question would be very
> |> | different if the reason was that the mapping was private than it would
> |> | be if the problem was that the mapping contains pointers and needs to
> |> | be at the same address for all processes.
> |
> |> The idea is to have some of the work spread out between processes and avoid
> |> having to use threads (because I don't want to share the whole VM). There
> |> is a way to transfer (already open) file descriptors between processes. I
> |> was hoping there might be a similar way to transfer already mapped pages
> |> between processes. But if there is not, I will just have to do things some
> |> other way.
> |
> | You didn't answer my question. I still don't know why you don't just
> | have each process open the mapping itself. Is it because you need the
> | mappings to be at the same address? Is it because the mappings are
> | private? What is the problem that you think this would solve?
>
> There's no need to open the mapping twice. It's only needed one. The
> idea is for one process to go ahead and open and map the file and have
> it all ready, maybe even with a few pages referenced, and hand it over
> to the next process. That's what I do now with open descriptors. One
> process opens the descriptor and hands it over to another process to
> read from.
>[/color]
Well that's the normal way of doing things (passing an fd between
processes), is it causing problems? There shouldn't be much of a
performance hit from remapping the file.

--
Ian Collins.

On Apr 28, 2:44 pm, phil-news-nos...@ipal.net wrote:
[color=blue]
> I do purposely try to avoid answering questions that go off track of what I
> an seeking answers for. I used to not do that, and that would almost always
> result in a pointless drawn out thread with someone else trying to suggest
> ways to do things that would not really work because of many issues in the
> program, that I didn't post in the first place because they were not relevant
> to[/color]

You can't use USENET that way. It's just not fair.

If you ask a really unusual question without any discussion of why
it's unusual, and you get (maybe even by luck) exactly the right
answer, the exchange will help nobody but you. Worse, other people
looking at the exchange may think the answer is the normal or
appropriate way, though it's only right because of all your "many
issues in the program", which they don't even know exists.

If you want help from USENET, you have to explain your situation such
that it's, at a minimum, not misleading to people in the future who
hunt around for useful information.
[color=blue]
> | Any time you ask a question, expect the response "why don't you do it
> | the usual way?" and give the answer in the question. Why not just have
> | each process map it itself? That's more efficient, because it doesn't
> | require extra coordination.[/color]
[color=blue]
> When I ask, expect that I have checked out the "usual" way, and that either
> that way doesn't work for some reason, or that I am already doing it that way
> and am exploring to find another way.[/color]

Unfortunately, that doesn't work for two reasons. First, the time when
you haven't checked out the usual way, it will lead you further
astray. Second, people who need the more typical answer will be lead
astray by the exchange.

These are real, common, problems whose consequences have been learned
by hard and painful experience. You are asking for things that just
don't work on USENET.
[color=blue]
> If I describe some particular way to do something and ask if it is available
> in some system, then I'm not interested in other ways, even though one of the
> other ways might be the usual. If I describe what end result I want and ask
> how to accomplish it, then any method to accomplish it is fair game.[/color]

But you don't describe what end result you want. You describe how you
think you could accomplish that end result.

For example, in this thread, was the end result the mapping at the
same address in both processes or not?

The only way to know what result you wanted was to understand what you
were trying to accomplish. Otherwise, we'd have to tease through every
tedious permutation of how two processes could wind up with the "same
mapping".
[color=blue]
> If what I want to do cannot be done, then just say "no" or don't say. But do
> not make assumptions about what other stuff I may or may not have explored.
> If you are curious what I have explored, you can always ask. I may or may not
> be interested in pursing that subthread.[/color]

That's not a fair attitude on USENET. USENET is a give and take, and
you cannot refuse to give. Spend the extra ten minutes explaining your
problem. Not only will that help novices to better understand whether
the answers you get apply to them or not, you will get better answers.
Plus, you will spend less time explaining why the answers you get
aren't helpful or saying "I already tried X, that was obvious" because
you didn't make clear why X wasn't what you wanted.

And this post is not just for you. It's more for anyone reading this
thread who might think that your post serves as a good example of how
to ask a question.

This is probably the most frequently posted link on USENET:
[url]http://catb.org/~esr/faqs/smart-questions.html[/url]
See especially the section no describing the ultimate goal and the
section on being specific.

DS

[email]phil-news-nospam@ipal.net[/email] wrote:
[color=blue]
> There's no need to open the mapping twice. It's only needed one. The
> idea is for one process to go ahead and open and map the file and have
> it all ready, maybe even with a few pages referenced, and hand it over
> to the next process. That's what I do now with open descriptors. One
> process opens the descriptor and hands it over to another process to
> read from.[/color]

Why set up the mapping in the first process if it isn't going to
actually use it?

Setting up mappings is relatively expensive, so it should only be done
in the process that is going to use the mapping.

Chris

In article <fv5gf0520m3@news2.newsguy.com>, [email]phil-news-nospam@ipal.net[/email]
wrote:
[color=blue]
> I do purposely try to avoid answering questions that go off track of
> what I an seeking answers for. I used to not do that, and that would
> almost always result in a pointless drawn out thread with someone
> else trying to suggest ways to do things that would not really work
> because of many issues in the program, that I didn't post in the
> first place because they were not relevant to[/color]

If those issues prevent you from doing it the usual way, they ARE
relevant to the discussion. They're part of the problem you're trying
to solve.

You want to avoid having the discussion go off track? Then describe the
problem as fully as possible, the issues that restrict what you can do,
what methods you've rejected and why you've rejected them. Otherwise
people WILL waste time giving you solutions you've already rejected,
because you've given them no reason to believe it won't work.

On Tue, 29 Apr 2008 09:54:26 +1200 Ian Collins <ian-news@hotmail.com> wrote:

|> There's no need to open the mapping twice. It's only needed one. The
|> idea is for one process to go ahead and open and map the file and have
|> it all ready, maybe even with a few pages referenced, and hand it over
|> to the next process. That's what I do now with open descriptors. One
|> process opens the descriptor and hands it over to another process to
|> read from.
|>
| Well that's the normal way of doing things (passing an fd between
| processes), is it causing problems? There shouldn't be much of a
| performance hit from remapping the file.

I'm just exploring to see if there is something better. Threading should be
better, but I want to explore possible ways to avoid threading.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Mon, 28 Apr 2008 15:16:47 -0700 (PDT) David Schwartz <davids@webmaster.com> wrote:
| On Apr 28, 2:44 pm, phil-news-nos...@ipal.net wrote:
|
|> I do purposely try to avoid answering questions that go off track of what I
|> an seeking answers for. I used to not do that, and that would almost always
|> result in a pointless drawn out thread with someone else trying to suggest
|> ways to do things that would not really work because of many issues in the
|> program, that I didn't post in the first place because they were not relevant
|> to
|
| You can't use USENET that way. It's just not fair.
|
| If you ask a really unusual question without any discussion of why
| it's unusual, and you get (maybe even by luck) exactly the right
| answer, the exchange will help nobody but you. Worse, other people
| looking at the exchange may think the answer is the normal or
| appropriate way, though it's only right because of all your "many
| issues in the program", which they don't even know exists.
|
| If you want help from USENET, you have to explain your situation such
| that it's, at a minimum, not misleading to people in the future who
| hunt around for useful information.

If you want the whole can of worms, I'm writing a program to rapidly install
a system during the time the system is booting up. If you want to discuss
that whole thing, feel free to ... in another thread that I will not take
any part in. Until such time as I have completed the exploration of it, I
do don't plan to participate it what I suspect such a thread will become
(based on past Usenet experience). But if it interests you and others, go
right ahead on your own.

And you might wonder, if it is for Linux, why not ask in a Linux system
group. I'd rather find portable ways to do it in case I might ever want
to do the same thing with BSD.


|> | Any time you ask a question, expect the response "why don't you do it
|> | the usual way?" and give the answer in the question. Why not just have
|> | each process map it itself? That's more efficient, because it doesn't
|> | require extra coordination.
|
|> When I ask, expect that I have checked out the "usual" way, and that either
|> that way doesn't work for some reason, or that I am already doing it that way
|> and am exploring to find another way.
|
| Unfortunately, that doesn't work for two reasons. First, the time when
| you haven't checked out the usual way, it will lead you further
| astray. Second, people who need the more typical answer will be lead
| astray by the exchange.

If they misunderstood what I ask, and how narrow it is, I supposed so.
"How do I do X?" ... "To do X, there is method Y and also method Z".
Why would someone wanting to do something other than X try to apply
method Y or method Z?


| These are real, common, problems whose consequences have been learned
| by hard and painful experience. You are asking for things that just
| don't work on USENET.
|
|> If I describe some particular way to do something and ask if it is available
|> in some system, then I'm not interested in other ways, even though one of the
|> other ways might be the usual. If I describe what end result I want and ask
|> how to accomplish it, then any method to accomplish it is fair game.
|
| But you don't describe what end result you want. You describe how you
| think you could accomplish that end result.

That depends on the scope of what is being considered. It can be looked
at in a broad way or a narrow way. When you are considering just the
application, then what you are asking is what the application needs to
accomplish. What if no one else is doing that application? But if I am
asking the "how to" in a more narrow way ...


| For example, in this thread, was the end result the mapping at the
| same address in both processes or not?

No. Just that the mapped pages got transferred. They could have become
mapped into a different address. I did not confine the goal to include
keeping the same address. Clearly process B would need to know what the
addresses are. I can envision a number of ways to transfer mapped pages
between processes and by that, a number of ways for process B to find out
the addresses it got them at, or potentially even a way to specify where
to put them (options it has in the mmap call now).


| The only way to know what result you wanted was to understand what you
| were trying to accomplish. Otherwise, we'd have to tease through every
| tedious permutation of how two processes could wind up with the "same
| mapping".

If a mechanism had existed to do this, the specifics of that mechanism
would have been the answer. There would have been no pondering. If the
mechanism only supported inserting the mapping into process B at addresses
not yet mapped to anythingdetermined by the kernel, then whoever would
know about and suggest such a mechanism would not be pondering other ways
such as letting process B pick the addresses. If I has narrowed what I
was asking for to one where process B can pick the addresses, but the only
solution was one that did not, then that would not fit what I was asking.


|> If what I want to do cannot be done, then just say "no" or don't say. But do
|> not make assumptions about what other stuff I may or may not have explored.
|> If you are curious what I have explored, you can always ask. I may or may not
|> be interested in pursing that subthread.
|
| That's not a fair attitude on USENET. USENET is a give and take, and
| you cannot refuse to give. Spend the extra ten minutes explaining your
| problem. Not only will that help novices to better understand whether
| the answers you get apply to them or not, you will get better answers.
| Plus, you will spend less time explaining why the answers you get
| aren't helpful or saying "I already tried X, that was obvious" because
| you didn't make clear why X wasn't what you wanted.

It has been my experience that this approach has led to very long discourse,
often times rather heated; far more than this thread is even approaching.
I want to avoid that kind of thing.

OTOH, I do give in other ways. I do post answers to what other people ask.
Sometimes I do "guess" at what they are trying to do, and think maybe they
have overlooked a better path if they just back up one and go another way.
Maybe they didn't. If I am confident of my guess, I will make such a
suggestion. Most of the time my guesses have been wrong. Occaisionally
they are right. If I were to post such alternatives all the time, then I
would be more of a pest than I seem to be now.


| And this post is not just for you. It's more for anyone reading this
| thread who might think that your post serves as a good example of how
| to ask a question.
|
| This is probably the most frequently posted link on USENET:
| [url]http://catb.org/~esr/faqs/smart-questions.html[/url]
| See especially the section no describing the ultimate goal and the
| section on being specific.

Now my goal is to find the words to get people to not drag me unto subthreads
of discussion that get off the subject of what I was asking. And these need
to be polite enough that I can stick them in every initiated question I ask.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen <cbf123@mail.usask.ca> wrote:
| [email]phil-news-nospam@ipal.net[/email] wrote:
|
|> There's no need to open the mapping twice. It's only needed one. The
|> idea is for one process to go ahead and open and map the file and have
|> it all ready, maybe even with a few pages referenced, and hand it over
|> to the next process. That's what I do now with open descriptors. One
|> process opens the descriptor and hands it over to another process to
|> read from.
|
| Why set up the mapping in the first process if it isn't going to
| actually use it?

Well, if it can't transfer it to the second process, then there is no need to
map the file in the first process.


| Setting up mappings is relatively expensive, so it should only be done
| in the process that is going to use the mapping.

I wanted to overlap setting up the mapping with the work already taking place
in process B. Much of the work of setting up a mapping (like reading block
pointer records for the file) would not need to be repeated in such a transfer,
so the time delay in process B to merely accept the mapping from process A
means process B can get on to other things more quickly. On a slow machine,
it might not be any advantage. On a fast machine, maybe there would be.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris <wayne.morris@this.is.invalid> wrote:
| In article <fv5gf0520m3@news2.newsguy.com>, [email]phil-news-nospam@ipal.net[/email]
| wrote:
|
|> I do purposely try to avoid answering questions that go off track of
|> what I an seeking answers for. I used to not do that, and that would
|> almost always result in a pointless drawn out thread with someone
|> else trying to suggest ways to do things that would not really work
|> because of many issues in the program, that I didn't post in the
|> first place because they were not relevant to
|
| If those issues prevent you from doing it the usual way, they ARE
| relevant to the discussion. They're part of the problem you're trying
| to solve.
|
| You want to avoid having the discussion go off track? Then describe the
| problem as fully as possible, the issues that restrict what you can do,
| what methods you've rejected and why you've rejected them. Otherwise
| people WILL waste time giving you solutions you've already rejected,
| because you've given them no reason to believe it won't work.

That would be too long of a post.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> On Tue, 29 Apr 2008 03:38:45 GMT Wayne C. Morris <wayne.morris@this.is.invalid> wrote:[/color]
[color=blue]
> | You want to avoid having the discussion go off track? Then describe the
> | problem as fully as possible, the issues that restrict what you can do,
> | what methods you've rejected and why you've rejected them. Otherwise
> | people WILL waste time giving you solutions you've already rejected,
> | because you've given them no reason to believe it won't work.[/color]
[color=blue]
> That would be too long of a post.[/color]

It would be shorter than what you've now written in this long thread,
and everyone's time would have been saved.

Seriously, the above advice is excellent.

Chris

[email]phil-news-nospam@ipal.net[/email] wrote:[color=blue]
> On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen <cbf123@mail.usask.ca> wrote:[/color]
[color=blue]
> | Setting up mappings is relatively expensive, so it should only be done
> | in the process that is going to use the mapping.
>
> I wanted to overlap setting up the mapping with the work already taking place
> in process B. Much of the work of setting up a mapping (like reading block
> pointer records for the file) would not need to be repeated in such a transfer,
> so the time delay in process B to merely accept the mapping from process A
> means process B can get on to other things more quickly.[/color]

A large part of the cost of setting up a mapping is in the fact that the
tlb needs to be flushed (to force the cpu to see the new mapping). This
would need to be done in the second process no matter what since it is
in a separate memory map. This overhead (as well as the overhead of
transferring file descriptors via unix sockets) could be minimized by
simply using threads.

Chris

On Mon, 28 Apr 2008 23:08:54 -0600 Chris Friesen <cbf123@mail.usask.ca> wrote:
| [email]phil-news-nospam@ipal.net[/email] wrote:
|> On Mon, 28 Apr 2008 16:46:16 -0600 Chris Friesen <cbf123@mail.usask.ca> wrote:
|
|> | Setting up mappings is relatively expensive, so it should only be done
|> | in the process that is going to use the mapping.
|>
|> I wanted to overlap setting up the mapping with the work already taking place
|> in process B. Much of the work of setting up a mapping (like reading block
|> pointer records for the file) would not need to be repeated in such a transfer,
|> so the time delay in process B to merely accept the mapping from process A
|> means process B can get on to other things more quickly.
|
| A large part of the cost of setting up a mapping is in the fact that the
| tlb needs to be flushed (to force the cpu to see the new mapping). This
| would need to be done in the second process no matter what since it is
| in a separate memory map. This overhead (as well as the overhead of
| transferring file descriptors via unix sockets) could be minimized by
| simply using threads.

But isn't this a cost associated with context switching?

Yes, I know threads is an option, and is a good one with respect to this
aspect. I have other reasons to not use threads. One is that at the time
my program will run, no libraries are available. Conceivably, I could still
do "threads" via a clone() call. But I don't want to go into that territory
right now. So I'm exploring what options, few as they may be, exist for
processes.

--
|WARNING: Due to extreme spam, I no longer see any articles originating from |
| Google Groups. If you want your postings to be seen by more readers |
| you will need to find a different place to post on Usenet. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |