pthread and process space sharing---Linux

I have a multithreaded program (using pthread implementation), and if I
do a ps -aux, the systems show all the threads as processes (Linux maps
pthreads to kernel threads, which are counted as processes).

For example, my program has one main thread and two worker theads, the
'ps -aux' shows something like

pid memory cpu
84004 ... 10.2 0.5 ....
84005 ... 10.2 2.5.....
84006 ... 10.2 1.2.....

The above one shows that all three threads share the same address space
(which is also indicated in the same memory consumption). Is there any
way to find each thread's private memory consumption (private stack
size), so that I can further break 10.2% to 10.2%= (1) code and
bss/global size + (2)heap size + (3) stack of thread 1 + (4)
stack of thread 2 + (5) stack of thread 3 ?

though part (2) is not-determinable, if we can get the number of
(1)(3)(4)(5), itis also helpful to get a sense of how program behaves.
Any approach? thanks,

Sean wrote:[color=blue]
> I have a multithreaded program (using pthread implementation), and if I
> do a ps -aux, the systems show all the threads as processes (Linux maps
> pthreads to kernel threads, which are counted as processes).[/color]

Yep, this is a bug in 'ps'.
[color=blue]
> For example, my program has one main thread and two worker theads, the
> 'ps -aux' shows something like
>
> pid memory cpu
> 84004 ... 10.2 0.5 ....
> 84005 ... 10.2 2.5.....
> 84006 ... 10.2 1.2.....
>
> The above one shows that all three threads share the same address space
> (which is also indicated in the same memory consumption).[/color]

Yep. Unfortunately, 'ps' is not smart enough to know that two KSEs that
share a vm are not two processes.
[color=blue]
> Is there any
> way to find each thread's private memory consumption (private stack
> size), so that I can further break 10.2% to 10.2%= (1) code and
> bss/global size + (2)heap size + (3) stack of thread 1 + (4)
> stack of thread 2 + (5) stack of thread 3 ?[/color]

Thread's do not have private stacks. Threads share all their memory.
[color=blue]
> though part (2) is not-determinable, if we can get the number of
> (1)(3)(4)(5), itis also helpful to get a sense of how program behaves.
> Any approach? thanks,[/color]

Threads do not have private memory consumption. The number you are
looking for does not exist.

DS

David Schwartz wrote:[color=blue]
> Sean wrote:[color=green]
> > I have a multithreaded program (using pthread implementation), and if I
> > do a ps -aux, the systems show all the threads as processes (Linux maps
> > pthreads to kernel threads, which are counted as processes).[/color]
>
> Yep, this is a bug in 'ps'.
>[color=green]
> > For example, my program has one main thread and two worker theads, the
> > 'ps -aux' shows something like
> >
> > pid memory cpu
> > 84004 ... 10.2 0.5 ....
> > 84005 ... 10.2 2.5.....
> > 84006 ... 10.2 1.2.....
> >
> > The above one shows that all three threads share the same address space
> > (which is also indicated in the same memory consumption).[/color]
>
> Yep. Unfortunately, 'ps' is not smart enough to know that two KSEs that
> share a vm are not two processes.
>[color=green]
> > Is there any
> > way to find each thread's private memory consumption (private stack
> > size), so that I can further break 10.2% to 10.2%= (1) code and
> > bss/global size + (2)heap size + (3) stack of thread 1 + (4)
> > stack of thread 2 + (5) stack of thread 3 ?[/color]
>
> Thread's do not have private stacks. Threads share all their memory.
>[color=green]
> > though part (2) is not-determinable, if we can get the number of
> > (1)(3)(4)(5), itis also helpful to get a sense of how program behaves.
> > Any approach? thanks,[/color]
>
> Threads do not have private memory consumption. The number you are
> looking for does not exist.[/color]
their own stacks, don't they exist?
[color=blue]
>
> DS[/color]

On 9 Oct 2006 09:09:58 -0700, Sean <seanatpurdue@hotmail.com> wrote:[color=blue]
> David Schwartz wrote:[color=green]
>> Thread's do not have private stacks. Threads share all their memory.[/color]
>
> their own stacks, don't they exist?[/color]

What David here was referring to is probably the fact that the thread
stacks aren't exactly private, since all other threads can access them
at will. (This is usually a bad idea in practice, of course.)

Threads _do_ indeed have their own stack spaces set aside, and one
presumes that one could dig up the portion of that space that is in
use at a given point in time using a capable debugger, eg. gdb, though
I don't know how to accomplish that in practice. Just thought I'd clarify
the apparent terminology issue here anyways.

--
Mikko Rauhala - [email]mjr@iki.fi[/email] - <URL:http://www.iki.fi/mjr/>
Transhumanist - WTA member - <URL:http://www.transhumanism.org/>
Singularitarian - SIAI supporter - <URL:http://www.singinst.org/>

Sean wrote:
[color=blue]
> David Schwartz wrote:[/color]
[color=blue][color=green]
> > Threads do not have private memory consumption. The number you are
> > looking for does not exist.[/color][/color]
[color=blue]
> their own stacks, don't they exist?[/color]

Not in the sense that you mean it. Here's the proof:

#include "pthread.h"
volatile int *p;

void *thread1(void *ignored)
{
int v=0; // on thread1's stack
p=&v;
while(1)
{
printf("v=%d\n", v);
sleep(1);
}
}

void *thread2(void *ignored)
{
while(1)
{
(*p)++;
sleep(1);
}
}

int main(void)
{
pthread_t t1, t2;
pthread_create(&t1, NULL, thread1, NULL);
sleep(1);
pthread_create(&t2, NULL, thread2, NULL);
sleep(5);
}

This program is for example purposes only, it does break a rule because
it doesn't use a mutex to protect the shared variable. But the point is
to show that 'thread2' has no problem incrementing a variable on
'thread1's stack.

Threads share all their memory, including their stacks. Obviously, it
would cause chaos if one thread wrote unpredictably into another's
stack, but that's a gentleman's agreement. Threads can modify each
others' stacks when that makes sense.

DS

To be a bit more helpful, if you look at /proc/<pid>/maps, you will see
the mapping for each thread's stack. Exactly how it shows depends upon
what version of Linux and threading library you are using.

A thread can determine its own stack information by the ugly mechanism
of taking the address of a variable on its stack and then finding the
entry that address is in. The size of the mapping the address is in is
the maximum size the stack can grow to. How far a newly-created entry
is into that mapping will tell you how much stack is currently used.

Yuck.

DS

Mikko Rauhala wrote:[color=blue]
> On 9 Oct 2006 09:09:58 -0700, Sean <seanatpurdue@hotmail.com> wrote:[color=green]
> > David Schwartz wrote:[color=darkred]
> >> Thread's do not have private stacks. Threads share all their memory.[/color]
> >
> > their own stacks, don't they exist?[/color]
>
> What David here was referring to is probably the fact that the thread
> stacks aren't exactly private, since all other threads can access them
> at will. (This is usually a bad idea in practice, of course.)[/color]

There are useful algorithms which depend on one thread accessing
another thread's stack.

For instance blocking message passing:

{
message_t request, reply;
fill_in(&request);
if (sendrequest(other_thread, &request, &reply)) {
// process reply
}
}

The request and reply data structures get embroiled in a multi-threaded
IPC mechanism, which is fine since the thread is suspended in
sendrequest(), ensuring that the stack remains valid.

In the kernel, wait queue nodes are declared in automatic storage all
over the place.