On 2007-06-22, RezaRob wrote:
> Is there a way to generate a brief view of the stack without dumping
> entire core?

attach gdb to the process?

> How about generating a core file which is truncated to a
> given size?

using ulimit?

Bye.
Jasen

On Jun 22, 7:04 am, Jasen wrote:
> On 2007-06-22, RezaRob wrote:
>
> > Is there a way to generate a brief view of the stack without dumping
> > entire core?
>
> attach gdb to the process?

This isn't an option because it's in "production mode" and only in the
event of an exception I need a stack trace.

>
> > How about generating a core file which is truncated to a
> > given size?
>
> using ulimit?

When I use ulimit, instead of getting a truncated core, I seem to be
getting none at all. Am I doing something wrong?

Thanks a lot for replying Jasen.

Reza.

>
> Bye.
> Jasen

On Jun 22, 11:33 am, RezaRob wrote:
> On Jun 22, 7:04 am, Jasen wrote:
>
> > On 2007-06-22, RezaRob wrote:
>
> > > Is there a way to generate a brief view of the stack without dumping
> > > entire core?
>
> > attach gdb to the process?
>
> This isn't an option because it's in "production mode" and only in the
> event of an exception I need a stack trace.

The simplest solution is to 'fork' and have the child call 'abort' to
generate a core dump. There are more complex solutions.

If you dynamically link and keep symbols, you can 'fork' and have the
child dump its stack directly. The idea is to call
'__builtin_return_address' and then call 'dladdr' on the address you
get back.

Note that if you have a real exception, you can't rely on anything
working because your process environment might be corrupt. For
example, after you 'fork', you should probably set all fatal signal
handlers to their default in the child. (You don't want to invoke your
own stack dump routine if the stack dump routine faults!)

DS

On Jun 23, 2:33 am, RezaRob wrote:
> On Jun 22, 7:04 am, Jasen wrote:
>
> > On 2007-06-22, RezaRob wrote:
>
> > > Is there a way to generate a brief view of the stack without dumping
> > > entire core?
>
> > attach gdb to the process?
>
> This isn't an option because it's in "production mode" and only in the
> event of an exception I need a stack trace.
>
It is an option because you can attach gdb to yourself when some
exception occurs.

void segv_handler(int no)
{
int pid, status;

VoIPLog("segv_handler.");
char buf[128];

pid = getpid();

sprintf(buf, "/usr/bin/gdbserver :9988 --attach %d", pid);

pid = fork();
if (pid == 0) {
system(buf);
} else
::wait((int *)&status);

return;
}

On Jun 22, 3:50 pm, David Schwartz wrote:
> On Jun 22, 11:33 am, RezaRob wrote:
>
> > On Jun 22, 7:04 am, Jasen wrote:
>
> > > On 2007-06-22, RezaRob wrote:
>
> > > > Is there a way to generate a brief view of the stack without dumping
> > > > entire core?
>
> > > attach gdb to the process?
>
> > This isn't an option because it's in "production mode" and only in the
> > event of an exception I need a stack trace.
>
> The simplest solution is to 'fork' and have the child call 'abort' to
> generate a core dump. There are more complex solutions.
>
> If you dynamically link and keep symbols, you can 'fork' and have the
> child dump its stack directly. The idea is to call
> '__builtin_return_address' and then call 'dladdr' on the address you
> get back.
>
> Note that if you have a real exception, you can't rely on anything
> working because your process environment might be corrupt. For
> example, after you 'fork', you should probably set all fatal signal
> handlers to their default in the child. (You don't want to invoke your
> own stack dump routine if the stack dump routine faults!)
>
> DS

Sorry I didn't clarify this: It's okay for the main process to be
killed; however, it's core may be gigabytes huge, and I only want a
small "stack trace."

Really, I want some kind of "oobs!" message, like the one the kernel
dumps. I guess a signal handler can just copy the stack into a "core"
file, but then, how do you prevent gdb from complaining "this is not a
core file?"

Thank you so much for your responses.

Reza.

On Jun 22, 8:58 pm, RezaRob wrote:

> Sorry I didn't clarify this: It's okay for the main process to be
> killed; however, it's core may be gigabytes huge, and I only want a
> small "stack trace."

Did you read the part about using __builtin_return_address and dladdr?

> Really, I want some kind of "oobs!" message, like the one the kernel
> dumps. I guess a signal handler can just copy the stack into a "core"
> file, but then, how do you prevent gdb from complaining "this is not a
> core file?"

Can you ship an executable with symbols that's dynamically linked? In
that case, just call 'dladdr'.

DS

On Jun 23, 6:05 pm, David Schwartz wrote:
> On Jun 22, 8:58 pm,RezaRob wrote:
>
> > Sorry I didn't clarify this: It's okay for the main process to be
> > killed; however, it's core may be gigabytes huge, and I only want a
> > small "stack trace."
>
> Did you read the part about using __builtin_return_address and dladdr?

Actually, not carefully. Sorry.

>
> > Really, I want some kind of "oobs!" message, like the one the kernel
> > dumps. I guess a signal handler can just copy the stack into a "core"
> > file, but then, how do you prevent gdb from complaining "this is not a
> > core file?"
>
> Can you ship an executable with symbols that's dynamically linked? In
> that case, just call 'dladdr'.

I would prefer not to do that. I wonder, is that how mozilla does
that "do you want to send crash info" thing?

If I just dump the stack, is there any way to tell gdb to reconstruct
the thing after the fact? For instance, can gcc generate a separate
file similar to the "system.map" file that the kernel uses?

I really appreciate your suggestions and help :-)

Reza.

On Jun 23, 11:43 pm, RezaRob wrote:

> If I just dump the stack, is there any way to tell gdb to reconstruct
> the thing after the fact? For instance, can gcc generate a separate
> file similar to the "system.map" file that the kernel uses?

I don't see what you would want 'gdb' for at that point. If you must
do it that way, just write the stack to a file using
__builtin_return_address. It may help to also write out a well-known
address in case the executable has been relocated. You can them
compare those addresses to a symbol table.

Compile/link the executable with symbols. Run 'nm' on the executable
and save the output. Then strip and ship. If you get a dump file, just
compare the addresses in the file to the ones in your 'nm' output.

Are you shipping a dynamically-linked or a statically-linked
executable? If dynamically-linked, I would still try to 'dlsym' the
addresses, because if they're in a library on that user's machine,
your symbol table won't help you -- you need the symbol table for the
library on the customer's machine.

DS

On Jun 24, 12:44 pm, David Schwartz wrote:
> On Jun 23, 11:43 pm,RezaRob wrote:
>
> > If I just dump the stack, is there any way to tell gdb to reconstruct
> > the thing after the fact? For instance, can gcc generate a separate
> > file similar to the "system.map" file that the kernel uses?
>
> I don't see what you would want 'gdb' for at that point. If you must
> do it that way, just write the stack to a file using
> __builtin_return_address. It may help to also write out a well-known
> address in case the executable has been relocated. You can them
> compare those addresses to a symbol table.

Good point about relocation :-) If it helps solve such a potential
issue it's great.

>
> Compile/link the executable with symbols. Run 'nm' on the executable
> and save the output. Then strip and ship. If you get a dump file, just
> compare the addresses in the file to the ones in your 'nm' output.

I wanted a stack dump + gdb solution, because that can also give me
_some_ idea about the specific function parameters that initiated the
crash. This can help greatly with debugging, and acts more similar to
a kernel oobs, i.e. a very brief "snapshot" of what was happening,
without the disturbing noise.

>
> Are you shipping a dynamically-linked or a statically-linked
> executable? If dynamically-linked, I would still try to 'dlsym' the
> addresses, because if they're in a library on that user's machine,
> your symbol table won't help you -- you need the symbol table for the
> library on the customer's machine.

I'm not concerned about the user's library being buggy. "gdb" or
whatever can just give me a raw address for unknown symbols, as long
as I can get the general pattern of what _my_ code was up to at the
time. So that it gives me a starting point to think about :-)

Thanks again.

Reza.

On 2007-06-23, Bin Chen wrote:
> On Jun 23, 2:33 am, RezaRob wrote:
>> On Jun 22, 7:04 am, Jasen wrote:
>>
>> > On 2007-06-22, RezaRob wrote:
>>
>> > > Is there a way to generate a brief view of the stack without dumping
>> > > entire core?
>>
>> > attach gdb to the process?
>>
>> This isn't an option because it's in "production mode" and only in the
>> event of an exception I need a stack trace.
>>
> It is an option because you can attach gdb to yourself when some
> exception occurs.
[...]
> sprintf(buf, "/usr/bin/gdbserver :9988 --attach %d", pid);
>
> pid = fork();
> if (pid == 0) {
> system(buf);


Well. I've never used the gdbserver, but using regular gdb the whole
thing is possible. Generally you want to install signal handler that
does

echo "bt\ndetach\nquit\n" | gdb -batch -x /dev/stdin path_to_file pid

Then you need to catch the output from gdb.

One can use this simple piece of code for that

void
write_stack(int no)
{
char path[4096];
char str[256];
path[readlink("/proc/self/exe", path, -1 + sizeof(path))] = 0;
sprintf(str,
"echo 'bt\ndetach\nquit\n' | gdb -batch -x /dev/stdin %s %d",
path, getpid());
system(str);
}

If you catch stdout in some file already, then you'll get it right
there.

There's a catch here though. At least in my case. Somehow I was getting
only first line of the stack trace and nothing else. So I had to get rid
of "system" call and arrange piping of the input and output thru pipes
between processes.

--
Minds, like parachutes, function best when open