Extracting data symbols from an archive using dlsym - Unix

This is a discussion on Extracting data symbols from an archive using dlsym - Unix ; Hi Everybody, I have declared a static variable in a header file of a library that times certain routines in the library. My application links to this library. Before terminating, I want to print the value of this variable. The ...

+ Reply to Thread
Results 1 to 12 of 12

Thread: Extracting data symbols from an archive using dlsym

  1. Extracting data symbols from an archive using dlsym

    Hi Everybody,

    I have declared a static variable in a header file of a library that
    times certain routines in the library. My application links to this
    library. Before terminating, I want to print the value of this
    variable. The library is in archive format. When I try using dlopen on
    this library on Linux, I get the error, "library: invalid ELF header".

    Does anybody have any idea, how to extract symbols from an archive
    format?

    Or does anybody have any other ideas about how to collect the timing
    of routines in the library in a better way?

    I would really appreciate any help and suggestion that anybody could
    offer,

    Warm Regards,
    Christina.

  2. Re: Extracting data symbols from an archive using dlsym

    On Dec 3, 8:50 am, Christina wrote:
    > Hi Everybody,
    >
    > I have declared a static variable in a header file of a library that
    > times certain routines in the library. My application links to this
    > library. Before terminating, I want to print the value of this
    > variable. The library is in archive format. When I try using dlopen on
    > this library on Linux, I get the error, "library: invalid ELF header".
    >
    > Does anybody have any idea, how to extract symbols from an archive
    > format?
    >
    > Or does anybody have any other ideas about how to collect the timing
    > of routines in the library in a better way?


    I don't really understand what you're trying to do. Can you post an
    example?

  3. Re: Extracting data symbols from an archive using dlsym

    Christina writes:
    >Hi Everybody,
    >
    >I have declared a static variable in a header file of a library that
    >times certain routines in the library. My application links to this
    >library. Before terminating, I want to print the value of this
    >variable. The library is in archive format. When I try using dlopen on
    >this library on Linux, I get the error, "library: invalid ELF header".
    >
    >Does anybody have any idea, how to extract symbols from an archive
    >format?
    >
    >Or does anybody have any other ideas about how to collect the timing
    >of routines in the library in a better way?
    >
    >I would really appreciate any help and suggestion that anybody could
    >offer,
    >
    >Warm Regards,
    >Christina.


    If it is a static archive, and is linked with your application, simply
    declare the variable 'extern' and access it directly.

    scott

  4. Re: Extracting data symbols from an archive using dlsym

    I tried declaring it extern. But it is not able to execute. That is
    why I tried going for the dlopen/dlsym approach. I think the
    components of the archive are dynamic, it is not static. I know that
    in AIX, members of a dynamic archive can be obtained using dlopen/
    dlsym. But I am not sure that it can be done in Linux.

    Regards,
    Christina.

    On Dec 3, 3:02 pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
    > Christina writes:
    > >Hi Everybody,

    >
    > >I have declared a static variable in a header file of a library that
    > >times certain routines in the library. My application links to this
    > >library. Before terminating, I want to print the value of this
    > >variable. The library is in archive format. When I try using dlopen on
    > >this library on Linux, I get the error, "library: invalid ELF header".

    >
    > >Does anybody have any idea, how to extract symbols from an archive
    > >format?

    >
    > >Or does anybody have any other ideas about how to collect the timing
    > >of routines in the library in a better way?

    >
    > >I would really appreciate any help and suggestion that anybody could
    > >offer,

    >
    > >Warm Regards,
    > >Christina.

    >
    > If it is a static archive, and is linked with your application, simply
    > declare the variable 'extern' and access it directly.
    >
    > scott- Hide quoted text -
    >
    > - Show quoted text -



  5. Re: Extracting data symbols from an archive using dlsym

    I know the question is a bit unclear. But it is difficult for me to
    write sample code. I'll try to explain it as well as I can.

    I have a library code and a header file in that code. A routine in the
    code is called repetitively. I want the cumulative time spent by a
    function that is called within that routine that is part of another
    library. So I inserted timing calls around that routine. At the end of
    the program run, I want to see the value of this variable.

    Eg:
    -------- lib.h file ---------
    static double tvar = 0.0; <== newly introduced var

    -------- lib.c file ---------
    library_routine() {
    .....

    stime = timing_routine();
    third_party_lib_routine();
    etime = timing_routine();
    tvar += etime - stime;
    .....
    }
    ---------- my src code --------
    main() {
    Access the value of tvar;
    }

    I was unable to access the value of 'tvar' using extern and the
    library is part of an archive. So dlopen on the archive is not
    working.

    I hope I was able to expain the problem.

    Regards,
    Christina.

    On Dec 3, 2:26 pm, fjbl...@yahoo.com wrote:
    > On Dec 3, 8:50 am, Christina wrote:
    >
    > > Hi Everybody,

    >
    > > I have declared a static variable in a header file of a library that
    > > times certain routines in the library. My application links to this
    > > library. Before terminating, I want to print the value of this
    > > variable. The library is in archive format. When I try using dlopen on
    > > this library on Linux, I get the error, "library: invalid ELF header".

    >
    > > Does anybody have any idea, how to extract symbols from an archive
    > > format?

    >
    > > Or does anybody have any other ideas about how to collect the timing
    > > of routines in the library in a better way?

    >
    > I don't really understand what you're trying to do. Can you post an
    > example?



  6. Re: Extracting data symbols from an archive using dlsym

    On Dec 4, 9:35 am, Christina wrote:
    > I know the question is a bit unclear. But it is difficult for me to
    > write sample code. I'll try to explain it as well as I can.
    >
    > I have a library code and a header file in that code. A routine in the
    > code is called repetitively. I want the cumulative time spent by a
    > function that is called within that routine that is part of another
    > library. So I inserted timing calls around that routine. At the end of
    > the program run, I want to see the value of this variable.
    >
    > Eg:
    > -------- lib.h file ---------
    > static double tvar = 0.0; <== newly introduced var
    >
    > -------- lib.c file ---------
    > library_routine() {
    > ....
    >
    > stime = timing_routine();
    > third_party_lib_routine();
    > etime = timing_routine();
    > tvar += etime - stime;
    > ....}
    >
    > ---------- my src code --------
    > main() {
    > Access the value of tvar;
    >
    > }
    >
    > I was unable to access the value of 'tvar' using extern and the
    > library is part of an archive. So dlopen on the archive is not
    > working.


    Ok, I understand now.

    I'm guessing you originally had `double tvar = 0.0;' in lib.h, and it
    didn't compile, so you added `static' for no particular reason. Now
    it compiles but doesn't work.

    Using `static' creates a variable which is visible only to the current
    "compilation unit" (normally a single source file, as seen after
    preprocessing, and therefore including all header files). So what
    this does is to create a *different* variable named `tvar' for every
    source file. The `tvar' that the library sets is different from the
    one that the main program reads, so the main program never sees the
    change.

    You really want tvar to be a single global variable. The reason it
    didn't compile at first is because the compiler sees a definition,
    with initialization, for tvar in every source file that includes
    lib.h. It doesn't allow multiple initializations because it can't
    check that they are all initializing to the same value, and it would
    certainly be an error if they were different.

    The correct way to do this is to have lib.h contain only a
    *declaration* of the variable:

    ---lib.h---
    extern double tvar;

    and then to have a *single* library file contain the *definition*:

    ---lib.c---
    double tvar = 0.0;
    /* more code */

    Then you can just access tvar in your program like any other global
    variable.

    Forget about dlopen, it doesn't do what you seem to think it does, and
    won't help you here.

  7. Re: Extracting data symbols from an archive using dlsym

    I have managed to compile the code and am aware of the connotations of
    static. I can change the code to have a single definition.

    What I was looking for was whether we can dlopen an archive in Linux?
    This is because using extern is not giving me linkage to tvar for some
    reason.

    Regards,
    Christina.

    On Dec 4, 2:26 pm, fjbl...@yahoo.com wrote:
    > On Dec 4, 9:35 am, Christina wrote:
    >
    >
    >
    >
    >
    > > I know the question is a bit unclear. But it is difficult for me to
    > > write sample code. I'll try to explain it as well as I can.

    >
    > > I have a library code and a header file in that code. A routine in the
    > > code is called repetitively. I want the cumulative time spent by a
    > > function that is called within that routine that is part of another
    > > library. So I inserted timing calls around that routine. At the end of
    > > the program run, I want to see the value of this variable.

    >
    > > Eg:
    > > -------- lib.h file ---------
    > > static double tvar = 0.0; <== newly introduced var

    >
    > > -------- lib.c file ---------
    > > library_routine() {
    > > ....

    >
    > > stime = timing_routine();
    > > third_party_lib_routine();
    > > etime = timing_routine();
    > > tvar += etime - stime;
    > > ....}

    >
    > > ---------- my src code --------
    > > main() {
    > > Access the value of tvar;

    >
    > > }

    >
    > > I was unable to access the value of 'tvar' using extern and the
    > > library is part of an archive. So dlopen on the archive is not
    > > working.

    >
    > Ok, I understand now.
    >
    > I'm guessing you originally had `double tvar = 0.0;' in lib.h, and it
    > didn't compile, so you added `static' for no particular reason. Now
    > it compiles but doesn't work.
    >
    > Using `static' creates a variable which is visible only to the current
    > "compilation unit" (normally a single source file, as seen after
    > preprocessing, and therefore including all header files). So what
    > this does is to create a *different* variable named `tvar' for every
    > source file. The `tvar' that the library sets is different from the
    > one that the main program reads, so the main program never sees the
    > change.
    >
    > You really want tvar to be a single global variable. The reason it
    > didn't compile at first is because the compiler sees a definition,
    > with initialization, for tvar in every source file that includes
    > lib.h. It doesn't allow multiple initializations because it can't
    > check that they are all initializing to the same value, and it would
    > certainly be an error if they were different.
    >
    > The correct way to do this is to have lib.h contain only a
    > *declaration* of the variable:
    >
    > ---lib.h---
    > extern double tvar;
    >
    > and then to have a *single* library file contain the *definition*:
    >
    > ---lib.c---
    > double tvar = 0.0;
    > /* more code */
    >
    > Then you can just access tvar in your program like any other global
    > variable.
    >
    > Forget about dlopen, it doesn't do what you seem to think it does, and
    > won't help you here.- Hide quoted text -
    >
    > - Show quoted text -



  8. Re: Extracting data symbols from an archive using dlsym

    Christina writes:
    >I have managed to compile the code and am aware of the connotations of
    >static. I can change the code to have a single definition.
    >
    >What I was looking for was whether we can dlopen an archive in Linux?
    >This is because using extern is not giving me linkage to tvar for some
    >reason.
    >


    How about providing the error message you are getting presumably from
    the linker?

    You may also need to specify the archive more than once to the linker
    depending on how many passes it makes over the .a. Unix 'ld' would
    never read a .a twice, so if the reference occurred in a .o/.a which
    was read by the linker after your .a, then the reference would be missed.

    I don't know if binutils (gnu ld) has similar behavior.

    dlopen cannot be used on an archive library on linux or any other
    unix except, apparently, aix.

    scott

  9. Re: Extracting data symbols from an archive using dlsym

    Christina writes:

    > I have managed to compile the code and am aware of the connotations of
    > static. I can change the code to have a single definition.
    >
    > What I was looking for was whether we can dlopen an archive in
    > Linux?


    I can't answer this, but I don't think it matters...

    > This is because using extern is not giving me linkage to tvar for some
    > reason.


    It seems to me you should push to find out why this relatively
    ordinary operation (linking to an external variable defined in a
    library) is not working as you expect. It is a common usage and
    should be sorted out rather than try to find exotic fixes.

    As usual, a small example would probably help clarify matters.

    --
    Ben.

  10. Re: Extracting data symbols from an archive using dlsym

    On Dec 5, 10:48 am, Christina wrote:
    > I have managed to compile the code and am aware of the connotations of
    > static. I can change the code to have a single definition.
    >
    > What I was looking for was whether we can dlopen an archive in Linux?
    > This is because using extern is not giving me linkage to tvar for some
    > reason.


    No, you can't. dlopen opens a *dynamic* library. Furthermore, using
    dlopen would load a new copy, which would be independent of the copy
    you linked against. For instance, if you call a library function
    foo() which sets a variable `bar', calling dlsym("bar") will return a
    pointer to a different variable which isn't the one that was set.

    I think you are seriously confused about how linking works. If "using
    extern is not giving you linkage to tvar for some reason", I think you
    had better figure out why that is. It's almost certainly something
    you're doing wrong, because what I described in my previous post is
    supposed to work; either that or your compilation environment is badly
    messed up. I can't imagine any way that manually loading a library
    with dlopen() could help with the sort of thing you mention; it's
    completely irrelevant.

  11. Re: Extracting data symbols from an archive using dlsym

    fjblurt@yahoo.com writes:
    >On Dec 5, 10:48 am, Christina wrote:
    >> I have managed to compile the code and am aware of the connotations of
    >> static. I can change the code to have a single definition.
    >>
    >> What I was looking for was whether we can dlopen an archive in Linux?
    >> This is because using extern is not giving me linkage to tvar for some
    >> reason.

    >
    >No, you can't. dlopen opens a *dynamic* library.


    > Furthermore, using
    >dlopen would load a new copy, which would be independent of the copy
    >you linked against.


    Not always. dlopen(NULL) will allow access to the main program
    and all libraries loaded up to that point.

    However, you are correct in that dlopen() is pretty useless with an
    archive library.

    scott


  12. Re: Extracting data symbols from an archive using dlsym

    On Dec 6, 4:51 pm, sc...@slp53.sl.home (Scott Lurndal) wrote:
    > fjbl...@yahoo.com writes:
    > >On Dec 5, 10:48 am, Christina wrote:
    > >> I have managed to compile the code and am aware of the connotations of
    > >> static. I can change the code to have a single definition.

    >
    > >> What I was looking for was whether we can dlopen an archive in Linux?
    > >> This is because using extern is not giving me linkage to tvar for some
    > >> reason.

    >
    > >No, you can't. dlopen opens a *dynamic* library.
    > > Furthermore, using
    > >dlopen would load a new copy, which would be independent of the copy
    > >you linked against.

    >
    > Not always. dlopen(NULL) will allow access to the main program
    > and all libraries loaded up to that point.
    >
    > However, you are correct in that dlopen() is pretty useless with an
    > archive library.
    >
    > scott


    I have opened libraries before using dlopen(), but I was never aware
    that it will open a new copy of the library. I have never had any
    problem accessing the values of variables before. But I think the
    problem is that I used static keyword and hence there are multiple
    copies of the variable in the library. Also, one of the posts mentions
    that the linker does not read the archive more than once. At this
    point I need to do a whole build again to test this and I got around
    the problem by printing the value of the variable in a temporary file
    and then seeing the value of the variable after my program runs. I
    guess I was trying to do fancy things

    Thanks,
    Christina.

+ Reply to Thread