Itanium porting question - VMS

This is a discussion on Itanium porting question - VMS ; This is a second attempt to post this question. My news server sometimes fails to post messages and I've waited 4 hours for it to appears. My apologies if it shows up twice. I've been called in to help with ...

+ Reply to Thread
Results 1 to 7 of 7

Thread: Itanium porting question

  1. Itanium porting question

    This is a second attempt to post this question. My news server sometimes
    fails to post messages and I've waited 4 hours for it to appears. My apologies
    if it shows up twice.

    I've been called in to help with a port to Itanium on some software that I was
    the prime developer for many many years. I haven't touched a VMS system in
    4 years, plus I only worked part time on VMS for the previous 6 years. So here
    goes.

    The application is a commerical tool sold into the VMS marketplace. It supports
    many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
    customers have all databases, let alone even one database. In order to make
    one set of images work for all customers we would go through the following
    generalized build procedure:

    1. Compile all source code
    2. Link executable images with all databases up and running.
    3. Shut down database A.
    4. Link again, and note all unresolved symbols.
    5. Create shareable image(s) with stub routines containing entry points from step 4.
    6. Redefine logicals relevant to database A to point to shareable image from stop 5
    7. Run and test software with database A missing
    8. Repeat steps 3 through 7 for every database system.
    9. At this point we now have an executable that can run with or without various
    databases being present.

    Notes:
    1. All databases used shareable images.
    2. During startup time, DCL procedures would look to see which databases were
    present and define logicals to point to stub images for missing databases.

    This general build process would work for almost all databases on Vax and Alpha.
    There were some strange things regarding CDD that I don't recall right now.

    The developers involved in the port are telling me that this general process no longer
    works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
    get the exact error message from them.)

    Any idea if this general approach should still work, or do we need to look at a
    different method?

    --
    Posted via a free Usenet account from http://www.teranews.com


  2. Re: Itanium porting question

    Randy Park wrote:
    > This is a second attempt to post this question. My news server sometimes
    > fails to post messages and I've waited 4 hours for it to appears. My apologies
    > if it shows up twice.
    >
    > I've been called in to help with a port to Itanium on some software that I was
    > the prime developer for many many years. I haven't touched a VMS system in
    > 4 years, plus I only worked part time on VMS for the previous 6 years. So here
    > goes.
    >
    > The application is a commerical tool sold into the VMS marketplace. It supports
    > many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
    > customers have all databases, let alone even one database. In order to make
    > one set of images work for all customers we would go through the following
    > generalized build procedure:
    >
    > 1. Compile all source code
    > 2. Link executable images with all databases up and running.
    > 3. Shut down database A.
    > 4. Link again, and note all unresolved symbols.
    > 5. Create shareable image(s) with stub routines containing entry points from step 4.
    > 6. Redefine logicals relevant to database A to point to shareable image from stop 5
    > 7. Run and test software with database A missing
    > 8. Repeat steps 3 through 7 for every database system.
    > 9. At this point we now have an executable that can run with or without various
    > databases being present.
    >
    > Notes:
    > 1. All databases used shareable images.
    > 2. During startup time, DCL procedures would look to see which databases were
    > present and define logicals to point to stub images for missing databases.
    >
    > This general build process would work for almost all databases on Vax and Alpha.
    > There were some strange things regarding CDD that I don't recall right now.
    >
    > The developers involved in the port are telling me that this general process no longer
    > works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
    > get the exact error message from them.)
    >
    > Any idea if this general approach should still work, or do we need to look at a
    > different method?


    Step 5 is problematic on a VMS system any version, if you are intending
    the same binary image to work with either your stub or with the real
    shared image from the database vendor.

    For this, not only must you have all the symbols that are in the shared
    image in the stub, you must also have them in the same place in the
    symbol vector. And with out cooperation from the database vendor, it is
    hard to get sufficient information to get it right.

    You will also run into problems where a vendor obsoletes a routine, and
    puts the replacement into a different location in the transfer vector.

    Otherwise, you have to link your application at the customer site, and
    if they ever add a database, you must relink your application to it.
    And if you are doing that, there is no need for your stubs to be shared
    images.

    My guess is that in the past someone worked out the minimum transfer
    vectors for VAX and ALPHA images in right order on VAX and ALPHA, and
    with Itanium as either the ELF format is too different, or one or more
    of the database vendors has reorganized how the symbol vector is set up.

    The hack you were using was high risk and high maintenance before, and
    Itanium has exposed it.

    A recommendation:

    Find out if the shared images for each database that you are currently
    linking against can be loaded through LIB$FIND_IMAGE_SYMBOL of dlsym()
    depending on the language used. If so, stop linking against them, and
    dynamically load them as needed.

    [Take a look at how Python on VMS supports various databases]

    If they do not support direct dynamic loading, then it may be possible
    to put wrappers around them that you can dynamically load.

    This way you do not have to provide stub routines, and you do not have
    to go through such a complex build procedure.

    -John
    wb8tyw@qsl.net
    Personal Opinion Only

  3. Re: Itanium porting question

    On Sat, 25 Aug 2007 13:59:47 GMT, "John E. Malmberg" wrote:

    >Randy Park wrote:
    >> This is a second attempt to post this question. My news server sometimes
    >> fails to post messages and I've waited 4 hours for it to appears. My apologies
    >> if it shows up twice.
    >>
    >> I've been called in to help with a port to Itanium on some software that I was
    >> the prime developer for many many years. I haven't touched a VMS system in
    >> 4 years, plus I only worked part time on VMS for the previous 6 years. So here
    >> goes.
    >>
    >> The application is a commerical tool sold into the VMS marketplace. It supports
    >> many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
    >> customers have all databases, let alone even one database. In order to make
    >> one set of images work for all customers we would go through the following
    >> generalized build procedure:
    >>
    >> 1. Compile all source code
    >> 2. Link executable images with all databases up and running.
    >> 3. Shut down database A.
    >> 4. Link again, and note all unresolved symbols.
    >> 5. Create shareable image(s) with stub routines containing entry points from step 4.
    >> 6. Redefine logicals relevant to database A to point to shareable image from stop 5
    >> 7. Run and test software with database A missing
    >> 8. Repeat steps 3 through 7 for every database system.
    >> 9. At this point we now have an executable that can run with or without various
    >> databases being present.
    >>
    >> Notes:
    >> 1. All databases used shareable images.
    >> 2. During startup time, DCL procedures would look to see which databases were
    >> present and define logicals to point to stub images for missing databases.
    >>
    >> This general build process would work for almost all databases on Vax and Alpha.
    >> There were some strange things regarding CDD that I don't recall right now.
    >>
    >> The developers involved in the port are telling me that this general process no longer
    >> works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
    >> get the exact error message from them.)
    >>
    >> Any idea if this general approach should still work, or do we need to look at a
    >> different method?

    >
    >Step 5 is problematic on a VMS system any version, if you are intending
    >the same binary image to work with either your stub or with the real
    >shared image from the database vendor.
    >
    >For this, not only must you have all the symbols that are in the shared
    >image in the stub, you must also have them in the same place in the
    >symbol vector. And with out cooperation from the database vendor, it is
    >hard to get sufficient information to get it right.
    >
    >You will also run into problems where a vendor obsoletes a routine, and
    >puts the replacement into a different location in the transfer vector.
    >
    >Otherwise, you have to link your application at the customer site, and
    >if they ever add a database, you must relink your application to it.
    >And if you are doing that, there is no need for your stubs to be shared
    >images.
    >
    >My guess is that in the past someone worked out the minimum transfer
    >vectors for VAX and ALPHA images in right order on VAX and ALPHA, and
    >with Itanium as either the ELF format is too different, or one or more
    >of the database vendors has reorganized how the symbol vector is set up.
    >
    >The hack you were using was high risk and high maintenance before, and
    >Itanium has exposed it.
    >
    >A recommendation:
    >
    >Find out if the shared images for each database that you are currently
    >linking against can be loaded through LIB$FIND_IMAGE_SYMBOL of dlsym()
    >depending on the language used. If so, stop linking against them, and
    >dynamically load them as needed.
    >
    >[Take a look at how Python on VMS supports various databases]
    >
    >If they do not support direct dynamic loading, then it may be possible
    >to put wrappers around them that you can dynamically load.
    >
    >This way you do not have to provide stub routines, and you do not have
    >to go through such a complex build procedure.
    >
    >-John
    >wb8tyw@qsl.net
    >Personal Opinion Only


    This method has worked for many versions of VMS over the last 15 years
    because the stub routines are never called. They are there so that the
    image activator will resolve the symbols.

    The software is in low to no maintenance mode and changing the code to
    support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
    all the databases with very little return on the investment. If we have
    to link on site during initial installation and relink when their database
    environment changes then that's what we will do.

    It would be nice to understand what has changed from vax/alpha to
    itanium and why our approach no longer works.

    --
    Posted via a free Usenet account from http://www.teranews.com


  4. Re: Itanium porting question

    rvfulltime wrote:

    >
    > This method has worked for many versions of VMS over the last 15 years
    > because the stub routines are never called. They are there so that the
    > image activator will resolve the symbols.


    On VAX and Alpha, the image activator does not resolve the symbols, it
    uses the offset coded into the main image to to access a vector in the
    shared image. It then provides fixups to the offsets to match where the
    image actually got loaded.

    It is the offset that absolutely must be the same in the stub and the
    real shared image.

    Inside the shared image, effectively there is a giant branch table that
    jumps to the start of where the routines actually are.

    Since you were just using the stub to keep the image activator happy and
    not actually calling the routines in the stub, or if all the routines
    ignored their parameters and just returned the same value, then you
    could get away with the hack on VAX and ALPHA.

    > The software is in low to no maintenance mode and changing the code to
    > support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
    > all the databases with very little return on the investment. If we have
    > to link on site during initial installation and relink when their database
    > environment changes then that's what we will do.
    >
    > It would be nice to understand what has changed from vax/alpha to
    > itanium and why our approach no longer works.


    Itanium uses the ELF image format. And that probably has changed a lot
    on how the image activator works. Someone more familiar with the image
    activator would probably be able to give some details.

    -John
    wb8tyw@qsl.network
    Personal Opinion Only

  5. Re: Itanium porting question

    On Sat, 25 Aug 2007 17:09:35 GMT, "John E. Malmberg" wrote:

    >rvfulltime wrote:
    >
    >>
    >> This method has worked for many versions of VMS over the last 15 years
    >> because the stub routines are never called. They are there so that the
    >> image activator will resolve the symbols.

    >
    >On VAX and Alpha, the image activator does not resolve the symbols, it
    >uses the offset coded into the main image to to access a vector in the
    >shared image. It then provides fixups to the offsets to match where the
    >image actually got loaded.
    >
    >It is the offset that absolutely must be the same in the stub and the
    >real shared image.
    >
    >Inside the shared image, effectively there is a giant branch table that
    >jumps to the start of where the routines actually are.
    >
    >Since you were just using the stub to keep the image activator happy and
    >not actually calling the routines in the stub, or if all the routines
    >ignored their parameters and just returned the same value, then you
    >could get away with the hack on VAX and ALPHA.
    >
    >> The software is in low to no maintenance mode and changing the code to
    >> support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
    >> all the databases with very little return on the investment. If we have
    >> to link on site during initial installation and relink when their database
    >> environment changes then that's what we will do.
    >>
    >> It would be nice to understand what has changed from vax/alpha to
    >> itanium and why our approach no longer works.

    >
    >Itanium uses the ELF image format. And that probably has changed a lot
    >on how the image activator works. Someone more familiar with the image
    >activator would probably be able to give some details.
    >
    >-John
    >wb8tyw@qsl.network
    >Personal Opinion Only


    I was misusing the term "symbol resolution". And yes, the hack we were
    performing was done simply to keep the image activator from complaining.
    Maybe someone else will read this and tell me about a different hack.

    --
    Posted via a free Usenet account from http://www.teranews.com


  6. Re: Itanium porting question

    On Aug 25, 3:54 pm, Randy Park wrote:
    > I was misusing the term "symbol resolution". And yes, the hack we were
    > performing was done simply to keep the image activator from complaining.
    > Maybe someone else will read this and tell me about a different hack.


    Without knowing how your applications are written it's a little
    difficult to come up with an
    alternative solution that won't require a lot of rewriting. Of
    course, given that the first solution
    wasn't too great then maybe it's time to bite the bullet and invest a
    little effort in getting it
    corrected.

    My suggestion would be to add a layer of abstraction above the
    database/RMS access
    routines.

    That is, create subroutines of your own that wrap the calls to the
    database products
    for attach, read, write, commit, etcetera. Name the subroutines
    uniquely so they match
    the database product name. For example, MYRDB$ATTACH, MYSYBASE
    $ATTACH,
    MYRMS$OPEN, etcetera. Put all of these wrapper routines into
    shareable images,
    one per database product.

    Each of those routines would mask the underlying database entry points
    (which appear
    to be changing on Integrity) and give you a constant, known set of
    names.

    Now, in addition to the working MYRDB$ATTACH, etcetera, create a stub
    version of them
    into a seperate shareable image. Again, one shareable set of stub
    routines per database
    product.

    You can ship all the database stubs and working wrappers. One single
    build. During
    installation use your existing methods to determine which database
    products are
    installed and define logical names to point to either a stub shareable
    or a working
    wrapper shareable.

    The image activator will always find your entry point names in
    whichever shareable
    the logicals reference. If it's the stub shareable then nothing else
    happens. If it's the
    working wrapper shareable then it, in turn, will reference the
    database-specific entry
    point names and the image activator will go out and resolve them.

    How does that sound?

    It's a lot like what you're doing now, except the abstraction layer
    hides the unknown
    entry points (database product specific).



  7. Re: Itanium porting question

    On Aug 24, 8:30 pm, Randy Park wrote:
    > This is a second attempt to post this question. My news server sometimes
    > fails to post messages and I've waited 4 hours for it to appears. My apologies
    > if it shows up twice.
    >
    > I've been called in to help with a port to Itanium on some software that I was
    > the prime developer for many many years. I haven't touched a VMS system in
    > 4 years, plus I only worked part time on VMS for the previous 6 years. So here
    > goes.
    >
    > The application is a commerical tool sold into the VMS marketplace. It supports
    > many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
    > customers have all databases, let alone even one database. In order to make
    > one set of images work for all customers we would go through the following
    > generalized build procedure:
    >
    > 1. Compile all source code
    > 2. Link executable images with all databases up and running.
    > 3. Shut down database A.
    > 4. Link again, and note all unresolved symbols.
    > 5. Create shareable image(s) with stub routines containing entry points from step 4.
    > 6. Redefine logicals relevant to database A to point to shareable image from stop 5
    > 7. Run and test software with database A missing
    > 8. Repeat steps 3 through 7 for every database system.
    > 9. At this point we now have an executable that can run with or without various
    > databases being present.
    >
    > Notes:
    > 1. All databases used shareable images.
    > 2. During startup time, DCL procedures would look to see which databases were
    > present and define logicals to point to stub images for missing databases.
    >
    > This general build process would work for almost all databases on Vax and Alpha.
    > There were some strange things regarding CDD that I don't recall right now.
    >
    > The developers involved in the port are telling me that this general process no longer
    > works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
    > get the exact error message from them.)
    >
    > Any idea if this general approach should still work, or do we need to look at a
    > different method?
    >
    > --
    > Posted via a free Usenet account fromhttp://www.teranews.com


    Randy,

    I have used precisely this type of procedure many times, most publicly
    in my recent OpenVMS Technical Journal article "Strategies for
    Migrating from Alpha and VAX systems to HP Integrity Servers on
    OpenVMS" (link at http://www.rlgsc.com/publications/vm...tionstrategies....
    ). I have seen no direct problems.

    My first impression from this post is that one of the build procedures
    is doing something incorrectly. In my examples, I was able to use
    essentially the same build procedures and options files on Alpha and
    Itanium.

    I admit that I would have to look at precisely what you are doing to
    determine where the problem is. However, I CAN say that the general
    approach should work flawlessly.

    - Bob Gezelter, http://www.rlgsc.com


+ Reply to Thread