Randy Park wrote:
> This is a second attempt to post this question. My news server sometimes
> fails to post messages and I've waited 4 hours for it to appears. My apologies
> if it shows up twice.
>
> I've been called in to help with a port to Itanium on some software that I was
> the prime developer for many many years. I haven't touched a VMS system in
> 4 years, plus I only worked part time on VMS for the previous 6 years. So here
> goes.
>
> The application is a commerical tool sold into the VMS marketplace. It supports
> many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
> customers have all databases, let alone even one database. In order to make
> one set of images work for all customers we would go through the following
> generalized build procedure:
>
> 1. Compile all source code
> 2. Link executable images with all databases up and running.
> 3. Shut down database A.
> 4. Link again, and note all unresolved symbols.
> 5. Create shareable image(s) with stub routines containing entry points from step 4.
> 6. Redefine logicals relevant to database A to point to shareable image from stop 5
> 7. Run and test software with database A missing
> 8. Repeat steps 3 through 7 for every database system.
> 9. At this point we now have an executable that can run with or without various
> databases being present.
>
> Notes:
> 1. All databases used shareable images.
> 2. During startup time, DCL procedures would look to see which databases were
> present and define logicals to point to stub images for missing databases.
>
> This general build process would work for almost all databases on Vax and Alpha.
> There were some strange things regarding CDD that I don't recall right now.
>
> The developers involved in the port are telling me that this general process no longer
> works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
> get the exact error message from them.)
>
> Any idea if this general approach should still work, or do we need to look at a
> different method?

Step 5 is problematic on a VMS system any version, if you are intending
the same binary image to work with either your stub or with the real
shared image from the database vendor.

For this, not only must you have all the symbols that are in the shared
image in the stub, you must also have them in the same place in the
symbol vector. And with out cooperation from the database vendor, it is
hard to get sufficient information to get it right.

You will also run into problems where a vendor obsoletes a routine, and
puts the replacement into a different location in the transfer vector.

Otherwise, you have to link your application at the customer site, and
if they ever add a database, you must relink your application to it.
And if you are doing that, there is no need for your stubs to be shared
images.

My guess is that in the past someone worked out the minimum transfer
vectors for VAX and ALPHA images in right order on VAX and ALPHA, and
with Itanium as either the ELF format is too different, or one or more
of the database vendors has reorganized how the symbol vector is set up.

The hack you were using was high risk and high maintenance before, and
Itanium has exposed it.

A recommendation:

Find out if the shared images for each database that you are currently
linking against can be loaded through LIB$FIND_IMAGE_SYMBOL of dlsym()
depending on the language used. If so, stop linking against them, and
dynamically load them as needed.

[Take a look at how Python on VMS supports various databases]

If they do not support direct dynamic loading, then it may be possible
to put wrappers around them that you can dynamically load.

This way you do not have to provide stub routines, and you do not have
to go through such a complex build procedure.

-John
wb8tyw@qsl.net
Personal Opinion Only

On Sat, 25 Aug 2007 13:59:47 GMT, "John E. Malmberg" wrote:

>Randy Park wrote:
>> This is a second attempt to post this question. My news server sometimes
>> fails to post messages and I've waited 4 hours for it to appears. My apologies
>> if it shows up twice.
>>
>> I've been called in to help with a port to Itanium on some software that I was
>> the prime developer for many many years. I haven't touched a VMS system in
>> 4 years, plus I only worked part time on VMS for the previous 6 years. So here
>> goes.
>>
>> The application is a commerical tool sold into the VMS marketplace. It supports
>> many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
>> customers have all databases, let alone even one database. In order to make
>> one set of images work for all customers we would go through the following
>> generalized build procedure:
>>
>> 1. Compile all source code
>> 2. Link executable images with all databases up and running.
>> 3. Shut down database A.
>> 4. Link again, and note all unresolved symbols.
>> 5. Create shareable image(s) with stub routines containing entry points from step 4.
>> 6. Redefine logicals relevant to database A to point to shareable image from stop 5
>> 7. Run and test software with database A missing
>> 8. Repeat steps 3 through 7 for every database system.
>> 9. At this point we now have an executable that can run with or without various
>> databases being present.
>>
>> Notes:
>> 1. All databases used shareable images.
>> 2. During startup time, DCL procedures would look to see which databases were
>> present and define logicals to point to stub images for missing databases.
>>
>> This general build process would work for almost all databases on Vax and Alpha.
>> There were some strange things regarding CDD that I don't recall right now.
>>
>> The developers involved in the port are telling me that this general process no longer
>> works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
>> get the exact error message from them.)
>>
>> Any idea if this general approach should still work, or do we need to look at a
>> different method?
>
>Step 5 is problematic on a VMS system any version, if you are intending
>the same binary image to work with either your stub or with the real
>shared image from the database vendor.
>
>For this, not only must you have all the symbols that are in the shared
>image in the stub, you must also have them in the same place in the
>symbol vector. And with out cooperation from the database vendor, it is
>hard to get sufficient information to get it right.
>
>You will also run into problems where a vendor obsoletes a routine, and
>puts the replacement into a different location in the transfer vector.
>
>Otherwise, you have to link your application at the customer site, and
>if they ever add a database, you must relink your application to it.
>And if you are doing that, there is no need for your stubs to be shared
>images.
>
>My guess is that in the past someone worked out the minimum transfer
>vectors for VAX and ALPHA images in right order on VAX and ALPHA, and
>with Itanium as either the ELF format is too different, or one or more
>of the database vendors has reorganized how the symbol vector is set up.
>
>The hack you were using was high risk and high maintenance before, and
>Itanium has exposed it.
>
>A recommendation:
>
>Find out if the shared images for each database that you are currently
>linking against can be loaded through LIB$FIND_IMAGE_SYMBOL of dlsym()
>depending on the language used. If so, stop linking against them, and
>dynamically load them as needed.
>
>[Take a look at how Python on VMS supports various databases]
>
>If they do not support direct dynamic loading, then it may be possible
>to put wrappers around them that you can dynamically load.
>
>This way you do not have to provide stub routines, and you do not have
>to go through such a complex build procedure.
>
>-John
>wb8tyw@qsl.net
>Personal Opinion Only

This method has worked for many versions of VMS over the last 15 years
because the stub routines are never called. They are there so that the
image activator will resolve the symbols.

The software is in low to no maintenance mode and changing the code to
support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
all the databases with very little return on the investment. If we have
to link on site during initial installation and relink when their database
environment changes then that's what we will do.

It would be nice to understand what has changed from vax/alpha to
itanium and why our approach no longer works.

--
Posted via a free Usenet account from http://www.teranews.com

rvfulltime wrote:

>
> This method has worked for many versions of VMS over the last 15 years
> because the stub routines are never called. They are there so that the
> image activator will resolve the symbols.

On VAX and Alpha, the image activator does not resolve the symbols, it
uses the offset coded into the main image to to access a vector in the
shared image. It then provides fixups to the offsets to match where the
image actually got loaded.

It is the offset that absolutely must be the same in the stub and the
real shared image.

Inside the shared image, effectively there is a giant branch table that
jumps to the start of where the routines actually are.

Since you were just using the stub to keep the image activator happy and
not actually calling the routines in the stub, or if all the routines
ignored their parameters and just returned the same value, then you
could get away with the hack on VAX and ALPHA.

> The software is in low to no maintenance mode and changing the code to
> support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
> all the databases with very little return on the investment. If we have
> to link on site during initial installation and relink when their database
> environment changes then that's what we will do.
>
> It would be nice to understand what has changed from vax/alpha to
> itanium and why our approach no longer works.

Itanium uses the ELF image format. And that probably has changed a lot
on how the image activator works. Someone more familiar with the image
activator would probably be able to give some details.

-John
wb8tyw@qsl.network
Personal Opinion Only

On Sat, 25 Aug 2007 17:09:35 GMT, "John E. Malmberg" wrote:

>rvfulltime wrote:
>
>>
>> This method has worked for many versions of VMS over the last 15 years
>> because the stub routines are never called. They are there so that the
>> image activator will resolve the symbols.
>
>On VAX and Alpha, the image activator does not resolve the symbols, it
>uses the offset coded into the main image to to access a vector in the
>shared image. It then provides fixups to the offsets to match where the
>image actually got loaded.
>
>It is the offset that absolutely must be the same in the stub and the
>real shared image.
>
>Inside the shared image, effectively there is a giant branch table that
>jumps to the start of where the routines actually are.
>
>Since you were just using the stub to keep the image activator happy and
>not actually calling the routines in the stub, or if all the routines
>ignored their parameters and just returned the same value, then you
>could get away with the hack on VAX and ALPHA.
>
>> The software is in low to no maintenance mode and changing the code to
>> support a LIB$FIND_IMAGE_SYMBOL method would be a lot of work for
>> all the databases with very little return on the investment. If we have
>> to link on site during initial installation and relink when their database
>> environment changes then that's what we will do.
>>
>> It would be nice to understand what has changed from vax/alpha to
>> itanium and why our approach no longer works.
>
>Itanium uses the ELF image format. And that probably has changed a lot
>on how the image activator works. Someone more familiar with the image
>activator would probably be able to give some details.
>
>-John
>wb8tyw@qsl.network
>Personal Opinion Only

I was misusing the term "symbol resolution". And yes, the hack we were
performing was done simply to keep the image activator from complaining.
Maybe someone else will read this and tell me about a different hack.

--
Posted via a free Usenet account from http://www.teranews.com

On Aug 25, 3:54 pm, Randy Park wrote:
> I was misusing the term "symbol resolution". And yes, the hack we were
> performing was done simply to keep the image activator from complaining.
> Maybe someone else will read this and tell me about a different hack.

Without knowing how your applications are written it's a little
difficult to come up with an
alternative solution that won't require a lot of rewriting. Of
course, given that the first solution
wasn't too great then maybe it's time to bite the bullet and invest a
little effort in getting it
corrected.

My suggestion would be to add a layer of abstraction above the
database/RMS access
routines.

That is, create subroutines of your own that wrap the calls to the
database products
for attach, read, write, commit, etcetera. Name the subroutines
uniquely so they match
the database product name. For example, MYRDB$ATTACH, MYSYBASE
$ATTACH,
MYRMS$OPEN, etcetera. Put all of these wrapper routines into
shareable images,
one per database product.

Each of those routines would mask the underlying database entry points
(which appear
to be changing on Integrity) and give you a constant, known set of
names.

Now, in addition to the working MYRDB$ATTACH, etcetera, create a stub
version of them
into a seperate shareable image. Again, one shareable set of stub
routines per database
product.

You can ship all the database stubs and working wrappers. One single
build. During
installation use your existing methods to determine which database
products are
installed and define logical names to point to either a stub shareable
or a working
wrapper shareable.

The image activator will always find your entry point names in
whichever shareable
the logicals reference. If it's the stub shareable then nothing else
happens. If it's the
working wrapper shareable then it, in turn, will reference the
database-specific entry
point names and the image activator will go out and resolve them.

How does that sound?

It's a lot like what you're doing now, except the abstraction layer
hides the unknown
entry points (database product specific).

On Aug 24, 8:30 pm, Randy Park wrote:
> This is a second attempt to post this question. My news server sometimes
> fails to post messages and I've waited 4 hours for it to appears. My apologies
> if it shows up twice.
>
> I've been called in to help with a port to Itanium on some software that I was
> the prime developer for many many years. I haven't touched a VMS system in
> 4 years, plus I only worked part time on VMS for the previous 6 years. So here
> goes.
>
> The application is a commerical tool sold into the VMS marketplace. It supports
> many databases (Rdb, Oracle, etc.) plus RMS. All at the same time. But not all
> customers have all databases, let alone even one database. In order to make
> one set of images work for all customers we would go through the following
> generalized build procedure:
>
> 1. Compile all source code
> 2. Link executable images with all databases up and running.
> 3. Shut down database A.
> 4. Link again, and note all unresolved symbols.
> 5. Create shareable image(s) with stub routines containing entry points from step 4.
> 6. Redefine logicals relevant to database A to point to shareable image from stop 5
> 7. Run and test software with database A missing
> 8. Repeat steps 3 through 7 for every database system.
> 9. At this point we now have an executable that can run with or without various
> databases being present.
>
> Notes:
> 1. All databases used shareable images.
> 2. During startup time, DCL procedures would look to see which databases were
> present and define logicals to point to stub images for missing databases.
>
> This general build process would work for almost all databases on Vax and Alpha.
> There were some strange things regarding CDD that I don't recall right now.
>
> The developers involved in the port are telling me that this general process no longer
> works on Itanium. They're getting some kind of "ELF" error message. (I'm trying to
> get the exact error message from them.)
>
> Any idea if this general approach should still work, or do we need to look at a
> different method?
>
> --
> Posted via a free Usenet account fromhttp://www.teranews.com

Randy,

I have used precisely this type of procedure many times, most publicly
in my recent OpenVMS Technical Journal article "Strategies for
Migrating from Alpha and VAX systems to HP Integrity Servers on
OpenVMS" (link at http://www.rlgsc.com/publications/vm...tionstrategies....
). I have seen no direct problems.

My first impression from this post is that one of the build procedures
is doing something incorrectly. In my examples, I was able to use
essentially the same build procedures and options files on Alpha and
Itanium.

I admit that I would have to look at precisely what you are doing to
determine where the problem is. However, I CAN say that the general
approach should work flawlessly.

- Bob Gezelter, http://www.rlgsc.com