Hi all!

I have this problem, that I want to build an app or a shareable image, which is linked with another shared library, but which also implements its own variant of symbols that are defined in that other shared library.

This is intentional. It works absolutely fine to do this on the Unixen I've access to, as well as on Windows... but VMS fails me.

Is this at all possible, or must I seek another method to get what I want?
(there is the possibilty to link with a .OLB library instead of a shareable image, but I'd like to explore the possibilities a little more before I resort to that)

An example below. legacy-dso-legacyprov.OBJ defines all the ERR symbols that LINK complains about, and they are also defined by OSSL$libcrypto03_shr.EXE.

----------8<----------
OPEN/WRITE/SHARE=READ OPT_FILE [.providers]legacy-components.OPT
WRITE OPT_FILE "[.crypto]legacy-dso-cpuid.OBJ,-"
WRITE OPT_FILE "[.crypto]legacy-dso-ctype.OBJ,-"
WRITE OPT_FILE "[.crypto]legacy-dso-mem_clr.OBJ,-"
WRITE OPT_FILE "[.providers]legacy-dso-legacyprov.OBJ"
WRITE OPT_FILE "[.providers]liblegacy.OLB/LIB"
WRITE OPT_FILE "[.providers]libcommon.OLB/LIB"
WRITE OPT_FILE "[]OSSL$libcrypto03_shr.EXE/SHARE"
CLOSE OPT_FILE
LINK /MAP='F$PARSE(".MAP","[.providers]legacy.EXE")' /NODEBUG/NOTRACEBACK/SHARE=[.providers]legacy.EXE [.providers]legacy.OPT/OPT, [.providers]legacy-components.OPT/OPT
%ILINK-W-MULDEF, symbol CRYPTO_memcmp multiply defined
module: CPUID
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.crypto]legacy-dso-cpuid.OBJ;1
%ILINK-W-MULDEF, symbol OPENSSL_cleanse multiply defined
module: MEM_CLR
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.crypto]legacy-dso-mem_clr.OBJ;1
%ILINK-W-MULDEF, symbol ERR_clear_last_mark multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_new multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_pop_to_mark multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_set_debug multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_set_error multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_set_mark multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
%ILINK-W-MULDEF, symbol ERR_vset_error multiply defined
module: LEGACYPROV
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
----------8<----------

Cheers,
Richard

On 9/28/22 9:33 AM, Richard Levitte wrote:

> I have this problem, that I want to build an app or a shareable image, which is linked with another shared library, but which also implements its own variant of symbols that are defined in that other shared library.
>

If this is a simplified example of what you want to do:

$ cre foo.c
foo() {}
foobar() {}
^Z
$ cc foo
$ link/share foo,tt:/opt
symbol_vector=(foo=proc,foobar=proc)
^Z
$

$ cre bar.c
extern foo();
foobar(){}
bar(){
foo();
foobar();
} ^Z
$ cc bar
$ link bar,tt:/opt/map=bar/cross/full
foo/share
^Z
%LINK-W-MULDEF, symbol FOOBAR multiply defined
in module BAR file DSA3:[DECUSERVE_USER]BAR.OBJ;3
$

then using a linker cluster and /selective will work for for you:

$ link/exe=bar/map=bar/cross/full tt:/opt
cluster=a,,,bar
foo/share/sel
^Z
$

In this example you get foobar resolved from bar.obj and foo from
foo.exe. And no, VMS does not support symbol pre-emption, whatever that is.

On 9/28/2022 3:33 AM, Richard Levitte wrote:
> I have this problem, that I want to build an app or a shareable
> image, which is linked with another shared library, but which also
> implements its own variant of symbols that are defined in that other
> shared library.
>
> This is intentional. It works absolutely fine to do this on the
> Unixen I've access to, as well as on Windows... but VMS fails me.
>
> Is this at all possible, or must I seek another method to get what I
> want? (there is the possibilty to link with a .OLB library instead of
> a shareable image, but I'd like to explore the possibilities a little
> more before I resort to that)
>
> An example below. legacy-dso-legacyprov.OBJ defines all the ERR
> symbols that LINK complains about, and they are also defined by
> OSSL$libcrypto03_shr.EXE.

Is this for VMS support of the OpenSSL 3 provider model?

Arne

On 2022-09-28 07:33:58 +0000, Richard Levitte said:

> I have this problem, that I want to build an app or a shareable image,
> which is linked with another shared library, but which also implements
> its own variant of symbols that are defined in that other shared
> library.
>
> This is intentional. It works absolutely fine to do this on the Unixen
> I've access to, as well as on Windows... but VMS fails me.

It's not so reliable on various Unix systems; the results can entail
indeterminate behavior even on some Unix systems, based on an error
message I've met.

On OpenVMS, I have a tool Shimmer, which creates a new shareable image
from an existing shareable image, and that allows interception of
selected (or all) calls in the shareable image. I open-sourced this
tool a while back, and probably still have a copy of it around. If
you're interested in this interception, let me know and I'll poke
around for a copy.

Otherwise, you're going to need to have some better hygiene around your
symbols, which usually means some combination of building shareable
images and using selective search qualifier, or maybe dynamic
activation of the shareable image you want with the symbol you want at
run-time using lib$find_image_symbol, this if linker clusters mentioned
earlier don't get you where you need.

--
Pure Personal Opinion | HoffmanLabs LLC

onsdag 28 september 2022 kl. 16:00:39 UTC+2 skrev Arne Vajhøj:
> Is this for VMS support of the OpenSSL 3 provider model?

Yes. That shareable image that serves as an example is the 'legacy' provider (a provider with implementations of algorithms that are considered, well, legacy).

On 2022-09-28 17:24, Stephen Hoffman wrote:
> On 2022-09-28 07:33:58 +0000, Richard Levitte said:
>
>> I have this problem, that I want to build an app or a shareable image,
>> which is linked with another shared library, but which also implements
>> its own variant of symbols that are defined in that other shared library.
>>
>> This is intentional.  It works absolutely fine to do this on the
>> Unixen I've access to, as well as on Windows...  but VMS fails me.
>
> It's not so reliable on various Unix systems; the results can entail
> indeterminate behavior even on some Unix systems, based on an error
> message I've met.

I was going to ask how you'd know which library you then get the symbol
from in the end in your application. Seems a fair risk you might end up
with the wrong one if you don't have more explicit control.

> On OpenVMS, I have a tool Shimmer, which creates a new shareable image
> from an existing shareable image, and that allows interception of
> selected (or all) calls in the shareable image. I open-sourced this tool
> a while back, and probably still have a copy of it around. If you're
> interested in this interception, let me know and I'll poke around for a
> copy.
>
> Otherwise, you're going to need to have some better hygiene around your
> symbols, which usually means some combination of building shareable
> images and using selective search qualifier, or maybe dynamic activation
> of the shareable image you want with the symbol you want at run-time
> using lib$find_image_symbol, this if linker clusters mentioned earlier
> don't get you where you need.

Not sure if this is helpful or not, but just to give an idea of what
could be something to look for.

In RSX, you would link the shareable library and list symbols you want
to be excluded from the symbol table (or overridden) of the shared
library. Mainly in order to not get symbol conflicts (that can happen
quite a lot otherwise, since external symbols in RSX are only 6
characters long in the first place).

However, that trick is not possible to use on plain object files.

Johnny

On 9/28/22 7:35 PM, Johnny Billquist wrote:
> On 2022-09-28 17:24, Stephen Hoffman wrote:
>> On 2022-09-28 07:33:58 +0000, Richard Levitte said:
>>
>>> I have this problem, that I want to build an app or a shareable
>>> image, which is linked with another shared library, but which also
>>> implements its own variant of symbols that are defined in that other
>>> shared library.
>>>
>>> This is intentional.  It works absolutely fine to do this on the
>>> Unixen I've access to, as well as on Windows...  but VMS fails me.
>>
> Not sure if this is helpful or not, but just to give an idea of what
> could be something to look for.
>
> In RSX, you would link the shareable library and list symbols you want
> to be excluded from the symbol table (or overridden) of the shared
> library. Mainly in order to not get symbol conflicts (that can happen
> quite a lot otherwise, since external symbols in RSX are only 6
> characters long in the first place).
>
> However, that trick is not possible to use on plain object files.

On VMS (all platforms except VAX) you would link the shareable image
once with all the symbols in the global symbol table (GST), which you
need when linking images and which do not exhibit the multiply defined
warning. For linking those images you would re-link the shareable and
exclude the multiply defined symbols from the GST, that is make them
"private" in the symbol vector option. Once linking is complete, any of
the shareable images can be used at run time. Obviously you need an
identical GSMATCH option when linking both shareable images.

To me this approach looks like more work/adjustments in the build
process than the previously suggested solution.

On 9/28/22 08:33, Richard Levitte wrote:
> Hi all!
>
> I have this problem, that I want to build an app or a shareable image, which is linked with another shared library, but which also implements its own variant of symbols that are defined in that other shared library.
>
> This is intentional. It works absolutely fine to do this on the Unixen I've access to, as well as on Windows... but VMS fails me.
>
> Is this at all possible, or must I seek another method to get what I want?
> (there is the possibilty to link with a .OLB library instead of a shareable image, but I'd like to explore the possibilities a little more before I resort to that)
>
> An example below. legacy-dso-legacyprov.OBJ defines all the ERR symbols that LINK complains about, and they are also defined by OSSL$libcrypto03_shr.EXE.
>
> ----------8<----------
> OPEN/WRITE/SHARE=READ OPT_FILE [.providers]legacy-components.OPT
> WRITE OPT_FILE "[.crypto]legacy-dso-cpuid.OBJ,-"
> WRITE OPT_FILE "[.crypto]legacy-dso-ctype.OBJ,-"
> WRITE OPT_FILE "[.crypto]legacy-dso-mem_clr.OBJ,-"
> WRITE OPT_FILE "[.providers]legacy-dso-legacyprov.OBJ"
> WRITE OPT_FILE "[.providers]liblegacy.OLB/LIB"
> WRITE OPT_FILE "[.providers]libcommon.OLB/LIB"
> WRITE OPT_FILE "[]OSSL$libcrypto03_shr.EXE/SHARE"
> CLOSE OPT_FILE
> LINK /MAP='F$PARSE(".MAP","[.providers]legacy.EXE")' /NODEBUG/NOTRACEBACK/SHARE=[.providers]legacy.EXE [.providers]legacy.OPT/OPT, [.providers]legacy-components.OPT/OPT
> %ILINK-W-MULDEF, symbol CRYPTO_memcmp multiply defined
> module: CPUID
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.crypto]legacy-dso-cpuid.OBJ;1
> %ILINK-W-MULDEF, symbol OPENSSL_cleanse multiply defined
> module: MEM_CLR
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.crypto]legacy-dso-mem_clr.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_clear_last_mark multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_new multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_pop_to_mark multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_set_debug multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_set_error multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_set_mark multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> %ILINK-W-MULDEF, symbol ERR_vset_error multiply defined
> module: LEGACYPROV
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]legacy-dso-legacyprov.OBJ;1
> ----------8<----------
>
> Cheers,
> Richard

Can you use the "weak" keyword with one of the sets of symbols, which
automatically gets over ridden at link time in favour of another
conventionally defined same ident symbol at link time ?.

It's typically used in embedded work as a placeholder, say in interrupt
vector tables to a provide a default handler, where the actual handler
has not yet been written...

Chris

On 9/29/22 5:09 PM, chris wrote:
> On 9/28/22 08:33, Richard Levitte wrote:
>> Hi all!
>>
>> I have this problem, that I want to build an app or a shareable image,
>> which is linked with another shared library, but which also implements
>> its own variant of symbols that are defined in that other shared library.
>>...
> Can you use the "weak" keyword with one of the sets of symbols, which
> automatically gets over ridden at link time in favour of another
> conventionally defined same ident symbol at link time ?
The short answer is no.

The VMS linker manual has some paragraphs about weak symbols. There it
is said: "In general, a symbol can have only one strong or one VMS-style
weak definition but it can have multiple UNIX-style weak definitions."

Yes, there is one exception for VMS-style weak definitions, but that
does not apply here.

Here you want to have multiple definitions - in the shareable image and
in the object module. So you would need to have UNIX-style weak
definitions. These weak definitions are IA64 and x86 specific. They are
created by the C++ compiler. If the OP's code were C++ code resulting in
UNIX-style weak definitions then the linker would not report a
%ILINK-W-MULDEF. Also, the IA64 shareable image's GST only contains
strong definitions. So the object module's definition would be
discarded. On x86 this can be different. So the source code is either
not C++ or the language constructs used, do not result in UNIX-style
weak definitions.

Creating a UNIX-style weak definition in other languages than C++ is not
possible. Converting any strong (or VMS-style weak) definition into a
UNIX-style weak definition is not possible (except you use the patch
utility).

On 9/29/22 23:21, hb wrote:
> On 9/29/22 5:09 PM, chris wrote:
>> On 9/28/22 08:33, Richard Levitte wrote:
>>> Hi all!
>>>
>>> I have this problem, that I want to build an app or a shareable image,
>>> which is linked with another shared library, but which also implements
>>> its own variant of symbols that are defined in that other shared library.
>>> ...
>> Can you use the "weak" keyword with one of the sets of symbols, which
>> automatically gets over ridden at link time in favour of another
>> conventionally defined same ident symbol at link time ?
> The short answer is no.
>
> The VMS linker manual has some paragraphs about weak symbols. There it
> is said: "In general, a symbol can have only one strong or one VMS-style
> weak definition but it can have multiple UNIX-style weak definitions."
>
> Yes, there is one exception for VMS-style weak definitions, but that
> does not apply here.
>
> Here you want to have multiple definitions - in the shareable image and
> in the object module. So you would need to have UNIX-style weak
> definitions. These weak definitions are IA64 and x86 specific. They are
> created by the C++ compiler. If the OP's code were C++ code resulting in
> UNIX-style weak definitions then the linker would not report a
> %ILINK-W-MULDEF. Also, the IA64 shareable image's GST only contains
> strong definitions. So the object module's definition would be
> discarded. On x86 this can be different. So the source code is either
> not C++ or the language constructs used, do not result in UNIX-style
> weak definitions.
>
> Creating a UNIX-style weak definition in other languages than C++ is not
> possible. Converting any strong (or VMS-style weak) definition into a
> UNIX-style weak definition is not possible (except you use the patch
> utility).

I guess you mean vms toolchain specific, as i've used the weak keyword
in C in several different projects and architectures. It's a toolchain
function, compiler and linker and not normally processor specific.

Dunno, seems like the op needs to refactor the codebase so that the
linker only ever sees a single instance of a given symbol, which may
or may not include the weak keyword, if the toolchain allows it.
Serious design error though, multiple definitions for the same symbol
are bound to lead to difficulties...

Chris

On 2022-09-28 23:08, hb wrote:
> On 9/28/22 7:35 PM, Johnny Billquist wrote:
>> On 2022-09-28 17:24, Stephen Hoffman wrote:
>>> On 2022-09-28 07:33:58 +0000, Richard Levitte said:
>>>
>>>> I have this problem, that I want to build an app or a shareable
>>>> image, which is linked with another shared library, but which also
>>>> implements its own variant of symbols that are defined in that other
>>>> shared library.
>>>>
>>>> This is intentional.  It works absolutely fine to do this on the
>>>> Unixen I've access to, as well as on Windows...  but VMS fails me.
>>>
>> Not sure if this is helpful or not, but just to give an idea of what
>> could be something to look for.
>>
>> In RSX, you would link the shareable library and list symbols you want
>> to be excluded from the symbol table (or overridden) of the shared
>> library. Mainly in order to not get symbol conflicts (that can happen
>> quite a lot otherwise, since external symbols in RSX are only 6
>> characters long in the first place).
>>
>> However, that trick is not possible to use on plain object files.
>
> On VMS (all platforms except VAX) you would link the shareable image
> once with all the symbols in the global symbol table (GST), which you
> need when linking images and which do not exhibit the multiply defined
> warning. For linking those images you would re-link the shareable and
> exclude the multiply defined symbols from the GST, that is make them
> "private" in the symbol vector option. Once linking is complete, any of
> the shareable images can be used at run time. Obviously you need an
> identical GSMATCH option when linking both shareable images.
>
> To me this approach looks like more work/adjustments in the build
> process than the previously suggested solution.

Maybe I wasn't paying attention. I don't think I've seen a previous
suggested solution that applies to VMS. And the solution for Unix from
which this started is questionable how well it actually works (not
entirely clear *which* symbol would in the end be the one your
application would resolve to).

Johnny

onsdag 28 september 2022 kl. 14:41:09 UTC+2 skrev hb:
> $ link bar,tt:/opt/map=bar/cross/full
> foo/share
> ^Z
> %LINK-W-MULDEF, symbol FOOBAR multiply defined
> in module BAR file DSA3:[DECUSERVE_USER]BAR.OBJ;3
> $
>
> then using a linker cluster and /selective will work for for you:
>
> $ link/exe=bar/map=bar/cross/full tt:/opt
> cluster=a,,,bar
> foo/share/sel
> ^Z
> $

So this is the part that's a mystery to me... and I gotta confess, 'cluster' is magic to me, I haven't been able to wrap my brains around it and have never really used it other than as an incantation that someone else gave me.

See, I've actually tried /SELECTIVE before, believing the documentation that essentially says "... only those symbols that have been referenced by previously processed input files". However, when I tried just adding it to the shareable image in my example, I ended up with a shitload of undefined symbol references instead, like this:

%ILINK-W-USEUNDEF, undefined symbol OSSL_PARAM_locate referenced
section: $CODE$
offset: %X0000000000000040 slot: 2
module: digestcommon
file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]LIBCOMMON.OLB;1

Looking at it all now, with your example in mind, I can only assume that my assumption that files given in the /OPT file are treated top to bottom as written is more like a delusion, and that CLUSTER helps getting things processed in the order I desire.

On 2022-09-30, Johnny Billquist <bqt@softjar.se> wrote:
>
> Maybe I wasn't paying attention. I don't think I've seen a previous
> suggested solution that applies to VMS. And the solution for Unix from
> which this started is questionable how well it actually works (not
> entirely clear *which* symbol would in the end be the one your
> application would resolve to).
>

_IF_ I understand what is being asked for here (two different source code
modules both defining the same global symbol name), then it doesn't matter
in Unix provided both symbols are the same size, and of the same type, as
the code in both modules will end up referencing the same memory area.

IIRC, if one symbol definition is of a different type, or a different size,
compared to the other symbol, doesn't the GNU linker issue an error during
linking ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 9/30/22 16:30, Richard Levitte wrote:
> onsdag 28 september 2022 kl. 14:41:09 UTC+2 skrev hb:
>> $ link bar,tt:/opt/map=bar/cross/full
>> foo/share
>> ^Z
>> %LINK-W-MULDEF, symbol FOOBAR multiply defined
>> in module BAR file DSA3:[DECUSERVE_USER]BAR.OBJ;3
>> $
>>
>> then using a linker cluster and /selective will work for for you:
>>
>> $ link/exe=bar/map=bar/cross/full tt:/opt
>> cluster=a,,,bar
>> foo/share/sel
>> ^Z
>> $
>
> So this is the part that's a mystery to me... and I gotta confess, 'cluster' is magic to me, I haven't been able to wrap my brains around it and have never really used it other than as an incantation that someone else gave me.
>
> See, I've actually tried /SELECTIVE before, believing the documentation that essentially says "... only those symbols that have been referenced by previously processed input files". However, when I tried just adding it to the shareable image in my example, I ended up with a shitload of undefined symbol references instead, like this:
>
> %ILINK-W-USEUNDEF, undefined symbol OSSL_PARAM_locate referenced
> section: $CODE$
> offset: %X0000000000000040 slot: 2
> module: digestcommon
> file: USR_DISK:[RICLE.TEST.openssl-3_1_0-dev.providers]LIBCOMMON.OLB;1
>
> Looking at it all now, with your example in mind, I can only assume that my assumption that files given in the /OPT file are treated top to bottom as written is more like a delusion, and that CLUSTER helps getting things processed in the order I desire.

Please have a look at the linker manual, 2.2.4. Processing Input Files
Selectively, and 2.3.1. Understanding Cluster Creation.

The linker processes input files in cluster order. With object modules
on the command line or in the options file, the object module is
collected into the "DEFAULT_CLUSTER". Any shareable image specified in
the options file is put in its own cluster (it's shareable image name)
in the order of the options file, no matter if it is selectively
processed or not. All such "user defined" clusters are put into the
cluster list before the default cluster.

So a
$ link/exe=bar/map=bar/cross/full bar,tt:/opt
foo/share/sel
^Z
will create the cluster FOO, which is processed before the
DEFAULT_CLUSTER, that is before the object module bar. Obviously at the
time processing the shareable foo, there is no undefined symbol which
can be "selectively" resolved from it. After processing bar, there is.
But the linker is done with cluster FOO, the shareable image, hence you
get the W-USEUNDEF.

$ link/exe=bar/map=bar/cross/full tt:/opt
cluster=a,,,bar
foo/share/sel
^Z
lets the linker put bar into the cluster A before the cluster FOO of the
shareable image. Now bar is processed before the shareable foo and the
unresolved symbol can be "selectively" resolved from the shareable image.

FWIW,
$ link/exe=bar/map=bar/cross/full tt:/opt
foo/share/sel
cluster=a,,,bar
^Z
also results in an W-USEUNDEF as well as
$ link/exe=bar/map=bar/cross/full tt:/opt
bar
foo/share/sel
^Z

And this processing of input files in cluster order explains why you
initially got a duplicate reported for the object module and not for the
shareable image, and why the symbol is resolved from the shareable image
and not from the object module.

On 9/30/22 19:24, Simon Clubley wrote:
> On 2022-09-30, Johnny Billquist <bqt@softjar.se> wrote:
>>
>> Maybe I wasn't paying attention. I don't think I've seen a previous
>> suggested solution that applies to VMS. And the solution for Unix from
>> which this started is questionable how well it actually works (not
>> entirely clear *which* symbol would in the end be the one your
>> application would resolve to).
>>
>
> _IF_ I understand what is being asked for here (two different source code
> modules both defining the same global symbol name), then it doesn't matter
> in Unix provided both symbols are the same size, and of the same type, as
> the code in both modules will end up referencing the same memory area.

Did you actually try this? On Linux with gcc and binutils I get
/usr/bin/ld: two.o:(.bss+0x0): multiple definition of `same';
one.o:(.bss+0x0): first defined here
collect2: error: ld returned 1 exit status

Two different source modules both defining the same global is not what
the OP is asking for, otherwise he could not link on all Unix systems.

On Unix, shared libraries and symbol pre-emption ensures that this works
for him as expected.

fredag 30 september 2022 kl. 19:39:03 UTC+2 skrev hb:
> The linker processes input files in cluster order. With object modules
> on the command line or in the options file, the object module is
> collected into the "DEFAULT_CLUSTER". Any shareable image specified in
> the options file is put in its own cluster (it's shareable image name)
> in the order of the options file, no matter if it is selectively
> processed or not. All such "user defined" clusters are put into the
> cluster list before the default cluster.

That last bit, "before the default cluster", is the bit that escaped me, obviously. I don't see that clearly in the manual, quite the contrary when I read this, for example (third paragraph of 2.3.1, emphasis of "after" mine):

> Clusters for shareable images, specified in shareable image libraries, appear /after/ the default cluster
> on the cluster list because they are created later in linker processing, when the linker knows which
> shareable images in the library are needed for the link operation.

Maybe I misunderstand... so I guess, thanks for the clarification, and I guess I'd better turn to explicit clustering.

Cheers,
Richard

On 10/1/22 21:13, Richard Levitte wrote:
> fredag 30 september 2022 kl. 19:39:03 UTC+2 skrev hb:
>> The linker processes input files in cluster order. With object modules
>> on the command line or in the options file, the object module is
>> collected into the "DEFAULT_CLUSTER". Any shareable image specified in
>> the options file is put in its own cluster (it's shareable image name)
>> in the order of the options file, no matter if it is selectively
>> processed or not. All such "user defined" clusters are put into the
>> cluster list before the default cluster.
>
> That last bit, "before the default cluster", is the bit that escaped me, obviously. I don't see that clearly in the manual, quite the contrary when I read this, for example (third paragraph of 2.3.1, emphasis of "after" mine):
>
>> Clusters for shareable images, specified in shareable image libraries, appear /after/ the default cluster
>> on the cluster list because they are created later in linker processing, when the linker knows which
>> shareable images in the library are needed for the link operation.

This is for libraries, for example IMAGELIB.OLB, which contains
DECC$SHR.EXE, etc.

In the previous paragraph it is said: "The linker may create additional
clusters, called named clusters, depending on the types of input files
you specify and the linker options you specify. If it creates additional
clusters, the linker places them on the cluster list ahead of the
default cluster, in the order in which it encounters them in the options
file."

And in Table 2.2. Linker Input File Cluster Processing, there is

Shareable image file (.EXE) - Always placed in a named cluster.

and

Library files (.OLB) - ... However, the linker puts shareable images
referenced from IMAGELIB.OLB into new clusters at the end of the cluster
list (after the default cluster).

Could the documentation be made clearer? It looks like.

Have a look at a full map of a hello-world type C program. You will see
the DECC$SHR after the DEFAULT_CLUSTER.

If you link /NOSYSSHR and add SYS$SHARE:DECC$SHR/SHARE (and
SYS$SHARE:SYS$PUBLIC_VECTORS/SHARE) you will find the named clusters
before the DEFAULT_CLUSTER.

You can have your own shareable image library. Put whatever shareable
image you need into the library. The one(s) with the same symbol(s) as
defined in the object module(s) need to be selectively processed. Then
add that image library to your command line.

For my example the commands are

$ libr /cre/share shrlib foo/sel
$ link bar,shrlib/lib
$

That should work as well.

On 2022-09-30, hb <end.of@inter.net> wrote:
> On 9/30/22 19:24, Simon Clubley wrote:
>> On 2022-09-30, Johnny Billquist <bqt@softjar.se> wrote:
>>>
>>> Maybe I wasn't paying attention. I don't think I've seen a previous
>>> suggested solution that applies to VMS. And the solution for Unix from
>>> which this started is questionable how well it actually works (not
>>> entirely clear *which* symbol would in the end be the one your
>>> application would resolve to).
>>>
>>
>> _IF_ I understand what is being asked for here (two different source code
>> modules both defining the same global symbol name), then it doesn't matter
>> in Unix provided both symbols are the same size, and of the same type, as
>> the code in both modules will end up referencing the same memory area.
>
> Did you actually try this? On Linux with gcc and binutils I get
> /usr/bin/ld: two.o:(.bss+0x0): multiple definition of `same';
> one.o:(.bss+0x0): first defined here
> collect2: error: ld returned 1 exit status
>

one.c:

/* Compile with: "gcc -Wall -o one one.c two.c" */

#include <stdio.h>

int gbl_val;

void two();

int main(int argc, char *argv[])
{
gbl_val = 1;
two();
printf("one: gbl_val = %d\n", gbl_val);
return 0;
}

two.c:

#include <stdio.h>

int gbl_val;

void two()
{
printf("two: gbl_val = %d\n", gbl_val);
gbl_val = 2;
}

That works just fine for me on Linux (unfortunately :-().

[deleted@deleted ~]$ ./one
two: gbl_val = 1
one: gbl_val = 2

If it's not working for you, then it's a recent change in one of Linux,
binutils, or gcc that's likely to break at least some code, because while
I would never write something so hideous, I have seen it present in other
code.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

söndag 2 oktober 2022 kl. 10:56:26 UTC+2 skrev hb:
> On 10/1/22 21:13, Richard Levitte wrote:
> > That last bit, "before the default cluster", is the bit that escaped me, obviously. I don't see that clearly in the manual, quite the contrary when I read this, for example (third paragraph of 2.3.1, emphasis of "after" mine):
> >
> >> Clusters for shareable images, specified in shareable image libraries, appear /after/ the default cluster
> >> on the cluster list because they are created later in linker processing, when the linker knows which
> >> shareable images in the library are needed for the link operation.
> This is for libraries, for example IMAGELIB.OLB, which contains
> DECC$SHR.EXE, etc.

Oh, right, now I see "shareable image libraries" in that quoted paragraph.
Somehow, I glossed over that detail and kept reading "shareable images"...

> Could the documentation be made clearer? It looks like.

It might be my way of reading that's a bit off, but yeah, I'm thinking that a discrete
sub-sectioning to emphasise a little more that one is about shareable images and
the other about shareable image libraries, that might help. If you've been immersed
in a world that talk in terms of "shared libraries" (which I have), it's easy to get lost
in the nuances between "image" and "library".

Anyway, thank you for your clarifications... and for your patience, which is noted.
I feel quite enlightened :-)

Cheers,
Richard

On 10/3/22 01:21, Simon Clubley wrote:
> On 2022-09-30, hb <end.of@inter.net> wrote:
>> On 9/30/22 19:24, Simon Clubley wrote:
>>> On 2022-09-30, Johnny Billquist <bqt@softjar.se> wrote:
>>>>
>>>> Maybe I wasn't paying attention. I don't think I've seen a previous
>>>> suggested solution that applies to VMS. And the solution for Unix from
>>>> which this started is questionable how well it actually works (not
>>>> entirely clear *which* symbol would in the end be the one your
>>>> application would resolve to).
>>>>
>>>
>>> _IF_ I understand what is being asked for here (two different source code
>>> modules both defining the same global symbol name), then it doesn't matter
>>> in Unix provided both symbols are the same size, and of the same type, as
>>> the code in both modules will end up referencing the same memory area.
>>
>> Did you actually try this? On Linux with gcc and binutils I get
>> /usr/bin/ld: two.o:(.bss+0x0): multiple definition of `same';
>> one.o:(.bss+0x0): first defined here
>> collect2: error: ld returned 1 exit status
>>
>
> one.c:
>
> /* Compile with: "gcc -Wall -o one one.c two.c" */
> ...
> If it's not working for you, then it's a recent change in one of Linux,
> binutils, or gcc that's likely to break at least some code, because while
> I would never write something so hideous, I have seen it present in other
> code.

$ gcc -v
....
gcc version 10.2.1 20210110 (Debian 10.2.1-6)
$ gcc -Wall -o one one.c two.c
/usr/bin/ld: /tmp/ccTDBf9g.o:(.bss+0x0): multiple definition of
`gbl_val'; /tmp/ccXuTT0f.o:(.bss+0x0): first defined here
collect2: error: ld returned 1 exit status
$ gcc -Wall -fcommon -o one one.c two.c
$

They changed the default, which is now -fno-common.

On 2022-10-03, hb <end.of@inter.net> wrote:
>
> They changed the default, which is now -fno-common.

Thanks. Now I see what is going on in the latest versions, I strongly
prefer that new behaviour as the default behaviour. Pity it wasn't the
out of the box behaviour in older versions.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.