Hey all,

I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.

Is there something inherent in the structure of a Basic array that doesn't work in a global section?

I've also tried an installed shared psect image and that works OK.

-John

On 2021-11-04 16:30:24 +0000, John Doppke said:

> I've been playing with some inter-process communication via Basic using
> a global section. It all works very well except for one thing - data
> in an array. Everything else comes across nicely - strings, longs,
> etc. But anything in an array written by one process never shows up in
> the other.
>
> Is there something inherent in the structure of a Basic array that
> doesn't work in a global section?
>
> I've also tried an installed shared psect image and that works OK.

Without using the debugger to look at what's been written into the
section, I'd guess that you've used a position-independent section and
have that section mapped into to different virtual address ranges and
the data structures you've then written data structures (e.g. string
and array descriptors) containing virtual addresses from one process
context that are mismatched with the virtual addresses from the other
process(es) accessing that same section.

BASIC doesn't do base-relative memory access, which means you'll have
to map the section at the same virtual address range in each process
(commons tend to be mapped at fixed addresses), or use only data and
not addresses, or do some work to wrap base-relative access for use
from BASIC.

C, C++, and ilk do better here than does BASIC, as pointers and pointer
math and pointer dereferencing are language-integrated, and as
base-relative memory addressing can be implemented with what's
available within the language. BASIC LOC() only gets you so far.

Mapping the section into an address range up in 64-bit P2 space can be
an alternative to the usually-more-crowded P0 space, but BASIC is again
less than useful there being a 32-bit language. BASIC LOC() returns a
longword, not a quadword.

One other wrinkle you'll encounter as you proceed: you will want to
understand and use interlocked queue instructions and interlocked
bitlocks just as soon as you start using a multiprocessor system, as
now you're fully exposed to processor caching. This detail and other
parts of resource coordination and locking are discussed in the
Programming Concepts manual. Some of that material starts around here:
https://docs.vmssoftware.com/vsi-openvms-programming-concepts-manual-volume-i/#SYNCH_ACCESS_DATA
etc.

In aggregate, using IP sockets, mailboxes, ICC, or other APIs, can be
preferable for all but the highest-performance requirements, as these
existing APIs deal with this low-level memory and cache stuff and
back-pressure and with notification support already available. Even
local DECnet, for the few of you that have that set up—though ICC is
patterned on DECnet, and requires no license on host or within a
cluster, and requires no network configuration and no startup.

--
Pure Personal Opinion | HoffmanLabs LLC

John Doppke <jdoppke@gmail.com> wrote:
>
>I've been playing with some inter-process communication via Basic using a g=
>lobal section. It all works very well except for one thing - data in an ar=
>ray. Everything else comes across nicely - strings, longs, etc. But anyth=
>ing in an array written by one process never shows up in the other.
>
>Is there something inherent in the structure of a Basic array that doesn't =
>work in a global section?

I don't know BASIC, but if the array is secretly a pointer like it is in
many languages, then passing the pointer to another process is likely not
useful since the process can't see the data.
--scott

--
"C'est un Nagra. C'est suisse, et tres, tres precis."

On Thursday, November 4, 2021 at 12:30:26 PM UTC-4, jdo...@gmail.com wrote:
> Hey all,
>
> I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.
>
> Is there something inherent in the structure of a Basic array that doesn't work in a global section?
>
> I've also tried an installed shared psect image and that works OK.
>
> -John
John,

I agree with many of the thoughts expressed by Hoff.

I have a bit of experience with BASIC, several clients have been very BASIC-centric. Combining shareable storage with BASIC and other languages always requires a serious degree of caution.

First, just compiling and linking is more involved, the sources have follow explicit guidelines to have the common areas actually shared between processes, and persist if one process terminates. You also need the tools to reset the shared area and other utility tasks. Manipulating data in the common area similarly requires caution, timing errors are difficult to reproduce. The various interlocked instructions must be used religiously.

Performance wise, the complexity often is not needed. Local TCP/UDP links, local DECnet links, even mailboxes have higher overhead than shared memory, but one must consider the actual traffic volume. On even slow systems, e.g.. first generation Alpha CPUs, I could process tens if not hundreds of thousands of requests/second using network links while avoiding all of the difficulties of using shared memory.

Been there, done that. My recommendation is that it is a path to be avoided if at all possible. There are better ways to accomplish almost all of the requirements.

- Bob Gezelter, http://www.rlgsc.com

On Thursday, November 4, 2021 at 12:30:26 PM UTC-4, jdo...@gmail.com wrote:
> Hey all,
>
> I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.
>
> Is there something inherent in the structure of a Basic array that doesn't work in a global section?
>
> I've also tried an installed shared psect image and that works OK.
>
> -John

It worked for me, back in the day!

The User guide has a small section dedication to this:
6.2.3 Creating Arrays with the COMMON Statement

So it's supposed to wrok one would think.

Please compile the writer, reader and declaration modules with /LIST/SHOW=MAP
Look for your (array) variables in the "Allocation information for COMMON <your common or map name>"
Look for similar 'sounding' variables outside the Psect.

Show us! I expect it to be a silly programming oversight.

If needed
Show us the MAP (or COMMON)
Show us the assignments from and to.
Remind us how you build the shareable (LINK /OPT ) and do the install.
And just in case, as with any and every support question:
Operating System version, Architecture, Compiler version.

I'm happy try it, but that's best done with an example close to your usage.

Cheers,
Hein

On 11/04/21 16:30, John Doppke wrote:
> Hey all,
>
> I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.
>
> Is there something inherent in the structure of a Basic array that doesn't work in a global section?
>
> I've also tried an installed shared psect image and that works OK.
>
> -John

You need to ensure that the array is copied to the global section, not
just a pointer to it, which will be meaningless to the other process.

Not familiar with Basic, but suggest check that out...

Chris

On Friday, November 5, 2021 at 2:27:47 PM UTC-4, chris wrote:
> On 11/04/21 16:30, John Doppke wrote:
> > Hey all,
> >
> > I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.
> >
> > Is there something inherent in the structure of a Basic array that doesn't work in a global section?
> >
> > I've also tried an installed shared psect image and that works OK.
> >
> > -John
> You need to ensure that the array is copied to the global section, not
> just a pointer to it, which will be meaningless to the other process.
>
> Not familiar with Basic, but suggest check that out...
>
> Chris
It ended up being a stupid mistake - I added the array to the common area it increased the section size, but I forgot to increase the number of mapped pages in my call. It all works well now.

-John

On Sunday, November 7, 2021 at 11:55:35 AM UTC+13, jdo...@gmail.com wrote:
> ... I added the array to the common area it increased the section size, but I forgot
> to increase the number of mapped pages in my call.

The instinctive response of an assembly-language programmer is to place an “end” symbol after the last object in the common area, and subtract its address from the start to compute how many pages you need, instead of working it out manually. That way it automatically adapts to any changes you might make.

On Saturday, November 6, 2021 at 6:55:35 PM UTC-4, jdo...@gmail.com wrote:
> On Friday, November 5, 2021 at 2:27:47 PM UTC-4, chris wrote:
> > On 11/04/21 16:30, John Doppke wrote:
> > > Hey all,
> > >
> > > I've been playing with some inter-process communication via Basic using a global section. It all works very well except for one thing - data in an array. Everything else comes across nicely - strings, longs, etc. But anything in an array written by one process never shows up in the other.
: :
> It ended up being a stupid mistake - I added the array to the common area it increased the section size, but I forgot to increase the number of mapped pages in my call. It all works well now.

I called it! :-).
Hein: Show us! I expect it to be a silly programming oversight.

Cheers,
Hein

Poor Man's MySQL Connector:

A while back, I needed to write some code that would allow DEC-BASIC to interface with mysql client libraries written in C. If you think about it, whenever you send off an SQL query all you know is that the result (if there is one) will be an array of strings. So I needed to be able to pass an array of strings back to BASIC. Those ideas began with a series of hybrid demos:

http://neilrieck.net/demo_vms_html/openvms_demo_index.html#hybrid1 (but I think you should look at #5)

Which culminated in these:

http://neilrieck.net/demo_vms_html/openvms_demo_index.html#mysql_demo11

Neil Rieck
Waterloo, Ontario, Canada.