On 12/11/2021 10:24 AM, MitchAlsup wrote:
> On Saturday, December 11, 2021 at 10:38:03 AM UTC-6, Stephen Fuld wrote:
>> On 12/11/2021 3:32 AM, Niklas Holsti wrote:
>>> On 2021-12-11 7:56, Stephen Fuld wrote:
>>>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
>>>>> On 2021-12-10 23:27, MitchAlsup wrote:
>>>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti wrote:
>>>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
>>>>
>>>> snip
>>>>
>>>>
>>>>>> What I was looking for is accept distribution. Thread[k] can call
>>>>>> the "read" accept point and this can be accepted by any of as
>>>>>> many accept "read" threads as are potentially running.
>>>>>
>>>>>
>>>>> That is not provided as primitive in Ada, but can be programmed, of
>>>>> course. For example, the primary rendez-vous code can "requeue" the
>>>>> call to another entry, of some task or some protected object, and
>>>>> that task or protected object can be selected from a pool of
>>>>> secondary "servers" in any way you want. The requeue statement
>>>>> completes the primary rendez-vous. The caller is still suspended on
>>>>> the requeued call, but the primary server is released to continue its
>>>>> work, usually by accepting the next call.
>>>>
>>>> But do you have to program all of this "requeueing" yourself?
>>>
>>>
>>> You do have to program the "requeue" statements, yes, and the logic to
>>> select the target of the requeue (the "secondary server").
>>>
>>>
>>>> And in particular, do you have to set up locks for the queues,
>>>> request exclusive access to them, etc.?
>>>
>>> Mainly no; the Ada run-time system handles the queueing of entry calls
>>> to entries, as well as the exlusive access or other protections
>>> intrinsic to Ada tasks and Ada protected objects. Of course that can
>>> need a significant amount of code in the run-time system.
>>>
>>>
>>>> This is just the kind of thing that the hardware assisted message
>>>> passing was designed to eliminate. :-(
>>>
>>> So perhaps this sub-thread is not much off-topic after all.
>> I was too quick in my reply. I shouldn't have said "hardware assisted"
>> above. It is really message passing that is designed to eliminate the
>> need to program the queues, etc. The hardware assist just makes it more
>> efficient.
>>
>> From my interpretation of your description, in the current Ada system,
>> it seems to require shared memory between the "sender", and all of the
>> "receivers", as they must access the common queues, etc. Is that true?
> <
> I can envision a system where one-guy owns and controls the queues
> and has the ability to send messages to <basically> wake up various
> threads.
> <
> I can envision that these woken up threads take a look at their CPU-affinity-set
> and decide whether and who to context switch.

I can too. But it was just my (perhaps mistaken) interpretation of
Nick's description that it wasn't the way Ada worked.

> <
> I further envision that this one-guy is NOT a CPU and is not running "instructions"
> but just queueing commands.

Could be. But somewhere there has to be some asynch (by my previous
definition) stuff going on, so that the originating thread can cause
multiple messages to be sent without waiting for a response from the
first one. I can envision several mechanisms for this.

>> If so, then a message passing based system could help, as it could
>> eliminate that requirement, allowing much more flexibility in allocating
>> processes to different processors not sharing memory with the "sending"
>> process.

Agreed, provided the asynch capability exists. And furthermore, if it
did exist, it would eliminate a lot of code, and be faster than a
software based system.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 2021-12-11 19:59, EricP wrote:
> Niklas Holsti wrote:
>> On 2021-12-11 4:08, EricP wrote:

[snip]

>>> Select statements with multiple OR options, possibly with guards,
>>> allow entrys and accepts to atomically wait in multiple queues at once,
>>> with multiple possible timers.
>>> When a client entry matches a server accept they bind and dequeue
>>> all other options and timers atomically at once.
>>
>>
>> Only the timers are dequeued (cancelled). The other (non-selected)
>> calls remain pending while the selected client call is executed in a
>> rendez-vous, and then also until the next execution of the select
>> statement.
>
> Yes. sorry, that was me misremembering how an entry select statement works.

And I did not notice that you were speaking of both the select-accept
statement and the select-(entry call) statement. The latter, while using
the "select" and "end select" syntax, is formally called a "conditional
entry call" if it has an "else", and a "timed entry call" if it has a
"delay".

> I remembered that Ada accept statement could have a select statement
> to allow multiple accept alternatives at once.
>
>   select
>    accept A (...);
>   or
>    accept B (...);
>   end select;
>
> I had, mistakenly, thought that an entry select statement also allowed
> multiple entry options to select different servers based on which accept
> is available first. Something like:
>
>    entry A (...);
>    entry B (...);
>    entry C (...);
>
>   select
>    A (...);
>   or
>    B (...);
>   or
>    C (...);
>   or
>    delay 123.4;
>   end select;
>
> But looking at the Ada LRM I see that it does _not_ allow multiple
> entry choices, just a delay or an ELSE option.

Right.

While I've never seen a full explanation of why multiple entry call
choices are not allowed, probably implementors felt it would be too
complicated and no Ada users came up with strong examples showing a need
for such constructs. The Ada language definition team tends not to
accept proposals for "nice to have" features.

On 2021-12-11 18:37, Stephen Fuld wrote:
> On 12/11/2021 3:32 AM, Niklas Holsti wrote:
>> On 2021-12-11 7:56, Stephen Fuld wrote:
>>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
>>>> On 2021-12-10 23:27, MitchAlsup wrote:
>>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti wrote:
>>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
>>>
>>> snip
>>>
>>>
>>>>> What I was looking for is accept distribution. Thread[k] can call
>>>>> the "read" accept point and this can be accepted by any of as
>>>>> many accept "read" threads as are potentially running.
>>>>
>>>>
>>>> That is not provided as primitive in Ada, but can be programmed, of
>>>> course. For example, the primary rendez-vous code can "requeue" the
>>>> call to another entry, of some task or some protected object, and
>>>> that task or protected object can be selected from a pool of
>>>> secondary "servers" in any way you want. The requeue statement
>>>> completes the primary rendez-vous. The caller is still suspended on
>>>> the requeued call, but the primary server is released to continue
>>>> its work, usually by accepting the next call.
>>>
>>> But do you have to program all of this "requeueing" yourself?
>>
>>
>> You do have to program the "requeue" statements, yes, and the logic to
>> select the target of the requeue (the "secondary server").
>>
>>
>>> And in particular, do you have to set up locks for the queues,
>>> request exclusive access to them, etc.?
>>
>> Mainly no; the Ada run-time system handles the queueing of entry calls
>> to entries, as well as the exlusive access or other protections
>> intrinsic to Ada tasks and Ada protected objects. Of course that can
>> need a significant amount of code in the run-time system.
>>
>>
>>> This is just the kind of thing that the hardware assisted message
>>> passing was designed to eliminate.  :-(
>>
>> So perhaps this sub-thread is not much off-topic after all.
>
> I was too quick in my reply.  I shouldn't have said "hardware assisted"
> above.  It is really message passing that is designed to eliminate the
> need to program the queues, etc.  The hardware assist just makes it more
> efficient.
>
> From my interpretation of your description, in the current Ada system,
> it seems to require shared memory between the "sender", and all of the
> "receivers", as they must access the common queues, etc.  Is that true?

Not only must the queues be shared, but the parameters in an entry call
can be passed by reference, which requires shared memory.

Ada tasking and inter-task communication does basically assume a
shared-memory system. For example, different tasks can access the same
global variables if the module structure makes them visible to those tasks.

>  If so, then a message passing based system could help, as it could
> eliminate that requirement, allowing much more flexibility in allocating
> processes to different processors not sharing memory with the "sending"
> process.

I suppose an Ada-like inter-task communication system could be built on
message passing, with the restriction that messages ("entry calls")
could use only pass-by-value parameters or pass by copy-in copy-out
parameters.

For communicating between program parts in separate memory spaces the
"distributed systems" feature was added to Ada
(http://www.ada-auth.org/standards/rm12_w_tc1/html/RM-E.html) but as
said earlier in this thread that feature is more or less obsolete. It
was based on RPC, so HW-assisted messages could certainly be used in its
implementation, at least if longish messages are possible.

On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti wrote:
> On 2021-12-11 18:37, Stephen Fuld wrote:
> > On 12/11/2021 3:32 AM, Niklas Holsti wrote:
> >> On 2021-12-11 7:56, Stephen Fuld wrote:
> >>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
> >>>> On 2021-12-10 23:27, MitchAlsup wrote:
> >>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti wrote:
> >>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
> >>>
> >>> snip
> >>>
> >>>
> >>>>> What I was looking for is accept distribution. Thread[k] can call
> >>>>> the "read" accept point and this can be accepted by any of as
> >>>>> many accept "read" threads as are potentially running.
> >>>>
> >>>>
> >>>> That is not provided as primitive in Ada, but can be programmed, of
> >>>> course. For example, the primary rendez-vous code can "requeue" the
> >>>> call to another entry, of some task or some protected object, and
> >>>> that task or protected object can be selected from a pool of
> >>>> secondary "servers" in any way you want. The requeue statement
> >>>> completes the primary rendez-vous. The caller is still suspended on
> >>>> the requeued call, but the primary server is released to continue
> >>>> its work, usually by accepting the next call.
> >>>
> >>> But do you have to program all of this "requeueing" yourself?
> >>
> >>
> >> You do have to program the "requeue" statements, yes, and the logic to
> >> select the target of the requeue (the "secondary server").
> >>
> >>
> >>> And in particular, do you have to set up locks for the queues,
> >>> request exclusive access to them, etc.?
> >>
> >> Mainly no; the Ada run-time system handles the queueing of entry calls
> >> to entries, as well as the exlusive access or other protections
> >> intrinsic to Ada tasks and Ada protected objects. Of course that can
> >> need a significant amount of code in the run-time system.
> >>
> >>
> >>> This is just the kind of thing that the hardware assisted message
> >>> passing was designed to eliminate. :-(
> >>
> >> So perhaps this sub-thread is not much off-topic after all.
> >
> > I was too quick in my reply. I shouldn't have said "hardware assisted"
> > above. It is really message passing that is designed to eliminate the
> > need to program the queues, etc. The hardware assist just makes it more
> > efficient.
> >
> > From my interpretation of your description, in the current Ada system,
> > it seems to require shared memory between the "sender", and all of the
> > "receivers", as they must access the common queues, etc. Is that true?
<
> Not only must the queues be shared, but the parameters in an entry call
> can be passed by reference, which requires shared memory.
<
Err, not quite.
<
If one passes a pointer, and there is a means to access through that pointer
in the callers address space (rather than the callees address space) one
does not actually need shared memory.
<
Consider traps in a PDP-11/70 where an application traps to the OS to perform
a <disk> read.
a) sure; the OS needs to be able to access (and verify permissions) to the
buffer supplied by application
b) but the OS uses the entire 16-bit virtual address space itself
c) so, there can be no actual sharing
d) but OS can still access pointer memory using the LD-from and ST-to
memory reference instructions.
<
Consider a similar trap as above where the OS resides on a computer
at the front of the airplane and the request came from a computer at
the back of the airplane, and the front of airplane computer performs the
read and uses the network to ship the <now full> buffer back to requestor.
<
Neither example needs shared memory, both pass pointers.
>
> Ada tasking and inter-task communication does basically assume a
> shared-memory system. For example, different tasks can access the same
> global variables if the module structure makes them visible to those tasks.
<
It seems to me that as long as the compiler binds the pointer with the address
space the memory model can be significantly freed up (PDP-11/70 style.)
After binding {to:from} with pointer, one can access his space far after any
rendezvous has transpired. Inside the rendezvous, no locking is required;
outside of the rendezvous locking may (MAY) be required.
<
> > If so, then a message passing based system could help, as it could
> > eliminate that requirement, allowing much more flexibility in allocating
> > processes to different processors not sharing memory with the "sending"
> > process.
<
> I suppose an Ada-like inter-task communication system could be built on
> message passing, with the restriction that messages ("entry calls")
> could use only pass-by-value parameters or pass by copy-in copy-out
> parameters.
<
Using PDP-11/70 mechanics, there is no need for this restriction.
>
> For communicating between program parts in separate memory spaces the
> "distributed systems" feature was added to Ada
> (http://www.ada-auth.org/standards/rm12_w_tc1/html/RM-E.html) but as
> said earlier in this thread that feature is more or less obsolete. It
> was based on RPC, so HW-assisted messages could certainly be used in its
> implementation, at least if longish messages are possible.

On 2021-12-11 22:33, MitchAlsup wrote:
> On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti wrote:
>> On 2021-12-11 18:37, Stephen Fuld wrote:
>>> On 12/11/2021 3:32 AM, Niklas Holsti wrote:
>>>> On 2021-12-11 7:56, Stephen Fuld wrote:
>>>>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
>>>>>> On 2021-12-10 23:27, MitchAlsup wrote:
>>>>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti wrote:
>>>>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
>>>>>
>>>>> snip
>>>>>
>>>>>
>>>>>>> What I was looking for is accept distribution. Thread[k] can call
>>>>>>> the "read" accept point and this can be accepted by any of as
>>>>>>> many accept "read" threads as are potentially running.
>>>>>>
>>>>>>
>>>>>> That is not provided as primitive in Ada, but can be programmed, of
>>>>>> course. For example, the primary rendez-vous code can "requeue" the
>>>>>> call to another entry, of some task or some protected object, and
>>>>>> that task or protected object can be selected from a pool of
>>>>>> secondary "servers" in any way you want. The requeue statement
>>>>>> completes the primary rendez-vous. The caller is still suspended on
>>>>>> the requeued call, but the primary server is released to continue
>>>>>> its work, usually by accepting the next call.
>>>>>
>>>>> But do you have to program all of this "requeueing" yourself?
>>>>
>>>>
>>>> You do have to program the "requeue" statements, yes, and the logic to
>>>> select the target of the requeue (the "secondary server").
>>>>
>>>>
>>>>> And in particular, do you have to set up locks for the queues,
>>>>> request exclusive access to them, etc.?
>>>>
>>>> Mainly no; the Ada run-time system handles the queueing of entry calls
>>>> to entries, as well as the exlusive access or other protections
>>>> intrinsic to Ada tasks and Ada protected objects. Of course that can
>>>> need a significant amount of code in the run-time system.
>>>>
>>>>
>>>>> This is just the kind of thing that the hardware assisted message
>>>>> passing was designed to eliminate. :-(
>>>>
>>>> So perhaps this sub-thread is not much off-topic after all.
>>>
>>> I was too quick in my reply. I shouldn't have said "hardware assisted"
>>> above. It is really message passing that is designed to eliminate the
>>> need to program the queues, etc. The hardware assist just makes it more
>>> efficient.
>>>
>>> From my interpretation of your description, in the current Ada system,
>>> it seems to require shared memory between the "sender", and all of the
>>> "receivers", as they must access the common queues, etc. Is that true?
> <
>> Not only must the queues be shared, but the parameters in an entry call
>> can be passed by reference, which requires shared memory.
> <
> Err, not quite.
> <
> If one passes a pointer, and there is a means to access through that pointer
> in the callers address space (rather than the callees address space) one
> does not actually need shared memory.
> <
> Consider traps in a PDP-11/70 where an application traps to the OS to perform
> a <disk> read.
> a) sure; the OS needs to be able to access (and verify permissions) to the
> buffer supplied by application
> b) but the OS uses the entire 16-bit virtual address space itself
> c) so, there can be no actual sharing
> d) but OS can still access pointer memory using the LD-from and ST-to
> memory reference instructions.

Ok, but then it becomes a question of defining what "shared memory"
actually means... if the callee can access the caller's memory almost as
immediately and rapidly as its own memory, that sounds like "shared"
even if the address spaces are not the same.

In the general case (not just application-to-OS calls) you would have to
allow for local calls (same address space) as well as remote calls
(different address spaces) so the call-by-reference parameter would have
to include also an address-space identifier, in effect making it an
extended address into a shared (but partitioned) memory.

> Consider a similar trap as above where the OS resides on a computer
> at the front of the airplane and the request came from a computer at
> the back of the airplane, and the front of airplane computer performs the
> read and uses the network to ship the <now full> buffer back to requestor.
> <
> Neither example needs shared memory, both pass pointers.

Huh? If you "ship the full buffer" over the network, that sounds to me
like a pass-by-value ("out") parameter.

On Saturday, December 11, 2021 at 3:11:30 PM UTC-6, Niklas Holsti wrote:
> On 2021-12-11 22:33, MitchAlsup wrote:
> > On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti wrote:
> >> On 2021-12-11 18:37, Stephen Fuld wrote:
> >>> On 12/11/2021 3:32 AM, Niklas Holsti wrote:
> >>>> On 2021-12-11 7:56, Stephen Fuld wrote:
> >>>>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
> >>>>>> On 2021-12-10 23:27, MitchAlsup wrote:
> >>>>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti wrote:
> >>>>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
> >>>>>
> >>>>> snip
> >>>>>
> >>>>>
> >>>>>>> What I was looking for is accept distribution. Thread[k] can call
> >>>>>>> the "read" accept point and this can be accepted by any of as
> >>>>>>> many accept "read" threads as are potentially running.
> >>>>>>
> >>>>>>
> >>>>>> That is not provided as primitive in Ada, but can be programmed, of
> >>>>>> course. For example, the primary rendez-vous code can "requeue" the
> >>>>>> call to another entry, of some task or some protected object, and
> >>>>>> that task or protected object can be selected from a pool of
> >>>>>> secondary "servers" in any way you want. The requeue statement
> >>>>>> completes the primary rendez-vous. The caller is still suspended on
> >>>>>> the requeued call, but the primary server is released to continue
> >>>>>> its work, usually by accepting the next call.
> >>>>>
> >>>>> But do you have to program all of this "requeueing" yourself?
> >>>>
> >>>>
> >>>> You do have to program the "requeue" statements, yes, and the logic to
> >>>> select the target of the requeue (the "secondary server").
> >>>>
> >>>>
> >>>>> And in particular, do you have to set up locks for the queues,
> >>>>> request exclusive access to them, etc.?
> >>>>
> >>>> Mainly no; the Ada run-time system handles the queueing of entry calls
> >>>> to entries, as well as the exlusive access or other protections
> >>>> intrinsic to Ada tasks and Ada protected objects. Of course that can
> >>>> need a significant amount of code in the run-time system.
> >>>>
> >>>>
> >>>>> This is just the kind of thing that the hardware assisted message
> >>>>> passing was designed to eliminate. :-(
> >>>>
> >>>> So perhaps this sub-thread is not much off-topic after all.
> >>>
> >>> I was too quick in my reply. I shouldn't have said "hardware assisted"
> >>> above. It is really message passing that is designed to eliminate the
> >>> need to program the queues, etc. The hardware assist just makes it more
> >>> efficient.
> >>>
> >>> From my interpretation of your description, in the current Ada system,
> >>> it seems to require shared memory between the "sender", and all of the
> >>> "receivers", as they must access the common queues, etc. Is that true?
> > <
> >> Not only must the queues be shared, but the parameters in an entry call
> >> can be passed by reference, which requires shared memory.
> > <
> > Err, not quite.
> > <
> > If one passes a pointer, and there is a means to access through that pointer
> > in the callers address space (rather than the callees address space) one
> > does not actually need shared memory.
> > <
> > Consider traps in a PDP-11/70 where an application traps to the OS to perform
> > a <disk> read.
> > a) sure; the OS needs to be able to access (and verify permissions) to the
> > buffer supplied by application
> > b) but the OS uses the entire 16-bit virtual address space itself
> > c) so, there can be no actual sharing
> > d) but OS can still access pointer memory using the LD-from and ST-to
> > memory reference instructions.
<
> Ok, but then it becomes a question of defining what "shared memory"
> actually means... if the callee can access the caller's memory almost as
> immediately and rapidly as its own memory, that sounds like "shared"
> even if the address spaces are not the same.
<
Even since the definition of mmap we have had this "what does shared memory
actually mean" problem. Is multiple applications reading the same disk file using
shared memory ? What if the system performs the mmap "under the cover of the
disk cache" ? Network OSs can move disk sectors from node to node making
applications think they use shared memory when in reality they do not they are
using coherent disk sectors.
<
I think, in general, if several threads use LD and ST instructions to access
memory that all can see and modify rather efficiently, then the applications
see the area as shared--whether it is physically shared or not. The key words
here are "rather efficiently" and it is only memory when one uses LD and ST
instructions to access.
<
{{We have similar problems in the arena of multi-access ATOMIC events}}
>
> In the general case (not just application-to-OS calls) you would have to
> allow for local calls (same address space) as well as remote calls
> (different address spaces) so the call-by-reference parameter would have
> to include also an address-space identifier, in effect making it an
> extended address into a shared (but partitioned) memory.
<
Yes, I use PDP-11/70 to illustrate the mechanism (LD-from and ST-to)
However, I have the notion that there may be multiple spaces where
one can use the -from and -to semantics simultaneously. The binding
of the address and the ROOT pointer is a means to support the -from
and -to mechanisms far into the future.......
<
> > Consider a similar trap as above where the OS resides on a computer
> > at the front of the airplane and the request came from a computer at
> > the back of the airplane, and the front of airplane computer performs the
> > read and uses the network to ship the <now full> buffer back to requestor.
> > <
> > Neither example needs shared memory, both pass pointers.
<
> Huh? If you "ship the full buffer" over the network, that sounds to me
> like a pass-by-value ("out") parameter.
<
Caller did not ship the buffer to requestor !! This is the KEY point here.
The OS of the caller shipped the buffer after callee touched the page !
{Not a whole lot different from the case where the caller sends a page
that is currently residing on disk and no where in memory--except that
callee (¿acceptee?) may be processing on a DIFFERENT core than caller.}
<
Thus, it is delayed pass by value--the sending side passed its pointer
(this pointer is associated with a translation table (ROOT pointer) and
can be utilized any time after the original call has been performed--
including after it has returned.)

On 12/11/2021 12:15 PM, Niklas Holsti wrote:
> On 2021-12-11 18:37, Stephen Fuld wrote:
>> On 12/11/2021 3:32 AM, Niklas Holsti wrote:
>>> On 2021-12-11 7:56, Stephen Fuld wrote:
>>>> On 12/10/2021 2:15 PM, Niklas Holsti wrote:
>>>>> On 2021-12-10 23:27, MitchAlsup wrote:
>>>>>> On Friday, December 10, 2021 at 2:34:20 PM UTC-6, Niklas Holsti
>>>>>> wrote:
>>>>>>> On 2021-12-10 21:45, MitchAlsup wrote:
>>>>
>>>> snip
>>>>
>>>>
>>>>>> What I was looking for is accept distribution. Thread[k] can call
>>>>>> the "read" accept point and this can be accepted by any of as
>>>>>> many accept "read" threads as are potentially running.
>>>>>
>>>>>
>>>>> That is not provided as primitive in Ada, but can be programmed, of
>>>>> course. For example, the primary rendez-vous code can "requeue" the
>>>>> call to another entry, of some task or some protected object, and
>>>>> that task or protected object can be selected from a pool of
>>>>> secondary "servers" in any way you want. The requeue statement
>>>>> completes the primary rendez-vous. The caller is still suspended on
>>>>> the requeued call, but the primary server is released to continue
>>>>> its work, usually by accepting the next call.
>>>>
>>>> But do you have to program all of this "requeueing" yourself?
>>>
>>>
>>> You do have to program the "requeue" statements, yes, and the logic
>>> to select the target of the requeue (the "secondary server").
>>>
>>>
>>>> And in particular, do you have to set up locks for the queues,
>>>> request exclusive access to them, etc.?
>>>
>>> Mainly no; the Ada run-time system handles the queueing of entry
>>> calls to entries, as well as the exlusive access or other protections
>>> intrinsic to Ada tasks and Ada protected objects. Of course that can
>>> need a significant amount of code in the run-time system.
>>>
>>>
>>>> This is just the kind of thing that the hardware assisted message
>>>> passing was designed to eliminate.  :-(
>>>
>>> So perhaps this sub-thread is not much off-topic after all.
>>
>> I was too quick in my reply.  I shouldn't have said "hardware
>> assisted" above.  It is really message passing that is designed to
>> eliminate the need to program the queues, etc.  The hardware assist
>> just makes it more efficient.
>>
>>  From my interpretation of your description, in the current Ada
>> system, it seems to require shared memory between the "sender", and
>> all of the "receivers", as they must access the common queues, etc.
>> Is that true?
>
>
> Not only must the queues be shared, but the parameters in an entry call
> can be passed by reference, which requires shared memory.
>
> Ada tasking and inter-task communication does basically assume a
> shared-memory system. For example, different tasks can access the same
> global variables if the module structure makes them visible to those tasks.
>
>
>>   If so, then a message passing based system could help, as it could
>> eliminate that requirement, allowing much more flexibility in
>> allocating processes to different processors not sharing memory with
>> the "sending" process.
>
>
> I suppose an Ada-like inter-task communication system could be built on
> message passing, with the restriction that messages ("entry calls")
> could use only pass-by-value parameters or pass by copy-in copy-out
> parameters.

You could do pass-by-reference without shared memory, too, by using
thunks to turn it into pass-by-name like Algol60. A reference would look
to the program like a memory fetch, but under the hood is actually a
function call that sends a value-request message.

Slow of course, but if the messaging were hardware it should be fairly
close to a remote fetch in shared memory. Remote fetch is going to be
long latency due to SOL no matter how you do it.

> For communicating between program parts in separate memory spaces the
> "distributed systems" feature was added to Ada
> (http://www.ada-auth.org/standards/rm12_w_tc1/html/RM-E.html) but as
> said earlier in this thread that feature is more or less obsolete. It
> was based on RPC, so HW-assisted messages could certainly be used in its
> implementation, at least if longish messages are possible.

On 12/6/2021 12:30 PM, MitchAlsup wrote:
> On Monday, December 6, 2021 at 10:25:43 AM UTC-6, Stephen Fuld wrote:

snip

>>>> 2. Elxsi also used the message passing mechanism to process faults such
>>>> as divide by zero. The hardware sends a message to the offending
>>>> process, which can choose to process it or not. Thus another mechanism
>>>> can be subsumed into message passing.
>>> <
>>> Exceptions are not interrupts. Exception handlers are bound to the thread
>>> creating the exception and are synchronous, Interrupts are bound to a
>>> particular handler asynchronously and for a long duration (the SATA disk
>>> handler is associated with multiple SATA drive interrupts and for the entire
>>> time the system has power turned on.
> <
>> Yes, but I think not relevant to the way Elxsi defined its message
>> system. I don't remember all the details, but I was struck by the fact
>> that it treated exceptions the way they did. BTW, at least one of the
>> Elxsi manuals are available on bitsavers, and it gives some details. It
>> seems to have a lot of options. I don't remember, and haven't taken the
>> time to understand all the details, but I was struck by its subsuming
>> exceptions like divide by zero into the message passing system, so I
>> mentioned it in my description as a further "unification".
> <
> It is currently unclear if such unification retains value. Certainly, they all
> fall under the general concept of an unintentional subroutine call, but
> unifying the mechanics of traps with that of exceptions or context-switch
> seems dubious at best.

I tend to agree. I included it because Elixi did it, and it is an
example of unification. But it did seem odd to me.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 2021-12-11 23:36, MitchAlsup wrote:
> On Saturday, December 11, 2021 at 3:11:30 PM UTC-6, Niklas Holsti wrote:
>> On 2021-12-11 22:33, MitchAlsup wrote:
>>> On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti wrote:
>>>> On 2021-12-11 18:37, Stephen Fuld wrote:

[snip]

>>>>> From my interpretation of your description, in the current Ada system,
>>>>> it seems to require shared memory between the "sender", and all of the
>>>>> "receivers", as they must access the common queues, etc. Is that true?
>>> <
>>>> Not only must the queues be shared, but the parameters in an entry call
>>>> can be passed by reference, which requires shared memory.
>>> <
>>> Err, not quite.
>>> <
>>> If one passes a pointer, and there is a means to access through that pointer
>>> in the callers address space (rather than the callees address space) one
>>> does not actually need shared memory.

[snip]

>> Ok, but then it becomes a question of defining what "shared memory"
>> actually means... if the callee can access the caller's memory almost as
>> immediately and rapidly as its own memory, that sounds like "shared"
>> even if the address spaces are not the same.
> <
> Even since the definition of mmap we have had this "what does shared memory
> actually mean" problem. Is multiple applications reading the same disk file using
> shared memory ? What if the system performs the mmap "under the cover of the
> disk cache" ? Network OSs can move disk sectors from node to node making
> applications think they use shared memory when in reality they do not they are
> using coherent disk sectors.
> <
> I think, in general, if several threads use LD and ST instructions to access
> memory that all can see and modify rather efficiently, then the applications
> see the area as shared--whether it is physically shared or not. The key words
> here are "rather efficiently" and it is only memory when one uses LD and ST
> instructions to access.

The criterion of accessing the data with LD/ST instructions sounds like
a possible definition of "shared", yes. But if the time taken by those
instructions varies by one or more orders of magnitude, depending on the
address being accessed, the definition may not be very useful to
programmers who are concerned with execution times and responsiveness.

Granted that the variation may be large, due to caching, even in
single-processor, single-physical-memory systems.

[snip]

>>> Consider a similar trap as above where the OS resides on a computer
>>> at the front of the airplane and the request came from a computer at
>>> the back of the airplane, and the front of airplane computer performs the
>>> read and uses the network to ship the <now full> buffer back to requestor.
>>> <
>>> Neither example needs shared memory, both pass pointers.
> <
>> Huh? If you "ship the full buffer" over the network, that sounds to me
>> like a pass-by-value ("out") parameter.
> <
> Caller did not ship the buffer to requestor !! This is the KEY point here.
> The OS of the caller shipped the buffer after callee touched the page !
> {Not a whole lot different from the case where the caller sends a page
> that is currently residing on disk and no where in memory--except that
> callee (¿acceptee?) may be processing on a DIFFERENT core than caller.}
> <
> Thus, it is delayed pass by value--the sending side passed its pointer
> (this pointer is associated with a translation table (ROOT pointer) and
> can be utilized any time after the original call has been performed--
> including after it has returned.)

And on 2021-12-12 0:30, Ivan Godard wrote:

> You could do pass-by-reference without shared memory, too, by using
> thunks to turn it into pass-by-name like Algol60. A reference would
> look to the program like a memory fetch, but under the hood is
> actually a function call that sends a value-request message.
>
> Slow of course, but if the messaging were hardware it should be
> fairly close to a remote fetch in shared memory. Remote fetch is
> going to be long latency due to SOL no matter how you do it.

Yes, but that means twice the network latency for each access, compared
to once if the parameter's "in" value is shipped along with the call
message ("pushed" rather than "pulled"). And if the "out" value is
shipped along with the call-completed message, the caller can access it
faster than if the caller has to separately request it from the callee's
memory (as in Mitch's example).

So it becomes fifty shades of "shared", depending on how radically the
access time is lengthened by whatever remote access method is used. I
suppose the critical point is that the application programmers must
understand the implications of accessing such temporally distant but
logically shared memory and take it into account in their designs.

On 12/12/2021 1:29 AM, Niklas Holsti wrote:
> On 2021-12-11 23:36, MitchAlsup wrote:
>> On Saturday, December 11, 2021 at 3:11:30 PM UTC-6, Niklas Holsti wrote:
>>> On 2021-12-11 22:33, MitchAlsup wrote:
>>>> On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti
>>>> wrote:
>>>>> On 2021-12-11 18:37, Stephen Fuld wrote:
>
>    [snip]
>
>>>>>>  From my interpretation of your description, in the current Ada
>>>>>> system,
>>>>>> it seems to require shared memory between the "sender", and all of
>>>>>> the
>>>>>> "receivers", as they must access the common queues, etc. Is that
>>>>>> true?
>>>> <
>>>>> Not only must the queues be shared, but the parameters in an entry
>>>>> call
>>>>> can be passed by reference, which requires shared memory.
>>>> <
>>>> Err, not quite.
>>>> <
>>>> If one passes a pointer, and there is a means to access through that
>>>> pointer
>>>> in the callers address space (rather than the callees address space)
>>>> one
>>>> does not actually need shared memory.
>
>    [snip]
>
>>> Ok, but then it becomes a question of defining what "shared memory"
>>> actually means... if the callee can access the caller's memory almost as
>>> immediately and rapidly as its own memory, that sounds like "shared"
>>> even if the address spaces are not the same.
>> <
>> Even since the definition of mmap we have had this "what does shared
>> memory
>> actually mean" problem. Is multiple applications reading the same disk
>> file using
>> shared memory ? What if the system performs the mmap "under the cover
>> of the
>> disk cache" ?  Network OSs can move disk sectors from node to node making
>> applications think they use shared memory when in reality they do not
>> they are
>> using coherent disk sectors.
>> <
>> I think, in general, if several threads use LD and ST instructions to
>> access
>> memory that all can see and modify rather efficiently, then the
>> applications
>> see the area as shared--whether it is physically shared or not. The
>> key words
>> here are "rather efficiently" and it is only memory when one uses LD
>> and ST
>> instructions to access.
>
>
> The criterion of accessing the data with LD/ST instructions sounds like
> a possible definition of "shared", yes. But if the time taken by those
> instructions varies by one or more orders of magnitude, depending on the
> address being accessed, the definition may not be very useful to
> programmers who are concerned with execution times and responsiveness.
>
> Granted that the variation may be large, due to caching, even in
> single-processor, single-physical-memory systems.
>
>   [snip]
>
>>>> Consider a similar trap as above where the OS resides on a computer
>>>> at the front of the airplane and the request came from a computer at
>>>> the back of the airplane, and the front of airplane computer
>>>> performs the
>>>> read and uses the network to ship the <now full> buffer back to
>>>> requestor.
>>>> <
>>>> Neither example needs shared memory, both pass pointers.
>> <
>>> Huh? If you "ship the full buffer" over the network, that sounds to me
>>> like a pass-by-value ("out") parameter.
>> <
>> Caller did not ship the buffer to requestor !! This is the KEY point
>> here.
>> The OS of the caller shipped the buffer after callee touched the page !
>> {Not a whole lot different from the case where the caller sends a page
>> that is currently residing on disk and no where in memory--except that
>> callee (¿acceptee?) may be processing on a DIFFERENT core than caller.}
>> <
>> Thus, it is delayed pass by value--the sending side passed its pointer
>> (this pointer is associated with a translation table (ROOT pointer) and
>> can be utilized any time after the original call has been performed--
>> including after it has returned.)
>
>
> And on 2021-12-12 0:30, Ivan Godard wrote:
>
>
>> You could do pass-by-reference without shared memory, too, by using
>> thunks to turn it into pass-by-name like Algol60. A reference would
>> look to the program like a memory fetch, but under the hood is
>> actually a function call that sends a value-request message.
>>
>> Slow of course, but if the messaging were hardware it should be
>> fairly close to a remote fetch in shared memory. Remote fetch is
>> going to be long latency due to SOL no matter how you do it.
>
> Yes, but that means twice the network latency for each access, compared
> to once if the parameter's "in" value is shipped along with the call
> message ("pushed" rather than "pulled"). And if the "out" value is
> shipped along with the call-completed message, the caller can access it
> faster than if the caller has to separately request it from the callee's
> memory (as in Mitch's example).

But that is pass-by-copy, not pass-by-reference. If in-out semantics is
all you want then you have a big win over true pass-by-reference, but
it's not shared memory. To me, shared memory must give the same
semantics as local memory; only wall-clock differs. You can fake shared
memory with copy if you know that the value doesn't mutate during the
RPC - but if it does, or might, then you've got to go re-read it at
every program reference.

> So it becomes fifty shades of "shared", depending on how radically the
> access time is lengthened by whatever remote access method is used. I
> suppose the critical point is that the application programmers must
> understand the implications of accessing such temporally distant but
> logically shared memory and take it into account in their designs.

Yes. And indeed, pass-by-copy is only distinguishable from
pass-by-reference if the referend is mutating, which is by definition a
race condition. If the local memory model is defined only in race-free
then you can freely use copy to extend it to remote access. But if the
local model is sequential consistency then you can't get away with that.

On 2021-12-12 12:47, Ivan Godard wrote:
> On 12/12/2021 1:29 AM, Niklas Holsti wrote:

[snipped discussion of Ada rendez-vous and shared memory]

>> And on 2021-12-12 0:30, Ivan Godard wrote:
>>
>>
>>> You could do pass-by-reference without shared memory, too, by using
>>> thunks to turn it into pass-by-name like Algol60. A reference would
>>> look to the program like a memory fetch, but under the hood is
>>> actually a function call that sends a value-request message.
>>>
>>> Slow of course, but if the messaging were hardware it should be
>>> fairly close to a remote fetch in shared memory. Remote fetch is
>>> going to be long latency due to SOL no matter how you do it.
>>
>> Yes, but that means twice the network latency for each access,
>> compared to once if the parameter's "in" value is shipped along with
>> the call message ("pushed" rather than "pulled"). And if the "out"
>> value is shipped along with the call-completed message, the caller can
>> access it faster than if the caller has to separately request it from
>> the callee's memory (as in Mitch's example).
>
> But that is pass-by-copy, not pass-by-reference.

Yes. Pass-by-copy works better in the message-passing system.

> To me, shared memory must give the same semantics as local memory;
> only wall-clock differs.

I agree.

> You can fake shared memory with copy if you know that the value
> doesn't mutate during the RPC - but if it does, or might, then you've
> got to go re-read it at every program reference.

I agree.

> And indeed, pass-by-copy is only distinguishable from
> pass-by-reference if the referend is mutating,

Yes.

> which is by definition a race condition.

No. The referend may be mutated by actions taken serially by the callee,
for example if there are several access paths from parameters to the
referend. Some languages try to prevent such aliasing to some extent,
but if the parameters are sure to be passed by reference (for example,
tagged objects in Ada), or explicitly passed by reference (for example,
as pointers), they do allow it, I believe.

>>>>>> [Some time ago,] On 2021-12-11 18:37, Stephen Fuld wrote:
>>>>>>>
>>>>>>> From my interpretation of your description, in the
>>>>>>> current Ada system, it seems to require shared memory
>>>>>>> between the "sender", and all of the "receivers", as
>>>>>>> they must access the common queues, etc. Is that true?

Perhaps Stephen should be the final judge of whether the current Ada
system requires "shared memory" -- of whatever shade.

On 12/12/2021 7:01 AM, Niklas Holsti wrote:
>
> On 2021-12-12 12:47, Ivan Godard wrote:

<snip>

>> And indeed, pass-by-copy is only distinguishable from
>> pass-by-reference if the referend is mutating,
>
>
> Yes.
>
>
>> which is by definition a race condition.
>
> No. The referend may be mutated by actions taken serially by the callee,
> for example if there are several access paths from parameters to the
> referend. Some languages try to prevent such aliasing to some extent,
> but if the parameters are sure to be passed by reference (for example,
> tagged objects in Ada), or explicitly passed by reference (for example,
> as pointers), they do allow it, I believe.

True - I was thinking of remote mutation. Thank you.

On Sunday, December 12, 2021 at 3:29:52 AM UTC-6, Niklas Holsti wrote:
> On 2021-12-11 23:36, MitchAlsup wrote:
> > On Saturday, December 11, 2021 at 3:11:30 PM UTC-6, Niklas Holsti wrote:
> >> On 2021-12-11 22:33, MitchAlsup wrote:
> >>> On Saturday, December 11, 2021 at 2:15:13 PM UTC-6, Niklas Holsti wrote:
> >>>> On 2021-12-11 18:37, Stephen Fuld wrote:
> [snip]
> >>>>> From my interpretation of your description, in the current Ada system,
> >>>>> it seems to require shared memory between the "sender", and all of the
> >>>>> "receivers", as they must access the common queues, etc. Is that true?
> >>> <
> >>>> Not only must the queues be shared, but the parameters in an entry call
> >>>> can be passed by reference, which requires shared memory.
> >>> <
> >>> Err, not quite.
> >>> <
> >>> If one passes a pointer, and there is a means to access through that pointer
> >>> in the callers address space (rather than the callees address space) one
> >>> does not actually need shared memory.
> [snip]
> >> Ok, but then it becomes a question of defining what "shared memory"
> >> actually means... if the callee can access the caller's memory almost as
> >> immediately and rapidly as its own memory, that sounds like "shared"
> >> even if the address spaces are not the same.
> > <
> > Even since the definition of mmap we have had this "what does shared memory
> > actually mean" problem. Is multiple applications reading the same disk file using
> > shared memory ? What if the system performs the mmap "under the cover of the
> > disk cache" ? Network OSs can move disk sectors from node to node making
> > applications think they use shared memory when in reality they do not they are
> > using coherent disk sectors.
> > <
> > I think, in general, if several threads use LD and ST instructions to access
> > memory that all can see and modify rather efficiently, then the applications
> > see the area as shared--whether it is physically shared or not. The key words
> > here are "rather efficiently" and it is only memory when one uses LD and ST
> > instructions to access.
> The criterion of accessing the data with LD/ST instructions sounds like
> a possible definition of "shared", yes. But if the time taken by those
> instructions varies by one or more orders of magnitude, depending on the
> address being accessed, the definition may not be very useful to
> programmers who are concerned with execution times and responsiveness.
>
> Granted that the variation may be large, due to caching, even in
> single-processor, single-physical-memory systems.
>
> [snip]
> >>> Consider a similar trap as above where the OS resides on a computer
> >>> at the front of the airplane and the request came from a computer at
> >>> the back of the airplane, and the front of airplane computer performs the
> >>> read and uses the network to ship the <now full> buffer back to requestor.
> >>> <
> >>> Neither example needs shared memory, both pass pointers.
> > <
> >> Huh? If you "ship the full buffer" over the network, that sounds to me
> >> like a pass-by-value ("out") parameter.
> > <
> > Caller did not ship the buffer to requestor !! This is the KEY point here.
> > The OS of the caller shipped the buffer after callee touched the page !
> > {Not a whole lot different from the case where the caller sends a page
> > that is currently residing on disk and no where in memory--except that
> > callee (¿acceptee?) may be processing on a DIFFERENT core than caller.}
> > <
> > Thus, it is delayed pass by value--the sending side passed its pointer
> > (this pointer is associated with a translation table (ROOT pointer) and
> > can be utilized any time after the original call has been performed--
> > including after it has returned.)
> And on 2021-12-12 0:30, Ivan Godard wrote:
>
>
> > You could do pass-by-reference without shared memory, too, by using
> > thunks to turn it into pass-by-name like Algol60. A reference would
> > look to the program like a memory fetch, but under the hood is
> > actually a function call that sends a value-request message.
> >
> > Slow of course, but if the messaging were hardware it should be
> > fairly close to a remote fetch in shared memory. Remote fetch is
> > going to be long latency due to SOL no matter how you do it.
<
> Yes, but that means twice the network latency for each access, compared
> to once if the parameter's "in" value is shipped along with the call
> message ("pushed" rather than "pulled").
<
Two trips through the network is probably commensurate with reading
a missing page from the disk. As long as you get to access all the data
on the page for this single occurrence of overhead--it probably meets the
criterion of "rather efficient".
<
> And if the "out" value is
> shipped along with the call-completed message, the caller can access it
> faster than if the caller has to separately request it from the callee's
> memory (as in Mitch's example).
>
> So it becomes fifty shades of "shared", depending on how radically the
> access time is lengthened by whatever remote access method is used. I
> suppose the critical point is that the application programmers must
> understand the implications of accessing such temporally distant but
> logically shared memory and take it into account in their designs.

On 12/12/2021 7:01 AM, Niklas Holsti wrote:
>
> On 2021-12-12 12:47, Ivan Godard wrote:
>> On 12/12/2021 1:29 AM, Niklas Holsti wrote:
>
>
>    [snipped discussion of Ada rendez-vous and shared memory]
>
>
>>> And on 2021-12-12 0:30, Ivan Godard wrote:
>>>
>>>
>>>> You could do pass-by-reference without shared memory, too, by using
>>>> thunks to turn it into pass-by-name like Algol60. A reference would
>>>> look to the program like a memory fetch, but under the hood is
>>>> actually a function call that sends a value-request message.
>>>>
>>>> Slow of course, but if the messaging were hardware it should be
>>>> fairly close to a remote fetch in shared memory. Remote fetch is
>>>> going to be long latency due to SOL no matter how you do it.
>>>
>>> Yes, but that means twice the network latency for each access,
>>> compared to once if the parameter's "in" value is shipped along with
>>> the call message ("pushed" rather than "pulled"). And if the "out"
>>> value is shipped along with the call-completed message, the caller
>>> can access it faster than if the caller has to separately request it
>>> from the callee's memory (as in Mitch's example).
>>
>> But that is pass-by-copy, not pass-by-reference.
>
>
> Yes. Pass-by-copy works better in the message-passing system.
>
>
>> To me, shared memory must give the same semantics as local memory;
>> only wall-clock differs.
>
> I agree.
>
>
>> You can fake shared memory with copy if you know that the value
>> doesn't mutate during the RPC - but if it does, or might, then you've
>> got to go re-read it at every program reference.
>
>
> I agree.
>
>
>> And indeed, pass-by-copy is only distinguishable from
>> pass-by-reference if the referend is mutating,
>
>
> Yes.
>
>
>> which is by definition a race condition.
>
> No. The referend may be mutated by actions taken serially by the callee,
> for example if there are several access paths from parameters to the
> referend. Some languages try to prevent such aliasing to some extent,
> but if the parameters are sure to be passed by reference (for example,
> tagged objects in Ada), or explicitly passed by reference (for example,
> as pointers), they do allow it, I believe.
>
>
>>>>>>> [Some time ago,] On 2021-12-11 18:37, Stephen Fuld wrote:
>>>>>>>>
>>>>>>>> From my interpretation of your description, in the current Ada
>>>>>>>> system, it seems to require shared memory between the "sender",
>>>>>>>> and all of the "receivers", as
>>>>>>>> they must access the common queues, etc. Is that true?
>
> Perhaps Stephen should be the final judge of whether the current Ada
> system requires "shared memory" -- of whatever shade.

Oh No! I am certainly not the final judge of anything, least of all
what seems like a semantic argument.

However, IMHO, these various mechanisms of using some kind of messaging
to simulate/implement shared memory are not a good idea. The idea (at
least in its initial high performance applications) of message passing
is to eliminate the need for shared memory at all. Using it get back to
shared memory seems, at least to me, a step backward.

I should also say, that it seems that the a message passing
implementation should implement more than the currently implemented Ada
rendezvous mechanism, specifically some way to address multiple targets
for a source, and probably multiple sources for a target. Support of
asynch operations seems like one way to do at least part of that.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Monday, December 13, 2021 at 4:31:58 PM UTC-6, Stephen Fuld wrote:
> On 12/12/2021 7:01 AM, Niklas Holsti wrote:
> >
> > On 2021-12-12 12:47, Ivan Godard wrote:
> >
> > Perhaps Stephen should be the final judge of whether the current Ada
> > system requires "shared memory" -- of whatever shade.
> Oh No! I am certainly not the final judge of anything, least of all
> what seems like a semantic argument.
>
> However, IMHO, these various mechanisms of using some kind of messaging
> to simulate/implement shared memory are not a good idea. The idea (at
> least in its initial high performance applications) of message passing
> is to eliminate the need for shared memory at all. Using it get back to
> shared memory seems, at least to me, a step backward.
<
I see shared Memory and Message Passing as different means to an end.
Sort of like a Fourier Transform of some input data--it still contains the
original "meaning" of the data set, just in a different form.
<
Message passing can be used to give the illusion of shared memory.
Shared memory can be used to give the illusion of message passing.
<
Shared memory is more appropriate to the single mother board systems
Message passing is more appropriate to systems the size of a gymnasium.
<
The language definition should not preclude EITHER or make one harder
than the other.
>
> I should also say, that it seems that the a message passing
> implementation should implement more than the currently implemented Ada
> rendezvous mechanism, specifically some way to address multiple targets
> for a source, and probably multiple sources for a target. Support of
> asynch operations seems like one way to do at least part of that.
<
Yes to multiple acceptors from a single task pending simultaneously
Yes to multiple callers calling a vector of acceptors
<
1-binding (rendezvous) between a caller-task and acceptor-task
at any instant in time.
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

Short versus long message passing.
<
At an ISA level, one could disguise one instruction to serve as a means to send
short messages. Keyword--disguise.
<
< -----------------------------I use these so the vertical white space eater does ---------------
< ------------------------------not lose my formatting---------------------------------------------------------
<
Consider a long message, first. A long message must be passed as a pointer to
a buffer and a length (or equivalently bounds). HW applies some kind of RDMA
and delivers the message.
<
Now consider a short message. This could be a STM (store multiple) instruction
which carries its own length (first register, last register, with end around continuance).
Such a message is necessarily limited, but likely to suffices for "lots of messages"
It might even be used as a courier for a long message RDMAed from the receiving
end rather than the sending end.
<
{It is the multiple registers that enable STM to encode this service request to HW}
<
Said STM instruction could target a non-cacheable* address known by HW to be
supplying message passing services. So, no actual additions to the ISA are
required in order to give messages first class citizenship.
<
(*) or Memory Mapped I/O space. My 66000 has a 64-bit cacheable virtual address
space AND 64-bit disjoint memory mapped I/O space (non-cacheable) AND a 64-bit
virtual Configuration space, AND a 64-bit virtual BOOT ROM address space. Only the
virtual addresses are limited to 64-bits, the rest of the system works with 66-bit
addresses all controlled by the PTEs.
<
Mitch

On 12/14/2021 3:18 PM, MitchAlsup wrote:
> Short versus long message passing.
> <
> At an ISA level, one could disguise one instruction to serve as a means to send
> short messages. Keyword--disguise.
> <
> < -----------------------------I use these so the vertical white space eater does ---------------
> < ------------------------------not lose my formatting---------------------------------------------------------
> <
> Consider a long message, first. A long message must be passed as a pointer to
> a buffer and a length (or equivalently bounds). HW applies some kind of RDMA
> and delivers the message.

There is the issue of whether the message is pushed or pulled, that is,
is the RDMA done on the source or destination side.

An advantage of doing it on the source side is it improves performance
by some implementation dependent, but non zero amount. It also reduces
the hardware requirements on a receiver, as no RDMA hardware is required
there. These may be minor, but I have yet to see any advantage in doing
the RDMA on the receiving side.

Then there is the question what is the boundary between short and long,
and is that boundary technology dependent, that is, is it likely to move
in the future with newer technology. If it is likely to move, that
argues for the same mechanism to be used for any (reasonable) length, in
order to prevent long term incompatibilities.

> <
> Now consider a short message. This could be a STM (store multiple) instruction
> which carries its own length (first register, last register, with end around continuance).
> Such a message is necessarily limited, but likely to suffices for "lots of messages"
> It might even be used as a courier for a long message RDMAed from the receiving
> end rather than the sending end.
> <
> {It is the multiple registers that enable STM to encode this service request to HW}

All true, but if I understand your proposal, it requires the data
comprising the short message to be pre-loaded into consecutive
registers. This seems like a lot of extra work.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Wednesday, December 15, 2021 at 12:11:55 PM UTC-6, Stephen Fuld wrote:
> On 12/14/2021 3:18 PM, MitchAlsup wrote:
> > Short versus long message passing.
> > <
> > At an ISA level, one could disguise one instruction to serve as a means to send
> > short messages. Keyword--disguise.
> > <
> > < -----------------------------I use these so the vertical white space eater does ---------------
> > < ------------------------------not lose my formatting---------------------------------------------------------
> > <
> > Consider a long message, first. A long message must be passed as a pointer to
> > a buffer and a length (or equivalently bounds). HW applies some kind of RDMA
> > and delivers the message.
> There is the issue of whether the message is pushed or pulled, that is,
> is the RDMA done on the source or destination side.
>
> An advantage of doing it on the source side is it improves performance
> by some implementation dependent, but non zero amount. It also reduces
> the hardware requirements on a receiver, as no RDMA hardware is required
> there. These may be minor, but I have yet to see any advantage in doing
> the RDMA on the receiving side.
<
But if the targeted device also has to use messages to "talk" back at the originator,
that RDMA already has to be there. So, for example, a SATA disk or network cont-
roller already have these RDMAs.
<
>
> Then there is the question what is the boundary between short and long,
<
The boundary seems to be the size of the useable register file--something
between about 16 and 24 GPRs.
<
> and is that boundary technology dependent,
<
I don't see why the number of registers in the file are going to change with
technology.
<
> that is, is it likely to move
> in the future with newer technology. If it is likely to move, that
> argues for the same mechanism to be used for any (reasonable) length, in
> order to prevent long term incompatibilities.
<
Your argument is that if we can't solve the general problem, give up.
Not a suitable engineering response.
<
> > <
> > Now consider a short message. This could be a STM (store multiple) instruction
> > which carries its own length (first register, last register, with end around continuance).
> > Such a message is necessarily limited, but likely to suffices for "lots of messages"
> > It might even be used as a courier for a long message RDMAed from the receiving
> > end rather than the sending end.
> > <
> > {It is the multiple registers that enable STM to encode this service request to HW}
<
> All true, but if I understand your proposal, it requires the data
> comprising the short message to be pre-loaded into consecutive
> registers. This seems like a lot of extra work.
<
I think much of the time, especially for short messages, the data will be passing
through the CPU registers shortly before the message is sent. I suspect that
much of the time the address and length of the long message passes through
the GPRs not long before the message is sent, too.
<
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

On 12/15/2021 11:31 AM, MitchAlsup wrote:
> On Wednesday, December 15, 2021 at 12:11:55 PM UTC-6, Stephen Fuld wrote:
>> On 12/14/2021 3:18 PM, MitchAlsup wrote:
>>> Short versus long message passing.
>>> <
>>> At an ISA level, one could disguise one instruction to serve as a means to send
>>> short messages. Keyword--disguise.
>>> <
>>> < -----------------------------I use these so the vertical white space eater does ---------------
>>> < ------------------------------not lose my formatting---------------------------------------------------------
>>> <
>>> Consider a long message, first. A long message must be passed as a pointer to
>>> a buffer and a length (or equivalently bounds). HW applies some kind of RDMA
>>> and delivers the message.
>> There is the issue of whether the message is pushed or pulled, that is,
>> is the RDMA done on the source or destination side.
>>
>> An advantage of doing it on the source side is it improves performance
>> by some implementation dependent, but non zero amount. It also reduces
>> the hardware requirements on a receiver, as no RDMA hardware is required
>> there. These may be minor, but I have yet to see any advantage in doing
>> the RDMA on the receiving side.
> <
> But if the targeted device also has to use messages to "talk" back at the originator,
> that RDMA already has to be there. So, for example, a SATA disk or network cont-
> roller already have these RDMAs.

That is a refutation of my second point, but not the first. And it
doesn't present an advantage of doing it on the receiving side.

>>
>> Then there is the question what is the boundary between short and long,
> <
> The boundary seems to be the size of the useable register file--something
> between about 16 and 24 GPRs.
> <
>> and is that boundary technology dependent,
> <
> I don't see why the number of registers in the file are going to change with
> technology.

Well, after looking it up, Elxsi's short messages were up to 4 or 8
bytes (depending upon whether the receiver was hardware or software), so
there is some evidence that the boundary has moved.

> <
>> that is, is it likely to move
>> in the future with newer technology. If it is likely to move, that
>> argues for the same mechanism to be used for any (reasonable) length, in
>> order to prevent long term incompatibilities.
> <
> Your argument is that if we can't solve the general problem, give up.
> Not a suitable engineering response.

No, not my argument at all. If anything, I am arguing for the
programmer interface be the same for as many different types of messages
as possible. Thus, for example, I want messages to hardware to differ
from messages to software only in destination address. I guess if short
messages can be done faster (not counting transferring fewer bytes) by a
different mechanism, that might be OK, but in general, I don't want the
programmer to have to do anything different if the length of the message
changes (of course other than changing the length indicator).

>>> <
>>> Now consider a short message. This could be a STM (store multiple) instruction
>>> which carries its own length (first register, last register, with end around continuance).
>>> Such a message is necessarily limited, but likely to suffices for "lots of messages"
>>> It might even be used as a courier for a long message RDMAed from the receiving
>>> end rather than the sending end.
>>> <
>>> {It is the multiple registers that enable STM to encode this service request to HW}
> <
>> All true, but if I understand your proposal, it requires the data
>> comprising the short message to be pre-loaded into consecutive
>> registers. This seems like a lot of extra work.
> <
> I think much of the time, especially for short messages, the data will be passing
> through the CPU registers shortly before the message is sent.

Perhaps, but you are adding the requirement that the registers they pass
through are consecutive and in order. This either adds extra work to
load it, or extra work in the compiler's register allocator to insure it
is so.

> I suspect that
> much of the time the address and length of the long message passes through
> the GPRs not long before the message is sent, too.

Again, perhaps, but I can see the information describing the target
address, message length and pointer to the data as a predefined "packet"
existing in memory, with the software just changing the values in some
of the fields in the packet, or even just changing the contents of the
data in the message with each invocation.

Of course, predicting usage of a yet to be implemented feature is, at
best, error prone. :-(

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Wednesday, December 15, 2021 at 2:22:25 PM UTC-6, Stephen Fuld wrote:
> On 12/15/2021 11:31 AM, MitchAlsup wrote:
> > On Wednesday, December 15, 2021 at 12:11:55 PM UTC-6, Stephen Fuld wrote:
> >> On 12/14/2021 3:18 PM, MitchAlsup wrote:
> >>> Short versus long message passing.
> >>> <
> >>> At an ISA level, one could disguise one instruction to serve as a means to send
> >>> short messages. Keyword--disguise.
> >>> <
> >>> < -----------------------------I use these so the vertical white space eater does ---------------
> >>> < ------------------------------not lose my formatting---------------------------------------------------------
> >>> <
> >>> Consider a long message, first. A long message must be passed as a pointer to
> >>> a buffer and a length (or equivalently bounds). HW applies some kind of RDMA
> >>> and delivers the message.
> >> There is the issue of whether the message is pushed or pulled, that is,
> >> is the RDMA done on the source or destination side.
> >>
> >> An advantage of doing it on the source side is it improves performance
> >> by some implementation dependent, but non zero amount. It also reduces
> >> the hardware requirements on a receiver, as no RDMA hardware is required
> >> there. These may be minor, but I have yet to see any advantage in doing
> >> the RDMA on the receiving side.
> > <
> > But if the targeted device also has to use messages to "talk" back at the originator,
> > that RDMA already has to be there. So, for example, a SATA disk or network cont-
> > roller already have these RDMAs.
<
> That is a refutation of my second point, but not the first. And it
> doesn't present an advantage of doing it on the receiving side.
> >>
> >> Then there is the question what is the boundary between short and long,
> > <
> > The boundary seems to be the size of the useable register file--something
> > between about 16 and 24 GPRs.
> > <
> >> and is that boundary technology dependent,
> > <
> > I don't see why the number of registers in the file are going to change with
> > technology.
<
> Well, after looking it up, Elxsi's short messages were up to 4 or 8
> bytes (depending upon whether the receiver was hardware or software), so
> there is some evidence that the boundary has moved.
> > <
> >> that is, is it likely to move
> >> in the future with newer technology. If it is likely to move, that
> >> argues for the same mechanism to be used for any (reasonable) length, in
> >> order to prevent long term incompatibilities.
> > <
> > Your argument is that if we can't solve the general problem, give up.
> > Not a suitable engineering response.
<
> No, not my argument at all. If anything, I am arguing for the
> programmer interface be the same for as many different types of messages
> as possible. Thus, for example, I want messages to hardware to differ
> from messages to software only in destination address. I guess if short
> messages can be done faster (not counting transferring fewer bytes) by a
> different mechanism, that might be OK, but in general, I don't want the
> programmer to have to do anything different if the length of the message
> changes (of course other than changing the length indicator).
> >>> <
> >>> Now consider a short message. This could be a STM (store multiple) instruction
> >>> which carries its own length (first register, last register, with end around continuance).
> >>> Such a message is necessarily limited, but likely to suffices for "lots of messages"
> >>> It might even be used as a courier for a long message RDMAed from the receiving
> >>> end rather than the sending end.
> >>> <
> >>> {It is the multiple registers that enable STM to encode this service request to HW}
> > <
> >> All true, but if I understand your proposal, it requires the data
> >> comprising the short message to be pre-loaded into consecutive
> >> registers. This seems like a lot of extra work.
> > <
> > I think much of the time, especially for short messages, the data will be passing
> > through the CPU registers shortly before the message is sent.
<
> Perhaps, but you are adding the requirement that the registers they pass
> through are consecutive and in order. This either adds extra work to
> load it, or extra work in the compiler's register allocator to insure it
> is so.
<
If the task wanting to send a message has to call the message transport service
the consecutive nature of pointer and count can be set by simply making the
parameters appear in argument list order; the compiler has already been trained
to use the ABI and here is where designing the interface properly means both
HW and SW are efficient.
<
for example: read( file_des, buffer, len ) will have &buffer in R2 and len in R3
according to ABI.
<
I find it far easier to get these pairing and sharing of registers into object code
by using the call/return interface properly (than by pounding on the register
allocator.)
<
> > I suspect that
> > much of the time the address and length of the long message passes through
> > the GPRs not long before the message is sent, too.
> Again, perhaps, but I can see the information describing the target
> address, message length and pointer to the data as a predefined "packet"
> existing in memory, with the software just changing the values in some
> of the fields in the packet, or even just changing the contents of the
> data in the message with each invocation.
>
> Of course, predicting usage of a yet to be implemented feature is, at
> best, error prone. :-(
> --
> - Stephen Fuld
> (e-mail address disguised to prevent spam)

A couple of years ago I made a network-on-chip consisting of 56 nodes
connected as a torus. One node was dedicated to shared memory and I/O.
Each node had a small local memory and ROM.
Packets of information which could be considered to be messages were
passed through a routing network. I used to following small packets
(96 bits in size):

// Shared memory / IO is node 62
// Packets automatically deleted by hardware if too old.

typedef struct tagPacket
{ unsigned int rid : 6; // intended reciever (63 = global broadcast)
unsigned int tid : 6; // transmitting node (0 = empty packet)
unsigned int age : 6;
unsigned int ack : 1;
unsigned int type : 6; // read,write,interrupt, etc.
unsigned int pad : 2; // unused bits
unsigned int we : 1; // write enable
unsigned int sel : 4; // byte lane selects
unsigned int addr : 32; // memory / IO address
unsigned int data : 32;
} Packet;

Messsages always moved to the right or downwards. An entire message was
transferred every clock cycle between routers. Packets were queued in a
fifo at the destination node.

It turns out that the NoC did not work that well. There were always about
a half-dozen cores not responding. I had each core displaying a status
character in a grid on the display.

A lot can be done with even a small size message.

I had three 32-bit words as an I/O device mapped into the local memory
space of a node to access network packets. A fourth IO word was for
control and status. Ordinary load / store instructions were used to
access data. Excepting the dedicated shared memory / IO node which was
entirely hardware access. I think programmatic access to the message
packet took much longer than the time required to transmit or recieve
a message. IIRC the maximum distance a message would travel is 15 nodes,
eight across and seven down. That would be only 15 clock cycles unless
the message needed to cycle around again.

On Wednesday, December 15, 2021 at 6:07:53 PM UTC-6, robf...@gmail.com wrote:
> A couple of years ago I made a network-on-chip consisting of 56 nodes
> connected as a torus. One node was dedicated to shared memory and I/O.
> Each node had a small local memory and ROM.
> Packets of information which could be considered to be messages were
> passed through a routing network. I used to following small packets
> (96 bits in size):
>
> // Shared memory / IO is node 62
> // Packets automatically deleted by hardware if too old.
>
> typedef struct tagPacket
> {
> unsigned int rid : 6; // intended reciever (63 = global broadcast)
> unsigned int tid : 6; // transmitting node (0 = empty packet)
> unsigned int age : 6;
> unsigned int ack : 1;
> unsigned int type : 6; // read,write,interrupt, etc.
> unsigned int pad : 2; // unused bits
> unsigned int we : 1; // write enable
> unsigned int sel : 4; // byte lane selects
> unsigned int addr : 32; // memory / IO address
> unsigned int data : 32;
> } Packet;
>
> Messsages always moved to the right or downwards. An entire message was
> transferred every clock cycle between routers. Packets were queued in a
> fifo at the destination node.
>
> It turns out that the NoC did not work that well. There were always about
> a half-dozen cores not responding. I had each core displaying a status
> character in a grid on the display.
<
Cray XMP with 8 CPUs provided essentially full memory BW to 6 of the
8 (possible) CPUs, CPU[7] got 60%-ish of what it should have, CPU[8]
basically was left starving much of the time. CRAY YMP solved this by
having the next consumer of the BW allocated in a circular fashion per
port. So, CRAY left about 20% of system performance on the table--rather
similar to your data. 19.3% = 12/62
<
Interesting........
<
>
> A lot can be done with even a small size message.
>
> I had three 32-bit words as an I/O device mapped into the local memory
> space of a node to access network packets. A fourth IO word was for
> control and status. Ordinary load / store instructions were used to
> access data. Excepting the dedicated shared memory / IO node which was
> entirely hardware access. I think programmatic access to the message
> packet took much longer than the time required to transmit or recieve
> a message. IIRC the maximum distance a message would travel is 15 nodes,
> eight across and seven down. That would be only 15 clock cycles unless
> the message needed to cycle around again.