The discussions going on about register to/from stack and load/store
multiple instructions has got me vaguely remembering that there was some
talk about old mainframes that could save to stack automatically any
registers in danger of being overwritten after a jump to subroutine or such.

Anyone else remember such a thing?, or am I just making things up by
reason of senility and/or madness? (entirely possible I'm afraid :-( )

I wonder, that if they were real, is it possible given today's
transistor core counts those mechanisms could be revived in modern clean
sheet cpu designs?, to at least help alleviate some of the issues we see
in the current state of the art perhaps?

Or were there some really big downsides to the automagic things that I'm
not remembering well enough?

Andy wrote:
> The discussions going on about register to/from stack and load/store
> multiple instructions has got me vaguely remembering that there was some
> talk about old mainframes that could save to stack automatically any
> registers in danger of being overwritten after a jump to subroutine or
> such.
>
> Anyone else remember such a thing?, or am I just making things up by
> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>
>
> I wonder, that if they were real, is it possible given today's
> transistor core counts those mechanisms could be revived in modern clean
> sheet cpu designs?, to at least help alleviate some of the issues we see
> in the current state of the art perhaps?
>
> Or were there some really big downsides to the automagic things that I'm
> not remembering well enough?

Sparc register windows. They were opaque, asynchronous lazy spill/fill
which was done by kernel mode traps. Reportedly it had... issues.

A quicky search of 'sparc "register window" ' finds

[paywalled]
Error behavior comparison of multiple computing systems: A case study
using linux on pentium, solaris on sparc, and aix on power, 2008
https://ieeexplore.ieee.org/abstract/document/4725314/

I found a copy available in a online viewer with download option:

https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkY2hlbjhyZXNlYXJjaHxneDoxNzdkNWMxOWVmOWNhMDdm

says:
"The registers introduced to manage the register window in the
SPARC architecture are particularly error-sensitive and contribute
to more than 50% of the hang cases for the Solaris System.
This result indicates that, while using the register window may
improve performance, it can create unexpected reliability problems."

On Monday, July 4, 2022 at 12:48:36 AM UTC-5, Andy wrote:
> The discussions going on about register to/from stack and load/store
> multiple instructions has got me vaguely remembering that there was some
> talk about old mainframes that could save to stack automatically any
> registers in danger of being overwritten after a jump to subroutine or such.
>
> Anyone else remember such a thing?, or am I just making things up by
> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>
IBM 360 series::
caller allocated a register save area and always kept it in R13
upon any arrival (Callee, interruptee) would perform STM 12,12(13)
which would dump all 16 registers in the save area, as the first step
in properly receiving control.
<
VAX did something similar, but performed in in the CALL side of ISA
processing.
>
> I wonder, that if they were real, is it possible given today's
> transistor core counts those mechanisms could be revived in modern clean
> sheet cpu designs?, to at least help alleviate some of the issues we see
> in the current state of the art perhaps?
<
Effectively that is what ENTER and EXIT do in My 66000 architecture.
>
> Or were there some really big downsides to the automagic things that I'm
> not remembering well enough?
<
Automagic was so slow in VAX that the more modern compilers used JSR
instead of CALL and got rid of ½ of the cycles in calling/returning.

According to MitchAlsup <MitchAlsup@aol.com>:
>> Anyone else remember such a thing?, or am I just making things up by
>> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>>
>IBM 360 series::
>caller allocated a register save area and always kept it in R13
>upon any arrival (Callee, interruptee) would perform STM 12,12(13)
>which would dump all 16 registers in the save area, as the first step
>in properly receiving control.

Actually it was STM 14,12,12(13). You chained save area pointers in
R13 in other ways depending on whether your routine was reentrant or
recursive or neither. The first "12" might be a smaller number if your
routine didn't change the high numbered registers. On TSS the sequence
was messier since every routine had both a code pointer (V address) and a
data pointer (R address.)

For obvious reasons interrupts could not depend on register contents
and saved registers in a static place in low memory.

>VAX did something similar, but performed in in the CALL side of ISA
>processing.

The VAX calling sequence had a bit mask of registers to save at
the start of the procedure and very complex CALL/RET instructions that
did the save and set up the stack frame. They were so slow that a lot
of software used the much simpler JSB/RSB that just saved the return
address on the stack and jumped.

SPARC used the old pcc compiler which didn't do very sophisticated
register management, so the SPARC register windows were a way to
save and restore all the registers fast. The contemporary 801
project used the much better PL.8 compiler so their calling sequence
just saved the registers it needed to. The 801 did not have load
or store multiple but the ROMP put it back in so they could get
full memory speed on the load/store traffic.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On 2022-07-04 19:14, EricP wrote:
> Andy wrote:
>> The discussions going on about register to/from stack and load/store
>> multiple instructions has got me vaguely remembering that there was
>> some talk about old mainframes that could save to stack automatically
>> any registers in danger of being overwritten after a jump to
>> subroutine or such.
>>
>> Anyone else remember such a thing?, or am I just making things up by
>> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>>
>>
>> I wonder, that if they were real, is it possible given today's
>> transistor core counts those mechanisms could be revived in modern
>> clean sheet cpu designs?, to at least help alleviate some of the
>> issues we see in the current state of the art perhaps?
>>
>> Or were there some really big downsides to the automagic things that
>> I'm not remembering well enough?
>
> Sparc register windows. They were opaque, asynchronous lazy spill/fill
> which was done by kernel mode traps. Reportedly it had... issues.

What sort of "issues"? Performance wrt alternatives? SPARC processors
are still used extensively in space systems, in particular on ESA
missions. I've written embedded SW for several such systems and I'm not
aware of any serious issues.

Register windows do complicate static WCET analysis a bit, because it is
not easy to predict exactly which call or which return can cause a trap.
But an upper bound can be calculated that is not very pessimistic. And
who uses static WCET analysis any more :-(

I have read that one port of gcc for SPARC offers a "flat"
register-usage model as an alternative to register windows. In the
"flat" model the same set of 32 registers is used at all points in the
program, and the register windows are never rotated (except for trap
handling, I assume). I suppose this "flat" option was implemented for a
reason, but the reason was not explained, IIRC.

A minor annoyance of register windows, and of the standard trap handlers
for rotating windows, is that every non-leaf subroutine must allocate 96
octets of stack for saving 24 registers. This area is used if, and only
if, the subroutine executes a call that causes a register-ring-overflow
trap. A programming style that uses many small subroutines can lead to
suprisingly large stack usage.

> A quicky search of 'sparc "register window" ' finds
>
> [paywalled]
> Error behavior comparison of multiple computing systems: A case study
> using linux on pentium, solaris on sparc, and aix on power, 2008
> https://ieeexplore.ieee.org/abstract/document/4725314/
>
> I found a copy available in a online viewer with download option:
>
> https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkY2hlbjhyZXNlYXJjaHxneDoxNzdkNWMxOWVmOWNhMDdm

Thanks for this reference...

> says:
> "The registers introduced to manage the register window in the
> SPARC architecture are particularly error-sensitive and contribute
> to more than 50% of the hang cases for the Solaris System.
> This result indicates that, while using the register window may
> improve performance, it can create unexpected reliability problems."

These are not "natural", real-life errors and hangs. That study injects
single-bit errors into memory words and system registers and sees what
happens. It is to be expected that critical system registers, such as
the ones involved in register-ring management, are very sensititive to
such simulated HW errors. This is not a flaw in the logical
architecture, but HW architects may want to make these registers
especially robust.

SPARC processors used in space applications are radiation-tolerant and
use triple modular redundancy for critical registers.

On 2022-07-04, MitchAlsup wrote:
> On Monday, July 4, 2022 at 12:48:36 AM UTC-5, Andy wrote:
>> The discussions going on about register to/from stack and load/store
>> multiple instructions has got me vaguely remembering that there was some
>> talk about old mainframes that could save to stack automatically any
>> registers in danger of being overwritten after a jump to subroutine or such.
>>
>> Anyone else remember such a thing?, or am I just making things up by
>> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>>
> IBM 360 series::
> caller allocated a register save area and always kept it in R13
> upon any arrival (Callee, interruptee) would perform STM 12,12(13)
> which would dump all 16 registers in the save area, as the first step
> in properly receiving control.

How does this differ from a regular STM to stack (except that the
allocation is done by the caller instead of the callee)? E.g. 68k style:

movem.l d0-d7/a0-a6,-(sp)

> <
> VAX did something similar, but performed in in the CALL side of ISA
> processing.
>>
>> I wonder, that if they were real, is it possible given today's
>> transistor core counts those mechanisms could be revived in modern clean
>> sheet cpu designs?, to at least help alleviate some of the issues we see
>> in the current state of the art perhaps?
> <
> Effectively that is what ENTER and EXIT do in My 66000 architecture.
>>
>> Or were there some really big downsides to the automagic things that I'm
>> not remembering well enough?
> <
> Automagic was so slow in VAX that the more modern compilers used JSR
> instead of CALL and got rid of ½ of the cycles in calling/returning.

On 2022-07-04 21:56, John Levine wrote:
>
> SPARC used the old pcc compiler which didn't do very sophisticated
> register management, so the SPARC register windows were a way to
> save and restore all the registers fast.

Nitpick: not "all the registers". When a SPARC rotates the register
window in connection with a call, 16 registers visible to the caller are
saved (become inaccessible). These are the 8 "in" registers and the 8
"local" registers of the caller. The callee gets 16 new registers: 8 new
"local" registers and 8 new "out" registers.

The 8 "out" registers of the caller are seen in the callee as the 8 "in"
registers of the callee. Successive windows overlap on those 8
registers, which of course are used to pass parameters.

In addition there are 8 "global" registers that are not held in the
register ring and are always accessible with the same names.

IIRC...

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>And who uses static WCET analysis any more :-(

I would presume that those applications that needed WCET (worst-case
execution time) analysis in the past still need it. What would they
use instead?

>I have read that one port of gcc for SPARC offers a "flat"
>register-usage model as an alternative to register windows. In the
>"flat" model the same set of 32 registers is used at all points in the
>program, and the register windows are never rotated (except for trap
>handling, I assume). I suppose this "flat" option was implemented for a
>reason, but the reason was not explained, IIRC.

I read somewhere that the register windows could be used for fast task
switching instead of for fast calling. To make use of that you would
need a compiler that does not use the windows for calling; it would
also have to limit its register usage to 16 (one window per task) or
24 (two windows per task) registers.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 7/5/2022 12:54 AM, Anton Ertl wrote:
> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>> And who uses static WCET analysis any more :-(
>
> I would presume that those applications that needed WCET (worst-case
> execution time) analysis in the past still need it. What would they
> use instead?
>
>> I have read that one port of gcc for SPARC offers a "flat"
>> register-usage model as an alternative to register windows. In the
>> "flat" model the same set of 32 registers is used at all points in the
>> program, and the register windows are never rotated (except for trap
>> handling, I assume). I suppose this "flat" option was implemented for a
>> reason, but the reason was not explained, IIRC.
>
> I read somewhere that the register windows could be used for fast task
> switching instead of for fast calling. To make use of that you would
> need a compiler that does not use the windows for calling; it would
> also have to limit its register usage to 16 (one window per task) or
> 24 (two windows per task) registers.

Shouldn't need such limits. You can flush the windows by a recursive
call (making a dummy window) that stops when the end interrupt happens.
Then switch the stack and do a recursive return of the number of dummy
windows that were injected when the new stack was previously switched
out of, saved somewhere (possibly in the dummy). The flush/restore can
tell how many dummy windows there are by counting the recursive calls
before the interrupt; the overflow routine doesn't have to spill any of
the dummies, nor the restore refill them. It does have to spill/refill
the non-dummy windows, but that's inevitable.

It would go even faster if the hardware refilled lazy, assuming that the
spilled windows don't need to be window-block aligned; do you know it
that was required by the implementation?

On Tuesday, July 5, 2022 at 1:33:03 AM UTC-5, Marcus wrote:
> On 2022-07-04, MitchAlsup wrote:
> > On Monday, July 4, 2022 at 12:48:36 AM UTC-5, Andy wrote:
> >> The discussions going on about register to/from stack and load/store
> >> multiple instructions has got me vaguely remembering that there was some
> >> talk about old mainframes that could save to stack automatically any
> >> registers in danger of being overwritten after a jump to subroutine or such.
> >>
> >> Anyone else remember such a thing?, or am I just making things up by
> >> reason of senility and/or madness? (entirely possible I'm afraid :-( )
> >>
> > IBM 360 series::
> > caller allocated a register save area and always kept it in R13
> > upon any arrival (Callee, interruptee) would perform STM 12,12(13)
> > which would dump all 16 registers in the save area, as the first step
> > in properly receiving control.
> How does this differ from a regular STM to stack (except that the
> allocation is done by the caller instead of the callee)? E.g. 68k style:
<
It was a linked list not a stack.
>
> movem.l d0-d7/a0-a6,-(sp)
> > <
> > VAX did something similar, but performed in in the CALL side of ISA
> > processing.
> >>
> >> I wonder, that if they were real, is it possible given today's
> >> transistor core counts those mechanisms could be revived in modern clean
> >> sheet cpu designs?, to at least help alleviate some of the issues we see
> >> in the current state of the art perhaps?
> > <
> > Effectively that is what ENTER and EXIT do in My 66000 architecture.
> >>
> >> Or were there some really big downsides to the automagic things that I'm
> >> not remembering well enough?
> > <
> > Automagic was so slow in VAX that the more modern compilers used JSR
> > instead of CALL and got rid of ½ of the cycles in calling/returning.

Niklas Holsti wrote:
> On 2022-07-04 19:14, EricP wrote:
>> Andy wrote:
>>> The discussions going on about register to/from stack and load/store
>>> multiple instructions has got me vaguely remembering that there was
>>> some talk about old mainframes that could save to stack automatically
>>> any registers in danger of being overwritten after a jump to
>>> subroutine or such.
>>>
>>> Anyone else remember such a thing?, or am I just making things up by
>>> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>>>
>>>
>>> I wonder, that if they were real, is it possible given today's
>>> transistor core counts those mechanisms could be revived in modern
>>> clean sheet cpu designs?, to at least help alleviate some of the
>>> issues we see in the current state of the art perhaps?
>>>
>>> Or were there some really big downsides to the automagic things that
>>> I'm not remembering well enough?
>>
>> Sparc register windows. They were opaque, asynchronous lazy spill/fill
>> which was done by kernel mode traps. Reportedly it had... issues.
>
>
> What sort of "issues"? Performance wrt alternatives? SPARC processors
> are still used extensively in space systems, in particular on ESA
> missions. I've written embedded SW for several such systems and I'm not
> aware of any serious issues.
>
> Register windows do complicate static WCET analysis a bit, because it is
> not easy to predict exactly which call or which return can cause a trap.
> But an upper bound can be calculated that is not very pessimistic. And
> who uses static WCET analysis any more :-(
>
> I have read that one port of gcc for SPARC offers a "flat"
> register-usage model as an alternative to register windows. In the
> "flat" model the same set of 32 registers is used at all points in the
> program, and the register windows are never rotated (except for trap
> handling, I assume). I suppose this "flat" option was implemented for a
> reason, but the reason was not explained, IIRC.
>
> A minor annoyance of register windows, and of the standard trap handlers
> for rotating windows, is that every non-leaf subroutine must allocate 96
> octets of stack for saving 24 registers. This area is used if, and only
> if, the subroutine executes a call that causes a register-ring-overflow
> trap. A programming style that uses many small subroutines can lead to
> suprisingly large stack usage.
>
>
>> A quicky search of 'sparc "register window" ' finds
>>
>> [paywalled]
>> Error behavior comparison of multiple computing systems: A case study
>> using linux on pentium, solaris on sparc, and aix on power, 2008
>> https://ieeexplore.ieee.org/abstract/document/4725314/
>>
>> I found a copy available in a online viewer with download option:
>>
>> https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkY2hlbjhyZXNlYXJjaHxneDoxNzdkNWMxOWVmOWNhMDdm
>
>
>
> Thanks for this reference...
>
>
>> says:
>> "The registers introduced to manage the register window in the
>> SPARC architecture are particularly error-sensitive and contribute
>> to more than 50% of the hang cases for the Solaris System.
>> This result indicates that, while using the register window may
>> improve performance, it can create unexpected reliability problems."
>
>
> These are not "natural", real-life errors and hangs. That study injects
> single-bit errors into memory words and system registers and sees what
> happens. It is to be expected that critical system registers, such as
> the ones involved in register-ring management, are very sensititive to
> such simulated HW errors. This is not a flaw in the logical
> architecture, but HW architects may want to make these registers
> especially robust.

Right, the errors were simulated but as I read it they contend the
fragility is due to the register windows and the way were manipulated.

> SPARC processors used in space applications are radiation-tolerant and
> use triple modular redundancy for critical registers.

I can't find the references just now but I have seen it
discussed in the past so I'll see what I can dig up.

IIRC the primary issue was the phase delay in window save/restore
was opaque so code could only know for sure that memory was up to
date by explicit flushing through OS traps -
something akin to manual cache coherence.
I would be paranoid about things like general OS calls, exceptions,
interrupts, user mode task switches not seeing a coherent view of memory,
and would flush the pending register window on entry to these
until each can be proved safe, on a case-by-case basis.

One issue is even for in-order implementations it can require a large
number of hardware registers but programmers can only use a small number,
similar to costs in bank-switched register sets like Arm32.

H&P note of Sparc:
- Given that each window has 16 unique registers, an implementation of
SPARC can have as few as 40 physical registers and as many as 520,
although most have 128 to 136.
...
The danger of register windows is that the larger number of
registers could slow down the clock rate."

One compiler writer notes:
- "The [Sparc] architecture requires a designer to implement 120 registers,
of which only 24 to 29 are available for use by the compiler writer."

- "Setjmp is an exceptional case for saving register windows.
Because setjmp saves processor state, it is necessary for it to
force the hidden register state to the stack and to save the current
state into the jump buffer. ...
This makes setjmp an exceptionally slow operation."

- some comments in comp.compilers
https://compilers.iecc.com/comparch/article/94-02-130
https://compilers.iecc.com/comparch/article/94-02-134

Ivan Godard wrote:
> On 7/5/2022 12:54 AM, Anton Ertl wrote:
>> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>>> And who uses static WCET analysis any more :-(
>>
>> I would presume that those applications that needed WCET (worst-case
>> execution time) analysis in the past still need it. What would they
>> use instead?
>>
>>> I have read that one port of gcc for SPARC offers a "flat"
>>> register-usage model as an alternative to register windows. In the
>>> "flat" model the same set of 32 registers is used at all points in the
>>> program, and the register windows are never rotated (except for trap
>>> handling, I assume). I suppose this "flat" option was implemented for a
>>> reason, but the reason was not explained, IIRC.
>>
>> I read somewhere that the register windows could be used for fast task
>> switching instead of for fast calling. To make use of that you would
>> need a compiler that does not use the windows for calling; it would
>> also have to limit its register usage to 16 (one window per task) or
>> 24 (two windows per task) registers.
>
>
> Shouldn't need such limits. You can flush the windows by a recursive
> call (making a dummy window) that stops when the end interrupt happens.
> Then switch the stack and do a recursive return of the number of dummy
> windows that were injected when the new stack was previously switched
> out of, saved somewhere (possibly in the dummy). The flush/restore can
> tell how many dummy windows there are by counting the recursive calls
> before the interrupt; the overflow routine doesn't have to spill any of
> the dummies, nor the restore refill them. It does have to spill/refill
> the non-dummy windows, but that's inevitable.
>
> It would go even faster if the hardware refilled lazy, assuming that the
> spilled windows don't need to be window-block aligned; do you know it
> that was required by the implementation?

The size of the Sparc saved HW register window is model dependent
from 2..32 so to flush it you would have to do 32 SAVE instructions,
which could potentially trigger multiple OS traps.

The hardware doesn't seem to have the tracking registers necessary
to optimize save/restore, though maybe later models did.

One improvment might be to have a model register that can be
read to find the HW window size in user mode.
Another could be a FLUSH instruction.

It looks like it just has a Valid bit per register window.
It also needs a Modified bit to track changes since last spill/fill,
and low-water and high-water spill/refill stack pointers.
That would allow it to only flush windows modified since reload,
spill-ahead when the window is 3/4 full, refill-ahead when 1/4 full,
and minimize the window management stalls.

According to MitchAlsup <MitchAlsup@aol.com>:
>> > IBM 360 series::
>> > caller allocated a register save area and always kept it in R13
>> > upon any arrival (Callee, interruptee) would perform STM 12,12(13)
>> > which would dump all 16 registers in the save area, as the first step
>> > in properly receiving control.
>> How does this differ from a regular STM to stack (except that the
>> allocation is done by the caller instead of the callee)? E.g. 68k style:
><
>It was a linked list not a stack.

Depends on the details. In PL/I it was a stack, in Fortran and COBOL which
did not allow recursion, it was a linked list of static save areas.

>> > Automagic was so slow in VAX that the more modern compilers used JSR
>> > instead of CALL and got rid of ½ of the cycles in calling/returning.

JSB actually, but yes, many of the complex VAX instructions were so slow that
they weren't useful.

VAX appears to have been designed in an era when microcode was much faster
than RAM so everything was limited by memory speed. Unfortunately, by the
time there were actual VAX computers, that wasn't true any more.
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On 2022-07-05 19:38, EricP wrote:
> Niklas Holsti wrote:
>> On 2022-07-04 19:14, EricP wrote:
>>> Andy wrote:
>>>> The discussions going on about register to/from stack and load/store
>>>> multiple instructions has got me vaguely remembering that there was
>>>> some talk about old mainframes that could save to stack
>>>> automatically any registers in danger of being overwritten after a
>>>> jump to subroutine or such.
>>>>
>>>> Anyone else remember such a thing?, or am I just making things up by
>>>> reason of senility and/or madness? (entirely possible I'm afraid :-( )
>>>>
>>>>
>>>> I wonder, that if they were real, is it possible given today's
>>>> transistor core counts those mechanisms could be revived in modern
>>>> clean sheet cpu designs?, to at least help alleviate some of the
>>>> issues we see in the current state of the art perhaps?
>>>>
>>>> Or were there some really big downsides to the automagic things that
>>>> I'm not remembering well enough?
>>>
>>> Sparc register windows. They were opaque, asynchronous lazy spill/fill
>>> which was done by kernel mode traps. Reportedly it had... issues.
>>
>>
>> What sort of "issues"? Performance wrt alternatives? SPARC processors
>> are still used extensively in space systems, in particular on ESA
>> missions. I've written embedded SW for several such systems and I'm
>> not aware of any serious issues.
>>
>> Register windows do complicate static WCET analysis a bit, because it
>> is not easy to predict exactly which call or which return can cause a
>> trap. But an upper bound can be calculated that is not very
>> pessimistic. And who uses static WCET analysis any more :-(
>>
>> I have read that one port of gcc for SPARC offers a "flat"
>> register-usage model as an alternative to register windows. In the
>> "flat" model the same set of 32 registers is used at all points in the
>> program, and the register windows are never rotated (except for trap
>> handling, I assume). I suppose this "flat" option was implemented for
>> a reason, but the reason was not explained, IIRC.
>>
>> A minor annoyance of register windows, and of the standard trap
>> handlers for rotating windows, is that every non-leaf subroutine must
>> allocate 96 octets of stack for saving 24 registers. This area is used
>> if, and only if, the subroutine executes a call that causes a
>> register-ring-overflow trap. A programming style that uses many small
>> subroutines can lead to suprisingly large stack usage.
>>
>>
>>> A quicky search of 'sparc "register window" ' finds
>>>
>>> [paywalled]
>>> Error behavior comparison of multiple computing systems: A case study
>>> using linux on pentium, solaris on sparc, and aix on power, 2008
>>> https://ieeexplore.ieee.org/abstract/document/4725314/
>>>
>>> I found a copy available in a online viewer with download option:
>>>
>>> https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxkY2hlbjhyZXNlYXJjaHxneDoxNzdkNWMxOWVmOWNhMDdm
>>
>>
>>
>>
>> Thanks for this reference...
>>
>>
>>> says:
>>> "The registers introduced to manage the register window in the
>>> SPARC architecture are particularly error-sensitive and contribute
>>> to more than 50% of the hang cases for the Solaris System.
>>> This result indicates that, while using the register window may
>>> improve performance, it can create unexpected reliability problems."
>>
>>
>> These are not "natural", real-life errors and hangs. That study
>> injects single-bit errors into memory words and system registers and
>> sees what happens. It is to be expected that critical system
>> registers, such as the ones involved in register-ring management, are
>> very sensititive to such simulated HW errors. This is not a flaw in
>> the logical architecture, but HW architects may want to make these
>> registers especially robust.
>
> Right, the errors were simulated but as I read it they contend the
> fragility is due to the register windows and the way were manipulated.

But they also say: "Injections into the system registers indicate that
each of the three processors (the Pentium 4, UltraSPARC IIIi, and
POWER5) have two to three critical registers that are very sensitive to
errors (see Figure 9 through Figure 11), those being IDTR, GDTR and CR4
in the Pentium 4; TBA and SET_SOFTINT in the UltraSPARCIIIi; and SPRG1,
IAR, and MSR in the POWER5."

I still don't think that this paper shows an intrinsic reliability flaw
in the register-window idea, at least not one that cannot be compensated
by increasing the robustness of the critical register HW.

However, I see now that UltraSPARC IIIi, used in the paper, uses the
SPARC V9 (or JPS1) architecture, while my experience is with the earlier
SPARC versions V7 and V8. I think the register-window mechanism is the
same in V9, but the paper's description of the mechanism disagrees in
some details with my understanding -- for example, the paper says that
each window has 32 registers, not 24 (visible) or 16 (owned by the window).

>> SPARC processors used in space applications are radiation-tolerant and
>> use triple modular redundancy for critical registers.
>
> I can't find the references just now but I have seen it
> discussed in the past so I'll see what I can dig up.
>
> IIRC the primary issue was the phase delay in window save/restore
> was opaque so code could only know for sure that memory was up to
> date by explicit flushing through OS traps -
> something akin to manual cache coherence.

All the occupied windows in the register ring have to be stored into
memory (into the areas reserved for this in each stack frame) when a
thread is suspended, and at least one windowful has to be reloaded when
the thread is resumed. This is in addition to storing and reloading the
8 "global" registers and any other thread-specific context.

A debugger may also prefer to store the ring into memory so that it can
access the saved SP, FP, and register-allocated locals of upper-level
calls. But at least in SPARC V7 and V8 that is all deterministic and
program-controlled. AFAIK the SPARC processors do not autonomously store
register-ring contents into memory in the background, it has to be
programmed explicitly.

A modern OoO processor may have hundreds of working registers, not
architecturally visible to the programmer. I don't know how those are
handled on thread switch, but it is probably complicated.

> I would be paranoid about things like general OS calls, exceptions,
> interrupts, user mode task switches not seeing a coherent view of memory,
> and would flush the pending register window on entry to these
> until each can be proved safe, on a case-by-case basis.

Traps and interrupts work in the same way as subroutine calls. OS calls
are typically done by a triggering a SW trap. Exceptions may have to
unwind the stack which involves both RESTORE instructions (as in normal
returns from calls) and reloading register windows that had been pushed
out of the register ring. Some work, but not horribly complex.

> One issue is even for in-order implementations it can require a large
> number of hardware registers but programmers can only use a small number,
> similar to costs in bank-switched register sets like Arm32.

But the same issue arises in OoO systems without register windows,
right? Hundreds of working registers, invisible to the programmer.

> H&P note of Sparc:
> - Given that each window has 16 unique registers, an implementation of
>   SPARC can have as few as 40 physical registers and as many as 520,
>   although most have 128 to 136.
>   ...
>   The danger of register windows is that the larger number of
>   registers could slow down the clock rate."

I suppose the OoO processors with large numbers of registers should have
the same clock-rate risk, but perhaps the OoO parallelism compensates.

> One compiler writer notes:
> - "The [Sparc] architecture requires a designer to implement 120 registers,
>   of which only 24 to 29 are available for use by the compiler writer."

Same issue for OoO machines.

> - "Setjmp is an exceptional case for saving register windows.
>   Because setjmp saves processor state, it is necessary for it to
>   force the hidden register state to the stack and to save the current
>   state into the jump buffer. ...
>   This makes setjmp an exceptionally slow operation."

On 2022-07-05 10:54, Anton Ertl wrote:
> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>> And who uses static WCET analysis any more :-(
>
> I would presume that those applications that needed WCET (worst-case
> execution time) analysis in the past still need it. What would they
> use instead?

They can seldom use /static/ WCET analysis because the execution speed
of modern processors is too unpredictable, thanks to all the
acceleration mechanism that are ever-evolving and poorly documented, not
to mention multi-cores and their resource-contention problems, not to
mention out-of-order and speculative processing...

The only pleasure aficionados of static WCET analysis (like myself) have
nowadays is the schadenfreude we feel when all those acceleration
mechanisms turn out to be side channels leaking secrets -- Spectre and
Meltdown etc.

People who need WCETs instead use "hybrid" methods that combine
fine-grained execution-time measurements with some static control-flow
analysis to compute a probabilistic "WCET estimate". Possibly combined
with randomized HW to motivate their use of "extreme-value statistics"
for computing the reliability of that probabilistic WCET estimate.

>> I have read that one port of gcc for SPARC offers a "flat"
>> register-usage model as an alternative to register windows. In the
>> "flat" model the same set of 32 registers is used at all points in the
>> program, and the register windows are never rotated (except for trap
>> handling, I assume). I suppose this "flat" option was implemented for a
>> reason, but the reason was not explained, IIRC.
>
> I read somewhere that the register windows could be used for fast task
> switching instead of for fast calling. To make use of that you would
> need a compiler that does not use the windows for calling; it would
> also have to limit its register usage to 16 (one window per task) or
> 24 (two windows per task) registers.

That seems possible, but I haven't come across a real example.

In addition to the cases of one or two windows per task in a "flat"
model without any within-task use of SAVE/RESTORE instructions, one
could instead partition the register ring so that each task could use a
task-specific sector of the ring, containing any number of windows, with
the task using SAVE/RESTORE for call/return in the usual way (non-"flat").

Replying to myself:

On 2022-07-05 22:04, Niklas Holsti wrote:

> A modern OoO processor may have hundreds of working registers, not
> architecturally visible to the programmer. I don't know how those are
> handled on thread switch, but it is probably complicated.

On second thought, the invisible working registers should be handled
fully by the OoO HW if the thread switch is implemented by normal code
that only manipulates the architecturally visible registers. So the
SPARC register windows do complicate thread switching more than these
invisible working registers do.

On Tuesday, July 5, 2022 at 2:04:11 PM UTC-5, Niklas Holsti wrote:
> On 2022-07-05 19:38, EricP wrote:
> > Niklas Holsti wrote:
<snip>
> >
> > Right, the errors were simulated but as I read it they contend the
> > fragility is due to the register windows and the way were manipulated.
> But they also say: "Injections into the system registers indicate that
> each of the three processors (the Pentium 4, UltraSPARC IIIi, and
> POWER5) have two to three critical registers that are very sensitive to
> errors (see Figure 9 through Figure 11), those being IDTR, GDTR and CR4
> in the Pentium 4; TBA and SET_SOFTINT in the UltraSPARCIIIi; and SPRG1,
> IAR, and MSR in the POWER5."
<
I don't know of any processors that can usefully withstand someone coming
along and randomly flipping bits in PSR, Root Pointers, Page table structures,
and other control registers. SPARC should be little different.
>
> I still don't think that this paper shows an intrinsic reliability flaw
> in the register-window idea, at least not one that cannot be compensated
> by increasing the robustness of the critical register HW.
<
Agreed.
<
>
> However, I see now that UltraSPARC IIIi, used in the paper, uses the
> SPARC V9 (or JPS1) architecture, while my experience is with the earlier
> SPARC versions V7 and V8. I think the register-window mechanism is the
> same in V9, but the paper's description of the mechanism disagrees in
> some details with my understanding -- for example, the paper says that
> each window has 32 registers, not 24 (visible) or 16 (owned by the window).
<
SPARC V9 added some new functionality of OS over register windows that
was supposed to make some of the background overhead go away.
<
> >> SPARC processors used in space applications are radiation-tolerant and
> >> use triple modular redundancy for critical registers.
> >
> > I can't find the references just now but I have seen it
> > discussed in the past so I'll see what I can dig up.
> >
> > IIRC the primary issue was the phase delay in window save/restore
> > was opaque so code could only know for sure that memory was up to
> > date by explicit flushing through OS traps -
> > something akin to manual cache coherence.
<
> All the occupied windows in the register ring have to be stored into
> memory (into the areas reserved for this in each stack frame) when a
> thread is suspended, and at least one windowful has to be reloaded when
> the thread is resumed. This is in addition to storing and reloading the
> 8 "global" registers and any other thread-specific context.
<
Imagine passing a pointer to the argument which arrives in a register window.
>
> A debugger may also prefer to store the ring into memory so that it can
> access the saved SP, FP, and register-allocated locals of upper-level
> calls. But at least in SPARC V7 and V8 that is all deterministic and
> program-controlled. AFAIK the SPARC processors do not autonomously store
> register-ring contents into memory in the background, it has to be
> programmed explicitly.
>
> A modern OoO processor may have hundreds of working registers, not
> architecturally visible to the programmer. I don't know how those are
> handled on thread switch, but it is probably complicated.
<
But those OoO registers are in use, whereas the invisible windows of SPARC
cannot be "in use".
<
> > I would be paranoid about things like general OS calls, exceptions,
> > interrupts, user mode task switches not seeing a coherent view of memory,
> > and would flush the pending register window on entry to these
> > until each can be proved safe, on a case-by-case basis.
<
When we shut down a company, we had a server (SPARC V8) that had been up
for 7 years 4 months and several days. It had every one of its CPU modules
replaced, every one of its memory modules replaced, and most of the disk
farm replaced; yet the system was never out of service or "down". Thus, I don't
see how register windows did those kinds of systems "any harm".
<
> Traps and interrupts work in the same way as subroutine calls.
<
I have to fundamentally disagree with this statement. subroutine calls know
who they are transferring control to, and that it is within its own address space.
Traps can be made to smell like subroutine calls, but are spurious in nature.
Interrupts are not like subroutine calls, because they are entirely asynchronous
with the thread being interrupted. You can make them smell like unexpected
subroutine calls if you desire.
<
> OS calls
> are typically done by a triggering a SW trap. Exceptions may have to
> unwind the stack which involves both RESTORE instructions (as in normal
> returns from calls) and reloading register windows that had been pushed
> out of the register ring. Some work, but not horribly complex.
<
> > One issue is even for in-order implementations it can require a large
> > number of hardware registers but programmers can only use a small number,
> > similar to costs in bank-switched register sets like Arm32.
<
> But the same issue arises in OoO systems without register windows,
> right? Hundreds of working registers, invisible to the programmer.
<
But in OoO design, one may have hundreds of physical registers, but over the
span of 200-ish clocks, every one of them can be used, without ever crossing
a subroutine boundary. Yes, the access time will be similar due to size/area/ports,
but the registers are in use, whereas in a register window, they are cannot be in use.
<
To build an OoO SPARC one would need the large window of the OoO and then
add to that the excesses of the register window. Worst of all cases (except Itanic-
like rotating register.....)....
<
> > H&P note of Sparc:
> > - Given that each window has 16 unique registers, an implementation of
> > SPARC can have as few as 40 physical registers and as many as 520,
> > although most have 128 to 136.
> > ...
> > The danger of register windows is that the larger number of
> > registers could slow down the clock rate."
<
Whereas MIPS got to high frequencies rather easily, SPARC never did.
Read into that what you will.
<
> I suppose the OoO processors with large numbers of registers should have
> the same clock-rate risk, but perhaps the OoO parallelism compensates.
<
You can "pipeline" the register file access time away. What you cannot pipeline
away is the trap handling complexity.
<
> > One compiler writer notes:
> > - "The [Sparc] architecture requires a designer to implement 120 registers,
> > of which only 24 to 29 are available for use by the compiler writer."
> Same issue for OoO machines.
> > - "Setjmp is an exceptional case for saving register windows.
> > Because setjmp saves processor state, it is necessary for it to
> > force the hidden register state to the stack and to save the current
> > state into the jump buffer. ...
> > This makes setjmp an exceptionally slow operation."
> Hm. That may be necessary for some weird uses of setjmp/longjmp, such as
> implementing some cheapo threading or coroutining services, but I don't
> think it is needed for the more normal use of setjmp/longjmp as an
> exception-handling mechanism.
> > - some comments in comp.compilers
> > https://compilers.iecc.com/comparch/article/94-02-130
> > https://compilers.iecc.com/comparch/article/94-02-134
> Those discussions of setjmp/longjmp implementations reveal some
> complexity, but it certainly works on SPARCs, and IMO it can't be seen
> as a major draw-back. But I may be prejudiced because my applications
> use exceptions either never (embedded SW) or rarely (other SW).
<
So, while it is not a MAJOR DrawBack, it is onerous enough that the
designer should proceed with great caution.

On Tuesday, July 5, 2022 at 2:30:24 PM UTC-5, Niklas Holsti wrote:
> On 2022-07-05 10:54, Anton Ertl wrote:
> > Niklas Holsti <niklas...@tidorum.invalid> writes:
> >> And who uses static WCET analysis any more :-(
> >
> > I would presume that those applications that needed WCET (worst-case
> > execution time) analysis in the past still need it. What would they
> > use instead?
> They can seldom use /static/ WCET analysis because the execution speed
> of modern processors is too unpredictable, thanks to all the
> acceleration mechanism that are ever-evolving and poorly documented, not
> to mention multi-cores and their resource-contention problems, not to
> mention out-of-order and speculative processing...
>
> The only pleasure aficionados of static WCET analysis (like myself) have
> nowadays is the schadenfreude we feel when all those acceleration
> mechanisms turn out to be side channels leaking secrets -- Spectre and
> Meltdown etc.
<
Have we finally reached the point that we suspect ANY and EVERY new way
to extract IPL is LIKELY to create side-channels ?
<
So, the only sane architecture point is to define the boundary between
architecture (SW model) and microarchitecture (HW model) such that
microarchectural state is never visible even with a high precision closk.
>
> People who need WCETs instead use "hybrid" methods that combine
> fine-grained execution-time measurements with some static control-flow
> analysis to compute a probabilistic "WCET estimate". Possibly combined
> with randomized HW to motivate their use of "extreme-value statistics"
> for computing the reliability of that probabilistic WCET estimate.

On Tuesday, July 5, 2022 at 2:46:13 PM UTC-5, Niklas Holsti wrote:
> Replying to myself:
> On 2022-07-05 22:04, Niklas Holsti wrote:
>
> > A modern OoO processor may have hundreds of working registers, not
> > architecturally visible to the programmer. I don't know how those are
> > handled on thread switch, but it is probably complicated.
<
> On second thought, the invisible working registers should be handled
> fully by the OoO HW if the thread switch is implemented by normal code
> that only manipulates the architecturally visible registers. So the
> SPARC register windows do complicate thread switching more than these
> invisible working registers do.
<
What happens when context switches are not preformed by running of
instructions, but by the arrival of a "context switch" message ? and HW
does everything wrt saving old state and absorbing new state.

On Tuesday, July 5, 2022 at 11:07:33 PM UTC+3, MitchAlsup wrote:
> On Tuesday, July 5, 2022 at 2:04:11 PM UTC-5, Niklas Holsti wrote:
> > On 2022-07-05 19:38, EricP wrote:
> > > Niklas Holsti wrote:
> <snip>
> > >
> > > Right, the errors were simulated but as I read it they contend the
> > > fragility is due to the register windows and the way were manipulated.
> > But they also say: "Injections into the system registers indicate that
> > each of the three processors (the Pentium 4, UltraSPARC IIIi, and
> > POWER5) have two to three critical registers that are very sensitive to
> > errors (see Figure 9 through Figure 11), those being IDTR, GDTR and CR4
> > in the Pentium 4; TBA and SET_SOFTINT in the UltraSPARCIIIi; and SPRG1,
> > IAR, and MSR in the POWER5."
> <
> I don't know of any processors that can usefully withstand someone coming
> along and randomly flipping bits in PSR, Root Pointers, Page table structures,
> and other control registers. SPARC should be little different.
> >
> > I still don't think that this paper shows an intrinsic reliability flaw
> > in the register-window idea, at least not one that cannot be compensated
> > by increasing the robustness of the critical register HW.
> <
> Agreed.
> <
> >
> > However, I see now that UltraSPARC IIIi, used in the paper, uses the
> > SPARC V9 (or JPS1) architecture, while my experience is with the earlier
> > SPARC versions V7 and V8. I think the register-window mechanism is the
> > same in V9, but the paper's description of the mechanism disagrees in
> > some details with my understanding -- for example, the paper says that
> > each window has 32 registers, not 24 (visible) or 16 (owned by the window).
> <
> SPARC V9 added some new functionality of OS over register windows that
> was supposed to make some of the background overhead go away.
> <
> > >> SPARC processors used in space applications are radiation-tolerant and
> > >> use triple modular redundancy for critical registers.
> > >
> > > I can't find the references just now but I have seen it
> > > discussed in the past so I'll see what I can dig up.
> > >
> > > IIRC the primary issue was the phase delay in window save/restore
> > > was opaque so code could only know for sure that memory was up to
> > > date by explicit flushing through OS traps -
> > > something akin to manual cache coherence.
> <
> > All the occupied windows in the register ring have to be stored into
> > memory (into the areas reserved for this in each stack frame) when a
> > thread is suspended, and at least one windowful has to be reloaded when
> > the thread is resumed. This is in addition to storing and reloading the
> > 8 "global" registers and any other thread-specific context.
> <
> Imagine passing a pointer to the argument which arrives in a register window.
> >
> > A debugger may also prefer to store the ring into memory so that it can
> > access the saved SP, FP, and register-allocated locals of upper-level
> > calls. But at least in SPARC V7 and V8 that is all deterministic and
> > program-controlled. AFAIK the SPARC processors do not autonomously store
> > register-ring contents into memory in the background, it has to be
> > programmed explicitly.
> >
> > A modern OoO processor may have hundreds of working registers, not
> > architecturally visible to the programmer. I don't know how those are
> > handled on thread switch, but it is probably complicated.
> <
> But those OoO registers are in use, whereas the invisible windows of SPARC
> cannot be "in use".
> <
> > > I would be paranoid about things like general OS calls, exceptions,
> > > interrupts, user mode task switches not seeing a coherent view of memory,
> > > and would flush the pending register window on entry to these
> > > until each can be proved safe, on a case-by-case basis.
> <
> When we shut down a company, we had a server (SPARC V8) that had been up
> for 7 years 4 months and several days. It had every one of its CPU modules
> replaced, every one of its memory modules replaced, and most of the disk
> farm replaced; yet the system was never out of service or "down". Thus, I don't
> see how register windows did those kinds of systems "any harm".
> <
> > Traps and interrupts work in the same way as subroutine calls.
> <
> I have to fundamentally disagree with this statement. subroutine calls know
> who they are transferring control to, and that it is within its own address space.
> Traps can be made to smell like subroutine calls, but are spurious in nature.
> Interrupts are not like subroutine calls, because they are entirely asynchronous
> with the thread being interrupted. You can make them smell like unexpected
> subroutine calls if you desire.
> <
> > OS calls
> > are typically done by a triggering a SW trap. Exceptions may have to
> > unwind the stack which involves both RESTORE instructions (as in normal
> > returns from calls) and reloading register windows that had been pushed
> > out of the register ring. Some work, but not horribly complex.
> <
> > > One issue is even for in-order implementations it can require a large
> > > number of hardware registers but programmers can only use a small number,
> > > similar to costs in bank-switched register sets like Arm32.
> <
> > But the same issue arises in OoO systems without register windows,
> > right? Hundreds of working registers, invisible to the programmer.
> <
> But in OoO design, one may have hundreds of physical registers, but over the
> span of 200-ish clocks, every one of them can be used, without ever crossing
> a subroutine boundary. Yes, the access time will be similar due to size/area/ports,
> but the registers are in use, whereas in a register window, they are cannot be in use.
> <
> To build an OoO SPARC one would need the large window of the OoO and then
> add to that the excesses of the register window. Worst of all cases (except Itanic-
> like rotating register.....)....
> <
> > > H&P note of Sparc:
> > > - Given that each window has 16 unique registers, an implementation of
> > > SPARC can have as few as 40 physical registers and as many as 520,
> > > although most have 128 to 136.
> > > ...
> > > The danger of register windows is that the larger number of
> > > registers could slow down the clock rate."
> <
> Whereas MIPS got to high frequencies rather easily, SPARC never did.

Actually, it did, eventually.
Fujitsu got to 4250 MHz, Oracle to 5000 MHz.
MIPS never reached quite that high. But, then again, it didn't last that long and even at the peak of its power had much lower development budget.

> Read into that what you will.

> <
> > I suppose the OoO processors with large numbers of registers should have
> > the same clock-rate risk, but perhaps the OoO parallelism compensates.
> <
> You can "pipeline" the register file access time away. What you cannot pipeline
> away is the trap handling complexity.
> <
> > > One compiler writer notes:
> > > - "The [Sparc] architecture requires a designer to implement 120 registers,
> > > of which only 24 to 29 are available for use by the compiler writer."
> > Same issue for OoO machines.
> > > - "Setjmp is an exceptional case for saving register windows.
> > > Because setjmp saves processor state, it is necessary for it to
> > > force the hidden register state to the stack and to save the current
> > > state into the jump buffer. ...
> > > This makes setjmp an exceptionally slow operation."
> > Hm. That may be necessary for some weird uses of setjmp/longjmp, such as
> > implementing some cheapo threading or coroutining services, but I don't
> > think it is needed for the more normal use of setjmp/longjmp as an
> > exception-handling mechanism.
> > > - some comments in comp.compilers
> > > https://compilers.iecc.com/comparch/article/94-02-130
> > > https://compilers.iecc.com/comparch/article/94-02-134
> > Those discussions of setjmp/longjmp implementations reveal some
> > complexity, but it certainly works on SPARCs, and IMO it can't be seen
> > as a major draw-back. But I may be prejudiced because my applications
> > use exceptions either never (embedded SW) or rarely (other SW).
> <
> So, while it is not a MAJOR DrawBack, it is onerous enough that the
> designer should proceed with great caution.

Niklas Holsti wrote:
> Replying to myself:
>
> On 2022-07-05 22:04, Niklas Holsti wrote:
>
>> A modern OoO processor may have hundreds of working registers, not
>> architecturally visible to the programmer. I don't know how those are
>> handled on thread switch, but it is probably complicated.
>
>
> On second thought, the invisible working registers should be handled
> fully by the OoO HW if the thread switch is implemented by normal code
> that only manipulates the architecturally visible registers. So the
> SPARC register windows do complicate thread switching more than these
> invisible working registers do.

Yes, as Mitch also points out, these registers (mostly) contain
currently valid data - its just that their 5-bit architecture
register names are prefixed by the 5-bit Window Pointer that was
current when they were written. But user mode doesn't have access
to the full 10-bit architecture register numbers.

What an OoO implementation might do is free up the physical registers
after an older window was spilled, allowing them to be reused for
in-flight instructions. But it would still need to have enough
physical registers for the worse case window save sets,
plus a minimum to issue new instructions.
So that is about 32*16+16 = 528 + say 200 for in-flight = 728.

But I would want a fully automatic spill/fill mechanism to go with this
so when Renamer, which assigns free registers, sees the free list is too
small it can trigger a spill and get back a block of 16 free physicals.

The register window is a circular buffer, with 6-bit head and tail
circular indexes (5 bits index plus a wrap bit), Valid and Modify bits
for each set, and a LDM/STM mechanism attached to automatically
spill/fill sets without trapping to kernel mode.

The large number of effective architecture register could be a problem.
Using SRAM style Rename tables would require 10-bit arch reg id index,
1024 entries of say 10 bits each for the 728 physical registers,
to hold the architecture-to-physical map.
That's a pretty big table to checkpoint for each conditional branch.

OoO register windows gets expensive real fast.
That probably explains their scarcity in the wild.

On 5/07/22 05:56, MitchAlsup wrote:

> IBM 360 series::
> caller allocated a register save area and always kept it in R13
> upon any arrival (Callee, interruptee) would perform STM 12,12(13)
> which would dump all 16 registers in the save area, as the first step
> in properly receiving control.
> <
> VAX did something similar, but performed in in the CALL side of ISA
> processing.
>>
>> I wonder, that if they were real, is it possible given today's
>> transistor core counts those mechanisms could be revived in modern clean
>> sheet cpu designs?, to at least help alleviate some of the issues we see
>> in the current state of the art perhaps?
> <
> Effectively that is what ENTER and EXIT do in My 66000 architecture.
>>
Heh, okay then, problem solved I guess. (I really should get around to
emailing you for a copy of the My 66000 manual, but I've still got a ton
of PDFs littering my desktop at the moment)

However on the other hand I'm still wondering if a fully transparent
mechanism existed/is possible.
IE a program JSRs to a subroutine and just uses the registers it needs
without any special instructions or calling conventions needed.
When it hits the return instruction all the registers it disturbed are
restored to their former values automatically (except for the nominated
resultant register/s).

>> Or were there some really big downsides to the automagic things that I'm
>> not remembering well enough?
> <
> Automagic was so slow in VAX that the more modern compilers used JSR
> instead of CALL and got rid of ½ of the cycles in calling/returning.

So hopefully there's enough transistors to go around these days that
such a thing could be made faster than programed spill/fill maybe?

On Tuesday, July 5, 2022 at 7:09:18 PM UTC-5, Andy wrote:
> On 5/07/22 05:56, MitchAlsup wrote:
>
> > IBM 360 series::
> > caller allocated a register save area and always kept it in R13
> > upon any arrival (Callee, interruptee) would perform STM 12,12(13)
> > which would dump all 16 registers in the save area, as the first step
> > in properly receiving control.
> > <
> > VAX did something similar, but performed in in the CALL side of ISA
> > processing.
> >>
> >> I wonder, that if they were real, is it possible given today's
> >> transistor core counts those mechanisms could be revived in modern clean
> >> sheet cpu designs?, to at least help alleviate some of the issues we see
> >> in the current state of the art perhaps?
> > <
> > Effectively that is what ENTER and EXIT do in My 66000 architecture.
> >>
> Heh, okay then, problem solved I guess. (I really should get around to
> emailing you for a copy of the My 66000 manual, but I've still got a ton
> of PDFs littering my desktop at the moment)
>
> However on the other hand I'm still wondering if a fully transparent
> mechanism existed/is possible.
<
> IE a program JSRs to a subroutine and just uses the registers it needs
> without any special instructions or calling conventions needed.
<
Effectively, Mill does this without adding instructions.
My 66000 does this with instructions.
<
On the other hand, up to ½ of all subroutine calls are to leaf subroutines
and in My 66000 ISA, if a subroutine does not touch R16..R31 it neither
has to nor expends any effort to save/restore registers. In my opinion
this is where you want the overhead to be minimal--don't do stuff that
does not need doing. For such subroutines the overhead is:
<
a) setting up arguments to the yet to be called subroutine
b) calling the subroutine
c) performing the subroutine
d) RET
e) doing something with the return value.
<
Any and all subroutines that can be performed using R0..R16 and do not
need any Local_data_area on the stack are overhead free.
<
{You can't get rid of (a) and (e) without inlining the subroutine.
And you can't get rid of (c), leaving only the 2 transfers of control}
<
> When it hits the return instruction all the registers it disturbed are
> restored to their former values automatically (except for the nominated
> resultant register/s).
<
You are making the assumption that values in registers cost more
to recalculate than to save and restore--which is often untrue. It is
often the case that one can recalculate a value in one cycle where
saving and restoring might cost up to 6 cycles. Then there are
registers the compiler KNOWS are not holding a value that is still
alive. Allowing these to die at certain control transfer values, saves
overhead.
<
> >> Or were there some really big downsides to the automagic things that I'm
> >> not remembering well enough?
> > <
> > Automagic was so slow in VAX that the more modern compilers used JSR
> > instead of CALL and got rid of ½ of the cycles in calling/returning.
<
> So hopefully there's enough transistors to go around these days that
> such a thing could be made faster than programed spill/fill maybe?
<
I am betting in that direction with ENTER and EXIT, and zero-instruction
context switching. With our current transistor budgets, one should be able
to deliver/receive at least 4-8 registers per cycle to/from the stack. This
saves 3-7 AGENs and consumes no more than ¼ cache line per cycle.
It is hard to imaging a series of LDs and STs that would save this much
power, cycles, accesses,... Also with ENTER and EXIT one can put preserved
registers on a stack where they cannot be destroyed and guarantee that
the ABI is preserved in both directions across the CALL boundary.

On 7/5/2022 5:44 PM, MitchAlsup wrote:
> On Tuesday, July 5, 2022 at 7:09:18 PM UTC-5, Andy wrote:
>> On 5/07/22 05:56, MitchAlsup wrote:
>>
>>> IBM 360 series::
>>> caller allocated a register save area and always kept it in R13
>>> upon any arrival (Callee, interruptee) would perform STM 12,12(13)
>>> which would dump all 16 registers in the save area, as the first step
>>> in properly receiving control.
>>> <
>>> VAX did something similar, but performed in in the CALL side of ISA
>>> processing.
>>>>
>>>> I wonder, that if they were real, is it possible given today's
>>>> transistor core counts those mechanisms could be revived in modern clean
>>>> sheet cpu designs?, to at least help alleviate some of the issues we see
>>>> in the current state of the art perhaps?
>>> <
>>> Effectively that is what ENTER and EXIT do in My 66000 architecture.
>>>>
>> Heh, okay then, problem solved I guess. (I really should get around to
>> emailing you for a copy of the My 66000 manual, but I've still got a ton
>> of PDFs littering my desktop at the moment)
>>
>> However on the other hand I'm still wondering if a fully transparent
>> mechanism existed/is possible.
> <
>> IE a program JSRs to a subroutine and just uses the registers it needs
>> without any special instructions or calling conventions needed.
> <
> Effectively, Mill does this without adding instructions.
> My 66000 does this with instructions.
> <
> On the other hand, up to ½ of all subroutine calls are to leaf subroutines
> and in My 66000 ISA, if a subroutine does not touch R16..R31 it neither
> has to nor expends any effort to save/restore registers. In my opinion
> this is where you want the overhead to be minimal--don't do stuff that
> does not need doing. For such subroutines the overhead is:
> <
> a) setting up arguments to the yet to be called subroutine
> b) calling the subroutine
> c) performing the subroutine
> d) RET
> e) doing something with the return value.
> <
> Any and all subroutines that can be performed using R0..R16 and do not
> need any Local_data_area on the stack are overhead free.
> <
> {You can't get rid of (a) and (e) without inlining the subroutine.
> And you can't get rid of (c), leaving only the 2 transfers of control}
> <
>> When it hits the return instruction all the registers it disturbed are
>> restored to their former values automatically (except for the nominated
>> resultant register/s).
> <
> You are making the assumption that values in registers cost more
> to recalculate than to save and restore--which is often untrue. It is
> often the case that one can recalculate a value in one cycle where
> saving and restoring might cost up to 6 cycles. Then there are
> registers the compiler KNOWS are not holding a value that is still
> alive. Allowing these to die at certain control transfer values, saves
> overhead.
> <
>>>> Or were there some really big downsides to the automagic things that I'm
>>>> not remembering well enough?
>>> <
>>> Automagic was so slow in VAX that the more modern compilers used JSR
>>> instead of CALL and got rid of ½ of the cycles in calling/returning.
> <
>> So hopefully there's enough transistors to go around these days that
>> such a thing could be made faster than programed spill/fill maybe?
> <
> I am betting in that direction with ENTER and EXIT, and zero-instruction
> context switching. With our current transistor budgets, one should be able
> to deliver/receive at least 4-8 registers per cycle to/from the stack. This
> saves 3-7 AGENs and consumes no more than ¼ cache line per cycle.
> It is hard to imaging a series of LDs and STs that would save this much
> power, cycles, accesses,... Also with ENTER and EXIT one can put preserved
> registers on a stack where they cannot be destroyed and guarantee that
> the ABI is preserved in both directions across the CALL boundary.

Whereas if the HW at the call site knows which registers have live
content then it can silently do the save/restore without instruction -
and without compiler analysis. If the save/restore is inherently lazy
and the callee doesn't use the location then there's no overhead at all.
That's what Mill tries to do.

However, IMO the biggest value of Mill-style implicit save compared to
the my66 (and all others here) explicit save is future-proofing. The two
methods can be made overhead-equivalent with enough work, but I prefer
to keep the sausage-making hidden.

On Tuesday, July 5, 2022 at 7:58:33 PM UTC-5, Ivan Godard wrote:
> On 7/5/2022 5:44 PM, MitchAlsup wrote:
> > On Tuesday, July 5, 2022 at 7:09:18 PM UTC-5, Andy wrote:
> >> On 5/07/22 05:56, MitchAlsup wrote:
> >>
> >>> IBM 360 series::
> >>> caller allocated a register save area and always kept it in R13
> >>> upon any arrival (Callee, interruptee) would perform STM 12,12(13)
> >>> which would dump all 16 registers in the save area, as the first step
> >>> in properly receiving control.
> >>> <
> >>> VAX did something similar, but performed in in the CALL side of ISA
> >>> processing.
> >>>>
> >>>> I wonder, that if they were real, is it possible given today's
> >>>> transistor core counts those mechanisms could be revived in modern clean
> >>>> sheet cpu designs?, to at least help alleviate some of the issues we see
> >>>> in the current state of the art perhaps?
> >>> <
> >>> Effectively that is what ENTER and EXIT do in My 66000 architecture.
> >>>>
> >> Heh, okay then, problem solved I guess. (I really should get around to
> >> emailing you for a copy of the My 66000 manual, but I've still got a ton
> >> of PDFs littering my desktop at the moment)
> >>
> >> However on the other hand I'm still wondering if a fully transparent
> >> mechanism existed/is possible.
> > <
> >> IE a program JSRs to a subroutine and just uses the registers it needs
> >> without any special instructions or calling conventions needed.
> > <
> > Effectively, Mill does this without adding instructions.
> > My 66000 does this with instructions.
> > <
> > On the other hand, up to ½ of all subroutine calls are to leaf subroutines
> > and in My 66000 ISA, if a subroutine does not touch R16..R31 it neither
> > has to nor expends any effort to save/restore registers. In my opinion
> > this is where you want the overhead to be minimal--don't do stuff that
> > does not need doing. For such subroutines the overhead is:
> > <
> > a) setting up arguments to the yet to be called subroutine
> > b) calling the subroutine
> > c) performing the subroutine
> > d) RET
> > e) doing something with the return value.
> > <
> > Any and all subroutines that can be performed using R0..R16 and do not
> > need any Local_data_area on the stack are overhead free.
> > <
> > {You can't get rid of (a) and (e) without inlining the subroutine.
> > And you can't get rid of (c), leaving only the 2 transfers of control}
> > <
> >> When it hits the return instruction all the registers it disturbed are
> >> restored to their former values automatically (except for the nominated
> >> resultant register/s).
> > <
> > You are making the assumption that values in registers cost more
> > to recalculate than to save and restore--which is often untrue. It is
> > often the case that one can recalculate a value in one cycle where
> > saving and restoring might cost up to 6 cycles. Then there are
> > registers the compiler KNOWS are not holding a value that is still
> > alive. Allowing these to die at certain control transfer values, saves
> > overhead.
> > <
> >>>> Or were there some really big downsides to the automagic things that I'm
> >>>> not remembering well enough?
> >>> <
> >>> Automagic was so slow in VAX that the more modern compilers used JSR
> >>> instead of CALL and got rid of ½ of the cycles in calling/returning.
> > <
> >> So hopefully there's enough transistors to go around these days that
> >> such a thing could be made faster than programed spill/fill maybe?
> > <
> > I am betting in that direction with ENTER and EXIT, and zero-instruction
> > context switching. With our current transistor budgets, one should be able
> > to deliver/receive at least 4-8 registers per cycle to/from the stack. This
> > saves 3-7 AGENs and consumes no more than ¼ cache line per cycle.
> > It is hard to imaging a series of LDs and STs that would save this much
> > power, cycles, accesses,... Also with ENTER and EXIT one can put preserved
> > registers on a stack where they cannot be destroyed and guarantee that
> > the ABI is preserved in both directions across the CALL boundary.
<
> Whereas if the HW at the call site knows which registers have live
> content then it can silently do the save/restore without instruction -
> and without compiler analysis. If the save/restore is inherently lazy
> and the callee doesn't use the location then there's no overhead at all.
> That's what Mill tries to do.
<
It would be easy enough to have the ENTER instruction just associate
a stack address with the registers annotated, and then only move them
to stack if they get written; i.e., lazily. Then the EXIT instruction can
restore only those modified. However, this can cause "sequencing"
issues as the registers are not 'dense' whereas they were dense in
the encoding. I prefer not to have too many sequencing issues.
>
> However, IMO the biggest value of Mill-style implicit save compared to
> the my66 (and all others here) explicit save is future-proofing. The two
> methods can be made overhead-equivalent with enough work, but I prefer
> to keep the sausage-making hidden.