On 2/18/2022 8:42 AM, Paul A. Clayton wrote:
> On Friday, February 18, 2022 at 9:27:14 AM UTC-5, Ivan Godard wrote:
> [snip]
>> Still,
>> in principle an OOO can issue a load from inside a called body before
>> the call itself has been issued, and a Mill can't do that.
>
> In theory a caller could start a pick up load that is picked up by the callee. This would be similar to function specific interfaces rather than using a generic ABI. It is not obvious how useful this would be.

The architecture does not permit that - each frame has a distinct set of
load retire stations. While that could be changed, it's not clear that,
given the limited applicability to calls which were statically
resolvable (i.e. no polymorphic), the the game would be worth the
candle. That case can be addressed without architecture change using
out-lining, too.

>
>> Put all this together and the Mill ISA has eliminated all the OOO
>> advantages except: OOO will excel on 1) frequently called 2) polymorphic
>> functions that 3) immediately execute a load whose address arguments are
>> 4) ready and that 5) misses when 6) the hardware is not bandwidth
>> limited. For that case a Mill will stall in the callee and an OOO likely
>> will not.
>
> I am not convinced this is the case, but writing on Google groups from a tablet is painful.
>
>> We decided that this case was not worth 12x.
>
> I doubt the Mill will get a 12x PPA advantage over an OoO implementation of a conventional ISA like AArch64.

12x is not my number (see Mitch), and in any case we have given up some
of whatever it is to pay for width.

On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
> On Friday, February 18, 2022 at 12:32:05 AM UTC-7, Anton Ertl wrote:
> > BGB <cr8...@gmail.com> writes:
>
> > >People will also have a lot more time to try to work out the "magic
> > >compiler" issues, ...
>
> > We have had from the appearance of microcode in the 1960s through
> > Metaflow and Cydrome in the 1980s until people lost faith in IA-64
> > around 2010 to work them out. It's unlikely that a breakthrough will
> > be found later.
> This is an issue that continues to confuse me.
>
> There is no way that a compiler can take the place of out-of-order
> circuitry for allowing a computer to deal with cache misses. These aren't
> predictable.
<
Actually, they are predictable in HW.
<
Say we have a loop that does 2 LDDs and one STD: and we run the loop
"lots" of times;
LDD[1] takes a miss every 8 cycles in iteration MOD 3
LDD[2] takes a miss every 8 cycles in iteration MOD 6
ST....... takes a miss every 8 cycles in iteration MOD 1
<
HW can handle this perfectly after a couple of 16+ loops, and it does not
have to be OoO HW to do this. It could be detected between L1<->L2
and just schedule a 3 prefetches every 8×latency-of-loop.
>
> But when it comes to register hazards, _that_ part of what OoO does,
> I would have thought, can of course be fully matched by changing the
> rename registers to explicit registers, and having the compiler allocate
> them and schedule the instructions accordingly. So I would have thought
> this was already a solved problem.
>
> If the burden of circuitry of OoO is very heavy, therefore, I would have
> thought that the burden of making instructions wider, of having more
> registers to save and restore, and so on, would be well worth it. But
<
History indicates a lot of people put a lot of investment here and failed.
<
> maybe it isn't, since memory bandwidth is a big issue in its own right
> as well.
>
> What I _suspect_, though, is that the main reason the choice is usually
> made to go with OoO rather than to go with big register files... is because
> of what happened with RISC. Going from 8 or 16 registers to 32 registers
> was supposed to be an example of this approach. And at one level of
> hardware complexity and performance, it was.
<
The ancient statistics showed going from 8->16 registers was fairly big
going from 16-32 was modest, going from 32-64 was negligible:: on compiled
code using pre-MIPS-company architecture. Stanford very early in RISC era.
>
> But quickly, as more transistors became available, and more performance
> was desired, we got OoO implementations of RISC architectures. Along
> came Itanium - and even I quickly saw that this design was built around
> one particular generation of microelectronics, and it would soon have to
> be replaced with something even bigger and hence incompatible.
>
> Whereas, if you just take a popular CISC architecture, and implement it
> with the aid of OoO execution, nobody has to change all their programs, and
> everyone is happy.
>
> So the issue isn't that doing away with OoO by explicitly allocating registers
> is technically infeasible, or technically inferior. You might be able to get
> better performance at lower cost that way. But as long as transistor counts
> keep going up, doing it that way means you keep having to change your ISA,
> whereas using OoO means you don't.
>
> In that case, the end of Moore's Law doesn't give us "time to solve the
> compiler problem" - that problem is already solved. Instead, it means we
> can finally stop at 256 registers or however many, as the next generation of
> silicon isn't going to come along shortly and make the ISA obsolete.
>
> So that's how I see the end of Moore's Law making VLIW designs possible
> once again.
>
> John Savard

On Friday, February 18, 2022 at 3:57:35 AM UTC-6, Anton Ertl wrote:
> Ivan Godard <iv...@millcomputing.com> writes:
> >EPIC hardware assumed that there are no intra-bundle hazards,
>

> >This makes the Mill suitable for general-purpose work that would cause a
> >classic VLIW to spend too much time in stall, yet still take advantage
> >of any schedule variability to do other work while (possibly) waiting
> >for external data, and without needing OOO retire hazard hardware.
>
> I assume you mean in-order readyness checking (absent in the MIPS
> R2000 load, which gave the architecture its name), sometimes called a
> scoreboard (although Mitch Alsup tells us that that name is
> inappropriate because the original CDC-6600 scoreboard was a more
> sophisticated feature with some OoO potential). The question is how
> expensive this is. They could afford this feature in the earliest
> RISCs, such as ARM and SPARC, so the cost cannot be that high.
<
A Scoreboard does not start a calculation until all RAW hazards disappear
{that is its operands are ready}
A Scoreboard does not finish a calculation until all RAW hazards disappear.
{that is all consumers of the previous value have gotten that value}
<
Expense:
<
The entire CDC 6600 scoreboard can be implemented in about 2½K gates
managed 10 function units. Had the scoreboard have to manage 32 flat
register file this number would have been about 15K gates. A scoreboard
is quadratic in register management, 3 8-entry files, the A file only being
used by increment, makes its control tiny. 1-32 entry file has 32×32 hazard
points, the 6600 only had 7×7 + 7×7 + 8×8 = 192 compared to 1024.
<
One Tomasulo station costs 1.5K gates. 360/91 had ~12 of them.
>
> >The gain from Mill's split load is limited by the maximal gap (in time)
> >between the load-issue and load-retire instructions, which is in turn
> >determined by how much work is available to do that neither depends on
> >the load result nor is depended on by the load issue. That's easily
> >determined by conventional dataflow analysis in the compiler, and is
> >exactly the same as the amount of work that an OOO can do while waiting
> >for a load to retire.
>
> Do you mean an in-order superscalar? An OoO superscalar often has
> fetched and decoded hundreds of instructions ahead, many of which have
> already been processed and advanced to the retire queue, many of which
> are blocked by dependencies on the load or other non-ready
> instructions, but there are also often a number of instructions that
> have just become ready (e.g., because they depended on the same
> instruction as the load, or because they depended on an instruction
> that produced its result in the same cycle as the last parent of the
> load), and which may be from places that a compiler cannot easily
> schedule (e.g., from behind a polymorphic method call).
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 2/18/2022 11:18 AM, MitchAlsup wrote:
> On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
>> On Friday, February 18, 2022 at 12:32:05 AM UTC-7, Anton Ertl wrote:
>>> BGB <cr8...@gmail.com> writes:
>>
>>>> People will also have a lot more time to try to work out the "magic
>>>> compiler" issues, ...
>>
>>> We have had from the appearance of microcode in the 1960s through
>>> Metaflow and Cydrome in the 1980s until people lost faith in IA-64
>>> around 2010 to work them out. It's unlikely that a breakthrough will
>>> be found later.
>> This is an issue that continues to confuse me.
>>
>> There is no way that a compiler can take the place of out-of-order
>> circuitry for allowing a computer to deal with cache misses. These aren't
>> predictable.
> <
> Actually, they are predictable in HW.
> <
> Say we have a loop that does 2 LDDs and one STD: and we run the loop
> "lots" of times;
> LDD[1] takes a miss every 8 cycles in iteration MOD 3
> LDD[2] takes a miss every 8 cycles in iteration MOD 6
> ST....... takes a miss every 8 cycles in iteration MOD 1
> <
> HW can handle this perfectly after a couple of 16+ loops, and it does not
> have to be OoO HW to do this. It could be detected between L1<->L2
> and just schedule a 3 prefetches every 8×latency-of-loop.
>>
>> But when it comes to register hazards, _that_ part of what OoO does,
>> I would have thought, can of course be fully matched by changing the
>> rename registers to explicit registers, and having the compiler allocate
>> them and schedule the instructions accordingly. So I would have thought
>> this was already a solved problem.
>>
>> If the burden of circuitry of OoO is very heavy, therefore, I would have
>> thought that the burden of making instructions wider, of having more
>> registers to save and restore, and so on, would be well worth it. But
> <
> History indicates a lot of people put a lot of investment here and failed.
> <
>> maybe it isn't, since memory bandwidth is a big issue in its own right
>> as well.
>>
>> What I _suspect_, though, is that the main reason the choice is usually
>> made to go with OoO rather than to go with big register files... is because
>> of what happened with RISC. Going from 8 or 16 registers to 32 registers
>> was supposed to be an example of this approach. And at one level of
>> hardware complexity and performance, it was.
> <
> The ancient statistics showed going from 8->16 registers was fairly big
> going from 16-32 was modest, going from 32-64 was negligible:: on compiled
> code using pre-MIPS-company architecture. Stanford very early in RISC era.

My testing would mostly agree, but I have found a partial edge case:
This assumes that one is not allocating the registers as even-pairs;
Even-pair allocation nearly doubles the register pressure, so now 32
registers is back to behaving like 16, adding incentive to move to 64 to
compensate.

But, yeah, if one values uses one register, then 32 seems to be mostly
sufficient.

Granted, one could argue that pair-allocating registers so that they can
use a 64-bit ISA for working with 128-bit pointers is "kinda stupid" but
alas...

I am not sure the long term effect of this experiment, I kinda expect it
will suck bad enough that I can throw it in a closet and forget about
it, but this is what I also thought about the "expand the address space
to 96 bits" idea...

Though, maybe piling on random crazy ideas over time isn't great in the
sense that it results in the design getting a bit messy.

....

On Friday, February 18, 2022 at 10:18:16 AM UTC-7, MitchAlsup wrote:
> On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:

> > There is no way that a compiler can take the place of out-of-order
> > circuitry for allowing a computer to deal with cache misses. These aren't
> > predictable.
> <
> Actually, they are predictable in HW.
> <
> Say we have a loop that does 2 LDDs and one STD: and we run the loop
> "lots" of times;
> LDD[1] takes a miss every 8 cycles in iteration MOD 3
> LDD[2] takes a miss every 8 cycles in iteration MOD 6
> ST....... takes a miss every 8 cycles in iteration MOD 1
> <
> HW can handle this perfectly after a couple of 16+ loops, and it does not
> have to be OoO HW to do this. It could be detected between L1<->L2
> and just schedule a 3 prefetches every 8×latency-of-loop.

Of course, being able to predict it using techniques
similar to those used in branch prediction when the program
is running... doesn't change my point that it isn't predictable
long before, when the compiler is generating machine code
from source code.

John Savard

>> In theory a caller could start a pick up load that is picked up by the
>> callee. This would be similar to function specific interfaces rather than
>> using a generic ABI. It is not obvious how useful this would be.
> The architecture does not permit that - each frame has a distinct set of
> load retire stations. While that could be changed, it's not clear that,
> given the limited applicability to calls which were statically resolvable
> (i.e. no polymorphic), the the game would be worth the candle.

Such calls are very common in some languages, and it seems like it would
be fairly easy to add a simple analysis to the compiler to annotate
functions with ad-hoc "in-flight loads" that the callers need
to provide.

Intuitively I'd expect it could be quite useful in languages with many
small functions, since it would allow starting accessing fields of an
argument before entering the function where there might not be much more
to do for the first few cycles otherwise.

> That case can be addressed without architecture change using
> out-lining, too.

At least, on the compiler side, it's much easier to accommodate
specialized per-function calling conventions, than using outlining
or inlining.

Stefan

On Friday, February 18, 2022 at 10:32:18 AM UTC-7, MitchAlsup wrote:

> Expense:
> <
> The entire CDC 6600 scoreboard can be implemented in about 2½K gates
> managed 10 function units. Had the scoreboard have to manage 32 flat
> register file this number would have been about 15K gates. A scoreboard
> is quadratic in register management, 3 8-entry files, the A file only being
> used by increment, makes its control tiny. 1-32 entry file has 32×32 hazard
> points, the 6600 only had 7×7 + 7×7 + 8×8 = 192 compared to 1024.
> <
> One Tomasulo station costs 1.5K gates. 360/91 had ~12 of them.

Useful information!

While the CDC 6600 had 2,500 gates of overhead, and the 360/91
had 18,000 gates of overhead, then, and a modern OoO computer would
need more (the 360 architecture just had four floating-point registers,
and modern OoO processors, unlike the Pentium II, wish to apply OoO
techniques to integer processing as well)...

Instead of being an argument for using the CDC 6600 scoreboard, this
seems to be an argument for OoO, since its costs are so modest!

And, of course, they _are_ - depending on how ambitious one is. But
the goals of today's "great big out-of-order" processors, as you call them,
are, of course, anything but modest.

Given that they've gone well past the point of diminishing returns on the
one hand, and parallel computing just doesn't seem to work for programmers
on the other, it seems that we're stuck with no good solutions.

Of course, that's only true when one's goals are not modest. People can
still design processors for smartphones without feeling guilty, desperate,
or confused.

John Savard

MitchAlsup <MitchAlsup@aol.com> schrieb:

> The ancient statistics showed going from 8->16 registers was fairly big
> going from 16-32 was modest, going from 32-64 was negligible:: on compiled
> code using pre-MIPS-company architecture. Stanford very early in RISC era.

I've been told that these statistics exist, but I didn't read the
original papers (and would like a pointer to them). It would be
interesting to read details such as which code they used for
benchmarks.

Most RISC architectures have 32 floating point registers in addition
to their 32 integer registers, so a total number of 64 registers
is not outlandish. Your designs are pretty unique in combining
these two. VVM eliminates the need for unrolling inner loops,
so at least My 66000 can get by with less.

MitchAlsup wrote:
> On Friday, February 18, 2022 at 3:57:35 AM UTC-6, Anton Ertl wrote:
>> Ivan Godard <iv...@millcomputing.com> writes:
>>> EPIC hardware assumed that there are no intra-bundle hazards,
>
>>> This makes the Mill suitable for general-purpose work that would cause a
>>> classic VLIW to spend too much time in stall, yet still take advantage
>>> of any schedule variability to do other work while (possibly) waiting
>>> for external data, and without needing OOO retire hazard hardware.
>> I assume you mean in-order readyness checking (absent in the MIPS
>> R2000 load, which gave the architecture its name), sometimes called a
>> scoreboard (although Mitch Alsup tells us that that name is
>> inappropriate because the original CDC-6600 scoreboard was a more
>> sophisticated feature with some OoO potential). The question is how
>> expensive this is. They could afford this feature in the earliest
>> RISCs, such as ARM and SPARC, so the cost cannot be that high.
> <
> A Scoreboard does not start a calculation until all RAW hazards disappear
> {that is its operands are ready}
> A Scoreboard does not finish a calculation until all RAW hazards disappear.
> {that is all consumers of the previous value have gotten that value}
^WAR^
> <
> Expense:
> <
> The entire CDC 6600 scoreboard can be implemented in about 2½K gates
> managed 10 function units. Had the scoreboard have to manage 32 flat
> register file this number would have been about 15K gates. A scoreboard
> is quadratic in register management, 3 8-entry files, the A file only being
> used by increment, makes its control tiny. 1-32 entry file has 32×32 hazard
> points, the 6600 only had 7×7 + 7×7 + 8×8 = 192 compared to 1024.
> <
> One Tomasulo station costs 1.5K gates. 360/91 had ~12 of them.

But the CDC6600 scoreboard allowed writeback to *different* registers
to be performed out of order, making interrupts imprecise.

To make interrupts precise, wouldn't it need an extra matrix to impose
program order on writebacks? This creates a WAW dependency chain to
*different* registers based on program order but one that stalls at
writeback, whereas normal WAW to the same register stalls at issue.

Then having multiple write ports allows multiple writebacks to
*different* registers at once, allowing it to catch up after a bubble.

On 2/18/2022 1:20 PM, Thomas Koenig wrote:
> MitchAlsup <MitchAlsup@aol.com> schrieb:
>
>> The ancient statistics showed going from 8->16 registers was fairly big
>> going from 16-32 was modest, going from 32-64 was negligible:: on compiled
>> code using pre-MIPS-company architecture. Stanford very early in RISC era.
>
> I've been told that these statistics exist, but I didn't read the
> original papers (and would like a pointer to them). It would be
> interesting to read details such as which code they used for
> benchmarks.
>
> Most RISC architectures have 32 floating point registers in addition
> to their 32 integer registers, so a total number of 64 registers
> is not outlandish. Your designs are pretty unique in combining
> these two. VVM eliminates the need for unrolling inner loops,
> so at least My 66000 can get by with less.

I ended up merging them as well...

At first, this was for SIMD:
32 GPRs, used for ALU or SIMD
16 FPRs, for the FPU

Then I ended up eliminating the FPRs in favor of merging FPU ops into
the GPR space.

This was also the end of the existence of a dedicated FDIV instruction
as well; early forms of the FPU had FDIV and FSQRT, but they were
dropped due to being both "not free" nor offering any real speed
advantage over doing them in software. In both cases, variants of
Newton-Raphson were used, and doing it in the FPU doesn't make the N-R
converge any faster (the other overheads not significantly contributing
to clock-cycle cost).

MitchAlsup <MitchAlsup@aol.com> writes:

> On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
>> There is no way that a compiler can take the place of out-of-order
>> circuitry for allowing a computer to deal with cache misses. These aren't
>> predictable.
> <
> Actually, they are predictable in HW.
> <
> Say we have a loop that does 2 LDDs and one STD: and we run the loop
> "lots" of times;
> LDD[1] takes a miss every 8 cycles in iteration MOD 3
> LDD[2] takes a miss every 8 cycles in iteration MOD 6
> ST....... takes a miss every 8 cycles in iteration MOD 1
> <
> HW can handle this perfectly after a couple of 16+ loops, and it does not
> have to be OoO HW to do this. It could be detected between L1<->L2
> and just schedule a 3 prefetches every 8×latency-of-loop.

Say you have a loop with unpredictable cache misses like this:

for (i = 0; i < n; i++)
d[i] = f(s[get_addr(i)]);

If you had an "in_cache()" instruction, would a transformation like this
be a win?

// run loop for everything in cache, skip and prefetch stuff that missed
for (i = 0, j = 0; i < n; i++) {
p = get_addr(i);
if (in_cache(&s[p]))
d[i] = f(s[p]);
else {
prefetch(&s[p]);
skipped[j++] = i;
}
}

// finish entries that missed cache
for (i = 0; i < j; i++)
d[skipped[i]] = f(s[get_addr(skipped[i])]);

On Friday, February 18, 2022 at 1:20:24 PM UTC-6, Thomas Koenig wrote:
> MitchAlsup <Mitch...@aol.com> schrieb:
> > The ancient statistics showed going from 8->16 registers was fairly big
> > going from 16-32 was modest, going from 32-64 was negligible:: on compiled
> > code using pre-MIPS-company architecture. Stanford very early in RISC era.
> I've been told that these statistics exist, but I didn't read the
> original papers (and would like a pointer to them). It would be
> interesting to read details such as which code they used for
> benchmarks.
>
> Most RISC architectures have 32 floating point registers in addition
> to their 32 integer registers, so a total number of 64 registers
> is not outlandish. Your designs are pretty unique in combining
> these two. VVM eliminates the need for unrolling inner loops,
> so at least My 66000 can get by with less.
<
Also note: The C version of Livermore loops compiles without
overflowing the 32 registers--only one program needed more
~23 registers.
<
LLVM from-end Brian's back-end; probably set to -O3

On Friday, February 18, 2022 at 1:44:13 PM UTC-6, EricP wrote:
> MitchAlsup wrote:
> > On Friday, February 18, 2022 at 3:57:35 AM UTC-6, Anton Ertl wrote:
> >> Ivan Godard <iv...@millcomputing.com> writes:
> >>> EPIC hardware assumed that there are no intra-bundle hazards,
> >
> >>> This makes the Mill suitable for general-purpose work that would cause a
> >>> classic VLIW to spend too much time in stall, yet still take advantage
> >>> of any schedule variability to do other work while (possibly) waiting
> >>> for external data, and without needing OOO retire hazard hardware.
> >> I assume you mean in-order readyness checking (absent in the MIPS
> >> R2000 load, which gave the architecture its name), sometimes called a
> >> scoreboard (although Mitch Alsup tells us that that name is
> >> inappropriate because the original CDC-6600 scoreboard was a more
> >> sophisticated feature with some OoO potential). The question is how
> >> expensive this is. They could afford this feature in the earliest
> >> RISCs, such as ARM and SPARC, so the cost cannot be that high.
> > <
> > A Scoreboard does not start a calculation until all RAW hazards disappear
> > {that is its operands are ready}
> > A Scoreboard does not finish a calculation until all RAW hazards disappear.
> > {that is all consumers of the previous value have gotten that value}
> ^WAR^
Missed that one.
> > <
> > Expense:
> > <
> > The entire CDC 6600 scoreboard can be implemented in about 2½K gates
> > managed 10 function units. Had the scoreboard have to manage 32 flat
> > register file this number would have been about 15K gates. A scoreboard
> > is quadratic in register management, 3 8-entry files, the A file only being
> > used by increment, makes its control tiny. 1-32 entry file has 32×32 hazard
> > points, the 6600 only had 7×7 + 7×7 + 8×8 = 192 compared to 1024.
> > <
> > One Tomasulo station costs 1.5K gates. 360/91 had ~12 of them.
<
> But the CDC6600 scoreboard allowed writeback to *different* registers
> to be performed out of order, making interrupts imprecise.
<
Different Registers problem that can be renamed away. So you track register
dependencies in rename space rather than in architectural space.
>
> To make interrupts precise, wouldn't it need an extra matrix to impose
> program order on writebacks? This creates a WAW dependency chain to
> *different* registers based on program order but one that stalls at
> writeback, whereas normal WAW to the same register stalls at issue.
<
I think Luke was doing this.
>
> Then having multiple write ports allows multiple writebacks to
> *different* registers at once, allowing it to catch up after a bubble.
<
Colloquially known as "catch up bandwidth"

On Friday, February 18, 2022 at 3:25:51 PM UTC-6, BGB wrote:
> On 2/18/2022 1:20 PM, Thomas Koenig wrote:
> > MitchAlsup <Mitch...@aol.com> schrieb:
> >
> >> The ancient statistics showed going from 8->16 registers was fairly big
> >> going from 16-32 was modest, going from 32-64 was negligible:: on compiled
> >> code using pre-MIPS-company architecture. Stanford very early in RISC era.
> >
> > I've been told that these statistics exist, but I didn't read the
> > original papers (and would like a pointer to them). It would be
> > interesting to read details such as which code they used for
> > benchmarks.
> >
> > Most RISC architectures have 32 floating point registers in addition
> > to their 32 integer registers, so a total number of 64 registers
> > is not outlandish. Your designs are pretty unique in combining
> > these two. VVM eliminates the need for unrolling inner loops,
> > so at least My 66000 can get by with less.
> I ended up merging them as well...
>
> At first, this was for SIMD:
> 32 GPRs, used for ALU or SIMD
> 16 FPRs, for the FPU
>
> Then I ended up eliminating the FPRs in favor of merging FPU ops into
> the GPR space.
>
> This was also the end of the existence of a dedicated FDIV instruction
> as well; early forms of the FPU had FDIV and FSQRT, but they were
> dropped due to being both "not free" nor offering any real speed
> advantage over doing them in software. In both cases, variants of
> Newton-Raphson were used, and doing it in the FPU doesn't make the N-R
> converge any faster (the other overheads not significantly contributing
> to clock-cycle cost).
<
You can get the iterations started faster, and do the first n-1 iterations using
Goldschmidt finishing with a Newton-Raphson iteration.
<
Goldschmidt uses 2 multiplications that are not dependent on one
another. NR uses 2 multiplies that are dependent on one another.
So if you FPU is pipeline, in HW is faster. Whether your particular
use of your HW gains anything is unknown.

On Friday, February 18, 2022 at 5:09:29 PM UTC-6, Josh Vanderhoof wrote:
> MitchAlsup <Mitch...@aol.com> writes:
>
> > On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
> >> There is no way that a compiler can take the place of out-of-order
> >> circuitry for allowing a computer to deal with cache misses. These aren't
> >> predictable.
> > <
> > Actually, they are predictable in HW.
> > <
> > Say we have a loop that does 2 LDDs and one STD: and we run the loop
> > "lots" of times;
> > LDD[1] takes a miss every 8 cycles in iteration MOD 3
> > LDD[2] takes a miss every 8 cycles in iteration MOD 6
> > ST....... takes a miss every 8 cycles in iteration MOD 1
> > <
> > HW can handle this perfectly after a couple of 16+ loops, and it does not
> > have to be OoO HW to do this. It could be detected between L1<->L2
> > and just schedule a 3 prefetches every 8×latency-of-loop.
> Say you have a loop with unpredictable cache misses like this:
>
> for (i = 0; i < n; i++)
> d[i] = f(s[get_addr(i)]);
<
MOV Ri,#0
for_loop:
LDD R1,[Rsp+Ri<<2]
CALL f
ADD Ri,Ri,#1
CMP Rt,Ri,Rn
BLT for_loop
<
did you mean to call a function ?
>
> If you had an "in_cache()" instruction, would a transformation like this
> be a win?
>
> // run loop for everything in cache, skip and prefetch stuff that missed
> for (i = 0, j = 0; i < n; i++) {
> p = get_addr(i);
> if (in_cache(&s[p]))
> d[i] = f(s[p]);
> else {
> prefetch(&s[p]);
> skipped[j++] = i;
> }
> }
>
> // finish entries that missed cache
> for (i = 0; i < j; i++)
> d[skipped[i]] = f(s[get_addr(skipped[i])]);

On 2/18/2022 4:25 PM, MitchAlsup wrote:
> On Friday, February 18, 2022 at 1:44:13 PM UTC-6, EricP wrote:
>> MitchAlsup wrote:
>>> On Friday, February 18, 2022 at 3:57:35 AM UTC-6, Anton Ertl wrote:
>>>> Ivan Godard <iv...@millcomputing.com> writes:
>>>>> EPIC hardware assumed that there are no intra-bundle hazards,
>>>
>>>>> This makes the Mill suitable for general-purpose work that would cause a
>>>>> classic VLIW to spend too much time in stall, yet still take advantage
>>>>> of any schedule variability to do other work while (possibly) waiting
>>>>> for external data, and without needing OOO retire hazard hardware.
>>>> I assume you mean in-order readyness checking (absent in the MIPS
>>>> R2000 load, which gave the architecture its name), sometimes called a
>>>> scoreboard (although Mitch Alsup tells us that that name is
>>>> inappropriate because the original CDC-6600 scoreboard was a more
>>>> sophisticated feature with some OoO potential). The question is how
>>>> expensive this is. They could afford this feature in the earliest
>>>> RISCs, such as ARM and SPARC, so the cost cannot be that high.
>>> <
>>> A Scoreboard does not start a calculation until all RAW hazards disappear
>>> {that is its operands are ready}
>>> A Scoreboard does not finish a calculation until all RAW hazards disappear.
>>> {that is all consumers of the previous value have gotten that value}
>> ^WAR^
> Missed that one.
>>> <
>>> Expense:
>>> <
>>> The entire CDC 6600 scoreboard can be implemented in about 2½K gates
>>> managed 10 function units. Had the scoreboard have to manage 32 flat
>>> register file this number would have been about 15K gates. A scoreboard
>>> is quadratic in register management, 3 8-entry files, the A file only being
>>> used by increment, makes its control tiny. 1-32 entry file has 32×32 hazard
>>> points, the 6600 only had 7×7 + 7×7 + 8×8 = 192 compared to 1024.
>>> <
>>> One Tomasulo station costs 1.5K gates. 360/91 had ~12 of them.
> <
>> But the CDC6600 scoreboard allowed writeback to *different* registers
>> to be performed out of order, making interrupts imprecise.
> <
> Different Registers problem that can be renamed away. So you track register
> dependencies in rename space rather than in architectural space.
>>
>> To make interrupts precise, wouldn't it need an extra matrix to impose
>> program order on writebacks? This creates a WAW dependency chain to
>> *different* registers based on program order but one that stalls at
>> writeback, whereas normal WAW to the same register stalls at issue.
> <
> I think Luke was doing this.

What happened to Luke? - haven't seen him here in a donkey's age.

>>
>> Then having multiple write ports allows multiple writebacks to
>> *different* registers at once, allowing it to catch up after a bubble.
> <
> Colloquially known as "catch up bandwidth"

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Friday, February 18, 2022 at 1:20:24 PM UTC-6, Thomas Koenig wrote:
>> MitchAlsup <Mitch...@aol.com> schrieb:
>> > The ancient statistics showed going from 8->16 registers was fairly big
>> > going from 16-32 was modest, going from 32-64 was negligible:: on compiled
>> > code using pre-MIPS-company architecture. Stanford very early in RISC era.
>> I've been told that these statistics exist, but I didn't read the
>> original papers (and would like a pointer to them). It would be
>> interesting to read details such as which code they used for
>> benchmarks.
>>
>> Most RISC architectures have 32 floating point registers in addition
>> to their 32 integer registers, so a total number of 64 registers
>> is not outlandish. Your designs are pretty unique in combining
>> these two. VVM eliminates the need for unrolling inner loops,
>> so at least My 66000 can get by with less.
><
> Also note: The C version of Livermore loops compiles without
> overflowing the 32 registers--only one program needed more
> ~23 registers.

It would be really interesting to see what your architecture makes
of more typical Fortran code like the Polyhedron benchmarks.

The Livermore loops are a few decades old and not really
represenative of more modern sicentific workloads.

Thomas Koenig <tkoenig@netcologne.de> schrieb:
> MitchAlsup <MitchAlsup@aol.com> schrieb:
>> On Friday, February 18, 2022 at 1:20:24 PM UTC-6, Thomas Koenig wrote:
>>> MitchAlsup <Mitch...@aol.com> schrieb:
>>> > The ancient statistics showed going from 8->16 registers was fairly big
>>> > going from 16-32 was modest, going from 32-64 was negligible:: on compiled
>>> > code using pre-MIPS-company architecture. Stanford very early in RISC era.
>>> I've been told that these statistics exist, but I didn't read the
>>> original papers (and would like a pointer to them). It would be
>>> interesting to read details such as which code they used for
>>> benchmarks.
>>>
>>> Most RISC architectures have 32 floating point registers in addition
>>> to their 32 integer registers, so a total number of 64 registers
>>> is not outlandish. Your designs are pretty unique in combining
>>> these two. VVM eliminates the need for unrolling inner loops,
>>> so at least My 66000 can get by with less.
>><
>> Also note: The C version of Livermore loops compiles without
>> overflowing the 32 registers--only one program needed more
>> ~23 registers.
>
> It would be really interesting to see what your architecture makes
> of more typical Fortran code like the Polyhedron benchmarks.

To be more precise, I tried compiling a random sample of the
Polyhedron benchmarks - fatigue2.f90 with flang v7 to llvm code,
then used llc to translate it to My 66000 via a compiler built
from the source at https://github.com/bagel99/llvm-my66000.git .

When I hit an ICE (not entirely unexpected, if this is the
first time anybody tried flang with that :-)

VVMLoopPass: read_input_m_
Unknown f64 condition code!
UNREACHABLE executed at /home/tkoenig/llvm-my66000/llvm/lib/Target/My66000/My66000ISelLowering.cpp:343!
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0. Program arguments: llc -O3 --march=my66000 --enable-vvm fatigue2.ll
1. Running pass 'Function Pass Manager' on module 'fatigue2.ll'.
2. Running pass 'My66000 DAG->DAG Pattern Instruction Selection' on function '@read_input_m_read_input_'
#0 0x0000000014b1b4b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/tkoenig/llvm-my66000/llvm/lib/Support/Unix/Signals.inc:565:22

[...]

I did not find a way to raise an issue at github, or I would have
posted the information there.

On Saturday, February 19, 2022 at 8:18:40 AM UTC-6, Thomas Koenig wrote:
> Thomas Koenig <tko...@netcologne.de> schrieb:
> > MitchAlsup <Mitch...@aol.com> schrieb:
> >> On Friday, February 18, 2022 at 1:20:24 PM UTC-6, Thomas Koenig wrote:
> >>> MitchAlsup <Mitch...@aol.com> schrieb:
> >>> > The ancient statistics showed going from 8->16 registers was fairly big
> >>> > going from 16-32 was modest, going from 32-64 was negligible:: on compiled
> >>> > code using pre-MIPS-company architecture. Stanford very early in RISC era.
> >>> I've been told that these statistics exist, but I didn't read the
> >>> original papers (and would like a pointer to them). It would be
> >>> interesting to read details such as which code they used for
> >>> benchmarks.
> >>>
> >>> Most RISC architectures have 32 floating point registers in addition
> >>> to their 32 integer registers, so a total number of 64 registers
> >>> is not outlandish. Your designs are pretty unique in combining
> >>> these two. VVM eliminates the need for unrolling inner loops,
> >>> so at least My 66000 can get by with less.
> >><
> >> Also note: The C version of Livermore loops compiles without
> >> overflowing the 32 registers--only one program needed more
> >> ~23 registers.
> >
> > It would be really interesting to see what your architecture makes
> > of more typical Fortran code like the Polyhedron benchmarks.
> To be more precise, I tried compiling a random sample of the
> Polyhedron benchmarks - fatigue2.f90 with flang v7 to llvm code,
> then used llc to translate it to My 66000 via a compiler built
> from the source at https://github.com/bagel99/llvm-my66000.git .
>
> When I hit an ICE (not entirely unexpected, if this is the
> first time anybody tried flang with that :-)
>
> VVMLoopPass: read_input_m_
> Unknown f64 condition code!
> UNREACHABLE executed at /home/tkoenig/llvm-my66000/llvm/lib/Target/My66000/My66000ISelLowering.cpp:343!
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
> Stack dump:
> 0. Program arguments: llc -O3 --march=my66000 --enable-vvm fatigue2.ll
> 1. Running pass 'Function Pass Manager' on module 'fatigue2.ll'.
> 2. Running pass 'My66000 DAG->DAG Pattern Instruction Selection' on function '@read_input_m_read_input_'
> #0 0x0000000014b1b4b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/tkoenig/llvm-my66000/llvm/lib/Support/Unix/Signals.inc:565:22
>
> [...]
>
> I did not find a way to raise an issue at github, or I would have
> posted the information there.
<
To my knowledge, Brian has C working at a pretty good level, but
he has not done Fortran--probably has to do with LLVM-IR on stuff
C does not need or use.

On 2/19/22 08:18, Thomas Koenig wrote:
.... snip ...
> To be more precise, I tried compiling a random sample of the
> Polyhedron benchmarks - fatigue2.f90 with flang v7 to llvm code,
> then used llc to translate it to My 66000 via a compiler built
> from the source at https://github.com/bagel99/llvm-my66000.git .
>
> When I hit an ICE (not entirely unexpected, if this is the
> first time anybody tried flang with that :-)
>
> VVMLoopPass: read_input_m_
> Unknown f64 condition code!
> UNREACHABLE executed at /home/tkoenig/llvm-my66000/llvm/lib/Target/My66000/My66000ISelLowering.cpp:343!
> PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
> Stack dump:
> 0. Program arguments: llc -O3 --march=my66000 --enable-vvm fatigue2.ll
> 1. Running pass 'Function Pass Manager' on module 'fatigue2.ll'.
> 2. Running pass 'My66000 DAG->DAG Pattern Instruction Selection' on function '@read_input_m_read_input_'
> #0 0x0000000014b1b4b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/tkoenig/llvm-my66000/llvm/lib/Support/Unix/Signals.inc:565:22
>
> [...]
>
> I did not find a way to raise an issue at github, or I would have
> posted the information there.

You should be able to raise an issue at github, go to the issues tab and hit
new issue. It has worked in the past. Maybe you have to have an account at
github?

Meanwhile send the fatigue2.ll file to bagel99 AT gmail.com. Probably, as
Mitch said, something flang generates that's new to me. Or maybe some test for
NaN that I haven't implemented.

brian

Brian G. Lucas <bagel99@gmail.com> schrieb:
> On 2/19/22 08:18, Thomas Koenig wrote:
> ... snip ...
>> To be more precise, I tried compiling a random sample of the
>> Polyhedron benchmarks - fatigue2.f90 with flang v7 to llvm code,
>> then used llc to translate it to My 66000 via a compiler built
>> from the source at https://github.com/bagel99/llvm-my66000.git .
>>
>> When I hit an ICE (not entirely unexpected, if this is the
>> first time anybody tried flang with that :-)
>>
>> VVMLoopPass: read_input_m_
>> Unknown f64 condition code!
>> UNREACHABLE executed at /home/tkoenig/llvm-my66000/llvm/lib/Target/My66000/My66000ISelLowering.cpp:343!
>> PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
>> Stack dump:
>> 0. Program arguments: llc -O3 --march=my66000 --enable-vvm fatigue2.ll
>> 1. Running pass 'Function Pass Manager' on module 'fatigue2.ll'.
>> 2. Running pass 'My66000 DAG->DAG Pattern Instruction Selection' on function '@read_input_m_read_input_'
>> #0 0x0000000014b1b4b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/tkoenig/llvm-my66000/llvm/lib/Support/Unix/Signals.inc:565:22
>>
>> [...]
>>
>> I did not find a way to raise an issue at github, or I would have
>> posted the information there.
>
> You should be able to raise an issue at github, go to the issues tab and hit
> new issue. It has worked in the past. Maybe you have to have an account at
> github?

It is indeed strange. I have an account at github, and when
looking at another repository, I can see the "Issues" tab, but
not at your repository.

Maybe I got something wrong, my github-fu is as weak as my git-fu.

> Meanwhile send the fatigue2.ll file to bagel99 AT gmail.com. Probably, as
> Mitch said, something flang generates that's new to me. Or maybe some test for
> NaN that I haven't implemented.

Sent about 10 seconds ago.

Thanks for looking into this!

MitchAlsup <MitchAlsup@aol.com> writes:

> On Friday, February 18, 2022 at 5:09:29 PM UTC-6, Josh Vanderhoof wrote:
>> MitchAlsup <Mitch...@aol.com> writes:
>>
>> > On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
>> >> There is no way that a compiler can take the place of out-of-order
>> >> circuitry for allowing a computer to deal with cache misses. These aren't
>> >> predictable.
>> > <
>> > Actually, they are predictable in HW.
>> > <
>> > Say we have a loop that does 2 LDDs and one STD: and we run the loop
>> > "lots" of times;
>> > LDD[1] takes a miss every 8 cycles in iteration MOD 3
>> > LDD[2] takes a miss every 8 cycles in iteration MOD 6
>> > ST....... takes a miss every 8 cycles in iteration MOD 1
>> > <
>> > HW can handle this perfectly after a couple of 16+ loops, and it does not
>> > have to be OoO HW to do this. It could be detected between L1<->L2
>> > and just schedule a 3 prefetches every 8×latency-of-loop.
>> Say you have a loop with unpredictable cache misses like this:
>>
>> for (i = 0; i < n; i++)
>> d[i] = f(s[get_addr(i)]);
> <
> MOV Ri,#0
> for_loop:
> LDD R1,[Rsp+Ri<<2]
> CALL f
> ADD Ri,Ri,#1
> CMP Rt,Ri,Rn
> BLT for_loop
> <
> did you mean to call a function ?

Yes, the function doesn't matter and I assume it would have to be inline
for this to work. What I mean is the loop creates a hard to predict
address, "get_addr()", loads a value from that address, does some work,
"f()", with that value and then stores the result. Any time the load
misses the cache it's going to have to sit and wait for the value, where
an OoO machine would keep busy working on f() for the next loop
iteration that hit the cache.

Would it be worthwhile to break it into 2 loops, one that works on data
already in the cache and skips/prefetches the uncached data, and then a
second loop that runs on the data that the previous loop skipped, which
has hopefully now arrived in the cache due to the prefetches in the
previous loop.

I was thinking a compiler with access to a instruction that tests
whether a memory location is cached could do this, but it also seems
like something you might be able to do automatically in your vvm loops.

>>
>> If you had an "in_cache()" instruction, would a transformation like this
>> be a win?
>>
>> // run loop for everything in cache, skip and prefetch stuff that missed
>> for (i = 0, j = 0; i < n; i++) {
>> p = get_addr(i);
>> if (in_cache(&s[p]))
>> d[i] = f(s[p]);
>> else {
>> prefetch(&s[p]);
>> skipped[j++] = i;
>> }
>> }
>>
>> // finish entries that missed cache
>> for (i = 0; i < j; i++)
>> d[skipped[i]] = f(s[get_addr(skipped[i])]);

Josh Vanderhoof <x@y.z> schrieb:
> MitchAlsup <MitchAlsup@aol.com> writes:
>
>> On Friday, February 18, 2022 at 5:09:29 PM UTC-6, Josh Vanderhoof wrote:
>>> MitchAlsup <Mitch...@aol.com> writes:
>>>
>>> > On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
>>> >> There is no way that a compiler can take the place of out-of-order
>>> >> circuitry for allowing a computer to deal with cache misses. These aren't
>>> >> predictable.
>>> > <
>>> > Actually, they are predictable in HW.
>>> > <
>>> > Say we have a loop that does 2 LDDs and one STD: and we run the loop
>>> > "lots" of times;
>>> > LDD[1] takes a miss every 8 cycles in iteration MOD 3
>>> > LDD[2] takes a miss every 8 cycles in iteration MOD 6
>>> > ST....... takes a miss every 8 cycles in iteration MOD 1
>>> > <
>>> > HW can handle this perfectly after a couple of 16+ loops, and it does not
>>> > have to be OoO HW to do this. It could be detected between L1<->L2
>>> > and just schedule a 3 prefetches every 8×latency-of-loop.
>>> Say you have a loop with unpredictable cache misses like this:
>>>
>>> for (i = 0; i < n; i++)
>>> d[i] = f(s[get_addr(i)]);
>> <
>> MOV Ri,#0
>> for_loop:
>> LDD R1,[Rsp+Ri<<2]
>> CALL f
>> ADD Ri,Ri,#1
>> CMP Rt,Ri,Rn
>> BLT for_loop
>> <
>> did you mean to call a function ?
>
> Yes, the function doesn't matter and I assume it would have to be inline
> for this to work. What I mean is the loop creates a hard to predict
> address, "get_addr()",

Generating the address of an element in a hash table would probably
be the canonical example.

Josh Vanderhoof wrote:
> MitchAlsup <MitchAlsup@aol.com> writes:
>
>> On Friday, February 18, 2022 at 5:09:29 PM UTC-6, Josh Vanderhoof wrote:
>>> MitchAlsup <Mitch...@aol.com> writes:
>>>
>>>> On Friday, February 18, 2022 at 3:10:19 AM UTC-6, Quadibloc wrote:
>>>>> There is no way that a compiler can take the place of out-of-order
>>>>> circuitry for allowing a computer to deal with cache misses. These aren't
>>>>> predictable.
>>>> <
>>>> Actually, they are predictable in HW.
>>>> <
>>>> Say we have a loop that does 2 LDDs and one STD: and we run the loop
>>>> "lots" of times;
>>>> LDD[1] takes a miss every 8 cycles in iteration MOD 3
>>>> LDD[2] takes a miss every 8 cycles in iteration MOD 6
>>>> ST....... takes a miss every 8 cycles in iteration MOD 1
>>>> <
>>>> HW can handle this perfectly after a couple of 16+ loops, and it does not
>>>> have to be OoO HW to do this. It could be detected between L1<->L2
>>>> and just schedule a 3 prefetches every 8×latency-of-loop.
>>> Say you have a loop with unpredictable cache misses like this:
>>>
>>> for (i = 0; i < n; i++)
>>> d[i] = f(s[get_addr(i)]);
>> <
>> MOV Ri,#0
>> for_loop:
>> LDD R1,[Rsp+Ri<<2]
>> CALL f
>> ADD Ri,Ri,#1
>> CMP Rt,Ri,Rn
>> BLT for_loop
>> <
>> did you mean to call a function ?
>
> Yes, the function doesn't matter and I assume it would have to be inline
> for this to work. What I mean is the loop creates a hard to predict
> address, "get_addr()", loads a value from that address, does some work,
> "f()", with that value and then stores the result. Any time the load
> misses the cache it's going to have to sit and wait for the value, where
> an OoO machine would keep busy working on f() for the next loop
> iteration that hit the cache.
>
> Would it be worthwhile to break it into 2 loops, one that works on data
> already in the cache and skips/prefetches the uncached data, and then a
> second loop that runs on the data that the previous loop skipped, which
> has hopefully now arrived in the cache due to the prefetches in the
> previous loop.

That's basically what scout threads do.

https://en.wikipedia.org/wiki/Hardware_scout

BGB <cr88192@gmail.com> writes:
>On 2/15/2022 10:58 AM, Stefan Monnier wrote:
>> The OoO works on the trace of the actual execution, where the main limit
>> is the size of the window it can consider (linked to the accuracy of the
>> branch predictor), whereas the compiler is not limited to such a window
>> but instead it's limited to work on the non-unrolled code.
>>
>
>While this is true, I suspect it is a case of "best" vs "good enough".
>If one can get the in-order scheduling to a "good enough" stage, it may
>be possible to work around this deficiency by being able to throw
>(slightly) more cores at the problem.

If the code is not thread-limited (and a lot of code is not), throwing
more cores at the problem won't help.

In the following I'll assume a workload that is thread-limited (say,
serving many web clients).

>Though, I will note that I am assuming that core counts remain within
>the limits of plausible memory bandwidth, but consider this to be a weak
>argument because, if additional cores would be limited by bandwidth, a
>single faster core would also be limited by bandwidth.

That's an interesting aspect. Mitch Alsup likes to tell us about his
comparison of core sizes and IPC for a small in-order vs. a big OoO
core, but does not tell us about the memory bandwidth. IIRC his
numbers are a factor 16 in core size (including D-cache, but with a
smaller D-cache for the smaller core), factor 2 in IPC.

So if you put 256 of the small cores on a Zen CCD (assuming that a Zen
3 core takes twice as much area as the big core he looked at earlier;
it probably takes more), and if we assume that the little core has a
twice as high D-cache miss rate, which with the lower IPC comes out as
the same number of D-cache misses per cycle. Now you have 32 times as
many D-cache misses hitting the L2 per cycle, with accesses from
different working sets, so a lower hit rate in the L2 cache. So the
number of L2 misses reaching the L3 cache per cycles will be more than
32 times as high as with the big cores; again, you have more working
sets and therefore a higher L3 miss rate, so the number of main memory
accesses per cycle will be more than 32 times as many as with the big
cores. You could mitigate this problem by making the L2 and L3 a lot
larger, but that means that the area advantage of the little cores
becomes smaller.

Another way to look at this is that a big core keeps the same working
set for a smaller time in the caches, and can afterwards work on
another job with the same caches, making better use of the same amount
of cache than a smaller core. Of course, there is a crossover point
somewhere, but I think Mitch Alsup's number paint a too-rosy picture
for the little cores.

>One is more limited by how effectively one can parallelize the codebase,
>but if one is assuming an extended timeframe where no other
>hardware-level performance improvements are viable, programmers will
>"make it work" (IOW: if one assumes that the only other option is
>multiple decades of stagnation).

We have seen ~17 years of stagnation since the introduction of
multi-cores, and I see no signs of that changing in the foreseeable
future.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>