> Clever. I don't see what's wrong about that either, although register
> assignment in the compiler might be "interesting"

BTW, for those who like "interesting", you can go a step further and use
different groups of 8 registers depending on the "base" register so you
can avoid the "silo"ing problem.

Stefan

> The specializer puts a *lot* of work into stuffing as many instructions as
> possible into the bundle format - Mill is a (very) wide architecture. But
> for quick and dirty, i.e. JIT, code you can just put one instruction in each
> bundle and be done with it, and ignore the fact that you are wasting all
> that parallelism. The result will be no worse than code for any other
> one-instruction-at-a-time code.

There's a lot of money poured into making JIT'ed code (Java, Javascript,
....) run as fast as possible, so JIT should not be confused with "quick
and dirty".

Whether it will be harder to generate good JIT'ed code for Mill than for
other machines, I don't know. But the "one-instruction-at-a-time"
option will likely result in a very poor showing: the Mill will look bad
when compared to other architectures, since those will use superscalar
execution to get tolerable speed from the quick&dirty JIT code.

> BIG.little migration is straightforward, so long as both BIG and little
> accept the same binary format. We expect to make "little" configurations
> that are half- or quarter-size one of our large configurations, accepting
> the same binary, and double- or quad-pumping the FUs. After all, all our
> configs have the same instructions set. The differ in parallelism and
> binary format, but they all use the same FUs.

Indeed, another way to deal with the problem is to refrain from using
the flexibility offered by the difference between the abstract ISA and
the actual machine code.

>> My provisional verdict: Mill has been designed with the supercomputer
>> mindset, and suffers from the usual problems of such designs.
> There are answers; perhaps asking for them before verdict would be useful?

I wonder tho: is it the case that there's a "supercomputer mindset"?
That doesn't seem to match your background, at least.
I also got the impression that the Mill might target more
embedded-style markets (maybe because of your mention of DSPs in some
of your talks), but I can't remember mentions of Mills targeting HPC.

Stefan

Ivan Godard <ivan@millcomputing.com> schrieb:
> On 2/13/2022 11:20 PM, Thomas Koenig wrote:
>> Ivan Godard <ivan@millcomputing.com> schrieb:
>>> On 2/13/2022 10:30 AM, Thomas Koenig wrote:
>>
>>>> Which begs the question - what is Mill going to do for operating
>>>> system support, are there any plans? Plan 9 with its multiple
>>>> binaries might actually be an option.
>>>
>>> In progress, albeit slowly. We will provide a microkernel, essentially
>>> at or slightly above the HAL level - this is furthest along. On that
>>> will be a basic Linux port sufficient to provide a standard syscall
>>> interface.
>>
>> I was just about to write that Linux is pretty much tied to gcc, but
>> it seems that clang is also supported now:
>> https://www.kernel.org/doc/html/latest/kbuild/llvm.html
>>
>> so your clang-only compiler infrastructure should not be a problem.
>
> We are compiler-independent, although we have only done clang/LLVM so
> far. Everything past genAsm is our own code, and any compiler can emit
> genAsm.

A bit like nvptx, I guess, which is translated in a "device driver"
into native graphics gard code.

>
>>> The real issue comes from existing software that embeds
>>> hardware or structural assumptions - /proc, drivers, etc. The CHERI
>>> project suffers from the same problem.
>>
>> There is also the problem of which surrounding hardware to use.
>> Hm... are pin layouts for CPUs protected, or could you (for example)
>> lay out the connections on your CPU to be compatible with one of
>> the existing AMD or ARM chips?
>
> I understand that HW expects to adopt existing industry standards for
> such things, but IANAHWG and don't know the details, which won't be
> settled anyway until we reach the point of advancing from FPGA to a real
> chip.

This is the first time I read about you talking about an FPGA for Mill.
This could be quite interesting for soft core.

Do you see advantages in running a Mill of whatever metal persuasion as
a soft core vs the conventional soft cores like MicroBlaze?

Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
>> Clever. I don't see what's wrong about that either, although register
>> assignment in the compiler might be "interesting"
>
> BTW, for those who like "interesting", you can go a step further and use
> different groups of 8 registers depending on the "base" register so you
> can avoid the "silo"ing problem.

Not sure I understand that.

Could you elaborate?

On 2/14/2022 10:44 AM, Stefan Monnier wrote:
>> The specializer puts a *lot* of work into stuffing as many instructions as
>> possible into the bundle format - Mill is a (very) wide architecture. But
>> for quick and dirty, i.e. JIT, code you can just put one instruction in each
>> bundle and be done with it, and ignore the fact that you are wasting all
>> that parallelism. The result will be no worse than code for any other
>> one-instruction-at-a-time code.
>
> There's a lot of money poured into making JIT'ed code (Java, Javascript,
> ...) run as fast as possible, so JIT should not be confused with "quick
> and dirty".
>
> Whether it will be harder to generate good JIT'ed code for Mill than for
> other machines, I don't know. But the "one-instruction-at-a-time"
> option will likely result in a very poor showing: the Mill will look bad
> when compared to other architectures, since those will use superscalar
> execution to get tolerable speed from the quick&dirty JIT code.
>

FWIW: A lot of my code generators had operated "One 3AC op at a time"
with a fairly simplistic register allocator strategy:
Initially assume nothing in registers (at start of Basic Block);
Allocate registers and pull things in dynamically;
Spill registers at end of Basic Block.

BGBCC started out with such a strategy (for my current back-ends).

Though, has moved to also statically assigning a certain number of
variables on the function scope. How does it pick them?
It counts how many times they appear in the function and ranks them by this.
Notable drawback: Does not take account for how often these loads or
stores would be executed at run time.

All this seems to work OK on BJX2, and on x86 this strategy works well.

Having previously tried generating code for 32-bit ARM with this
strategy, its performance relative to GCC was terrible (and the
generated code tended to have a very large number of stack-relative
loads and stores). There also aren't really enough registers to
statically assign variables to them.

Comparably, GCC seems to be smarter, knowing what to keep in registers
and when, and how to make the values flow smoothly from one basic-block
to another.

Well, along with other things, such as GCC being able to inline called
functions, ...

Granted, all the ARM devices I have had access to have been using
in-order superscalar.

Throwing a lot of GPRs at the problem can at least reduce the number of
stack spills (or, mostly eliminate them if the number of GPRs is larger
than the number of variables+temporaries in the function, allowing all
of them to be statically assigned).

Note actually, that BGBCC first compiles to a stack-machine (like in JVM
or .NET), and then converts this to 3AC before feeding it into the
backend code-generator.

So, it sorta converts stack-ops one at a time into 3AC ops, though some
stack operations may be "absorbed" by this translation step.

I guess one can speculate for whether (or how much) the ISA could be
faster with a smarter compiler.

Though, if I added a portability layer, it is possible that a variant of
the stack IR might be used. However, the IR still can't entirely hide
the specifics of the target machine (things like "sizeof(void*)", etc,
are rather difficult to gloss over effectively for C code).

>> BIG.little migration is straightforward, so long as both BIG and little
>> accept the same binary format. We expect to make "little" configurations
>> that are half- or quarter-size one of our large configurations, accepting
>> the same binary, and double- or quad-pumping the FUs. After all, all our
>> configs have the same instructions set. The differ in parallelism and
>> binary format, but they all use the same FUs.
>
> Indeed, another way to deal with the problem is to refrain from using
> the flexibility offered by the difference between the abstract ISA and
> the actual machine code.
>
>>> My provisional verdict: Mill has been designed with the supercomputer
>>> mindset, and suffers from the usual problems of such designs.
>> There are answers; perhaps asking for them before verdict would be useful?
>
> I wonder tho: is it the case that there's a "supercomputer mindset"?
> That doesn't seem to match your background, at least.
> I also got the impression that the Mill might target more
> embedded-style markets (maybe because of your mention of DSPs in some
> of your talks), but I can't remember mentions of Mills targeting HPC.
>

I suspect it is mostly a question of how much is required to run the
specializer...

Something like LLVM is fairly heavyweight.

A stack IR could be a little lighter, but still depends on how much it
assumes the backend to be able to pull off.

For example, a stack IR which identifies entities symbolically, assumes
value types to be passed implicitly via the stack, ... will require a
more advanced VM than one which uses direct indices to the entities it
references, allows layouts to be determined statically, and which
encodes the value-type of each stack operator in the operator itself.

Similarly, for varying approaches to running the code:
Direct Interpreter (A1)
Uses a while loop and a big switch block.
Fairly common approach for interpreters.
Indirect Interpreter (A2)
Decoding IR to a list of structs and function pointers;
Trampoline loop calls the function pointers.
Faster than A1, but also more complicated.
Translation to call-threaded code (B1)
Generated code is a list of function calls to operator functions.
Little or no logic emitted directly.
Common operators are implemented natively (B2)
One may start using optimizations like a register allocator, etc.
A fair number of operators are still handled as function calls.
These calls are typically treated as implicit.
Nearly full native operators (B3)
Register allocation is fundamental to code-generation;
Relatively few runtime calls are used
Runtime calls are typically explicit, rather than implicit.
...

So, my emulators are typically A2 or B1.
For example, my BJX2 emulator uses the A2 approach.

Effective use of A1 or B1 requires the IR to be fairly explicit (all
types/etc need to be readily visible at the level of every IR
instruction). For B1, explicitly encoding the division points between
basic-blocks may also be helpful.

The added "indirection" in A2 and B2 better allow for things like
abstract structure layout and indirect handling of operator types. They
add costs in terms of added complexity and memory overhead (such as
needing to convert the IR into an intermediate internal format).

This would reflect a difference between, say, JVM Bytecode and .NET CIL,
where the former would be more friendly to an A1 strategy, whereas CIL
would likely require an A2, B2, or B3 approach.

Many of my past code generators were B1 or B2.

B1 code-generators are fairly simple.
For a loop over the list of IR instructions:
Figure out which runtime function to call;
Emit a call to said function.
One may also have "blobs" for the start and end of basic-blocks.
Set up initial state when called;
Restore state and return to caller (typically a trampoline loop).

A B1 strategy can be good for, say:
Low memory overhead in compiler;
Relatively fast.
But, drawback:
Generated code is rather slow (on par with an A2 interpreter).

B1 vs B2:
Code generation for B1 may assume an external trampoline loop.
High-level control flow is typically handled externally.
Code generation for B2 may assume function structuring.
May require a full function prolog and epilog.
Control flow is handled within the compiled function.
B2 may be organized around functions rather than individual blocks.

Though, a few of my code-generators have used parts of the B2 strategy
while still otherwise built around the assumption of control flow via an
external trampoline loop.

Differences between the B2 and B3 strategy relate mostly to the handling
of scratch registers:
In B2, they would be assumed to be purely volatile.
They can only be used within a given operator.
May be stomped at any time via an emitted runtime call.
In B3, one can assume that scratch registers are stable.
They may also be handled via the register allocator.
They may potentially be used for holding variables.
...

BGBCC started out closer to B2, but is now mostly using the B3 strategy.

For some types of operations (such as working with 'variant',
'float128', ...), it falls back to the B2 strategy. This is flagged on a
per-basic-block basis; So that the register allocator knows not to use
scratch registers. Scratch registers may not hold statically assigned
variables in normal functions (and are auto-evicted between basic blocks).

Something like GCC or LLVM seem to represent a somewhat more advanced
category.

On 2/14/2022 9:57 AM, Quadibloc wrote:
> On Monday, February 14, 2022 at 2:15:21 AM UTC-7, Anton Ertl wrote:
>
>> My provisional verdict: Mill has been designed with the supercomputer
>> mindset, and suffers from the usual problems of such designs.
>
> Hmm. While I suspect _my_ designs have that flaw, my perception was
> that the focus in Mill was low power consumption rather than maximum
> performance.
>
> Of course, a focus on throughput, rather than single-thread performance,
> may be exactly what you are referring to.
>

FWIW: In my case I am mostly trying to get decent performance while
being economical with resource costs.

Much as clock speeds couldn't get faster indefinitely, transistor
budgets are likely already rapidly approaching the limits of what is
practical.

Hitting this limit is not likely to bode well for the long term
"superiority" of OoO, and may well lead to push-back towards
simpler/cheaper cores.

Though, this may be limited to the extent that L1 cache can't really be
made much cheaper (lots of cores with tiny L1 caches wouldn't exactly be
great either).

IME, the L1 caches seem to reach a stable state at ~ 8K..32K, where hit
rates are upwards of 95%, and increasing cache size only slightly
increases hit rate. Much below this point, hit rate starts to drop off
pretty rapidly.

A core with a 2K L1 cache suffers from a fairly high miss rate.

I have yet to find an "optimal lower limit" for the TLB, as it seems to
depend a lot on the program. Have noted: 256..1K TLBEs works fairly
well, but 16/32/64 TLBE's (4x/8x/16x 4-way) kinda sucks.

However, for these smaller TLB sizes, have noted that address-modulo
indexing seems to have a better hit rate than hashed indexing.

Having a 16x 1-way TLB or similar within the L1 caches can also be help
(can do address translation locally within the L1 cache if it hits).

....

But, less clear are the optimal tradeoffs for a hardware implementation
(for example, 16K L1's with 16 TLBE's in each L1 might potentially be
"too expensive" for a cost-conscious core).

But, then one could also argue about the relative cost of spending the
cost of all of the bits in the register file (say, 32 or 64 GPRs, each
64 bits, isn't exactly free either, ...).

....

>> BTW, for those who like "interesting", you can go a step further and use
>> different groups of 8 registers depending on the "base" register so you
>> can avoid the "silo"ing problem.
> Not sure I understand that.
> Could you elaborate?

For example:

base register possible other registers
============= ========================
0 00,01,02,03,04,05,06,07
1 01,03,05,07,09,11,13,15
2 02,05,08,11,14,17,20,23
.... ......

-- Stefan

On 2/14/2022 9:11 AM, Thomas Koenig wrote:
> Ivan Godard <ivan@millcomputing.com> schrieb:
>> On 2/13/2022 11:20 PM, Thomas Koenig wrote:
>>> Ivan Godard <ivan@millcomputing.com> schrieb:
>>>> On 2/13/2022 10:30 AM, Thomas Koenig wrote:
>>>
>>>>> Which begs the question - what is Mill going to do for operating
>>>>> system support, are there any plans? Plan 9 with its multiple
>>>>> binaries might actually be an option.
>>>>
>>>> In progress, albeit slowly. We will provide a microkernel, essentially
>>>> at or slightly above the HAL level - this is furthest along. On that
>>>> will be a basic Linux port sufficient to provide a standard syscall
>>>> interface.
>>>
>>> I was just about to write that Linux is pretty much tied to gcc, but
>>> it seems that clang is also supported now:
>>> https://www.kernel.org/doc/html/latest/kbuild/llvm.html
>>>
>>> so your clang-only compiler infrastructure should not be a problem.
>>
>> We are compiler-independent, although we have only done clang/LLVM so
>> far. Everything past genAsm is our own code, and any compiler can emit
>> genAsm.
>
> A bit like nvptx, I guess, which is translated in a "device driver"
> into native graphics gard code.
>
>>
>>>> The real issue comes from existing software that embeds
>>>> hardware or structural assumptions - /proc, drivers, etc. The CHERI
>>>> project suffers from the same problem.
>>>
>>> There is also the problem of which surrounding hardware to use.
>>> Hm... are pin layouts for CPUs protected, or could you (for example)
>>> lay out the connections on your CPU to be compatible with one of
>>> the existing AMD or ARM chips?
>>
>> I understand that HW expects to adopt existing industry standards for
>> such things, but IANAHWG and don't know the details, which won't be
>> settled anyway until we reach the point of advancing from FPGA to a real
>> chip.
>
> This is the first time I read about you talking about an FPGA for Mill.
> This could be quite interesting for soft core.
>
> Do you see advantages in running a Mill of whatever metal persuasion as
> a soft core vs the conventional soft cores like MicroBlaze?

The FPGA is a low-cost testbed and proof-of-principle; there's no
present plan to productize it. But that's a business issue, not a
technical one.

On 2/14/2022 8:44 AM, Stefan Monnier wrote:
>> The specializer puts a *lot* of work into stuffing as many instructions as
>> possible into the bundle format - Mill is a (very) wide architecture. But
>> for quick and dirty, i.e. JIT, code you can just put one instruction in each
>> bundle and be done with it, and ignore the fact that you are wasting all
>> that parallelism. The result will be no worse than code for any other
>> one-instruction-at-a-time code.
>
> There's a lot of money poured into making JIT'ed code (Java, Javascript,
> ...) run as fast as possible, so JIT should not be confused with "quick
> and dirty".
>
> Whether it will be harder to generate good JIT'ed code for Mill than for
> other machines, I don't know. But the "one-instruction-at-a-time"
> option will likely result in a very poor showing: the Mill will look bad
> when compared to other architectures, since those will use superscalar
> execution to get tolerable speed from the quick&dirty JIT code.

Whether it is worthwhile to do target-independent optimization (inline,
outline, CFG folds, etc.) is the same tradeoff in a Mill as it is for
any other ISA target. Mill differs only because of static scheduling,
which the super-scalar does dynamically in hardware. How much you want
to pull the stops out in the scheduler is a market trade-off, not a
technical one, and we don't know where the balance point is. Current
guess is that scheduling cost will be in the noise.

>> BIG.little migration is straightforward, so long as both BIG and little
>> accept the same binary format. We expect to make "little" configurations
>> that are half- or quarter-size one of our large configurations, accepting
>> the same binary, and double- or quad-pumping the FUs. After all, all our
>> configs have the same instructions set. The differ in parallelism and
>> binary format, but they all use the same FUs.
>
> Indeed, another way to deal with the problem is to refrain from using
> the flexibility offered by the difference between the abstract ISA and
> the actual machine code.
>
>>> My provisional verdict: Mill has been designed with the supercomputer
>>> mindset, and suffers from the usual problems of such designs.
>> There are answers; perhaps asking for them before verdict would be useful?
>
> I wonder tho: is it the case that there's a "supercomputer mindset"?
> That doesn't seem to match your background, at least.
> I also got the impression that the Mill might target more
> embedded-style markets (maybe because of your mention of DSPs in some
> of your talks), but I can't remember mentions of Mills targeting HPC.

It's a general purpose architecture, suitable for all except the very
low end.

>
>
> Stefan

On Monday, February 14, 2022 at 9:29:35 AM UTC-7, Stefan Monnier wrote:
> > Clever. I don't see what's wrong about that either, although register
> > assignment in the compiler might be "interesting"

> BTW, for those who like "interesting", you can go a step further and use
> different groups of 8 registers depending on the "base" register so you
> can avoid the "silo"ing problem.

Unfortunately, my designs can't take advantage of that; they already only
use one set of eight base registers, as they don't have room for a larger
field. And I expect that most code would use all 32 registers in conjunction
with the area of memory to which a single base register points in the
most common case.

John Savard

BGB <cr88192@gmail.com> writes:
>Hitting this limit is not likely to bode well for the long term
>"superiority" of OoO,

Why? If OoO is superior now at the current transistor budget, why
would it stop being superior if the transistor budget does not
increase?

>and may well lead to push-back towards
>simpler/cheaper cores.

If/When the increase in transistors/$ finally stops, it will be
interesting to see what happens then. Will we see more specialized
hardware, because now you can amortize the development (and mask
costs) over a longer time, even if you only serve a niche? Will
software developers finally get around to designing software that
makes the maximum of the hardware, because inefficiency can no longer
be papered over with faster hardware?

I have my doubts. Network effects will neuter the advantages of
specialized hardware and of carefully designed software that is late
to the market.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Monday, February 14, 2022 at 2:26:46 PM UTC-8, Anton Ertl wrote:
> BGB <cr8...@gmail.com> writes:
> >Hitting this limit is not likely to bode well for the long term
> >"superiority" of OoO,
> Why? If OoO is superior now at the current transistor budget, why
> would it stop being superior if the transistor budget does not
> increase?

Not sure if the assumption of OoO superiority is justified at current transistor budget, but for the same transistor budget at the same clock rate, statically scheduled should win because it more efficiently uses the available transistors to do useful work.
Every time a program runs on an OoO machine, it wastes power to speculate about un-taken paths that can be avoided when source code compiles to a statically scheduled target instruction set.
Once the speculation transistors are removed, a statically scheduled chip can replace them with additional FUs to get more done each clock cycle.

> >and may well lead to push-back towards
> >simpler/cheaper cores.
> If/When the increase in transistors/$ finally stops, it will be
> interesting to see what happens then. Will we see more specialized
> hardware, because now you can amortize the development (and mask
> costs) over a longer time, even if you only serve a niche? Will
> software developers finally get around to designing software that
> makes the maximum of the hardware, because inefficiency can no longer
> be papered over with faster hardware?
>
> I have my doubts. Network effects will neuter the advantages of
> specialized hardware and of carefully designed software that is late
> to the market.
> - anton

Statically scheduled isn't specialized. Optimizing transistor budgets in a general purpose chip is, to a great degree, separable from optimizing the applications that run on it, and it should be.

Imo, that's what makes Mill so keen, but as you say, it will be interesting to see.

> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 2/14/2022 4:17 PM, Anton Ertl wrote:
> BGB <cr88192@gmail.com> writes:
>> Hitting this limit is not likely to bode well for the long term
>> "superiority" of OoO,
>
> Why? If OoO is superior now at the current transistor budget, why
> would it stop being superior if the transistor budget does not
> increase?
>

Advantages of OoO were mostly:
"Ye Olde" scalar code goes keeps getting faster;
Not requiring as much from the compiler;
Effective at getting maximizing single thread performance;
...

It achieved this at the expense of spending considerable transistor
budget and energy on the problem.

The demand for higher performance will continue past the point where
continuously increasing transistor budget is no longer viable.

However, the push-back may favor designs which can give more performance
relative to transistor budget and energy use. Namely, such as VLIW cores
with dynamic translation from some higher-level IR (could be a
stack-based IR, but also likely would be an existing ISA being
repurposed as an IR).

Ideally, one needs an ISA that "doesn't suck" as an IR, where I suspect
both x86-64 and ARM64 making use of condition-codes does not exactly
work in their favor in this use-case.

While RISC-V doesn't use condition codes (good in this sense), it also
kinda sucks in many other areas.

I have yet to figure out what the ideal IR would look like exactly (or
even if it would necessarily be singular).

Though, there were a few considered (but mostly stalled) sub-efforts
towards trying to run x86-64 code on top of BJX2.

>> and may well lead to push-back towards
>> simpler/cheaper cores.
>
> If/When the increase in transistors/$ finally stops, it will be
> interesting to see what happens then. Will we see more specialized
> hardware, because now you can amortize the development (and mask
> costs) over a longer time, even if you only serve a niche? Will
> software developers finally get around to designing software that
> makes the maximum of the hardware, because inefficiency can no longer
> be papered over with faster hardware?
>
> I have my doubts. Network effects will neuter the advantages of
> specialized hardware and of carefully designed software that is late
> to the market.
>

My prediction is more in the form of VLIW based manycore systems likely
running software on top of a dynamic translation layer.

Unlike traditional VLIW compilers, the dynamic translation later could
have access to live / real-time profiler data, so could make better
guesses about things like when and how to go about modulo scheduling
loops and similar.

Sadly, what I am imagining here, isn't all that much like my BJX2
project, but BJX2 is limited more to what I can fit on an FPGA, and
ended up where it is because I kept finding ways I could push it
"forwards" with mostly only small/incremental increases in resource cost.

Though, yes, a 1-wide pipelined RISC core is a little cheaper than a
3-wide VLIW, but if limited to the same clock-speed, a 3-wide core can
outperform a 1-wide core.

The situation might be different if the 1-wide RISC were running at
100MHz, but I have found that "reliably"/"easily" passing timing at
100MHz (on the Spartan-7 and Artix-7) seems to require a using a 32-bit
ISA design (such as RV32I).

But, personally, I don't find RV32I all that inspiring.
Like, RV32I is like some sort of weird phantom that does pretty good on
Dhrystone but seemingly kinda sucks at nearly everything else.

....

On Monday, February 14, 2022 at 9:57:52 AM UTC-6, Quadibloc wrote:
> On Monday, February 14, 2022 at 2:15:21 AM UTC-7, Anton Ertl wrote:
>
> > My provisional verdict: Mill has been designed with the supercomputer
> > mindset, and suffers from the usual problems of such designs.
<
> Hmm. While I suspect _my_ designs have that flaw, my perception was
> that the focus in Mill was low power consumption rather than maximum
> performance.
<
Other than not having a 32-bit subset: My 66000 ISA can be implemented
on a 1-wide in order implementation up through at least the 6-wide GBOoO
implementation I am looking at.
<
At the system level: we are at the density where we expect multiple cores
on a die, along with plenty of cache, multiple DRAM channels, several PCIe
links, and a Chip repeater.
<
This is no where near a Super Computer today. It takes a basket ball stadium
filled with racks, each rack filled with systems, all connected as a great big
network to qualify as a Super Computer today.
<
Now, with that in mind:: It seems to me that we are approaching the density
where one would box up 16-cores, a memory controller, and PCIe links as
<pick some name> and this becomes the unit of replication in the Chip--not
the cores themselves!
>
> Of course, a focus on throughput, rather than single-thread performance,
> may be exactly what you are referring to.
>
> John Savard

On Monday, February 14, 2022 at 2:02:56 PM UTC-6, BGB wrote:
> On 2/14/2022 9:57 AM, Quadibloc wrote:
> > On Monday, February 14, 2022 at 2:15:21 AM UTC-7, Anton Ertl wrote:
> >
> >> My provisional verdict: Mill has been designed with the supercomputer
> >> mindset, and suffers from the usual problems of such designs.
> >
> > Hmm. While I suspect _my_ designs have that flaw, my perception was
> > that the focus in Mill was low power consumption rather than maximum
> > performance.
> >
> > Of course, a focus on throughput, rather than single-thread performance,
> > may be exactly what you are referring to.
> >
> FWIW: In my case I am mostly trying to get decent performance while
> being economical with resource costs.
>
>
> Much as clock speeds couldn't get faster indefinitely, transistor
> budgets are likely already rapidly approaching the limits of what is
> practical.
<
Check out PCIe 6.0 with links operating at 64Gts (16 GHz, PAM4 modulation,
Double Data Rate) per wire (unidirectional twisted pair.)
>
> Hitting this limit is not likely to bode well for the long term
> "superiority" of OoO, and may well lead to push-back towards
> simpler/cheaper cores.
>
>
> Though, this may be limited to the extent that L1 cache can't really be
> made much cheaper (lots of cores with tiny L1 caches wouldn't exactly be
> great either).
>
> IME, the L1 caches seem to reach a stable state at ~ 8K..32K, where hit
> rates are upwards of 95%, and increasing cache size only slightly
> increases hit rate. Much below this point, hit rate starts to drop off
> pretty rapidly.
<
Ahem:: It is not hit rate that matters--it is the miss rate and the miss
multiplier which matter.
<
L1s are stable because if the get bigger than 64 KB they take more than
one cycle to "access", where SRAM and wire routing are both considered.
<
Speed Deamons use smaller caches and target numerical applications.
Servers use larger applications, lower clock rates, and target server
"stuff". A speed deamon running data base is at a performance disadvantage
to a server oriented design running the same.
>
> A core with a 2K L1 cache suffers from a fairly high miss rate.
<
<ahem> Long memory latency.
>
>
> I have yet to find an "optimal lower limit" for the TLB, as it seems to
> depend a lot on the program. Have noted: 256..1K TLBEs works fairly
> well, but 16/32/64 TLBE's (4x/8x/16x 4-way) kinda sucks.
>
> However, for these smaller TLB sizes, have noted that address-modulo
> indexing seems to have a better hit rate than hashed indexing.
>
> Having a 16x 1-way TLB or similar within the L1 caches can also be help
> (can do address translation locally within the L1 cache if it hits).
>
> ...
>
>
> But, less clear are the optimal tradeoffs for a hardware implementation
> (for example, 16K L1's with 16 TLBE's in each L1 might potentially be
> "too expensive" for a cost-conscious core).
>
> But, then one could also argue about the relative cost of spending the
> cost of all of the bits in the register file (say, 32 or 64 GPRs, each
> 64 bits, isn't exactly free either, ...).
>
> ...

On Monday, February 14, 2022 at 4:26:46 PM UTC-6, Anton Ertl wrote:
> BGB <cr8...@gmail.com> writes:
> >Hitting this limit is not likely to bode well for the long term
> >"superiority" of OoO,
> Why? If OoO is superior now at the current transistor budget, why
> would it stop being superior if the transistor budget does not
> increase?
<
Since 2003 the hope was that SW would figure out how to use
as many processors as chips could hold. So far this has been
mostly a failure.
<
> >and may well lead to push-back towards
> >simpler/cheaper cores.
<
> If/When the increase in transistors/$ finally stops, it will be
> interesting to see what happens then. Will we see more specialized
> hardware, because now you can amortize the development (and mask
> costs) over a longer time, even if you only serve a niche? Will
> software developers finally get around to designing software that
> makes the maximum of the hardware, because inefficiency can no longer
> be papered over with faster hardware?
<
Until the transistor wall gets hit, designers will simply consume transistors.
<
Afterwards, one must measure and use them wisely.
>
> I have my doubts. Network effects will neuter the advantages of
> specialized hardware and of carefully designed software that is late
> to the market.
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On Monday, February 14, 2022 at 6:22:45 PM UTC-6, BGB wrote:
> On 2/14/2022 4:17 PM, Anton Ertl wrote:
> > BGB <cr8...@gmail.com> writes:
> >> Hitting this limit is not likely to bode well for the long term
> >> "superiority" of OoO,
> >
> > Why? If OoO is superior now at the current transistor budget, why
> > would it stop being superior if the transistor budget does not
> > increase?
> >
> Advantages of OoO were mostly:
> "Ye Olde" scalar code goes keeps getting faster;
> Not requiring as much from the compiler;
<
A good OoO implementation wants the compiler to do LESS--encode
the subroutine with as few instructions as possible, don't bother to
schedule them, ... in the early 1990s Mc88120 was underperforming
on code optimized for Mc88100, where the code was heavily scheduled
loops software pipelined, instructions put in "inconsiderate" places
for the GBOoO machine we had. We actually god faster code most
of the time with -O1 and sometimes -O2 than -O3.
<
> Effective at getting maximizing single thread performance;
> ...
>
> It achieved this at the expense of spending considerable transistor
> budget and energy on the problem.
>
> The demand for higher performance will continue past the point where
> continuously increasing transistor budget is no longer viable.
>
>
> However, the push-back may favor designs which can give more performance
> relative to transistor budget and energy use. Namely, such as VLIW cores
> with dynamic translation from some higher-level IR (could be a
> stack-based IR, but also likely would be an existing ISA being
> repurposed as an IR).
>
>
>
> Ideally, one needs an ISA that "doesn't suck" as an IR, where I suspect
> both x86-64 and ARM64 making use of condition-codes does not exactly
> work in their favor in this use-case.
<
x86 basically has 3 condition codes C, O, and ZAPS and HW trance each
independently.
>
> While RISC-V doesn't use condition codes (good in this sense), it also
> kinda sucks in many other areas.
<
Academic quality.
>
> I have yet to figure out what the ideal IR would look like exactly (or
> even if it would necessarily be singular).
<
LLVM intermediate is actually quite good.
>
> Though, there were a few considered (but mostly stalled) sub-efforts
> towards trying to run x86-64 code on top of BJX2.
> >> and may well lead to push-back towards
> >> simpler/cheaper cores.
> >
> > If/When the increase in transistors/$ finally stops, it will be
> > interesting to see what happens then. Will we see more specialized
> > hardware, because now you can amortize the development (and mask
> > costs) over a longer time, even if you only serve a niche? Will
> > software developers finally get around to designing software that
> > makes the maximum of the hardware, because inefficiency can no longer
> > be papered over with faster hardware?
> >
> > I have my doubts. Network effects will neuter the advantages of
> > specialized hardware and of carefully designed software that is late
> > to the market.
> >
> My prediction is more in the form of VLIW based manycore systems likely
> running software on top of a dynamic translation layer.
>
> Unlike traditional VLIW compilers, the dynamic translation later could
> have access to live / real-time profiler data, so could make better
> guesses about things like when and how to go about modulo scheduling
> loops and similar.
>
>
> Sadly, what I am imagining here, isn't all that much like my BJX2
> project, but BJX2 is limited more to what I can fit on an FPGA, and
> ended up where it is because I kept finding ways I could push it
> "forwards" with mostly only small/incremental increases in resource cost.
>
>
> Though, yes, a 1-wide pipelined RISC core is a little cheaper than a
> 3-wide VLIW, but if limited to the same clock-speed, a 3-wide core can
> outperform a 1-wide core.
>
>
> The situation might be different if the 1-wide RISC were running at
> 100MHz, but I have found that "reliably"/"easily" passing timing at
> 100MHz (on the Spartan-7 and Artix-7) seems to require a using a 32-bit
> ISA design (such as RV32I).
>
> But, personally, I don't find RV32I all that inspiring.
> Like, RV32I is like some sort of weird phantom that does pretty good on
> Dhrystone but seemingly kinda sucks at nearly everything else.
>
> ...
<
Also note: I now have over 100 wed sites where JavaSxcript is turned off
just to get rid of annoying advertising.

On Monday, February 14, 2022 at 5:22:45 PM UTC-7, BGB wrote:
> On 2/14/2022 4:17 PM, Anton Ertl wrote:
> > BGB <cr8...@gmail.com> writes:
> >> Hitting this limit is not likely to bode well for the long term
> >> "superiority" of OoO,

> > Why? If OoO is superior now at the current transistor budget, why
> > would it stop being superior if the transistor budget does not
> > increase?

> Advantages of OoO were mostly:
> "Ye Olde" scalar code goes keeps getting faster;
> Not requiring as much from the compiler;
> Effective at getting maximizing single thread performance;

> It achieved this at the expense of spending considerable transistor
> budget and energy on the problem.

> The demand for higher performance will continue past the point where
> continuously increasing transistor budget is no longer viable.

You - and Mitch Alsup - are such *optimists*!

Given that out-of-order execution requires a _lot_ of transistors
for the extra performance it provides, then it logically follows
that one could get more throughput by putting a large number
of in-order processors on the same die, does it not?

If so, why did Intel change the Atom over from in-order to
out-of-order, and why did the Xeon Phi fail?

I have two answers to offer:

1) We don't know how to write parallel code well, and we
never will. At least for the values of "well" which would
allow speed to scale as the number of processors applied
over a much larger range than is the case at present for
a given application.

2) The potential memory bandwidth feeding a single die
is a serious constraint. Thus, it will not be possible to
make use of the maximum throughput that could be achieved
on a die through a very large number of in-order processors.
The best possible would be a lesser value, which is what
is required by the case of a smaller number of OoO processors,
which would offer the benefit of dealing better with constraint
(1) above.

John Savard

On Monday, February 14, 2022 at 7:13:13 PM UTC-7, MitchAlsup wrote:

> Speed Deamons use smaller caches and target numerical applications.
> Servers use larger applications, lower clock rates, and target server
> "stuff". A speed deamon running data base is at a performance disadvantage
> to a server oriented design running the same.

Currently, big.LITTLE designs are oriented around the big processors
being the ones designed to do the actual work, and the little processors
being enough to handle updating the GUI, providing the processor with
enough oomph to let the operating system handle its routine busywork,
and to run highly interactive applications which, not being computationally
intensive, could have run just as well on a 386SX running Windows 3.1.

However, the Efficiency cores in Intel's designs with P and E cores are
apparently somewhat more powerful than LITTLE cores in the ARM chips
in people's smartphones, so this generalization may be less true in that
case.

Still, though, given that a PC might be used for number-crunching at
one moment, and for database queries at another, to me this sounds
like an argument for putting both speed demon performance cores
and server performance cores in a design, as well as cores oriented
towards power efficiency.

That is, though, only if data base still needs OoO cores, just less
enormous than those suited to number-crunching. If data base
would prefer a larger number of really small cores, then we don't
need a third kind of core, we just need chips with, say, four performance
cores and sixteen efficiency cores instead of four performance cores
and two efficiency cores.

A high-performance chip would have four of those dies in it;
sixteen performance cores to keep up with an AMD Ryzen 9 5950
or its current successor, and sixty four efficiency cores to supply
a more appropriate hardware resource when database processing
is what is being done.

I presume that by the time one gets to 4nm, it will be possible to
have such a chip running at around 3 GHz without requiring any
exotic cooling solution? And while sixty-four efficiency cores would
require a lot of memory bandwith, there would be room for enough
cache so that the requirement could actually be met, even if a
modest exercise of ingenuity will be required?

And, of course, since AMD already makes Threadrippers, it
could be that I'm being entirely too cautious in predicting what
could be achieved in just a couple of years if they really tried.

John Savard

> Not sure if the assumption of OoO superiority is justified at current
> transistor budget, but for the same transistor budget at the same clock
> rate, statically scheduled should win because it more efficiently uses the
> available transistors to do useful work.

What makes you think so? AFAIK OoO cores are better at keeping their
FUs busy, and they're probably also better at keeping the memory
hierarchy busy. Maybe they do that at the cost of more transistors
"wasted" doing "administrative overhead", but it's far from obvious that
this overhead is the only thing that matters.

Stefan

> Unfortunately, my designs can't take advantage of that; they already
> only use one set of eight base registers, as they don't have room for
> a larger field.

My suggestion doesn't require more bits in the instruction.
[ I don't claim it's a good idea, tho. It might be, but if so, it's
probably at the cost of a nasty register allocation problem. ]

Stefan

On Monday, February 14, 2022 at 8:13:38 PM UTC-7, Stefan Monnier wrote:
> > Unfortunately, my designs can't take advantage of that; they already
> > only use one set of eight base registers, as they don't have room for
> > a larger field.

> My suggestion doesn't require more bits in the instruction.
> [ I don't claim it's a good idea, tho. It might be, but if so, it's
> probably at the cost of a nasty register allocation problem. ]

I wouldn't be qualified to say if the register allocation problem
would be so nasty as to negate its benefits.

My problem, instead, is that I can't see its potential benefits;
while I want multiple strands of execution going on, using
different registers for their work, I didn't imagine that they
would be using separate regions of memory.

If I _did_ have multiple strands of execution _like that_
to deal with, instead of playing with the base registers,
I would put them on different physical CPUs with different
buses to different arrays of DRAM, connected to each other
with Ethernet cables. That way, I could scale the memory
bandwidth with no headaches.

Now, of course, if they shared _some_ memory, but some
of the memory they used was different - and they really
were part of the same program - possibly enlarging the
pool of base registers available for a given number of
bits in the instruction by using the high bits of the
destination register... *would* make sense.

I'm just not familiar with a class of applications with such
a large thirst for memory.

Actually, though, I *did* come up with an idea to address
a case sort of like that. Array Mode.

Given that some programs require really big arrays, arrays
so big as to be equal to the memory expanse covered through
the use of a single base register...

instead of trying to figure out a way to use 32 base registers
instead of 8, I refused to settle for half measures, and instead
went with a form of indirect addressing - the displacement
field of the instruction is replaced by an array selector, which chooses
the memory location containing the base of the array, to which the
index register contents are added.

Registers are a scarce resource. Using *more* of them as static
base registers just didn't seem like a good idea to me. The pain of
a second memory access for indirect addressing can be taken care
of by a good cache.

So, basically, I didn't consider anything like your idea because I
went straight for the nuclear option.

John Savard

On 2/14/2022 8:49 PM, Quadibloc wrote:
> On Monday, February 14, 2022 at 5:22:45 PM UTC-7, BGB wrote:
>> On 2/14/2022 4:17 PM, Anton Ertl wrote:
>>> BGB <cr8...@gmail.com> writes:
>>>> Hitting this limit is not likely to bode well for the long term
>>>> "superiority" of OoO,
>
>>> Why? If OoO is superior now at the current transistor budget, why
>>> would it stop being superior if the transistor budget does not
>>> increase?
>
>> Advantages of OoO were mostly:
>> "Ye Olde" scalar code goes keeps getting faster;
>> Not requiring as much from the compiler;
>> Effective at getting maximizing single thread performance;
>
>> It achieved this at the expense of spending considerable transistor
>> budget and energy on the problem.
>
>> The demand for higher performance will continue past the point where
>> continuously increasing transistor budget is no longer viable.
>
> You - and Mitch Alsup - are such *optimists*!
>
> Given that out-of-order execution requires a _lot_ of transistors
> for the extra performance it provides, then it logically follows
> that one could get more throughput by putting a large number
> of in-order processors on the same die, does it not?
>
> If so, why did Intel change the Atom over from in-order to
> out-of-order, and why did the Xeon Phi fail?
>

One issue with x86 (and x86-64) is that the design of the ISA kinda
sucks for in-order cores as well.

What I am imagining here is not x86.

I am not aware of any (particularly mainstream) attempts at VLIW
processors, at least much beyond the (ill fated) Itanium.

Though, something more like a manycore Itanium is closer to what I am
imagining than something like an Atom or Xeon Phi.

> I have two answers to offer:
>
> 1) We don't know how to write parallel code well, and we
> never will. At least for the values of "well" which would
> allow speed to scale as the number of processors applied
> over a much larger range than is the case at present for
> a given application.
>
> 2) The potential memory bandwidth feeding a single die
> is a serious constraint. Thus, it will not be possible to
> make use of the maximum throughput that could be achieved
> on a die through a very large number of in-order processors.
> The best possible would be a lesser value, which is what
> is required by the case of a smaller number of OoO processors,
> which would offer the benefit of dealing better with constraint
> (1) above.
>

Possible, though there is also the possibility that memory bandwidth
could still be improved further, such as by making memory modules that
have large numbers of lanes.

Though, this partly becomes a question of how much IO is it viable to
route through a PCB, or if we start seeing RAM modules mounted on top of
the CPU.

> My problem, instead, is that I can't see its potential benefits;
> while I want multiple strands of execution going on, using
> different registers for their work, I didn't imagine that they
> would be using separate regions of memory.
[...]
> Registers are a scarce resource. Using *more* of them as static
> base registers just didn't seem like a good idea to me. The pain of
> a second memory access for indirect addressing can be taken care
> of by a good cache.

I don't understand what this has to do with my suggestion.

AFAIU the design mentioned earlier was two have 2-reg instructions
for things like add/sub/... but instead of allowing both of those
registers to be any of the 32 regs in the register file (eating up
2x5bit = 10bit of the instruction), one of the operands could be any of
the 32 registers, while the other was limited to only 8 registers, and
those 8 registers depend on which register is used for the
first operand.
IOW

base register allowed second register
============= =======================
00-07 00-07
08-15 08-15
16-23 16-23
23-31 23-31

So you need 5bits to encode the base register but only 3bits to encode
the second register, thus saving 2 previous bits, which can be quite
valuable in the tight space of 16bit instructions. The downside is that
combining data from registers 00-07 which data from registers 08-16
(for example) can't be done with those instructions, so you need to
resort to "full size" instructions for that.

My suggestion is to make the mapping more complex so that instead of
having 3 big walls splitting your register file into completely disjoint
sets, you have many small walls all over the place: they still hinder
data movement, but the register file is not partitioned any more, so you
might be able to avoid using a full size instruction by (instead)
carefully choosing which registers you use.

In a way this is a similar idea to Seznec's skewed associative caches,
applied in a different context.

Finding which registers to use for which operation so as to avoid extra
register moves then becomes a kind of search for the exit of
a labyrinth.

The example of mapping I gave is probably a very poor choice
(e.g. it doesn't seem to lend itself to a simple&efficient
implementation), but something like:

base register allowed second register
============= =======================
00-01 00-07
02-03 08-15
04-05 16-23
06-07 23-31
08-09 00-07
10-11 08-15
12-13 16-23
14-15 23-31
...

seems cheap to implement in hardware (it might still be challenging for
the register allocator, OTOH).

Stefan

On Monday, February 14, 2022 at 9:12:06 PM UTC-7, Stefan Monnier wrote:

> I don't understand what this has to do with my suggestion.

> My suggestion is to make the mapping more complex so that instead of
> having 3 big walls splitting your register file into completely disjoint
> sets, you have many small walls all over the place:

I am sorry. I completely misunderstood your suggestion. When I
saw the phrase "base register", I automatically took it to mean
a register used to indicate a region of memory, so that displacements
in instructions didn't need to be the full width of an address: like
segment registers in the x86, or like what are referred to as base
registers on the System/360.

You were using that phrase in a different sense, as the register
indicating which group of registers was to be used.

John Savard