Just read this article on HPC Wire:

https://www.hpcwire.com/2022/08/11/google-program-to-free-chips-boosts-university-semiconductor-design/

John Savard

On Friday, August 12, 2022 at 9:56:22 PM UTC-6, Quadibloc wrote:
> Just read this article on HPC Wire:
>
> https://www.hpcwire.com/2022/08/11/google-program-to-free-chips-boosts-university-semiconductor-design/

September 13 is the deadline for the next batch. It's 130nm, which may be old, but it's good enough - the Pentium
Pro was implemented in 350 to 500 nm, although it was called 0.35 um to 0.5 um at the time, so you can make a
big core at 130nm.

John Savard

Quadibloc <jsavard@ecn.ab.ca> schrieb:
> On Friday, August 12, 2022 at 9:56:22 PM UTC-6, Quadibloc wrote:
>> Just read this article on HPC Wire:
>>
>> https://www.hpcwire.com/2022/08/11/google-program-to-free-chips-boosts-university-semiconductor-design/
>
> September 13 is the deadline for the next batch. It's 130nm,
> which may be old, but it's good enough - the Pentium Pro was
> implemented in 350 to 500 nm, although it was called 0.35 um to
> 0.5 um at the time, so you can make a big core at 130nm.

https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
done in a 130 nm process, 512 kb L2 cache and a transistor count
of 58 million. It ran at up to 1.7 GHz.

Sounds like a 66130 would fit very comfortably, at least for size.

On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:

> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
> done in a 130 nm process, 512 kb L2 cache and a transistor count
> of 58 million. It ran at up to 1.7 GHz.

And one sees the press gloating that Russia is helpless, because they can't
make chips beyond 65nm, so sanctions have them at our mercy!

The Red Chinese designed their H-bomb on a 24-bit computer made out
of discrete transistors.

And there are even articles saying the Chinese 7nm chips are nothing to
worry about, because they're an early 7nm process that doesn't use EUV.
The Ryzen 9 3900 12-core processor in my daily driver was made that way,
and I've seen no need to update it with something later and greater.

Sanctions are good, but let's not kid ourselves; to be able to make fast
and powerful processors, a country doesn't need what constitutes a bleeding-edge
process node these days: we had fast and powerful processors for years now,
so even an out-of-date process node is enough to allow a country to make weapons
with which to make a serious nuisance of itself.

John Savard

On 8/13/2022 12:27 PM, Quadibloc wrote:
> On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:
>
>> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
>> done in a 130 nm process, 512 kb L2 cache and a transistor count
>> of 58 million. It ran at up to 1.7 GHz.
>
> And one sees the press gloating that Russia is helpless, because they can't
> make chips beyond 65nm, so sanctions have them at our mercy!
>
> The Red Chinese designed their H-bomb on a 24-bit computer made out
> of discrete transistors.

The Eniac was used for some US H-bomb calculations. And, of course, the
calculations for the A-bomb were done by Computers, when the word was a
job description, not a thing!

snip

> Sanctions are good, but let's not kid ourselves; to be able to make fast
> and powerful processors, a country doesn't need what constitutes a bleeding-edge
> process node these days: we had fast and powerful processors for years now,
> so even an out-of-date process node is enough to allow a country to make weapons
> with which to make a serious nuisance of itself.

Yes, and some weapons don't require any computing power, e.g. passenger
jets on 9/11, or biological weapons.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On Saturday, August 13, 2022 at 2:27:45 PM UTC-5, Quadibloc wrote:
> On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:
>
> > https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
> > done in a 130 nm process, 512 kb L2 cache and a transistor count
> > of 58 million. It ran at up to 1.7 GHz.
> And one sees the press gloating that Russia is helpless, because they can't
> make chips beyond 65nm, so sanctions have them at our mercy!
>
> The Red Chinese designed their H-bomb on a 24-bit computer made out
> of discrete transistors.
>
> And there are even articles saying the Chinese 7nm chips are nothing to
> worry about, because they're an early 7nm process that doesn't use EUV.
> The Ryzen 9 3900 12-core processor in my daily driver was made that way,
> and I've seen no need to update it with something later and greater.
<
I was designing for 5GHz in 65nm............oh so long ago.
<
If you are looking for a small in-order machine at reasonable power and
frequency, you can make a sub 1mm^2 super-macro in 45nm complete
with its cache hierarchy.
<
It is only the GBOoO machines that require bleeding edge lithography--
these are 15×-30× the size of a reasonable in-order core. Of course,
Caches are the same size per unit store for both small IO and GBOoO.
>
> Sanctions are good, but let's not kid ourselves; to be able to make fast
> and powerful processors, a country doesn't need what constitutes a bleeding-edge
> process node these days: we had fast and powerful processors for years now,
> so even an out-of-date process node is enough to allow a country to make weapons
> with which to make a serious nuisance of itself.
<
You must be talking about those IOT thingies................
>
> John Savard

On 8/13/2022 12:56 PM, MitchAlsup wrote:
> On Saturday, August 13, 2022 at 2:27:45 PM UTC-5, Quadibloc wrote:
>> On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:
>>
>>> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
>>> done in a 130 nm process, 512 kb L2 cache and a transistor count
>>> of 58 million. It ran at up to 1.7 GHz.
>> And one sees the press gloating that Russia is helpless, because they can't
>> make chips beyond 65nm, so sanctions have them at our mercy!
>>
>> The Red Chinese designed their H-bomb on a 24-bit computer made out
>> of discrete transistors.
>>
>> And there are even articles saying the Chinese 7nm chips are nothing to
>> worry about, because they're an early 7nm process that doesn't use EUV.
>> The Ryzen 9 3900 12-core processor in my daily driver was made that way,
>> and I've seen no need to update it with something later and greater.
> <
> I was designing for 5GHz in 65nm............oh so long ago.
> <
> If you are looking for a small in-order machine at reasonable power and
> frequency, you can make a sub 1mm^2 super-macro in 45nm complete
> with its cache hierarchy.
> <
> It is only the GBOoO machines that require bleeding edge lithography--
> these are 15×-30× the size of a reasonable in-order core. Of course,
> Caches are the same size per unit store for both small IO and GBOoO.

Total chip area doesn't matter: it's dominated by pins and pads and
caches, so for a given core count you get about as many dice per wafer
regardless of ISA and micro-architecture.

What matters is *yield* - how many *good* dice do you get per wafer. And
defect rates in different parts of the die have wildly disparate effects
on the yield. A defect in a pin pad is not even noticed. A defect in a
cache can be (often) fixed with a hot spare line/entry during fab; no
impact on profit. A defect in a core can bin to a model with fewer
cores: not a total loss, but still bottom line impact.

So the dollar impact of yield is dominated by the area of the die
occupied by logic that cannot use free sparing. Guess what is in that
area: the core. A chip with 10 GBOOO cores may have twice the mm^2 area
of one with 10 in-order, but it will have 30x the defect-vulnerable
area. Until the fab process has matured enough to be effectively
defect-free, the IO yield per wafer can easily be twice that of the GBOOO.

So, rough numbers IO/GBOOO:
1-2x chips/wafer
1-4x good/die (bleeding edge worst)
2-5x instructions/watt
.5-1x instructions/cycle (for legacy ISAs)
.1 marketing brags/product

Thomas Koenig <tkoenig@netcologne.de> wrote:
> Quadibloc <jsavard@ecn.ab.ca> schrieb:
> > On Friday, August 12, 2022 at 9:56:22 PM UTC-6, Quadibloc wrote:
> >> Just read this article on HPC Wire:
> >>
> >> https://www.hpcwire.com/2022/08/11/google-program-to-free-chips-boosts-university-semiconductor-design/
> >
> > September 13 is the deadline for the next batch. It's 130nm,
> > which may be old, but it's good enough - the Pentium Pro was
> > implemented in 350 to 500 nm, although it was called 0.35 um to
> > 0.5 um at the time, so you can make a big core at 130nm.
>
> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
> done in a 130 nm process, 512 kb L2 cache and a transistor count
> of 58 million. It ran at up to 1.7 GHz.

The usual problem with 'XYZ old high-end chip was made on that process' is that
they did, but they used a lot of mm2. The 7447 was, according to that
article 83.9mm2, and the 7457 98.3mm2. Google's programme offers you 10mm2 of
user space (total chip area probably about 13-15mm2).

Which is not nothing, but nobody is going to be building a Pentium 4 in that
space (131/146mm2 in 130nm)

> Sounds like a 66130 would fit very comfortably, at least for size.

What you can fit is one thing, how much cache you can manage to squeeze in,
is another question...

That said, a high end design from a previous process (say 0.3um or bigger) could
be shrunk (or reimplemented) and might fit within the mm2 limit. Just needs
scaling the expectations down a little.

Theo

On Saturday, August 13, 2022 at 3:43:35 PM UTC-5, Ivan Godard wrote:
> On 8/13/2022 12:56 PM, MitchAlsup wrote:
> > On Saturday, August 13, 2022 at 2:27:45 PM UTC-5, Quadibloc wrote:
> >> On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:
> >>
> >>> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
> >>> done in a 130 nm process, 512 kb L2 cache and a transistor count
> >>> of 58 million. It ran at up to 1.7 GHz.
> >> And one sees the press gloating that Russia is helpless, because they can't
> >> make chips beyond 65nm, so sanctions have them at our mercy!
> >>
> >> The Red Chinese designed their H-bomb on a 24-bit computer made out
> >> of discrete transistors.
> >>
> >> And there are even articles saying the Chinese 7nm chips are nothing to
> >> worry about, because they're an early 7nm process that doesn't use EUV..
> >> The Ryzen 9 3900 12-core processor in my daily driver was made that way,
> >> and I've seen no need to update it with something later and greater.
> > <
> > I was designing for 5GHz in 65nm............oh so long ago.
> > <
> > If you are looking for a small in-order machine at reasonable power and
> > frequency, you can make a sub 1mm^2 super-macro in 45nm complete
> > with its cache hierarchy.
> > <
> > It is only the GBOoO machines that require bleeding edge lithography--
> > these are 15×-30× the size of a reasonable in-order core. Of course,
> > Caches are the same size per unit store for both small IO and GBOoO.
<
> Total chip area doesn't matter: it's dominated by pins and pads and
> caches, so for a given core count you get about as many dice per wafer
> regardless of ISA and micro-architecture.
>
> What matters is *yield* - how many *good* dice do you get per wafer. And
> defect rates in different parts of the die have wildly disparate effects
> on the yield. A defect in a pin pad is not even noticed. A defect in a
> cache can be (often) fixed with a hot spare line/entry during fab; no
> impact on profit. A defect in a core can bin to a model with fewer
> cores: not a total loss, but still bottom line impact.
>
> So the dollar impact of yield is dominated by the area of the die
> occupied by logic that cannot use free sparing. Guess what is in that
> area: the core. A chip with 10 GBOOO cores may have twice the mm^2 area
> of one with 10 in-order, but it will have 30x the defect-vulnerable
> area. Until the fab process has matured enough to be effectively
> defect-free, the IO yield per wafer can easily be twice that of the GBOOO..
>
> So, rough numbers IO/GBOOO:
> 1-2x chips/wafer
> 1-4x good/die (bleeding edge worst)
> 2-5x instructions/watt
> .5-1x instructions/cycle (for legacy ISAs)
> .1 marketing brags/product
<
With respect to the offered 130nm processes, you get one (1) GBOoO
core per die at acceptable yield. You might be able to cram on a
SouthBridge (PCIe), since Opteron (SledgeHammer) was 180nm, and
bump the L2 to 2GB.
<
In the same footprint, you can put 12 SIO cores on the die, each
with reasonable L1s, and sharing a similarly sized L2 (and a South-
Bridge too).
<
1 fatal flaw in the GBOoO and its dead,
4 fatal flaws in the SIO and you still have 8 working cores.

On Saturday, August 13, 2022 at 4:13:17 PM UTC-6, MitchAlsup wrote:

> With respect to the offered 130nm processes, you get one (1) GBOoO
> core per die at acceptable yield.

Sounds right, given that the Pentium III was made on 250nm down to 130nm
during its lifetime.

Of course, while it was out-of-order, it may not have been a "Great Big Out-of-Order"
chip. Which is all right: while it seems that the benefits of out-of-order designs
are such that having twelve in-order cores instead of one out-of-order core is
_not_ seen as a desirable trade-off, enlarging things like the reorder window, the
number of physical registers, and so on surely comes with diminishing returns.

John Savard

Theo wrote:
[snip]
> What you can fit is one thing, how much cache you can manage to squeeze in,
> is another question...

I wonder if one could interface with AMD's L3 extending cache
chip? (I suspect such would require too much area even with the
much smaller area per connection for such tight integration.)

(Such would require AMD to offer such for independent sale and
even then the packaging costs would be painful. I think AMD also
has a separate I/O and memory controller chip which could reduce
the chip area needed for a research processor if AMD was willing
to sell such.)

Paul A. Clayton <paaronclayton@gmail.com> wrote:
> Theo wrote:
> [snip]
> > What you can fit is one thing, how much cache you can manage to squeeze in,
> > is another question...
>
> I wonder if one could interface with AMD's L3 extending cache
> chip? (I suspect such would require too much area even with the
> much smaller area per connection for such tight integration.)

I'd guess the 130nm process used here does not support the through silicon
vias that would be needed for the interconnect. I don't know much about how
TSVs are built in the fab, but would guess you'd need some help from the
process.

If you routed the signals off as pads and then did a wire-bond maybe you
could do that, but I suspect AMD's chip is using TSVs.

The other complicated is the voltages may be different - Vcore on 130nm is
probably higher than Vcore on 7nm. It would likely need some voltage
conversion (as pads typically do).

Theo

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> wrote:
> On 8/13/2022 12:27 PM, Quadibloc wrote:
> > On Saturday, August 13, 2022 at 7:19:08 AM UTC-6, Thomas Koenig wrote:
> >
> >> https://en.wikipedia.org/wiki/PowerPC_G4#PowerPC_7447_and_7457 was
> >> done in a 130 nm process, 512 kb L2 cache and a transistor count
> >> of 58 million. It ran at up to 1.7 GHz.
> >
> > And one sees the press gloating that Russia is helpless, because they can't
> > make chips beyond 65nm, so sanctions have them at our mercy!
> >
> > The Red Chinese designed their H-bomb on a 24-bit computer made out
> > of discrete transistors.
>
> The Eniac was used for some US H-bomb calculations. And, of course, the
> calculations for the A-bomb were done by Computers, when the word was a
> job description, not a thing!

Feynmann wrote that US A-bomb was done using IBM accounting
machines. IIUC Russians build their bomb without computers
or equivalent devices (supposedly one of physicists involved
in the program said that any computational problem can be
solved by sufficient number of students).

--
Waldek Hebisch

On Saturday, August 13, 2022 at 4:13:17 PM UTC-6, MitchAlsup wrote:

> With respect to the offered 130nm processes, you get one (1) GBOoO
> core per die at acceptable yield. You might be able to cram on a
> SouthBridge (PCIe), since Opteron (SledgeHammer) was 180nm, and
> bump the L2 to 2GB.

> In the same footprint, you can put 12 SIO cores on the die, each
> with reasonable L1s, and sharing a similarly sized L2 (and a South-
> Bridge too).

> 1 fatal flaw in the GBOoO and its dead,
> 4 fatal flaws in the SIO and you still have 8 working cores.

I think that you may have come up with a valuable idea...

At one point, some time back, you told me that a VLIW design
could never subsitute for out-of-order, because while having a
lot of registers to execute code that looks like register rename
was done by the compiler... is all well and good, but it doesn't
address the most important thing OoO deals with: cache
misses. Because, of course, they're unpredictable, unlike
register hazards.

Why that was valuable was because you _also_ noted, in a
different post, that the scoreboarding of the 6600, even if it
misses some register hazards, _does_ also address what
cache misses do.

Well, maybe I misunderstood you, but it sounded to me that
you just solved the problem that Ivan Godard and others are
trying to solve: get the high speed of OoO without the
humungous overhead in transistors. Just make a VLIW
design with a lot of registers... and make a scoreboarded
implementation of it... and you've got an OoO equivalent
without the transistor overhead!!!

Now, maybe that's a mistake on my part, and there are good
reasons why this wouldn't work.

But in any case, if by in-order cores, you mean just plain old
ordinary in-order cores, like the first Intel Atom, or like the
486, then...

while what I quoted above of your post was indeed
accurate and factual, I still have to admit I disagree
with what I think you were implying and advocating by
it - which you had also explicitly said in other posts long
ago.
The Star 100 failed where the Cray I succeeded. Because
the Cray I could be efficient working with shorter
vectors, since its vector registers gave it shorter set-up
times per vector, and because it was designed to be
fast on scalar code too, so that Amdahl's law wouldn't bite
it quite so hard.
And the Illiac IV failed even harder; so hard that it's
almost forgotten today.
And so, while the failure of Dennard Scaling, and the petering-out of
Moore's Law now means we have few other choices but
parallelism for more performance... it's not as if parallelism
is something _new_.
People have been trying to find ways to efficiently put multiple
processors working in parallel to work on problems in an
efficient manner for a long time.
And there's been no general solution, only the odd
parallel algorithm for the odd individual problem.

Until some Einstein comes along and _changes_ that situation,
the gains in single-thread performance provided by
Great Big Out-of-Order chip designs are worth every transistor
they cost.
Of course, _some_ applications (i.e. database) are embarassingly
parallel or throughput-based. More cores would be a simple
multiplier for them.
And so, with today's technology, instead of putting eight GBOoO
cores on a die, that more value might be obtained by putting,
say, _four_ GBOoO cores, and *sixty-four* in-order cores on that same
die, because it might be a little slower in some applications, but
it would be a lot faster in certain other applications.

So while I'm willing to, as it were, go with you
halfway, I'm in disagreement with disparaging the value of
GBOoO designs, however wretchedly excessive they may seem.

If, by combining the scoreboard with VLIW, one can get the
benefits of OoO without the cost, that's different.
But if we can't... the benefits of GBOoO are worth their
great cost.

John Savard

On Monday, August 15, 2022 at 12:25:37 PM UTC-5, Quadibloc wrote:
> On Saturday, August 13, 2022 at 4:13:17 PM UTC-6, MitchAlsup wrote:
> > With respect to the offered 130nm processes, you get one (1) GBOoO
> > core per die at acceptable yield. You might be able to cram on a
> > SouthBridge (PCIe), since Opteron (SledgeHammer) was 180nm, and
> > bump the L2 to 2GB.
>
> > In the same footprint, you can put 12 SIO cores on the die, each
> > with reasonable L1s, and sharing a similarly sized L2 (and a South-
> > Bridge too).
>
> > 1 fatal flaw in the GBOoO and its dead,
> > 4 fatal flaws in the SIO and you still have 8 working cores.
> I think that you may have come up with a valuable idea...
>
> At one point, some time back, you told me that a VLIW design
> could never subsitute for out-of-order, because while having a
> lot of registers to execute code that looks like register rename
> was done by the compiler... is all well and good, but it doesn't
> address the most important thing OoO deals with: cache
> misses. Because, of course, they're unpredictable, unlike
> register hazards.
<
Many of them are perfectly predictable:: 7-hits, 1-miss.
>
> Why that was valuable was because you _also_ noted, in a
> different post, that the scoreboarding of the 6600, even if it
> misses some register hazards, _does_ also address what
> cache misses do.
<
The ability to absorb latency.
>
> Well, maybe I misunderstood you, but it sounded to me that
> you just solved the problem that Ivan Godard and others are
> trying to solve: get the high speed of OoO without the
> humungous overhead in transistors. Just make a VLIW
> design with a lot of registers... and make a scoreboarded
> implementation of it... and you've got an OoO equivalent
> without the transistor overhead!!!
<
To a large part, this is true. I have a clean RISC instruction set,
I can CoIssue 30% of the instruction stream, and when backed
with reasonable caches, and TLBs these give me a machine
performing close to 1 IPC. It is my hope that VVM doubles
the overall performance--and I am within spitting distance of
a GBOoO machine while only paying for two SMIO machines
or area--so it should be on the order of 6× smaller.
>
> Now, maybe that's a mistake on my part, and there are good
> reasons why this wouldn't work.
>
> But in any case, if by in-order cores, you mean just plain old
> ordinary in-order cores, like the first Intel Atom, or like the
> 486, then...
<
No, as noted above.
>
> while what I quoted above of your post was indeed
> accurate and factual, I still have to admit I disagree
> with what I think you were implying and advocating by
> it - which you had also explicitly said in other posts long
> ago.
<
> The Star 100 failed where the Cray I succeeded. Because
> the Cray I could be efficient working with shorter
> vectors, since its vector registers gave it shorter set-up
> times per vector, and because it was designed to be
> fast on scalar code too, so that Amdahl's law wouldn't bite
> it quite so hard.
<
Agreed Star failed essentially due to Amdahl's law.
<
> And the Illiac IV failed even harder; so hard that it's
> almost forgotten today.
>
Vectors won over arrays, in essence.
>
> And so, while the failure of Dennard Scaling, and the petering-out of
> Moore's Law now means we have few other choices but
> parallelism for more performance... it's not as if parallelism
> is something _new_.
<
> People have been trying to find ways to efficiently put multiple
> processors working in parallel to work on problems in an
> efficient manner for a long time.
> And there's been no general solution, only the odd
> parallel algorithm for the odd individual problem.
<
The HW is there and works. What is failing is SW's ability to use
the kinds of HW that ahs been designed and produced in the past.
>
> Until some Einstein comes along and _changes_ that situation,
> the gains in single-thread performance provided by
> Great Big Out-of-Order chip designs are worth every transistor
> they cost.
<
For your application, yes.
<
> Of course, _some_ applications (i.e. database) are embarassingly
> parallel or throughput-based. More cores would be a simple
> multiplier for them.
> And so, with today's technology, instead of putting eight GBOoO
> cores on a die, that more value might be obtained by putting,
> say, _four_ GBOoO cores, and *sixty-four* in-order cores on that same
> die, because it might be a little slower in some applications, but
> it would be a lot faster in certain other applications.
>
> So while I'm willing to, as it were, go with you
> halfway, I'm in disagreement with disparaging the value of
> GBOoO designs, however wretchedly excessive they may seem.
<
I do not disparage GBOoO, I disparage them not making forward
progress faster--and this is a direct result of them being so <well>
complicated.
>
> If, by combining the scoreboard with VLIW, one can get the
> benefits of OoO without the cost, that's different.
> But if we can't... the benefits of GBOoO are worth their
> great cost.
>
> John Savard

On Monday, August 15, 2022 at 1:21:40 PM UTC-6, MitchAlsup wrote:

> The HW is there and works. What is failing is SW's ability to use
> the kinds of HW that ahs been designed and produced in the past.

In general, I found your reply interesting and informative, and had
little to take issue with. But _this_ statement is the sort of thing that
leads me to feel that I am in disagreement with you somewhere.

A pile of in-order CPUs, like, say, those in an old Xeon Phi, are there,
and work. Can't argue with that!

The fact, though, that computer programs aimed at solving certain
classes of problems can't make effective use of them... does not
necessarily indicate "failure" on the part of the software. It could just
as easily be that the hardware, which is there and working, all right,
is of an unsuitable kind, and those transistors would have been put to
more appropriate use building fewer GBOoO CPUs instead.

In other words, you seem to be making a value judgement... which
implies the assertion of "a fact not in evidence" - specifically, that
if those programmers were only doing their jobs, they would have
written better software.

Algorithms that haven't been discovered yet... will not necessarily,
with absolute inevitability, be discovered some time in the future.
For all we know, they may not exist at all.

Certainly, you're entitled to your opinion that optimism over the
prospects for software to exploit parallel computing facilities is
warranted. It's just that what I've quoted seems to elevate that
opinion to a statement of fact.

This is why I give you a hard time, even though I very much agree that
GBOoO designs do seem like a wretchedly excessive expenditure of
transistors and energy and the heat budget of chips... but if I don't
know of a better way, then I accept what I must.

John Savard

On Monday, August 15, 2022 at 4:15:36 PM UTC-6, Quadibloc wrote:

> Certainly, you're entitled to your opinion that optimism over the
> prospects for software to exploit parallel computing facilities is
> warranted. It's just that what I've quoted seems to elevate that
> opinion to a statement of fact.

And _why_ is it that whenever I run across it, it sticks in my craw
enough that I feel a need to comment on it?

I think I still need to explain myself some more.

Naturally, as an engineer working on chip designs for major
semiconductor manufacturers, your formal education would have
included courses in calculus and differential equations, and of
course you would be able to program computers in BASIC,
FORTRAN, C, and no doubt a few assembler languages as well.

But while you _are_ Mitcn Alsup, to whose great hardware
knowledge I defer, none of those things make you *Donald E.
Knuth*, who at least collected a lot of algorithms in his multi-volume
series _The Art of Computer Programming_.

Whether or not he also did important work on developing _new_
algorithms, I know that there are mathematicians who work in
that field. Every now and again, they will be mentioned in news
reports, say about progress being made in solving the travelling
salesman problem. (That's the one about the most efficient order
in which to visit a given set of cities; not the one faced by farmers'
daughters.)

If I want to know if there's any hope that breakthroughs in using
parallel computers are on the horizon, I'd be asking them.

John Savard

On Monday, August 15, 2022 at 5:15:36 PM UTC-5, Quadibloc wrote:
> On Monday, August 15, 2022 at 1:21:40 PM UTC-6, MitchAlsup wrote:
>
> > The HW is there and works. What is failing is SW's ability to use
> > the kinds of HW that ahs been designed and produced in the past.
<
> In general, I found your reply interesting and informative, and had
> little to take issue with. But _this_ statement is the sort of thing that
> leads me to feel that I am in disagreement with you somewhere.
>
> A pile of in-order CPUs, like, say, those in an old Xeon Phi, are there,
> and work. Can't argue with that!
>
> The fact, though, that computer programs aimed at solving certain
> classes of problems can't make effective use of them... does not
> necessarily indicate "failure" on the part of the software. It could just
> as easily be that the hardware, which is there and working, all right,
> is of an unsuitable kind, and those transistors would have been put to
> more appropriate use building fewer GBOoO CPUs instead.
>
> In other words, you seem to be making a value judgement... which
> implies the assertion of "a fact not in evidence" - specifically, that
> if those programmers were only doing their jobs, they would have
> written better software.
<
Over the past 40 years (has it been that long already) I have watched
the computer architects come up with a small variety of things that
purported to "solve the synchronization problem". Test and Set, Test
and test and Set, Compare and Swap, DCAS, Operation-on-Memory, Load
Locked-Store Conditional, My own ASF and my own ESM as extensions
of LL-SC. And then there is the failure of Transactional Memory,.....
<
While these provide an adequate way to control synchronization, they
are completely unattractive for controlling parallelism--which is the real
goal here.
<
We are at a point where the SW people don't know what to ask HW people
to build into the next CPU. Without being ask (told) and what the mechanics
of such an addition would be composed from, HW designers don't know
what to try next. I tried with ASF and ESM.
>
> Algorithms that haven't been discovered yet... will not necessarily,
> with absolute inevitability, be discovered some time in the future.
> For all we know, they may not exist at all.
<
It seems the old insurance model would be reasonable in spreading
work out over a large number of workers. A "bus boy" walks around
a matrix of desks, each desk manned by a worker. Bus boy drops
of a packet of work to the person at the desk, picks up completed
work, and continues on his rounds. The workers are supplied with
enough work to stay busy, work flows around fast enough that each
task is completed in 'reasonable' time. This model worked for nearly
100 years in insurance, advertising,...
<
But we don't have such a model for computer applications.
>
> Certainly, you're entitled to your opinion that optimism over the
> prospects for software to exploit parallel computing facilities is
> warranted. It's just that what I've quoted seems to elevate that
> opinion to a statement of fact.
<
As I stated her about 18 months ago (more targeting Ivan that the
group as a whole). What we have run out of is ideas as to what would
be a good model to perform work on a parallel possibly heterogenous
computing system(s).
<
This is not a HW problem--it may (MAY) not be a SW problem either.
But the ability to put zillions of cores on a die is already hear. How
we use them or abuse them is our problem not the HW guys.
>
> This is why I give you a hard time, even though I very much agree that
> GBOoO designs do seem like a wretchedly excessive expenditure of
> transistors and energy and the heat budget of chips... but if I don't
> know of a better way, then I accept what I must.
<
In the beginning GBOoO was a way around Amdahl's law--it got a factor
of 2× and then petered out. Now, to make the next transition, we need
a model for controlling parallelism, where user applications can spawn,
communicate, and control other threads (applications-threads,...) without
excursions through GuestOS.
>
> John Savard

On Monday, August 15, 2022 at 7:10:44 PM UTC-4, MitchAlsup wrote:
> On Monday, August 15, 2022 at 5:15:36 PM UTC-5, Quadibloc wrote:

> Over the past 40 years (has it been that long already) I have watched
> the computer architects come up with a small variety of things that
> purported to "solve the synchronization problem". Test and Set, Test
> and test and Set, Compare and Swap, DCAS, Operation-on-Memory, Load
> Locked-Store Conditional, My own ASF and my own ESM as extensions
> of LL-SC. And then there is the failure of Transactional Memory,.....
> <
> While these provide an adequate way to control synchronization, they
> are completely unattractive for controlling parallelism--which is the real
> goal here.
> <
> We are at a point where the SW people don't know what to ask HW people
> to build into the next CPU. Without being ask (told) and what the mechanics
> of such an addition would be composed from, HW designers don't know
> what to try next. I tried with ASF and ESM.
> It seems the old insurance model would be reasonable in spreading
> work out over a large number of workers. A "bus boy" walks around
> a matrix of desks, each desk manned by a worker. Bus boy drops
> of a packet of work to the person at the desk, picks up completed
> work, and continues on his rounds. The workers are supplied with
> enough work to stay busy, work flows around fast enough that each
> task is completed in 'reasonable' time. This model worked for nearly
> 100 years in insurance, advertising,...
> <
> But we don't have such a model for computer applications.

> This is not a HW problem--it may (MAY) not be a SW problem either.
> But the ability to put zillions of cores on a die is already hear. How
> we use them or abuse them is our problem not the HW guys.

Now, given a clean sheet for the OS and APIs, I would ask for a
per CPU queue, where I could push data from one CPU to another.
In fact, it one queue per CPU combination. Then you could
a) send a message from CPU A to CPU B with no synchronization
overhead.
b) it is a "push", so CPU A basically does a "fire and forget".
c) CPU B (somehow) knows there is no need to deal with the
message until the whole thing gets there.
(Yes, I am skipping over the details, but I'm sure they are solvable).

The usual response I get for this is that, "just put the message in memory".
That re-introduces synchronization overhead, memory bandwidth, memory latency, etc.
all over again.

With this feature you could efficiently build a real software pipeline, splitting processing
steps between CPUs and passing results on to the next step in the pipeline.
This does mean your typical multitasking OS with its semi-random CPU
scheduling is NOT appropriate, which is why I specified a clean sheet for the OS and the APIs above.

- Tim

On Monday, August 15, 2022 at 6:10:44 PM UTC-5, MitchAlsup wrote:
> On Monday, August 15, 2022 at 5:15:36 PM UTC-5, Quadibloc wrote:
> > On Monday, August 15, 2022 at 1:21:40 PM UTC-6, MitchAlsup wrote:
> >
> > > The HW is there and works. What is failing is SW's ability to use
> > > the kinds of HW that ahs been designed and produced in the past.
> <
> > In general, I found your reply interesting and informative, and had
> > little to take issue with. But _this_ statement is the sort of thing that
> > leads me to feel that I am in disagreement with you somewhere.
> >
> > A pile of in-order CPUs, like, say, those in an old Xeon Phi, are there,
> > and work. Can't argue with that!
> >
> > The fact, though, that computer programs aimed at solving certain
> > classes of problems can't make effective use of them... does not
> > necessarily indicate "failure" on the part of the software. It could just
> > as easily be that the hardware, which is there and working, all right,
> > is of an unsuitable kind, and those transistors would have been put to
> > more appropriate use building fewer GBOoO CPUs instead.
> >
> > In other words, you seem to be making a value judgement... which
> > implies the assertion of "a fact not in evidence" - specifically, that
> > if those programmers were only doing their jobs, they would have
> > written better software.
> <
> Over the past 40 years (has it been that long already) I have watched
> the computer architects come up with a small variety of things that
> purported to "solve the synchronization problem". Test and Set, Test
> and test and Set, Compare and Swap, DCAS, Operation-on-Memory, Load
> Locked-Store Conditional, My own ASF and my own ESM as extensions
> of LL-SC. And then there is the failure of Transactional Memory,.....
> <
> While these provide an adequate way to control synchronization, they
> are completely unattractive for controlling parallelism--which is the real
> goal here.
> <
> We are at a point where the SW people don't know what to ask HW people
> to build into the next CPU. Without being ask (told) and what the mechanics
> of such an addition would be composed from, HW designers don't know
> what to try next. I tried with ASF and ESM.
> >
> > Algorithms that haven't been discovered yet... will not necessarily,
> > with absolute inevitability, be discovered some time in the future.
> > For all we know, they may not exist at all.
> <
> It seems the old insurance model would be reasonable in spreading
> work out over a large number of workers. A "bus boy" walks around
> a matrix of desks, each desk manned by a worker. Bus boy drops
> of a packet of work to the person at the desk, picks up completed
> work, and continues on his rounds. The workers are supplied with
> enough work to stay busy, work flows around fast enough that each
> task is completed in 'reasonable' time. This model worked for nearly
> 100 years in insurance, advertising,...
> <
> But we don't have such a model for computer applications.
> >
> > Certainly, you're entitled to your opinion that optimism over the
> > prospects for software to exploit parallel computing facilities is
> > warranted. It's just that what I've quoted seems to elevate that
> > opinion to a statement of fact.
> <
> As I stated her about 18 months ago (more targeting Ivan that the
> group as a whole). What we have run out of is ideas as to what would
> be a good model to perform work on a parallel possibly heterogenous
> computing system(s).
> <
> This is not a HW problem--it may (MAY) not be a SW problem either.
> But the ability to put zillions of cores on a die is already hear. How
> we use them or abuse them is our problem not the HW guys.
> >
> > This is why I give you a hard time, even though I very much agree that
> > GBOoO designs do seem like a wretchedly excessive expenditure of
> > transistors and energy and the heat budget of chips... but if I don't
> > know of a better way, then I accept what I must.
> <
> In the beginning GBOoO was a way around Amdahl's law--it got a factor
> of 2× and then petered out. Now, to make the next transition, we need
> a model for controlling parallelism, where user applications can spawn,
> communicate, and control other threads (applications-threads,...) without
> excursions through GuestOS.
> >
> > John Savard

Ugh, my two cents:
> It seems the old insurance model would be reasonable in spreading
> work out over a large number of workers. A "bus boy" walks around
> a matrix of desks, each desk manned by a worker. Bus boy drops
> of a packet of work to the person at the desk, picks up completed
> work, and continues on his rounds. The workers are supplied with
> enough work to stay busy, work flows around fast enough that each
> task is completed in 'reasonable' time. This model worked for nearly
> 100 years in insurance, advertising,...

And why not?
The natural unit of work would be a very small number of DRAM bursts.
It is important to to realize one needs multiple "desks", eg processors
capable of working these "burst" units. E.g. the work (data and program blocks)
can go to any available processor. Chopping up data may not be optimal
for big data. IMHO there are sizeable workload percentages in internet browsing and
data base applications. A gigabit++ network takes care of moving the work
blocks to the nearest available processor or obtaining the next data or program
blocks. Block size assumed to be ~1KB or so?

On Monday, August 15, 2022 at 7:27:29 PM UTC-5, timca...@aol.com wrote:
> On Monday, August 15, 2022 at 7:10:44 PM UTC-4, MitchAlsup wrote:
> > On Monday, August 15, 2022 at 5:15:36 PM UTC-5, Quadibloc wrote:
>
> > Over the past 40 years (has it been that long already) I have watched
> > the computer architects come up with a small variety of things that
> > purported to "solve the synchronization problem". Test and Set, Test
> > and test and Set, Compare and Swap, DCAS, Operation-on-Memory, Load
> > Locked-Store Conditional, My own ASF and my own ESM as extensions
> > of LL-SC. And then there is the failure of Transactional Memory,.....
> > <
> > While these provide an adequate way to control synchronization, they
> > are completely unattractive for controlling parallelism--which is the real
> > goal here.
> > <
> > We are at a point where the SW people don't know what to ask HW people
> > to build into the next CPU. Without being ask (told) and what the mechanics
> > of such an addition would be composed from, HW designers don't know
> > what to try next. I tried with ASF and ESM.
> > It seems the old insurance model would be reasonable in spreading
> > work out over a large number of workers. A "bus boy" walks around
> > a matrix of desks, each desk manned by a worker. Bus boy drops
> > of a packet of work to the person at the desk, picks up completed
> > work, and continues on his rounds. The workers are supplied with
> > enough work to stay busy, work flows around fast enough that each
> > task is completed in 'reasonable' time. This model worked for nearly
> > 100 years in insurance, advertising,...
> > <
> > But we don't have such a model for computer applications.
> > This is not a HW problem--it may (MAY) not be a SW problem either.
> > But the ability to put zillions of cores on a die is already hear. How
> > we use them or abuse them is our problem not the HW guys.
<
> Now, given a clean sheet for the OS and APIs, I would ask for a
> per CPU queue, where I could push data from one CPU to another.
<
Change/CPU/thread/ and you have just defined my Messaging model.
Given a "method" any thread with said method can send a request to
any other thread (even across GuestOSs) and this request will be
queued on that other thread. Should that thread be at a priority where
it can find a core within its affinity set at a lower priority, that thread
can be scheduled--without anyone going through an excursion through
any of the GuestOSs.
<
> In fact, it one queue per CPU combination. Then you could
> a) send a message from CPU A to CPU B with no synchronization
> overhead.
<
Change CUP to thread. You don't want a CPU to tell another CPU
what to do, you want a thread to tell another thread to do something.
Then you have some kind of orderly mechanism whereby that thread
can pick up the work and start working on it. You don't want to be in
a position where 1 instruction of the sending thread runs on CPU[k]
and the next instruction runs on CPU[m]. So, by altering the notion
away from CPU and towards "the unit of scheduled state" you get
what you really want, without dragging the mechanism into the policy.
<
> b) it is a "push", so CPU A basically does a "fire and forget".
<
My 66000 can fire and forget, or fire and wait.
In both cases, the message arrives as if the target of the message
were CALLed from within his own thread--the std ABI. The return
follows this lead and sends back any response as if it came from
a subroutine within the callers thread.
<
> c) CPU B (somehow) knows there is no need to deal with the
> message until the whole thing gets there.
> (Yes, I am skipping over the details, but I'm sure they are solvable).
<
Do all of the above without having to pass through the GuestOS interfaces.
{No excursion though GuestOS is required. No Change in GuestOS is
required even if the GuesOSs are disjoint under different hypervisors.
>
> The usual response I get for this is that, "just put the message in memory".
> That re-introduces synchronization overhead, memory bandwidth, memory latency, etc.
> all over again.
<
In addition to the above, what if you wanted to send such a message, but you did
not want GuestOS to be able to read said message ? {The Paranoid Application}
>
> With this feature you could efficiently build a real software pipeline, splitting processing
> steps between CPUs and passing results on to the next step in the pipeline.
> This does mean your typical multitasking OS with its semi-random CPU
> scheduling is NOT appropriate, which is why I specified a clean sheet for the OS and the APIs above.
<
My 66000 messaging--all I require for you to read about it is a signed NDA.
>
> - Tim

On Monday, August 15, 2022 at 5:10:44 PM UTC-6, MitchAlsup wrote:

> It seems the old insurance model would be reasonable in spreading
> work out over a large number of workers. A "bus boy" walks around
> a matrix of desks, each desk manned by a worker. Bus boy drops
> of a packet of work to the person at the desk, picks up completed
> work, and continues on his rounds. The workers are supplied with
> enough work to stay busy, work flows around fast enough that each
> task is completed in 'reasonable' time. This model worked for nearly
> 100 years in insurance, advertising,...

> But we don't have such a model for computer applications.

I thought that's at least equivalent to the way computers work
now, when a lot of processors in parallel are being utilized to
provide maximum throughput.

Every processor is kept busy, and individual tasks are completed
as fast as their dependencies permit.

There's nothing in the "bus boy" problem that gets around
dependencies - the only thing it seems to be saying is that we
aren't splitting our programs into enough threads. Well,
AMD is making 128 core Threadripper and EPYC chips, and
so there is an incentive to split up programs into at least 128
threads. I agree we need to do that, but I doubt it will "solve
the problem" the way people are hoping.

John Savard

On Monday, August 15, 2022 at 5:10:44 PM UTC-6, MitchAlsup wrote:

> In the beginning GBOoO was a way around Amdahl's law--it got a factor
> of 2× and then petered out.

Absolutely. No disagreement there.

> Now, to make the next transition, we need
> a model for controlling parallelism, where user applications can spawn,
> communicate, and control other threads (applications-threads,...) without
> excursions through GuestOS.

I definitely agree that since we need to make applications more
multi-threaded, we also need to pare down the overhead for
multi-threading.

That many applications haven't been made as capable of using
multiple threads as they could and should be is certainly true.
But fixing that _is_ happening, and while when it is fixed, it's
going to result in significant performance improvements to the
affected applications on many-core systems, I don't see that
as the "next breakthrough".

It's the _current_ best practices state of the art, and people doing
fancy scientific processing on what passes for a supercomputer
these days are doing that. It will trickle down; by the time further
process shrinks change 128 cores from a top-of-the-line
Threadripper to a top-of-the-line Ryzen in consumer boxes, it will
be achieved.

John Savard

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Monday, August 15, 2022 at 12:25:37 PM UTC-5, Quadibloc wrote:

>> People have been trying to find ways to efficiently put multiple
>> processors working in parallel to work on problems in an
>> efficient manner for a long time.
>> And there's been no general solution, only the odd
>> parallel algorithm for the odd individual problem.
><
> The HW is there and works. What is failing is SW's ability to use
> the kinds of HW that ahs been designed and produced in the past.

At the time, I had the feeling that the hardware just increased the
number of cores because they could not increase frequency by much,
and they didn't know what else to do, in the hope that software
would catch up... and to a large degree, it has not.

Programming is hard, as can be seen by the numerous programming
bugs that plague us. And they are expensive to us all. I don't
quite believe the estimates putting the cost of software bugs at
2.E12 dollars per year, that would be more than 2% of the world's
GDP and more than the 6.E11 dollars of annual software revenue
cited by https://www.statista.com/outlook/tmo/software/worldwide ,
but there can be no doubt that the impact is huge.

And parallel programming is _much_ harder than serial programming.
it introduces potential race conditions which make bugs spurious,
and harder to track down.

There is also the question of how to program in parallel. I do
not particular like pthreads or OpenMP, because both share data by
default, which is a recipe for race conditions.

Something like Fortran's PGAS model, where data is only shared
explicitly, but where it is possible to access memory on another
image, and where there are strict rules for synchronization,
is far better (IMHO). It also scales well (also to clusters,
using MPI as underlying mechanism), but it is certainly not
a good fit for everything.

Functional programming may help in making things parallelizable,
but has not caught on to a very large extent, because people
are not quite used to the style, and this has its drawbacks (see
the absence of a direct DO loop in Haskell, you use recursion or
iterators instead).

So... no ready solution, unfortunately.

Timothy McCaffrey wrote:
> On Monday, August 15, 2022 at 7:10:44 PM UTC-4, MitchAlsup wrote:
>> On Monday, August 15, 2022 at 5:15:36 PM UTC-5, Quadibloc wrote:
>
>> Over the past 40 years (has it been that long already) I have watched
>> the computer architects come up with a small variety of things that
>> purported to "solve the synchronization problem". Test and Set, Test
>> and test and Set, Compare and Swap, DCAS, Operation-on-Memory, Load
>> Locked-Store Conditional, My own ASF and my own ESM as extensions
>> of LL-SC. And then there is the failure of Transactional Memory,.....
>> <
>> While these provide an adequate way to control synchronization, they
>> are completely unattractive for controlling parallelism--which is the real
>> goal here.
>> <
>> We are at a point where the SW people don't know what to ask HW people
>> to build into the next CPU. Without being ask (told) and what the mechanics
>> of such an addition would be composed from, HW designers don't know
>> what to try next. I tried with ASF and ESM.
>> It seems the old insurance model would be reasonable in spreading
>> work out over a large number of workers. A "bus boy" walks around
>> a matrix of desks, each desk manned by a worker. Bus boy drops
>> of a packet of work to the person at the desk, picks up completed
>> work, and continues on his rounds. The workers are supplied with
>> enough work to stay busy, work flows around fast enough that each
>> task is completed in 'reasonable' time. This model worked for nearly
>> 100 years in insurance, advertising,...
>> <
>> But we don't have such a model for computer applications.
>
>> This is not a HW problem--it may (MAY) not be a SW problem either.
>> But the ability to put zillions of cores on a die is already hear. How
>> we use them or abuse them is our problem not the HW guys.
>
> Now, given a clean sheet for the OS and APIs, I would ask for a
> per CPU queue, where I could push data from one CPU to another.
> In fact, it one queue per CPU combination. Then you could
> a) send a message from CPU A to CPU B with no synchronization
> overhead.
> b) it is a "push", so CPU A basically does a "fire and forget".
> c) CPU B (somehow) knows there is no need to deal with the
> message until the whole thing gets there.
> (Yes, I am skipping over the details, but I'm sure they are solvable).
>
> The usual response I get for this is that, "just put the message in memory".
> That re-introduces synchronization overhead, memory bandwidth, memory latency, etc.
> all over again.

Transputers, anyone?

>
> With this feature you could efficiently build a real software pipeline, splitting processing
> steps between CPUs and passing results on to the next step in the pipeline.
> This does mean your typical multitasking OS with its semi-random CPU
> scheduling is NOT appropriate, which is why I specified a clean sheet for the OS and the APIs above.

Afair, the Transputer failed to move on to upgraded versions?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"