On Thursday, February 17, 2022 at 6:52:27 AM UTC-7, Ivan Godard wrote:

> I think the problem is really a linguistic one: our programming
> languages have caused us to think about our programs in control flow
> terms, when they are more naturally (and more efficiently) thought of
> in dataflow terms.

That does raise another issue.

It's my opinion that the ENIAC was originally a dataflow computer.

A dataflow machine makes very efficient use of its available ALUs.

That's a good thing. But it doesn't seem to be as general-purpose as
a von Neumann machine; you set it up to process multiple streams
of data in a way somewhat analogous to what punched card machines
did with their data, so it doesn't work well with problems that don't
fit into that paradigm.

I may have already mentioned 'way back that VVM could be considered
a way to sugar-coat the description of a dataflow setup. (And I tend
to prefer making it explicit what you're really telling the computer to do.)

John Savard

On Thursday, February 17, 2022 at 3:24:20 AM UTC-6, BGB wrote:
> On 2/15/2022 3:09 PM, Thomas Koenig wrote:
> > Quadibloc <jsa...@ecn.ab.ca> schrieb:
> >> I noticed that in another thread I might have seemed to have contradicted myself.
> >>
> >> So I will clarify.
> >>
> >> In the near term, in two or three years, I think that it's entirely possible that we
> >> will have dies that combine four "GBOoO" performance cores with sixteen in-
> >> order efficiency cores, and chips that have four of those dies in a package, to
> >> give good performance on both number-crunching and database workloads.
> >
> > Who actually needs number crunching?
> >
> > I certainly do, even in the the company I work in (wich is in
> > the chemical industry, so rather technical) the number of people
> > actually running code which depends on floating point execution
> > speed is rather small, probably in the low single digit percent
> > range of all employees.
> >
> Probably depends. IME, integer operations tend to dominate, but in some
> workloads, floating point tends to make its presence known.
> > That does not mean that floating point is not important :-) but that
> > most users would not notice if they had a CPU with, let's say, a
> > reasonably efficient software emulation of floating point numbers.
> >
> Probably depends on "reasonably efficient".
> Say, ~ 20 cycles, probably only a minority of programs will notice.
> Say, ~ 500 cycles, probably nearly everything will notice.
>
> One big factor is having integer operations which are larger than the
> floating-point values being worked with.
>
>
> As noted, having an FPU which does ADD/SUB/MUL and a few conversion ops
> and similar is for the most part "sufficient" for most practical uses.
>
> Granted, having all the rest could be "better", but is more expensive.
> > Such a CPU would look horrible in SPECfp, and the savings from removing
> > floating point from a general purpose CPU are probably not that great so
> > it is not done, and I think that as an intensive user of floating point,
> > I have to be grateful for that.
> >
> Quick look, at least in my case, the FPU costs less than the L1 D$.
<
Point of order:
<
The actual calculation units of Opteron were, indeed, smaller that the
data storage area of DCache (or ICache).
<
But the FP register file of Opteron was larger than FMUL and FADD
combined as DECODE did some renaming to smooth the flow of
calculations.
<
Also note the LD and ST buffers of Opteron were larger than the
storage (64KB 2-banked) of Opteron.
<
So it depends if you mean "L1 D$" as L1-tag, L1-TLB, L1 data,
or whether you mean "L1 D$" as L1-tag, L1-TLB, L1 data, LD buffer, ST buffer.
And we still do not have the area of the miss buffer.
>
> For a 1-wide core, it may be tempting to omit the FPU and MMU for cost
> reasons. For a bigger core, omitting them may not be worthwhile if
> anything actually uses them.
<
Not any more. When you can fit 16 GBOoOs in a chip and a LBIO is 1/12
and that LBIO already has a FMAC fully pipelined, there is no reason to
leave out this kind of functionality unless your market is "lowest
possible power" and still, this decision cause you grief wrt IMUL and IDIV.
<
When you could put ~200 LBIOs in a die, making the LBIO 30% smaller
is like taking air conditioning and power widows out of your car.
>
>
> Or, at least within the limits of an FPU which is cost-cut enough to
> where the LSB being correctly rounded is a bit hit or miss.
> > Hmm, come to think of it, that is the fist positive thing about
> > SPEC that occurred to me in quite a few years...
> ...

On Thursday, February 17, 2022 at 7:43:22 AM UTC-6, Ivan Godard wrote:
> On 2/16/2022 10:25 AM, MitchAlsup wrote:

> > PCIe 6.0 uses 16 GHz clock to send 4 bits per wire per cycle using
> > double data rate and PAM4 modulation; and achieves 64GTs per wire
> > each direction. So 4 pins: true-comp out, true comp in: provide 8GB/s
> > out and 8GB/s in.
> > <
> > Now, remember from yesterday out 120 GB/s per core. You will need
> > 10 of these 4 pin wires to support inbound bandwidth and 5 to support
> > the outbound bandwidth.
> > <
> > But hey, if you want to provide 512 pins, I sure you can find some use
> > for this kind of bandwidth. {but try dealing with the heat.}
<
> If you don't really need the bandwidth, but have the pincount in the
> socket, can't you get less heat by say driving the pins eight at a time
> at an eighth the clock? (please forgive my HW ignorance)
<
Sure, heat is essentially proportional to GTs×pins

On Thursday, February 17, 2022 at 8:19:23 AM UTC-6, Stefan Monnier wrote:
> > Cray instructions in it will take input from memory and put output to
> > memory, while putting intermediate results in vector registers... and
> > a VVM loop would do exactly the same thing, except using ordinary
> > registers.
<
Point of misunderstanding:
<
VVM uses register specifiers to determine data flow within a loop.
It does not actually use GPRs while performing the calculations !
GPRs get written only when there is an interrupt/exception, or at
loop termination based on the "outs" clause of the VEC instruction.
<
This is the relaxation that enables SIMD VVM--not overburdening
the registers themselves.
<
> There is the important difference that Cray vector registers are
> significantly bigger than My 66000 registers, so really if you look at
> the ISA itself, VVM matches the behavior of those CPUs that had
> in-memory vectors rather than vector registers.
<
This needs a lot of justification before I'll buy it.
>
> IIRC none of those vector processors used caches, so maybe VVM can be
> compared to Cray-style in-register vectors except the vector registers
> are stored in the L1 cache (tho it might not give quite as much read
> bandwidth, admittedly).
>
>
> Stefan

On 2/17/2022 9:17 AM, MitchAlsup wrote:
> On Thursday, February 17, 2022 at 3:24:20 AM UTC-6, BGB wrote:
>> On 2/15/2022 3:09 PM, Thomas Koenig wrote:
>>> Quadibloc <jsa...@ecn.ab.ca> schrieb:
>>>> I noticed that in another thread I might have seemed to have contradicted myself.
>>>>
>>>> So I will clarify.
>>>>
>>>> In the near term, in two or three years, I think that it's entirely possible that we
>>>> will have dies that combine four "GBOoO" performance cores with sixteen in-
>>>> order efficiency cores, and chips that have four of those dies in a package, to
>>>> give good performance on both number-crunching and database workloads.
>>>
>>> Who actually needs number crunching?
>>>
>>> I certainly do, even in the the company I work in (wich is in
>>> the chemical industry, so rather technical) the number of people
>>> actually running code which depends on floating point execution
>>> speed is rather small, probably in the low single digit percent
>>> range of all employees.
>>>
>> Probably depends. IME, integer operations tend to dominate, but in some
>> workloads, floating point tends to make its presence known.
>>> That does not mean that floating point is not important :-) but that
>>> most users would not notice if they had a CPU with, let's say, a
>>> reasonably efficient software emulation of floating point numbers.
>>>
>> Probably depends on "reasonably efficient".
>> Say, ~ 20 cycles, probably only a minority of programs will notice.
>> Say, ~ 500 cycles, probably nearly everything will notice.
>>
>> One big factor is having integer operations which are larger than the
>> floating-point values being worked with.
>>
>>
>> As noted, having an FPU which does ADD/SUB/MUL and a few conversion ops
>> and similar is for the most part "sufficient" for most practical uses.
>>
>> Granted, having all the rest could be "better", but is more expensive.
>>> Such a CPU would look horrible in SPECfp, and the savings from removing
>>> floating point from a general purpose CPU are probably not that great so
>>> it is not done, and I think that as an intensive user of floating point,
>>> I have to be grateful for that.
>>>
>> Quick look, at least in my case, the FPU costs less than the L1 D$.
> <
> Point of order:
> <
> The actual calculation units of Opteron were, indeed, smaller that the
> data storage area of DCache (or ICache).
> <
> But the FP register file of Opteron was larger than FMUL and FADD
> combined as DECODE did some renaming to smooth the flow of
> calculations.
> <
> Also note the LD and ST buffers of Opteron were larger than the
> storage (64KB 2-banked) of Opteron.
> <
> So it depends if you mean "L1 D$" as L1-tag, L1-TLB, L1 data,
> or whether you mean "L1 D$" as L1-tag, L1-TLB, L1 data, LD buffer, ST buffer.
> And we still do not have the area of the miss buffer.

Yeah, I don't have FPU registers.

I dropped them initially (along with the original FMOV instructions)
because they added a fair bit of cost.

I have more recently re-added an "FMOV.S" instruction, which in this
form is implemented as a normal Load/Store with Single<->Double
converters glued on. A case could be made for only doing it load-side,
as this is both cheaper and more common.

Otherwise, loading/storing Single values requires using explicit
conversion ops (with the FADD/FMUL units only operating natively on
Double). Well, it is either that, or add (proper) single-precision ops.

I was counting the D$ and TLB separately, because in my case they are
represented as different modules, and the Vivado netlist view also keeps
them separate.

Some modules just disappear though, like the instruction decoder tends
to get absorbed into other things.

Between FADD and FMUL, FADD is larger.
Just the TLB (by itself) is larger than the FADD (in terms of LUTs).

Or, a few costs (from the netlist, argressively rounded):
FPU : ~ 5k (FADD/FMUL: ~ 2k each, ~ 1k other, *1).
TLB : ~ 3k (4-way, 256 x 4 x 128b)
L1 D$ : ~ 7k
L1 I$ : ~ 2k
ALU : ~ 4k (~ 1.3k per lane)
GPR RF: ~ 2k (64x64b, 6R3W)
...
L2 : ~ 10k (256K, 2-way, 64B cache lines, *2)
DDR : ~ 7k (64B cache lines)
VRAM : ~ 7k (RAM backed, structurally similar to an L1 cache)

*1: This is for Double precision with SIMD.
SIMD seems to have minimal effect on cost.
Enabling GFPX (widens FADD and FMUL to 96 bit, S.E15.M80) does add a
more significant cost increase.

The SIMD operations are internally implemented using pipelining:
Extract element from vector, convert to internal format;
Feed through FADD or FMUL;
Convert back to vector format, store into output vector;
Final result is the output vector.

*2: The size of the L2 cache and DDR PHY vary significantly based on the
use of 16B or 64B cache lines (along the L2<->DRAM interface). The
bigger lines are better for performance as they allow for (more
efficient) burst transfers.

The ringbus continues to operate with 16B logical transfers in either
case (so the L2 is presented to the ringbus in terms of 16B lines).

The RAM in this case is a DDR2 module with a 16-bit bus interface being
run at 50MHz (with 'DLL disabled'). Some other boards use RAM with LPDDR
or QSPI interfaces, but my Verilog code doesn't currently support these (*).

This build was with UTLB disabled, where UTLB adds a smaller 1-way TLB
to the L1 D$. This can (on a hit) allow requests to skip over the main
TLB and go (more directly) onto the L2 ring.

This was not done for the I$, as the I$ miss rate tends to be low enough
(relative to the D$) on average that it isn't likely to be gain much.

*: One of the boards I bought, I thought was going to have QSPI RAM, but
the model I got turned out to not have any RAM (the RAM came with the
XC7A35T variant but not the XC7S25 variant).

>>
>> For a 1-wide core, it may be tempting to omit the FPU and MMU for cost
>> reasons. For a bigger core, omitting them may not be worthwhile if
>> anything actually uses them.
> <
> Not any more. When you can fit 16 GBOoOs in a chip and a LBIO is 1/12
> and that LBIO already has a FMAC fully pipelined, there is no reason to
> leave out this kind of functionality unless your market is "lowest
> possible power" and still, this decision cause you grief wrt IMUL and IDIV.
> <
> When you could put ~200 LBIOs in a die, making the LBIO 30% smaller
> is like taking air conditioning and power widows out of your car.

Yeah.

I have little idea how many BJX2 cores could fit into an ASIC
implementation. In any case, probably a whole lot more than on an Artix-7.

It can make sense mostly for the lower-end Spartan chips (eg: XC7S25),
and is probably necessary for ICE40 based designs.

>>
>>
>> Or, at least within the limits of an FPU which is cost-cut enough to
>> where the LSB being correctly rounded is a bit hit or miss.
>>> Hmm, come to think of it, that is the fist positive thing about
>>> SPEC that occurred to me in quite a few years...
>> ...

On 2/17/2022 11:20 AM, BGB wrote:

big snip

> Or, a few costs (from the netlist, argressively rounded):
>   FPU   : ~ 5k (FADD/FMUL: ~ 2k each, ~ 1k other, *1).
>   TLB   : ~ 3k (4-way, 256 x 4 x 128b)
>   L1 D$ : ~ 7k
>   L1 I$ : ~ 2k

Why does the L1 D$ cache take 3.5 X as much as the L1 I$

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld wrote:
> On 2/17/2022 11:20 AM, BGB wrote:
>
> big snip
>
>> Or, a few costs (from the netlist, argressively rounded):
>>  Â  FPU   : ~ 5k (FADD/FMUL: ~ 2k each, ~ 1k other, *1).
>>  Â  TLB   : ~ 3k (4-way, 256 x 4 x 128b)
>>  Â  L1 D$ : ~ 7k
>>  Â  L1 I$ : ~ 2k
>
> Why does the L1 D$ cache take 3.5 X as much as the L1 I$

I$ is effectively read-only, vs D$ which has to handle writes all the time?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On 2/17/2022 2:38 PM, Stephen Fuld wrote:
> On 2/17/2022 11:20 AM, BGB wrote:
>
> big snip
>
>> Or, a few costs (from the netlist, argressively rounded):
>>    FPU   : ~ 5k (FADD/FMUL: ~ 2k each, ~ 1k other, *1).
>>    TLB   : ~ 3k (4-way, 256 x 4 x 128b)
>>    L1 D$ : ~ 7k
>>    L1 I$ : ~ 2k
>
> Why does the L1 D$ cache take 3.5 X as much as the L1 I$
>

Probably because it is:
Byte Addressable vs Word Addressable;
Supports a wider fetch (128-bit vs 96-bit);
Supports both Load and Store operations (vs just Fetch);
Also deals with MMIO operations (different behavior on the bus);
...

There is a lot of extra logic needed to deal with memory stores, dirty
cache lines, storing stuff back to RAM, ... which is effectively
irrelevant to the I$.

The I$ has logic to deal with things like instruction length and
instruction bundles, but this seems to be comparably much smaller than
the logic needed by the D$ to deal with things like byte-aligned Load
and Store.

....

Not shown in this stat is Block-RAM cost, where:
L1 D$ is 32K, organized as 1024 x 2 x 16B.
Each 16B line has: 128-bits of data and 108* bits of metadata (tag).
L1 I$ is 16K, organized as 512 x 2 x 16B.
Each 16B line has: 128-bits of data and 72 bits of metadata (tag).

Cache lines are organized as Even/Odd pairs (or A/B), mostly to deal
with Load/Store crossing a 16B boundary. An "Aligned Only" cache could
skip this step (I$ would still need it though to be able to support
variable-length / variable-alignment bundles).

*: In a newer/experimental version of the L1 D$ which adds epoch
flushing and some other tweaks (the prior version was 88 bits). However,
this version still has some bugs and I haven't released the code yet.

The D$ would actually be a lot more expensive if I did not impose an 8B
alignment restriction for 128-bit Load/Store. This would effectively
double the size of the Extract/Insert logic; as the 128b Load/Store
works by bypassing the logic normally used for Extract/Insert of values
64-bits or less.

The Extract/Insert logic fetches 16B from a 8B aligned position, and
then selects a byte-aligned 8B from this, and then zero or sign-extends
it to the requested length (for Load).

For Store, it combines this value with the value being stored (based on
the store width), inserts it (byte aligned) back into the 16B block, and
then builds a set of Store cache-lines which re-insert this block.

For 128-bit store, the stored value replaces the 16B block, which is
then inserted into the store cache lines.

If the request is Store type (and not MMIO), then the newly-updated
cache-lines may be stored back to the cache-line arrays (with the Dirty
Flag) being Set. On a Cache Miss, it will then be sent out on the bus
before requesting the replacement cache line.

....

On 2/17/2022 3:37 PM, Terje Mathisen wrote:
> Stephen Fuld wrote:
>> On 2/17/2022 11:20 AM, BGB wrote:
>>
>> big snip
>>
>>> Or, a few costs (from the netlist, argressively rounded):
>>>  Â  FPU   : ~ 5k (FADD/FMUL: ~ 2k each, ~ 1k other, *1).
>>>  Â  TLB   : ~ 3k (4-way, 256 x 4 x 128b)
>>>  Â  L1 D$ : ~ 7k
>>>  Â  L1 I$ : ~ 2k
>>
>> Why does the L1 D$ cache take 3.5 X as much as the L1 I$
>
> I$ is effectively read-only, vs D$ which has to handle writes all the time?
>

This is probably most of it.

There is a whole lot of logic which just goes poof and disappears when
the cache is read-only.

Also, in general, the logic in the I$ is simpler than in the D$.

Main unique thing that the I$ needs to do is look at instruction bits
and figure out how long the bundle is.

So, say (15:12):
0111: w32b=1, Wx=(11) //XGPR
1001: w32b=1, Wx=(11) //XGPR
1110: w32b=1, Wx=((11:9)==101) //PrWEX
1111: w32b=1, Wx=(10)
Else: w32b=0, Wx=0

Then look at this across several instruction words,
w32b=0,Wx=0: 16-bit
w32b=1,Wx=0: 32-bit
(w32b=1,Wx=1), (w32b=0,Wx=0): 48-bit (unused)
(w32b=1,Wx=1), (w32b=1,Wx=0): 64-bit
(w32b=1,Wx=1), (w32b=1,Wx=1), (w32b=1,Wx=0): 96-bit

The actual logic for this part, while not particularly concise, doesn't
have a particularly high LUT cost.

The I$ doesn't care all that much what the instructions "actually do"
(this part is for the Decode and Execute stages to figure out).
Similarly, the decode stage has to sort out all the stuff with Jumbo
Prefixes (the I$ mostly ignores these, treating them like a special case
of the normal bundle encoding).

....

Some of my other ISA designs (imagined) would have gone over to,
effectively (15,13):
0zz: 16b
10z: 16b
110: 32b (Wx=0)
111: 32b (Wx=1)

For something like WEX3W, this would allow determining bundle length by
looking at 6-bits:
{ (15:13), (47:45) }
{ (31:29), (63:61) }
...

Though, I have gone back and forth between using the high bits of each
16-bit word (as what BJX2 currently does), or the low order bits of each
word (like RISC-V).

The RISC-V ordering makes more sense if one assumes a (consistent)
little-ending ordering, but the high-bits scheme makes more sense for
laying out the encodings in hexadecimal (even if the resultant encoding
is mixed-endian).

The bit layout doesn't make any real difference to an FPGA though.

Partial limiting factors (that have partly kept BJX2 alive), is the
issue that I can't really fit 6 bit registers, predication, and WEX,
into a 32-bit instruction word, which still having a good amount of
encoding space.

Say:
zzzz-zzzz tttt-ttss ssss-nnnn nnqq-pw11

pw:
00: 32-bit, Unconditional
01: 32-bit, WEX, Unconditional
10: 32-bit, Predicated
11: 32-bit, WEX, Predicated
qq (p=0):
00: Block 0
01: Block 1
10: Block 2
11: Block 3
qq (p=1):
00: Block 0 ?T
01: Block 0 ?F
10: Block 0 ?ST
11: Block 0 ?SF

The opcode space would work out considerably smaller than what I
currently have (to compensate, would need to move over to using 40 or 48
bit instructions, which is also undesirable).

Another option being to gain one or two bits by dropping the possibility
of 16-bit instructions.
zzzz-zzzz tttt-ttss ssss-nnnn nnzz-qqpw
This could be at least roughly break-even in terms of opcode space.

The tradeoff being, that with the current ISA one has to get a little
mix/match:
Predicates + WEX, R0..R31 only.
WEX + R0..R63, no predicates.
Predicates + R0..R63, no WEX

But, did hack it more recently:
Predicates + WEX + R0..R63; 2-wide bundle (2 ops in 96 bits).
(The encoding here is a bit "questionable", but oh well).

Kinda ugly, basically works though...

A lot of this hair magically disappears if one assumes a subset which
only allows R0..R31 (all these extra special-case encodings become
"invalid" and the implementation can ignore their existence).

....

Ivan Godard <ivan@millcomputing.com> writes:

[...]

> I think the problem is really a linguistic one: our programming
> languages have caused us to think about our programs in control
> flow terms, [...]

Some of us. Not everyone, fortunately.

According to Ivan Godard <ivan@millcomputing.com>:
>I think the problem is really a linguistic one: our programming
>languages have caused us to think about our programs in control flow
>terms, when they are more naturally (and more efficiently) thought of
>in dataflow terms.

I blame von Neumann. Look at the First Draft of a Report on the EDVAC
and it's brutally serial, one step after another, one word at a time
loaded or stored from the iconoscope.

Compare it to the lovely ENIAC where you could plug anything into
anything else, cables permitting, and the data flowed as soon as it
was ready. Yeah, it was a little harder to program and only five
people in the world could do it, but you can't have everything.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

John Levine <johnl@taugh.com> writes:
>According to Ivan Godard <ivan@millcomputing.com>:
>>I think the problem is really a linguistic one: our programming
>>languages have caused us to think about our programs in control flow
>>terms, when they are more naturally (and more efficiently) thought of
>>in dataflow terms.
>
>I blame von Neumann. Look at the First Draft of a Report on the EDVAC
>and it's brutally serial, one step after another, one word at a time
>loaded or stored from the iconoscope.
>
>Compare it to the lovely ENIAC where you could plug anything into
>anything else, cables permitting, and the data flowed as soon as it
>was ready. Yeah, it was a little harder to program and only five
>people in the world could do it, but you can't have everything.

You can, and you could a quarter century ago: You specify the code in
a sequential way (so more than five people in the world can program
it), and the magic of OoO execution means that data flows as soon as
it is ready.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Tim Rentsch wrote:
> Ivan Godard <ivan@millcomputing.com> writes:
>
> [...]
>
>> I think the problem is really a linguistic one: our programming
>> languages have caused us to think about our programs in control
>> flow terms, [...]
>
> Some of us. Not everyone, fortunately.
>
I tend to think much more in data flow terms, as I've written before:

An optimal program is like a creek finding its path down from the
mountain, always searching for the path of least resistance.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On 17/02/22 13:52, Ivan Godard wrote:
> I think the problem is really a linguistic one: our programming languages have
> caused us to think about our programs in control flow terms, when they are  more
> naturally (and more efficiently) thought of in dataflow terms.

Not all languages, of course.

The commercially and technically important class of hardware
description languages (e.g. Verilog, VHDL) have a very large
component of data flowing (streaming) through "wires" connecting
"processes/FSMs" and event based computation. To anyone used
to "thinking in hardware" that, and its associated parallelism
is easy and natural.

Softies find a marked "impedance mismatch" when they encounter
HDLs, and try to "write Fortran" in the HDL. Hence there are
now moves to have more procedural concepts and constructs,
e.g. System C. The impedance mismatch is lower, but it is still
easy to create something that is poorly "synthesisable", i.e.
can't be translated into hardware/dataflow.

On 2022-02-15, Thomas Koenig wrote:
> Quadibloc <jsavard@ecn.ab.ca> schrieb:
>> I noticed that in another thread I might have seemed to have contradicted myself.
>>
>> So I will clarify.
>>
>> In the near term, in two or three years, I think that it's entirely possible that we
>> will have dies that combine four "GBOoO" performance cores with sixteen in-
>> order efficiency cores, and chips that have four of those dies in a package, to
>> give good performance on both number-crunching and database workloads.
>
> Who actually needs number crunching?
>
> I certainly do, even in the the company I work in (wich is in
> the chemical industry, so rather technical) the number of people
> actually running code which depends on floating point execution
> speed is rather small, probably in the low single digit percent
> range of all employees.
>
> That does not mean that floating point is not important :-) but that
> most users would not notice if they had a CPU with, let's say, a
> reasonably efficient software emulation of floating point numbers.

The problem is that AFAIK there exists no reasonably efficient software
emulation of IEEE-754 floating-point (where "reasonable" would be
something like 10x slower than using an FPU).

I recently spent some time optimizing function argument sanity checks in
an API layer of an embedded software. For instance some arguments were
orthonormal matrices, and the API made a crude check for that. Normally
this is not a problem, but in this case the API layer ran on an ARM M4
without an FPU, and so a single argument check could take more than
0.1 ms, which is a considerable time period in a real time system where
all work must finish in less than 16 ms.

It would be interesting to see an FP standard that is optimized for
software implementation, and/or CPU:s with "software FP aid"
instructions (fast handling of NaN:s and special cases,
(de)normalization instructions - like CLZ on steroids, etc), so that
common FP operations can be implemented with 5-20 instructions, for
instance.

/Marcus

On Monday, March 7, 2022 at 5:43:35 AM UTC-6, Tom Gardner wrote:
> On 17/02/22 13:52, Ivan Godard wrote:
> > I think the problem is really a linguistic one: our programming languages have
> > caused us to think about our programs in control flow terms, when they are more
> > naturally (and more efficiently) thought of in dataflow terms.
> Not all languages, of course.
>
> The commercially and technically important class of hardware
> description languages (e.g. Verilog, VHDL) have a very large
> component of data flowing (streaming) through "wires" connecting
> "processes/FSMs" and event based computation. To anyone used
> to "thinking in hardware" that, and its associated parallelism
> is easy and natural.
>
> Softies find a marked "impedance mismatch" when they encounter
> HDLs, and try to "write Fortran" in the HDL. Hence there are
> now moves to have more procedural concepts and constructs,
> e.g. System C. The impedance mismatch is lower, but it is still
> easy to create something that is poorly "synthesisable", i.e.
> can't be translated into hardware/dataflow.

I call HDL writers "time lords" as they can operate across both time and space.

On 3/7/2022 5:52 AM, JimBrakefield wrote:
> On Monday, March 7, 2022 at 5:43:35 AM UTC-6, Tom Gardner wrote:
>> On 17/02/22 13:52, Ivan Godard wrote:
>>> I think the problem is really a linguistic one: our programming languages have
>>> caused us to think about our programs in control flow terms, when they are more
>>> naturally (and more efficiently) thought of in dataflow terms.
>> Not all languages, of course.
>>
>> The commercially and technically important class of hardware
>> description languages (e.g. Verilog, VHDL) have a very large
>> component of data flowing (streaming) through "wires" connecting
>> "processes/FSMs" and event based computation. To anyone used
>> to "thinking in hardware" that, and its associated parallelism
>> is easy and natural.
>>
>> Softies find a marked "impedance mismatch" when they encounter
>> HDLs, and try to "write Fortran" in the HDL. Hence there are
>> now moves to have more procedural concepts and constructs,
>> e.g. System C. The impedance mismatch is lower, but it is still
>> easy to create something that is poorly "synthesisable", i.e.
>> can't be translated into hardware/dataflow.
>
> I call HDL writers "time lords" as they can operate across both time and space.

+1!

Marcus wrote:
> On 2022-02-15, Thomas Koenig wrote:
>> Quadibloc <jsavard@ecn.ab.ca> schrieb:
>>> I noticed that in another thread I might have seemed to have
>>> contradicted myself.
>>>
>>> So I will clarify.
>>>
>>> In the near term, in two or three years, I think that it's entirely
>>> possible that we
>>> will have dies that combine four "GBOoO" performance cores with
>>> sixteen in-
>>> order efficiency cores, and chips that have four of those dies in a
>>> package, to
>>> give good performance on both number-crunching and database workloads.
>>
>> Who actually needs number crunching?
>>
>> I certainly do, even in the the company I work in (wich is in
>> the chemical industry, so rather technical) the number of people
>> actually running code which depends on floating point execution
>> speed is rather small, probably in the low single digit percent
>> range of all employees.
>>
>> That does not mean that floating point is not important :-) but that
>> most users would not notice if they had a CPU with, let's say, a
>> reasonably efficient software emulation of floating point numbers.
>
> The problem is that AFAIK there exists no reasonably efficient software
> emulation of IEEE-754 floating-point (where "reasonable" would be
> something like 10x slower than using an FPU).

40-50 clock cycles for FADD/FMAC is just about doable.
>
> I recently spent some time optimizing function argument sanity checks in
> an API layer of an embedded software. For instance some arguments were
> orthonormal matrices, and the API made a crude check for that. Normally
> this is not a problem, but in this case the API layer ran on an ARM M4
> without an FPU, and so a single argument check could take more than
> 0.1 ms, which is a considerable time period in a real time system where
> all work must finish in less than 16 ms.

Could you do the checking in integer domain, or would that be meaningless?
>
> It would be interesting to see an FP standard that is optimized for
> software implementation, and/or CPU:s with "software FP aid"
> instructions (fast handling of NaN:s and special cases,
> (de)normalization instructions - like CLZ on steroids, etc), so that
> common FP operations can be implemented with 5-20 instructions, for
> instance.

We did quite a bit of work on sw/hw fp codesign for an absolute minimum
Mill, you are correct that it helps to have a small number of hw
helpers. In particular you need a saturating shift right in order to
implement the sticky bit for proper rounding.

It also helps to have a fast classifier for one or two inputs,
optionally sorting a pair of inputs by magnitude (needed for FADD/FSUB).
You do all the special cases in parallel with the normal calculations,
then select either that normal result or the special value at the very end.

Doing it this way you can get into a 5x slower speed ballpark.

Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Monday, March 7, 2022 at 5:43:35 AM UTC-6, Tom Gardner wrote:
> On 17/02/22 13:52, Ivan Godard wrote:
> > I think the problem is really a linguistic one: our programming languages have
> > caused us to think about our programs in control flow terms, when they are more
> > naturally (and more efficiently) thought of in dataflow terms.
> Not all languages, of course.
>
> The commercially and technically important class of hardware
> description languages (e.g. Verilog, VHDL) have a very large
> component of data flowing (streaming) through "wires" connecting
> "processes/FSMs" and event based computation. To anyone used
> to "thinking in hardware" that, and its associated parallelism
> is easy and natural.
>
> Softies find a marked "impedance mismatch" when they encounter
> HDLs, and try to "write Fortran" in the HDL.
<
Many Verilog "subroutines" can have their "assignment statements"
placed backwards and the subroutine still does the same job !
You HAVE to give up the vonNeumann paradigm that one thing
happens then another and another--they all happen based on
dependencies (some of which are not "in scope".)
<
> Hence there are
> now moves to have more procedural concepts and constructs,
> e.g. System C. The impedance mismatch is lower, but it is still
> easy to create something that is poorly "synthesisable", i.e.
> can't be translated into hardware/dataflow.

On Monday, March 7, 2022 at 6:48:44 AM UTC-6, Marcus wrote:
> On 2022-02-15, Thomas Koenig wrote:
> > Quadibloc <jsa...@ecn.ab.ca> schrieb:
> >> I noticed that in another thread I might have seemed to have contradicted myself.
> >>
> >> So I will clarify.
> >>
> >> In the near term, in two or three years, I think that it's entirely possible that we
> >> will have dies that combine four "GBOoO" performance cores with sixteen in-
> >> order efficiency cores, and chips that have four of those dies in a package, to
> >> give good performance on both number-crunching and database workloads.
> >
> > Who actually needs number crunching?
> >
> > I certainly do, even in the the company I work in (wich is in
> > the chemical industry, so rather technical) the number of people
> > actually running code which depends on floating point execution
> > speed is rather small, probably in the low single digit percent
> > range of all employees.
> >
> > That does not mean that floating point is not important :-) but that
> > most users would not notice if they had a CPU with, let's say, a
> > reasonably efficient software emulation of floating point numbers.
> The problem is that AFAIK there exists no reasonably efficient software
> emulation of IEEE-754 floating-point (where "reasonable" would be
> something like 10x slower than using an FPU).
>
> I recently spent some time optimizing function argument sanity checks in
> an API layer of an embedded software. For instance some arguments were
> orthonormal matrices, and the API made a crude check for that. Normally
> this is not a problem, but in this case the API layer ran on an ARM M4
> without an FPU, and so a single argument check could take more than
> 0.1 ms, which is a considerable time period in a real time system where
> all work must finish in less than 16 ms.
>
> It would be interesting to see an FP standard that is optimized for
> software implementation, and/or CPU:s with "software FP aid"
> instructions (fast handling of NaN:s and special cases,
> (de)normalization instructions - like CLZ on steroids, etc), so that
> common FP operations can be implemented with 5-20 instructions, for
> instance.
<
All you need is a native integer 2× as wide as the FP you simulate,
Find First 1 over 2×, and 2× Shifts, and some rounding instruction
also over 2×, and extract/insert exponents and fractions.
<
The problem is there is no 2× designed to emulate FP they are all
designed to emulate 2× integers.
>
> /Marcus

Terje Mathisen <terje.mathisen@tmsw.no> writes:

> Tim Rentsch wrote:
>
>> Ivan Godard <ivan@millcomputing.com> writes:
>>
>> [...]
>>
>>> I think the problem is really a linguistic one: our programming
>>> languages have caused us to think about our programs in control
>>> flow terms, [...]
>>
>> Some of us. Not everyone, fortunately.
>
> I tend to think much more in data flow terms, as I've written before:
>
> An optimal program is like a creek finding its path down from the
> mountain, always searching for the path of least resistance.

It's an interesting simile. Note by the way that creeks are
greedy algorithms, in the sense of being only locally optimal,
not necessarily globally optimal.

To me the more interesting question is how does this perspective
affect how the code looks? If reading your programs, would I see
something that looks pretty much like other imperative code, or
would there be some distinguishing characteristics that would
indicate your different thought mode? Can you say anything about
what those characteristics might be? Or perhaps give an example
or two? (Short is better if that is feasible.)

On Sunday, March 6, 2022 at 10:50:47 AM UTC-7, John Levine wrote:
> According to Ivan Godard <iv...@millcomputing.com>:
> >I think the problem is really a linguistic one: our programming
> >languages have caused us to think about our programs in control flow
> >terms, when they are more naturally (and more efficiently) thought of
> >in dataflow terms.

> I blame von Neumann. Look at the First Draft of a Report on the EDVAC
> and it's brutally serial, one step after another, one word at a time
> loaded or stored from the iconoscope.

> Compare it to the lovely ENIAC where you could plug anything into
> anything else, cables permitting, and the data flowed as soon as it
> was ready. Yeah, it was a little harder to program and only five
> people in the world could do it, but you can't have everything.

After the ENIAC, though, eventually more people learned how to
program in dataflow terms. Because that's how analog computers
were programmed.

And what about punched-card accounting machines?

Originally, the victory of the von Neumann machine came about
for a very simple reason: such a machine required far less hardware.
One ALU for a program of any length or complexity, instead of as many
arithmetic stages as there were steps in the problem.

But the other problem is that a lot of uses were found for computers.
Dataflow seems to be good for one thing: turning a stack of input numbers
into a stack of output numbers. Now that programs are written to work in
a GUI instead of from a command line, however, a paradigm that's at least
analogous to dataflow is needed on the highest level, instead of just in
the innermost loops.

Also, while a computer could be designed to be a dataflow engine on
one level, there is the brutal slowness of the interface to main memory,
to external DRAM.

So it would be simple enough to design a dataflow processor. You
put a pile of ALUs on a chip. You have a lot of switchable connections
linking them. And you also put a pile of separate memories in the chip,
so that you can link multiple inputs and outputs to the computation.

Some of those memories could be used as look-up tables instead of
input or output hoppers.

The problem is: how useful would such a processor be? Assume it
shares a die with a conventional processor, which is used to set it up
for problems. How often will setting it up for a problem be faster than
just having the serial processor do the problem?

I have tried to think of a way to define a dataflow instruction for
what is otherwise a conventional processor. Basically, the instruction
performs a vector operation, but with multiple opcodes, with all the
data forwarding specified.

John Savard

Tim Rentsch wrote:
> Terje Mathisen <terje.mathisen@tmsw.no> writes:
>
>> Tim Rentsch wrote:
>>
>>> Ivan Godard <ivan@millcomputing.com> writes:
>>>
>>> [...]
>>>
>>>> I think the problem is really a linguistic one: our programming
>>>> languages have caused us to think about our programs in control
>>>> flow terms, [...]
>>>
>>> Some of us. Not everyone, fortunately.
>>
>> I tend to think much more in data flow terms, as I've written before:
>>
>> An optimal program is like a creek finding its path down from the
>> mountain, always searching for the path of least resistance.
>
> It's an interesting simile. Note by the way that creeks are
> greedy algorithms, in the sense of being only locally optimal,
> not necessarily globally optimal.
>
> To me the more interesting question is how does this perspective
> affect how the code looks? If reading your programs, would I see
> something that looks pretty much like other imperative code, or
> would there be some distinguishing characteristics that would
> indicate your different thought mode? Can you say anything about
> what those characteristics might be? Or perhaps give an example
> or two? (Short is better if that is feasible.)
>
First example: Use (very) small lookup tables to get rid of branches,
typically trying to turn code state machines into data state machines.
This is driven by the fact that modern CPUs are much better at dependent
loads than unpredictable branches.

I typically try to write code that can run in SIMD mode, i.e. using
tables/branchless/predicated ops to handle any single-lane alternate paths.

Such code can also much more often scale across multiple cores/systems
even if that means that I have to do some redundant work across the
boundaries. I.e. when I process lidar data I can make the problem almost
embarrasingly parallelizable by splitting the input into tiles with
35-50m overlap: This is sufficient to effectively eliminate all edge
artifacts when I generate contours and vegetation classifications.

The most fun is when I find ways to remove all internal branching
related to exceptional data, i.e. anything which can impede that nicely
flowing stream of water going downhill.

BTW, looking for both local and global optimal paths/solutions is very
similar to what we do when competing in orienteering: What is my running
speed going to be along these paths vs going cross country? How is that
affected by the density of the vegetation and the amount of elevation
gain/loss? How much more time will I need to spend on the navigation
part when taking a more risky direct route without obvious intermediate
features to use for feedback? (In the latter case I also need to
incorporate the expected loss of time by not hitting the control
perfectly and then having to search and/or recover my position by
running to the nearest large/obvious feature.)

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Quadibloc <jsavard@ecn.ab.ca> schrieb:

> So it would be simple enough to design a dataflow processor. You
> put a pile of ALUs on a chip. You have a lot of switchable connections
> linking them. And you also put a pile of separate memories in the chip,
> so that you can link multiple inputs and outputs to the computation.
>
> Some of those memories could be used as look-up tables instead of
> input or output hoppers.

Sounds like an FPGA to me.

> The problem is: how useful would such a processor be? Assume it
> shares a die with a conventional processor, which is used to set it up
> for problems. How often will setting it up for a problem be faster than
> just having the serial processor do the problem?

> I have tried to think of a way to define a dataflow instruction for
> what is otherwise a conventional processor. Basically, the instruction
> performs a vector operation, but with multiple opcodes, with all the
> data forwarding specified.

Use a HDL.

If you want, you can have a softcore for your CPU and define special
instructions for your special needs. I don't think it is easy
to modify the FPGA programming on the fly.

On Wednesday, March 9, 2022 at 5:36:57 AM UTC-6, Thomas Koenig wrote:
> Quadibloc <jsa...@ecn.ab.ca> schrieb:
> > So it would be simple enough to design a dataflow processor. You
> > put a pile of ALUs on a chip. You have a lot of switchable connections
> > linking them. And you also put a pile of separate memories in the chip,
> > so that you can link multiple inputs and outputs to the computation.
> >
> > Some of those memories could be used as look-up tables instead of
> > input or output hoppers.
> Sounds like an FPGA to me.
> > The problem is: how useful would such a processor be? Assume it
> > shares a die with a conventional processor, which is used to set it up
> > for problems. How often will setting it up for a problem be faster than
> > just having the serial processor do the problem?
>
> > I have tried to think of a way to define a dataflow instruction for
> > what is otherwise a conventional processor. Basically, the instruction
> > performs a vector operation, but with multiple opcodes, with all the
> > data forwarding specified.
> Use a HDL.
>
> If you want, you can have a softcore for your CPU and define special
> instructions for your special needs. I don't think it is easy
> to modify the FPGA programming on the fly.

It is possible to modify block RAM on the fly
And it can be used to contain micro-code