A couple of weeks ago, someone with 20+ years of software
experience emailed me asking how to develop an "intuition"
or "way of thinking" in computer architecture to "come up
with elegant solutions that are implementable and scalable
in hardware", "jumping the chasm from absorbing and
understanding to being able to contribute and come up with
elegant architecture".

My response (which I will append at the end of this post)
was very weak; perhaps some more useful advice might be
offered here on comp.arch.

============== My email response ===========================

I myself am only a hobbyist — and not even a hobbyist who has
the skills to develop hardware — so I am not certain how much
useful advice I can offer. I will present a few thoughts.
Even if I were a professional, you would be wise to seek
others for advice; any creative field will have different
methods that work for different people and having the same
thing explained from multiple perspectives often aids
understanding.

It is commonly said that computer hardware differs most
fundamentally from software in parallelism, but I think
spatial reasoning is more significant. The structural support
of a bridge is parallel, but I doubt the engineers designing
a bridge think of each cable individually as acting in
parallel. Giving less attention to the aspect of time may
offer a more humanly natural means of conceiving of
the parallel (in space) operation; I suspect most people are
more familiar with spatial relationships (the toaster is to
the right of the sink; the conventional oven is to the left
of the sink; the microwave is above the oven) than parallel
action even when the functionality is specialized. (Each
piece of kitchen equipment can operate in parallel — and the
specialization of function may help convert "oven bakes,
microwave heats, toaster toasts" to "oven operates, microwave
operates, toaster operates" — but I suspect "is occupied" is
a more common conception.)

(Obviously, a bridge designer cannot ignore time; forces are
not constant over time. When preparing food one pays
attention to cooking time, equipment occupancy, and dependency
chains and not so much the locations of the equipment, but
visualizing the equipment spatially may make "structural
hazards" more obvious and this hazard detection seems to be
second nature.)

Familiarity with some rules of thumb, special techniques, and
trade-off axes would probably help in developing a mindset
for working in computer architecture. Smaller/closer is
faster/lower power is a commonly cited rule of thumb. The
trade-off axis of specialization (with efficiency) and
utilization is basic (as is the generalized application of
Amdahl's Law to fractional improvements). The trade-off axis
of caching results and redoing work (with communication
considered work) is also basic. (One might naively imagine
that one should always cache results and schedule work as
early as possible, but timely reuse is a speculation and even
if all inputs are available at an earlier time a later start
may be better — e.g., to reduce buffering if consumers of a
result will not be ready earlier.)

These do not seem especially alien to software design. (I
suspect software is more biased to favor utilization over
specialization; the total complexity of software seems
higher — as one would expect higher up an interface stack —
and the temporal and workload breadth of software use seems
higher, both factors discourage specialization which adds
complexity and conflicts with breadth of use. Being lower
on the interface stack, hardware design may be more able to
justify effort for correctness and fitness of purpose
[usually performance].)

Speculation and doing more work to produce a result faster
may be less common in software, but even software is not
typically designed to minimize variation from worst case
execution time. E.g., a simple hash table look-up "speculates"
that the primary bucket will have the matching key.

I suspect there is a rough equivalent of "data structures and
algorithms" for hardware development, where a familiarity
with a modest number of principles of the trade-offs is
sufficient and one would gradually gain awareness of
variations and use cases. Looking at good examples is also a
common learning technique, but the availability of technical
information about professional hardware designs seems to have
decreased significantly (and rationales are even less
commonly presented).

I fear little of the above has been helpful, but you are very
welcome to send a follow-up email.

If it would be helpful, I could try to walk through a couple
of textbook/exam-style exercises to present how I would work
on them.

On 2022-03-31, Paul A. Clayton wrote:
> A couple of weeks ago, someone with 20+ years of software
> experience emailed me asking how to develop an "intuition"
> or "way of thinking" in computer architecture to "come up
> with elegant solutions that are implementable and scalable
> in hardware", "jumping the chasm from absorbing and
> understanding to being able to contribute and come up with
> elegant architecture".
>
> My response (which I will append at the end of this post)
> was very weak; perhaps some more useful advice might be
> offered here on comp.arch.
>
> ============== My email response ===========================
> [snip - sorry]

One thing that I have noticed is that one major difference in mindset
between software and hardware development is that of resource reuse.

In software development it is often beneficial to make many different
specialized solutions. If you identify that you can do less work in
certain cases by skipping operations or replacing them with cheaper
operations, it's almost always a win to do so in software (mainly
because software size is rarely an issue, and the L1I cache quickly
absorbs new/different variants).

In hardware development, on the other hand, it's almost always
beneficial to make heavy use of a few different resources. For instance
it is usually more beneficial to use a single full-width multiplier for
many different purposes than to have many different width multipliers
for different purposes (unless they can all be used concurrently, that
is). This is because die area (or LUTs or whatever your measure is) is
a limited resource.

A similar discussion in this forum concerned the differences in hardware
and software implementations of reciprocals. E.g. in software it can
be beneficial to start off with a narrow word width and work your way
up to wider (and more costly) operations as the precision increases
after a few Newton-Raphson iterations, whereas in hardware it can be
better to just use full width all the way.

Thus it is important to construct your hardware design around powerful
primitives. A Fused Multiply-Add unit is a good example of a primitive
that can be used for many different purposes.

/Marcus

Paul A. Clayton <paaronclayton@gmail.com> wrote:
> A couple of weeks ago, someone with 20+ years of software
> experience emailed me asking how to develop an "intuition"
> or "way of thinking" in computer architecture to "come up
> with elegant solutions that are implementable and scalable
> in hardware", "jumping the chasm from absorbing and
> understanding to being able to contribute and come up with
> elegant architecture".
>
> My response (which I will append at the end of this post)
> was very weak; perhaps some more useful advice might be
> offered here on comp.arch.

I think a difference is that in software your foundation is a toolbox
provided by the architecture. You have *these* tools, and you can achieve
your desired result if you use tool A then tool B then tool C. There is
then a lot of scheming that takes place to achieve the desired result based
on the combination of tools that happen to be in the toolbox.

eg there are lots of virtual memory tricks (demand paging, page compression,
copy on write) that rely on non-obvious combinations of the MMU page table,
something cunning in the page fault handler and instruction resumption after
page faults. Lots of people come up with schemes that fit the same basic
pattern, because the basic pattern is set up by the architecture.

In hardware you can do *almost anything*, but that anything has an
implementation cost in performance, area, power, complexity and
compatibility. You don't need to come up with a cunning combination of
tools from the toolbox, because you can just add a new one. The problem is
that it's too expensive to do that for every idea unless the thing you're
offering has a really good justification or relatively minor costs.
So you end up needing to make a more generic tool to add to the toolbox than
a special purpose one for a specific problem.

> It is commonly said that computer hardware differs most
> fundamentally from software in parallelism, but I think
> spatial reasoning is more significant.

Software is increasingly spatial too, of course (threads/cores/servers/etc).

I think a flaw in hardware design is people taking spatiality too literally:
they have an algorithm with N nodes, so we map each node to a LUT/ALU/core
and then wire them in a 2/3/4D mesh, and... we end up with an unroutable
mess of wires which ends up very inefficient. Whereas it may be better to
have central core(s) and pump data through them, with memory addresses
representing dimensions. Parallelism is useful, but too much can be
problematic, just like overly serialising things in software can be
problematic.

Theo

On 3/31/2022 9:34 AM, Marcus wrote:
> On 2022-03-31, Paul A. Clayton wrote:
>> A couple of weeks ago, someone with 20+ years of software
>> experience emailed me asking how to develop an "intuition"
>> or "way of thinking" in computer architecture to "come up
>> with elegant solutions that are implementable and scalable
>> in hardware", "jumping the chasm from absorbing and
>> understanding to being able to contribute and come up with
>> elegant architecture".
>>
>> My response (which I will append at the end of this post)
>> was very weak; perhaps some more useful advice might be
>> offered here on comp.arch.
>>
>> ============== My email response ===========================
> > [snip - sorry]
>
> One thing that I have noticed is that one major difference in mindset
> between software and hardware development is that of resource reuse.
>
> In software development it is often beneficial to make many different
> specialized solutions. If you identify that you can do less work in
> certain cases by skipping operations or replacing them with cheaper
> operations, it's almost always a win to do so in software (mainly
> because software size is rarely an issue, and the L1I cache quickly
> absorbs new/different variants).
>
> In hardware development, on the other hand, it's almost always
> beneficial to make heavy use of a few different resources. For instance
> it is usually more beneficial to use a single full-width multiplier for
> many different purposes than to have many different width multipliers
> for different purposes (unless they can all be used concurrently, that
> is). This is because die area (or LUTs or whatever your measure is) is
> a limited resource.
>

Pros/Cons:
FPGA has far more DSP48's than one actually needs, whereas doing a
single big multiplier is slow/expensive.

So, instead of say, a big 64-bit multiplier, one might end up with
something that does, say:
One 32-bit widening multiply;
Two 32-bit narrow multiplies (SIMD);
Four 16-bit multiplies (SIMD).

Packed integer SIMD might just be normal ALU ops with carry propagation
disabled.

But, yeah, it is still true that one might end up with a few big units
that most everything else ends up being routed through, with different
sub-features being enabled or disabled.

Elsewhere, recently, an argument came up over scaled-index addressing
(as a hypothetical RISC-V extension), where the assumption seemed to be
that for an ISA to offer scaled-index at all, then all operations would
necessarily need to have 3 GPR ports at all times.

This isn't really how it works in BJX2 though, since for the 6R3W
regfile, I usually have the Lane 3 ports sitting around not doing
anything else, I can leverage these for the ports needed for
Scaled-Index and similar.

So, really, it would likely only be a significant burden for a 1-wide
core, which might need a 3R/1W regfile rather than a 2R1W regfile. But,
then again, as I see it, about as soon as the core is much larger than a
microcontroller, this whole debate becomes kinda moot (and if it were an
extension, a microcontroller could choose to omit it).

The index-scale + add doesn't really offer much cost otherwise over
doing the same thing with an immediate (and the AGU doesn't need to care
whether the displacement came from a register or immediate).

Or at least not enough to where I will be like, "Yeah, it doesn't matter
that these 15% or so of Load/Store ops will now need to be expressed as
multi-instruction sequences". This falls a bit outside my "small enough
that it doesn't matter" heuristic.

This is, admittedly, one of the bigger disagreements I had with RISC-V;
well, along with requiring a full-width multiplier (and a divide
instruction) for 'M' (as opposed to a narrower widening multiplier).

Say, for example, if we could have 'Mlite' instead, which provides a
half-width widening multiply (with no divide instruction). The widening
multiply could be either Signed+Unsigned, or Unsigned-Only (since one
mostly uses unsigned-multiply to compose a larger multiply).

But, why would I argue for scaled-index but against full-width 64-bit
multiply?:
Scaled index is cheap and has a visible effect on performance;
64-bit multiply is slow, expensive, and rarely used.

A hardware divide instruction could probably help with Dhrystone score,
but is less relevant in general (Dhrystone seems to significantly
over-represent the use of integer divide vs normal code).

One could (more affordably) provide full-width multiply (and divide) as
dedicated shift-check-add / shift-check-subtract logic (stalling the
pipeline until done), but these would be fairly slow.

I am half wondering if this is what the 'M' extension assumed (as
opposed to people being able to fit them into 1..3 pipeline stages).

OTOH: If you have a 3-cycle half-width multiply, or a 64..68-cycle full
width multiply, faking it in software is faster (well, unless you use an
emulation trap ISR, then 68 cycles is "almost free" in comparison).

(This algo could likely implement MUL as a hacked variant of the DIV
sequence, switching the input and output registers, using the "would be"
output of the DIV as the MUL input, with the accumulator initialized to
Zero, checking the MSB of the DIV output for whether or not to ADD
right-shift register to the accumulator, and then using the accumulator
as the MUL output).

I guess although it would be slow, it could allow for the 'M' extension
within a reasonable cost window.

....

Well, there was also the 'A' extension, which seems to be built under
the assumption that one has some sort of multi-core cache-coherency
protocol (could be faked, but would not behave as expected without this;
but is mostly N/A for single core).

....

> A similar discussion in this forum concerned the differences in hardware
> and software implementations of reciprocals. E.g. in software it can
> be beneficial to start off with a narrow word width and work your way
> up to wider (and more costly) operations as the precision increases
> after a few Newton-Raphson iterations, whereas in hardware it can be
> better to just use full width all the way.
>
> Thus it is important to construct your hardware design around powerful
> primitives. A Fused Multiply-Add unit is a good example of a primitive
> that can be used for many different purposes.
>

Granted.

I had considered switching to an FMAC design (in my BJX2 core), but cost
and timing tradeoffs favored sticking with an FMUL and FADD unit; which
may be internally "glued" to mimic an FMAC operation, albeit with some
numerical restrictions (a true FMAC would implement some calculations
which can't be emulated with split units because of "bits falling off
the bottom").

But, to get these properties with an FMAC unit would require an ~ 108
bit or so mantissa to be used internally (and/or split units which
internally operate on full-precision Binary128).

But, even S.E15.F80 (Float96, *) was seriously pushing it...

*: This format was (in storage) the same as Binary128, except that the
low-order 32 bits were ignored on inputs (treated as if they only
contained zeroes), and were filled with zeroes on output. This being
because the relative cost difference between an 80-bit mantissa and
112-bit mantissa (for FMUL purposes) is quite significant.

Granted, if one had such an FMAC unit, they could leverage it for a
64-bit integer multiplier or integer MAC (feeding integers as FP inputs
isn't that hard to pull off, and the integer-output logic likely also
exists if the FADD mechanism is being leveraged for FP->Int conversions).

The intermediate option being, say, to multiply two F80 inputs producing
a F144 intermediate output (vs F160), using the F144 to produce a
128-bit integer MUL/MAC output (for integer MAC, it would make sense to
support the 3rd input as either Float96 or Int128).

....

In theory, if FDIV exists, IDIV could be routed through the same logic
in this case (as with the above). Doesn't provide a good way to provide
for an integer Modulo/Remainder instruction though.

....

Elsewhere (I am still debating it), is whether my BJX2 project needs a
"real" OS port, or whether I am better off (for now) still hacking
features onto TestKern and going in an "OS like" direction with it.

I had looked into it, and realized that porting the kernel would
actually be the lesser of the issues, and that the "hard part" would be
trying to port over the userland (whether I went for either the GNU or
BSD userland).

More so, it would likely also require either modifying GCC to be able to
target BJX2, or making be BGBCC able mimic the external interface
expected for cross-compilers, ...

Eg:
'bjx2-testkern-cc' (likewise for as, 'ld', ...);
Producing '.o' files;
Accepting '.o' and '.a' files;
Ability to 'ar' multiple '.o' files into a '.a' file;
...

On 3/31/2022 8:58 AM, Theo Markettos wrote:
> Paul A. Clayton <paaronclayton@gmail.com> wrote:
>> A couple of weeks ago, someone with 20+ years of software
>> experience emailed me asking how to develop an "intuition"
>> or "way of thinking" in computer architecture to "come up
>> with elegant solutions that are implementable and scalable
>> in hardware", "jumping the chasm from absorbing and
>> understanding to being able to contribute and come up with
>> elegant architecture".
>>
>> My response (which I will append at the end of this post)
>> was very weak; perhaps some more useful advice might be
>> offered here on comp.arch.
>
> I think a difference is that in software your foundation is a toolbox
> provided by the architecture. You have *these* tools, and you can achieve
> your desired result if you use tool A then tool B then tool C. There is
> then a lot of scheming that takes place to achieve the desired result based
> on the combination of tools that happen to be in the toolbox.
>
> eg there are lots of virtual memory tricks (demand paging, page compression,
> copy on write) that rely on non-obvious combinations of the MMU page table,
> something cunning in the page fault handler and instruction resumption after
> page faults. Lots of people come up with schemes that fit the same basic
> pattern, because the basic pattern is set up by the architecture.
>
> In hardware you can do *almost anything*, but that anything has an
> implementation cost in performance, area, power, complexity and
> compatibility. You don't need to come up with a cunning combination of
> tools from the toolbox, because you can just add a new one. The problem is
> that it's too expensive to do that for every idea unless the thing you're
> offering has a really good justification or relatively minor costs.
> So you end up needing to make a more generic tool to add to the toolbox than
> a special purpose one for a specific problem.

I am not sure about your answer. The hardware guys have a set of
"tools" but they are different types of gates, storage cells,
interconnect characteristics, etc. They can be combined to create new
combinations i.e. instructions, similarly to the way software combines
various instructions to create new "primitives" i.e. functions.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> wrote:
> I am not sure about your answer. The hardware guys have a set of
> "tools" but they are different types of gates, storage cells,
> interconnect characteristics, etc. They can be combined to create new
> combinations i.e. instructions, similarly to the way software combines
> various instructions to create new "primitives" i.e. functions.

Of course, but you are typically designing hardware at a much higher level
than that. You're writing HDL which gets synthesised down to hardware, and
you typically don't know what kind of hardware they made (unless you take a
deep look).

I suppose the analogy for that would be someone who writes Python:
they're a software developer, but they're nowhere near the compiler so they
don't know what kind of instructions are being generated.

Hardware designers are not choosing what kind of transistor or NOR gate to
use, although they are deciding where to put an SRAM and how wide to make a
bus. If not quite Python, it's more like writing portable C or C++ without
targeting any particular OS or CPU.

The people who worry about transistors are the people who push backend tools
- they're taking the HDL from the hardware designer and making a chip come
out, worry about things like timing margin, clock distribution and power
supplies. They are generally not architects, they are more like human
versions of your C compiler.

(There is also a difference that there is a much stronger focus on
verification, because getting something wrong is very expensive.)

If we are talking about architecture, there are many more degrees of freedom
afforded to the hardware designer on, say, an FPGA, than there are to the
programmer on a CPU. So you are not constrained nearly as much as you are
by an ISA or the architectural features afforded by the current CPU.

Theo

> (There is also a difference that there is a much stronger focus on
> verification, because getting something wrong is very expensive.)

Tho, IIUC "verification" in the hardware world refers usually to what
the software world calls "testing". In the software world,
"verification" means the use of exhaustive tools such as theorem provers
or model checkers.

Stefan

On Thursday, March 31, 2022 at 5:17:08 PM UTC-5, BGB wrote:
> On 3/31/2022 9:34 AM, Marcus wrote:
> > On 2022-03-31, Paul A. Clayton wrote:

> Pros/Cons:
> FPGA has far more DSP48's than one actually needs, whereas doing a
> single big multiplier is slow/expensive.
>
> So, instead of say, a big 64-bit multiplier, one might end up with
> something that does, say:
> One 32-bit widening multiply;
> Two 32-bit narrow multiplies (SIMD);
> Four 16-bit multiplies (SIMD).
<
A computer architect understands that one can segment a multiplier tree
by preventing carries from one sub-tree to another. This architect is smart
enough to understand that clipping the carries doe snot even add gates of
delay to the multiplier tree's calculations. So all 3 above multipliers can be
built from one incarnation of a SIMD sliding width multiplier.
>
> Packed integer SIMD might just be normal ALU ops with carry propagation
> disabled.
>
You also have to have a means to clip the carries in the adder that inevitably
follows the tree.
>
> But, yeah, it is still true that one might end up with a few big units
> that most everything else ends up being routed through, with different
> sub-features being enabled or disabled.
>
>
>
> Elsewhere, recently, an argument came up over scaled-index addressing
> (as a hypothetical RISC-V extension), where the assumption seemed to be
> that for an ISA to offer scaled-index at all, then all operations would
> necessarily need to have 3 GPR ports at all times.
<
That is a poor argument,.......proposed by somebody over there.
>
> This isn't really how it works in BJX2 though, since for the 6R3W
> regfile, I usually have the Lane 3 ports sitting around not doing
> anything else, I can leverage these for the ports needed for
> Scaled-Index and similar.
<
Well, that and the index is almost always available on the forwarding path ....
>
> So, really, it would likely only be a significant burden for a 1-wide
> core, which might need a 3R/1W regfile rather than a 2R1W regfile. But,
> then again, as I see it, about as soon as the core is much larger than a
> microcontroller, this whole debate becomes kinda moot (and if it were an
> extension, a microcontroller could choose to omit it).
>
BTW I am getting 1.3-wide issue from a 3R1W RF in My66130 by taking
fuller than standard advantage of other necessities in My 66000.
>
> The index-scale + add doesn't really offer much cost otherwise over
> doing the same thing with an immediate (and the AGU doesn't need to care
> whether the displacement came from a register or immediate).
<
I would argue it is free to add (or insignificant burden)
>
> Or at least not enough to where I will be like, "Yeah, it doesn't matter
> that these 15% or so of Load/Store ops will now need to be expressed as
> multi-instruction sequences". This falls a bit outside my "small enough
> that it doesn't matter" heuristic.
>
Even if the gain is only 2% it is still a win, and it is brain dead easy to AGEN
And after AGEN there is no damage to the rest of the whole cache pipeline.
>
> This is, admittedly, one of the bigger disagreements I had with RISC-V;
> well, along with requiring a full-width multiplier (and a divide
> instruction) for 'M' (as opposed to a narrower widening multiplier).
>
> Say, for example, if we could have 'Mlite' instead, which provides a
> half-width widening multiply (with no divide instruction). The widening
> multiply could be either Signed+Unsigned, or Unsigned-Only (since one
> mostly uses unsigned-multiply to compose a larger multiply).
>
>
> But, why would I argue for scaled-index but against full-width 64-bit
> multiply?:
> Scaled index is cheap and has a visible effect on performance;
> 64-bit multiply is slow, expensive, and rarely used.
<
My 66100 (the smallest possible pipelined implementation) has the
full sized multiplier 64×64 so as to do i× in 4 cycles as well as FMAC
in 4 cycles. The trick, here, is that I run the multiplier pipeline at DIV-2.
When the multiplier is not busy, it can accept an instruction on any
clock, when it is busy multiplying it can accept a multiply every other
clock. This gets rid of either 2048 or 4096 flip-flops in the multiplier
tree (Athloon+Opteron) without losing performance because the
rest of the implementation cannot feed the multiplier continuously.
<
The flip-flops 2 :: 3-input adders cells above the end of the tree
were nearly as large of the multiplier tree itself !!!
>
> A hardware divide instruction could probably help with Dhrystone score,
> but is less relevant in general (Dhrystone seems to significantly
> over-represent the use of integer divide vs normal code).
>
>
> One could (more affordably) provide full-width multiply (and divide) as
> dedicated shift-check-add / shift-check-subtract logic (stalling the
> pipeline until done), but these would be fairly slow.
<
This is standard practice today.
>
> I am half wondering if this is what the 'M' extension assumed (as
> opposed to people being able to fit them into 1..3 pipeline stages).
>
> OTOH: If you have a 3-cycle half-width multiply, or a 64..68-cycle full
> width multiply, faking it in software is faster (well, unless you use an
> emulation trap ISR, then 68 cycles is "almost free" in comparison).
>
> (This algo could likely implement MUL as a hacked variant of the DIV
> sequence, switching the input and output registers, using the "would be"
> output of the DIV as the MUL input, with the accumulator initialized to
> Zero, checking the MSB of the DIV output for whether or not to ADD
> right-shift register to the accumulator, and then using the accumulator
> as the MUL output).
>
> I guess although it would be slow, it could allow for the 'M' extension
> within a reasonable cost window.
>
> ...
>
>
> Well, there was also the 'A' extension, which seems to be built under
> the assumption that one has some sort of multi-core cache-coherency
> protocol (could be faked, but would not behave as expected without this;
> but is mostly N/A for single core).
>
> ...
> > A similar discussion in this forum concerned the differences in hardware
> > and software implementations of reciprocals. E.g. in software it can
> > be beneficial to start off with a narrow word width and work your way
> > up to wider (and more costly) operations as the precision increases
> > after a few Newton-Raphson iterations, whereas in hardware it can be
> > better to just use full width all the way.
> >
> > Thus it is important to construct your hardware design around powerful
> > primitives. A Fused Multiply-Add unit is a good example of a primitive
> > that can be used for many different purposes.
> >
> Granted.
>
>
> I had considered switching to an FMAC design (in my BJX2 core), but cost
> and timing tradeoffs favored sticking with an FMUL and FADD unit; which
> may be internally "glued" to mimic an FMAC operation, albeit with some
> numerical restrictions (a true FMAC would implement some calculations
> which can't be emulated with split units because of "bits falling off
> the bottom").
>
> But, to get these properties with an FMAC unit would require an ~ 108
> bit or so mantissa to be used internally (and/or split units which
> internally operate on full-precision Binary128).
>
> But, even S.E15.F80 (Float96, *) was seriously pushing it...
>
> *: This format was (in storage) the same as Binary128, except that the
> low-order 32 bits were ignored on inputs (treated as if they only
> contained zeroes), and were filled with zeroes on output. This being
> because the relative cost difference between an 80-bit mantissa and
> 112-bit mantissa (for FMUL purposes) is quite significant.
>
>
> Granted, if one had such an FMAC unit, they could leverage it for a
> 64-bit integer multiplier or integer MAC (feeding integers as FP inputs
> isn't that hard to pull off, and the integer-output logic likely also
> exists if the FADD mechanism is being leveraged for FP->Int conversions).
>
> The intermediate option being, say, to multiply two F80 inputs producing
> a F144 intermediate output (vs F160), using the F144 to produce a
> 128-bit integer MUL/MAC output (for integer MAC, it would make sense to
> support the 3rd input as either Float96 or Int128).
>
> ...
>
>
> In theory, if FDIV exists, IDIV could be routed through the same logic
> in this case (as with the above). Doesn't provide a good way to provide
> for an integer Modulo/Remainder instruction though.
>
> ...
>
>
>
>
> Elsewhere (I am still debating it), is whether my BJX2 project needs a
> "real" OS port, or whether I am better off (for now) still hacking
> features onto TestKern and going in an "OS like" direction with it.
>
> I had looked into it, and realized that porting the kernel would
> actually be the lesser of the issues, and that the "hard part" would be
> trying to port over the userland (whether I went for either the GNU or
> BSD userland).
>
>
> More so, it would likely also require either modifying GCC to be able to
> target BJX2, or making be BGBCC able mimic the external interface
> expected for cross-compilers, ...
>
> Eg:
> 'bjx2-testkern-cc' (likewise for as, 'ld', ...);
> Producing '.o' files;
> Accepting '.o' and '.a' files;
> Ability to 'ar' multiple '.o' files into a '.a' file;
> ...
>
> Since this is what things like 'autoconf' and similar expect (well, as
> opposed to a tool which uses a CLI more like that used by MSVC...).
>
> Luckily, at least, these tools don't really care that much what the '.o'
> files contain (so, eg, it can be RIL rather than a COFF or ELF object).
>
> ...

On Friday, April 1, 2022 at 5:19:21 PM UTC-5, Stefan Monnier wrote:
> > (There is also a difference that there is a much stronger focus on
> > verification, because getting something wrong is very expensive.)
> Tho, IIUC "verification" in the hardware world refers usually to what
> the software world calls "testing". In the software world,
> "verification" means the use of exhaustive tools such as theorem provers
> or model checkers.
<
Err, no. HW verification is a suite of tests upon HW carried out with the
hope that there is never a need for a field update (Pentium FDVI bug)
<
Essentially no SW ever gets tested to this degree of correctness.
>
>
> Stefan

On Friday, April 1, 2022 at 5:14:14 PM UTC-5, Theo Markettos wrote:
> Stephen Fuld <sf...@alumni.cmu.edu.invalid> wrote:
> > I am not sure about your answer. The hardware guys have a set of
> > "tools" but they are different types of gates, storage cells,
> > interconnect characteristics, etc. They can be combined to create new
> > combinations i.e. instructions, similarly to the way software combines
> > various instructions to create new "primitives" i.e. functions.
> Of course, but you are typically designing hardware at a much higher level
> than that. You're writing HDL which gets synthesised down to hardware, and
> you typically don't know what kind of hardware they made (unless you take a
> deep look).
>
> I suppose the analogy for that would be someone who writes Python:
> they're a software developer, but they're nowhere near the compiler so they
> don't know what kind of instructions are being generated.
>
> Hardware designers are not choosing what kind of transistor or NOR gate to
> use, although they are deciding where to put an SRAM and how wide to make a
> bus. If not quite Python, it's more like writing portable C or C++ without
> targeting any particular OS or CPU.
>
> The people who worry about transistors are the people who push backend tools
> - they're taking the HDL from the hardware designer and making a chip come
> out, worry about things like timing margin, clock distribution and power
> supplies. They are generally not architects, they are more like human
> versions of your C compiler.
<
I submit that a "Real Computer Architect" must have an understanding that
traverses the field from::
Spice level modeling
Layout,
Gate design
Bus design
Select line design
State machine design
Calculation unit design,
Block design
Integration
Verification
Block management
Group management
Project management
Documentation design
Documentation update
HDL design
HDL coding
HDL integration
Software modeling
Compiler design
Reading of assembler code from a variety of compilers
<
And once they have spent 1 year writing code generation inside a compiler
and have been exposed to all of the list above--then they are in a position
to start being an apprentice to a computer architect.
<
The computer architect must be able to understand his design well enough
such that when [s]he enters a meeting and there is a problem between the gate
designer and a std-cell designer, he is in a position to understand the arguments,
and make suggestive comments to drive the design forward.
<
The computer architect does NOT throw a design over the cubical wall
and expect good things to happen. He has to own every nuance and aspect
of the design, he may find himself in a position where some aspect of what
was proposed simply cannot work, and he has to fix the bad concept OVERNIGHT
and get the team back working the next day--even if he has to stay up all night
getting it fixed, documented, complete with drawings of what needs to change.
>
> (There is also a difference that there is a much stronger focus on
> verification, because getting something wrong is very expensive.)
>
> If we are talking about architecture, there are many more degrees of freedom
> afforded to the hardware designer on, say, an FPGA, than there are to the
> programmer on a CPU. So you are not constrained nearly as much as you are
> by an ISA or the architectural features afforded by the current CPU.
>
> Theo
<
But I also admit that much of what Theo wrote is dead accurate--much to the
detriment of the state of computers today.

>> > (There is also a difference that there is a much stronger focus on
>> > verification, because getting something wrong is very expensive.)
>> Tho, IIUC "verification" in the hardware world refers usually to what
>> the software world calls "testing". In the software world,
>> "verification" means the use of exhaustive tools such as theorem provers
>> or model checkers.
> Err, no. HW verification is a suite of tests upon HW carried out with the
> hope that there is never a need for a field update (Pentium FDVI bug)
> Essentially no SW ever gets tested to this degree of correctness.

AFAIK it's not exhaustive, so in software terms it falls in the camp of
"testing" rather than "verification".

I know it's typically a much more extensive test suite than what is used
for software, and I also understand that "exhaustive" can make sense
only if you limit it by making assumptions about your hardware
(typically that starts by pretending the world is not analog).

It's not a criticism, rather a clarification about the kind of
tools used since verification (in the software-sense of the word) is
done using fundamentally very different tools than testing.

Stefan

On 4/4/2022 4:05 PM, MitchAlsup wrote:
> On Friday, April 1, 2022 at 5:14:14 PM UTC-5, Theo Markettos wrote:
>> Stephen Fuld <sf...@alumni.cmu.edu.invalid> wrote:
>>> I am not sure about your answer. The hardware guys have a set of
>>> "tools" but they are different types of gates, storage cells,
>>> interconnect characteristics, etc. They can be combined to create new
>>> combinations i.e. instructions, similarly to the way software combines
>>> various instructions to create new "primitives" i.e. functions.
>> Of course, but you are typically designing hardware at a much higher level
>> than that. You're writing HDL which gets synthesised down to hardware, and
>> you typically don't know what kind of hardware they made (unless you take a
>> deep look).
>>
>> I suppose the analogy for that would be someone who writes Python:
>> they're a software developer, but they're nowhere near the compiler so they
>> don't know what kind of instructions are being generated.
>>
>> Hardware designers are not choosing what kind of transistor or NOR gate to
>> use, although they are deciding where to put an SRAM and how wide to make a
>> bus. If not quite Python, it's more like writing portable C or C++ without
>> targeting any particular OS or CPU.
>>
>> The people who worry about transistors are the people who push backend tools
>> - they're taking the HDL from the hardware designer and making a chip come
>> out, worry about things like timing margin, clock distribution and power
>> supplies. They are generally not architects, they are more like human
>> versions of your C compiler.
> <
> I submit that a "Real Computer Architect" must have an understanding that
> traverses the field from::
> Spice level modeling
> Layout,
> Gate design
> Bus design
> Select line design
> State machine design
> Calculation unit design,
> Block design
> Integration
> Verification
> Block management
> Group management
> Project management
> Documentation design
> Documentation update
> HDL design
> HDL coding
> HDL integration
> Software modeling
> Compiler design
> Reading of assembler code from a variety of compilers

By this list I am not a Real Computer Architect (IANACA). Which I am
happy to concede.

However, the definition uses the singular article "a", implying that
there quality of Computer Architect is necessarily embodied in a single
person. However, that is false: Computer Architect may be a composite
form or two or more persons, joined at the hip, who together present
your list - *and can interoperate well enough to fill each other's lacunae*.

> And once they have spent 1 year writing code generation inside a compiler
> and have been exposed to all of the list above--then they are in a position
> to start being an apprentice to a computer architect.
> <
> The computer architect must be able to understand his design well enough
> such that when [s]he enters a meeting and there is a problem between the gate
> designer and a std-cell designer, he is in a position to understand the arguments,
> and make suggestive comments to drive the design forward.
> <
> The computer architect does NOT throw a design over the cubical wall
> and expect good things to happen. He has to own every nuance and aspect
> of the design, he may find himself in a position where some aspect of what
> was proposed simply cannot work, and he has to fix the bad concept OVERNIGHT
> and get the team back working the next day--even if he has to stay up all night
> getting it fixed, documented, complete with drawings of what needs to change.
>>
>> (There is also a difference that there is a much stronger focus on
>> verification, because getting something wrong is very expensive.)
>>
>> If we are talking about architecture, there are many more degrees of freedom
>> afforded to the hardware designer on, say, an FPGA, than there are to the
>> programmer on a CPU. So you are not constrained nearly as much as you are
>> by an ISA or the architectural features afforded by the current CPU.
>>
>> Theo
> <
> But I also admit that much of what Theo wrote is dead accurate--much to the
> detriment of the state of computers today.

On 4/4/2022 5:49 PM, MitchAlsup wrote:
> On Thursday, March 31, 2022 at 5:17:08 PM UTC-5, BGB wrote:
>> On 3/31/2022 9:34 AM, Marcus wrote:
>>> On 2022-03-31, Paul A. Clayton wrote:
>
>> Pros/Cons:
>> FPGA has far more DSP48's than one actually needs, whereas doing a
>> single big multiplier is slow/expensive.
>>
>> So, instead of say, a big 64-bit multiplier, one might end up with
>> something that does, say:
>> One 32-bit widening multiply;
>> Two 32-bit narrow multiplies (SIMD);
>> Four 16-bit multiplies (SIMD).
> <
> A computer architect understands that one can segment a multiplier tree
> by preventing carries from one sub-tree to another. This architect is smart
> enough to understand that clipping the carries doe snot even add gates of
> delay to the multiplier tree's calculations. So all 3 above multipliers can be
> built from one incarnation of a SIMD sliding width multiplier.

Yeah. These can be done on an FPGA mostly by reorganizing inputs/outputs
to the DSP's and skipping adders.

Nevermind if the FPGA provides way more DSPs than I actually need, so
their isn't a huge incentive to conserve them. Big limiting factor is
more timing and the latency of the adder trees.

>>
>> Packed integer SIMD might just be normal ALU ops with carry propagation
>> disabled.
>>
> You also have to have a means to clip the carries in the adder that inevitably
> follows the tree.

For ALU, I did carry select, with SIMD following behaving as if the
carries were all 0 (or 1 for subtract).

>>
>> But, yeah, it is still true that one might end up with a few big units
>> that most everything else ends up being routed through, with different
>> sub-features being enabled or disabled.
>>
>>
>>
>> Elsewhere, recently, an argument came up over scaled-index addressing
>> (as a hypothetical RISC-V extension), where the assumption seemed to be
>> that for an ISA to offer scaled-index at all, then all operations would
>> necessarily need to have 3 GPR ports at all times.
> <
> That is a poor argument,.......proposed by somebody over there.

Pretty much.

One drawback of RISC-V is that many of its supporters seemingly get a
bit cultish, and part of their religion is apparently believing that:
Scaled index is expensive
It is not, more so if compared with other stuff in the ISA.
It offers no advantage over faking it using ALU ops.
I also disagree, from my own experience.

>>
>> This isn't really how it works in BJX2 though, since for the 6R3W
>> regfile, I usually have the Lane 3 ports sitting around not doing
>> anything else, I can leverage these for the ports needed for
>> Scaled-Index and similar.
> <
> Well, that and the index is almost always available on the forwarding path ...

Possibly.

>>
>> So, really, it would likely only be a significant burden for a 1-wide
>> core, which might need a 3R/1W regfile rather than a 2R1W regfile. But,
>> then again, as I see it, about as soon as the core is much larger than a
>> microcontroller, this whole debate becomes kinda moot (and if it were an
>> extension, a microcontroller could choose to omit it).
>>
> BTW I am getting 1.3-wide issue from a 3R1W RF in My66130 by taking
> fuller than standard advantage of other necessities in My 66000.

This is harder if using hard-wired port assignments.

In my case, the shared ports are effectively duplicated across the
relevant lanes, and where the register number/... comes from is left up
to the instruction decoder.

So, say:
1-wide: 3R1W
2-wide: 4R2W | 5R2W
3-wide: 6R3W

1-wide is cheap-ish, 2-wide a bit more, and 3-wide slightly more than
2-wide.

However, trying to go wider than 3, the situation seems to become "no
longer favorable".

Ironically, the Control Registers are a lot simpler (~ 1R1W), but due to
being mostly implemented using flip-flops and routed all over the place,
they seem to be a fair bit more expensive.

If I were doing it again, might try to come up with a way to reduce the
number or CRs and SPRs. But, the limitation is mostly how to best deal
with registers which don't really behave like normal GPRs (such as those
which are directly effected by or used to control architectural state).

Though, the main alternative used by SH and MSP430, namely putting a lot
of this stuff into MMIO, is not exactly all that likely to be much
better on this front (I didn't really feel that CPU internal state
belongs in MMIO).

>>
>> The index-scale + add doesn't really offer much cost otherwise over
>> doing the same thing with an immediate (and the AGU doesn't need to care
>> whether the displacement came from a register or immediate).
> <
> I would argue it is free to add (or insignificant burden)

Yeah.

Main relative difference here is "simple adder", vs "adder with ability
to shift the index by 0..3 bits".

>>
>> Or at least not enough to where I will be like, "Yeah, it doesn't matter
>> that these 15% or so of Load/Store ops will now need to be expressed as
>> multi-instruction sequences". This falls a bit outside my "small enough
>> that it doesn't matter" heuristic.
>>
> Even if the gain is only 2% it is still a win, and it is brain dead easy to AGEN
> And after AGEN there is no damage to the rest of the whole cache pipeline.

Pretty much.

This seems like a bit much of a hit to take for sake of philosophy or
aesthetic (or for keeping the listing "as small as possible").

Maybe makes sense for a microcontroller, but by the time one has an FPU
and similar, not having scaled-index no longer really makes much sense IMO.

>>
>> This is, admittedly, one of the bigger disagreements I had with RISC-V;
>> well, along with requiring a full-width multiplier (and a divide
>> instruction) for 'M' (as opposed to a narrower widening multiplier).
>>
>> Say, for example, if we could have 'Mlite' instead, which provides a
>> half-width widening multiply (with no divide instruction). The widening
>> multiply could be either Signed+Unsigned, or Unsigned-Only (since one
>> mostly uses unsigned-multiply to compose a larger multiply).
>>
>>
>> But, why would I argue for scaled-index but against full-width 64-bit
>> multiply?:
>> Scaled index is cheap and has a visible effect on performance;
>> 64-bit multiply is slow, expensive, and rarely used.
> <
> My 66100 (the smallest possible pipelined implementation) has the
> full sized multiplier 64×64 so as to do i× in 4 cycles as well as FMAC
> in 4 cycles. The trick, here, is that I run the multiplier pipeline at DIV-2.
> When the multiplier is not busy, it can accept an instruction on any
> clock, when it is busy multiplying it can accept a multiply every other
> clock. This gets rid of either 2048 or 4096 flip-flops in the multiplier
> tree (Athloon+Opteron) without losing performance because the
> rest of the implementation cannot feed the multiplier continuously.
> <
> The flip-flops 2 :: 3-input adders cells above the end of the tree
> were nearly as large of the multiplier tree itself !!!

OK.

>>
>> A hardware divide instruction could probably help with Dhrystone score,
>> but is less relevant in general (Dhrystone seems to significantly
>> over-represent the use of integer divide vs normal code).
>>
>>
>> One could (more affordably) provide full-width multiply (and divide) as
>> dedicated shift-check-add / shift-check-subtract logic (stalling the
>> pipeline until done), but these would be fairly slow.
> <
> This is standard practice today.

Apparently, some people are providing a pipelined small multiplier
(MULW), while using Shift-Add/Shift-Sub unit for 64-bit MUL/DIV.

I guess, comparing ISAs here (RISC-V vs BJX2):
MUL / MULS.Q (UI,1)
NE / MULU.Q (UI,1)
MULH / NE,1 (64x64->64 High-Bits Signed)
MULHU / NE,1 (64x64->64 High-Bits Unsigned)
MULW / MULS.L (32*32->32, Sign-Extend)
NE / MULU.L (32*32->32, Zero-Extend)
NE / DMULS.L (32*32->64, Signed)
NE / DMULU.L (32*32->64, Unsigned)

DIV / NE,1
DIVU / NE,1
DIVW / NE,1
DIVUW / NE,1

NE: Non-Encodable / Not-Defined
UI: Encodings defined, but unimplemented.

1: These cases could be mapped to a Slow Shift-Add/Sub unit.

But, I guess maybe it could make sense to add a slow MUL/DIV unit (could
also allow the dual-ISA mode to support the 'M' extension).

The 'F' and 'D' extensions would still be a problem given the BJX2 FPU
is significantly different from the RISC-V FPU.

MitchAlsup wrote:
> On Friday, April 1, 2022 at 5:19:21 PM UTC-5, Stefan Monnier wrote:
>>> (There is also a difference that there is a much stronger focus on
>>> verification, because getting something wrong is very expensive.)
>> Tho, IIUC "verification" in the hardware world refers usually to what
>> the software world calls "testing". In the software world,
>> "verification" means the use of exhaustive tools such as theorem provers
>> or model checkers.
> <
> Err, no. HW verification is a suite of tests upon HW carried out with the
> hope that there is never a need for a field update (Pentium FDVI bug)
> <
> Essentially no SW ever gets tested to this degree of correctness.

And we can still get effects like the RowHammer attack where a
sufficiently non-random pattern of writes to a DRAM array can spill over
into neighboring cells!

I.e. it is probably impossible to detect all possible analog
interactions of this type, using signal propagation patterns as a GHz
scale radio transmitter, even without using DRAM for last level cache.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On 2022-04-05 1:52, MitchAlsup wrote:
> On Friday, April 1, 2022 at 5:19:21 PM UTC-5, Stefan Monnier wrote:
>>> (There is also a difference that there is a much stronger focus on
>>> verification, because getting something wrong is very expensive.)
>> Tho, IIUC "verification" in the hardware world refers usually to what
>> the software world calls "testing". In the software world,
>> "verification" means the use of exhaustive tools such as theorem provers
>> or model checkers.
> <
> Err, no. HW verification is a suite of tests upon HW carried out with the
> hope that there is never a need for a field update (Pentium FDVI bug)
> <

Perhaps you see "verification" as different from "formal verification",
which is what model-checkers and theorem-provers do?

Intel and others have applied formal verification to HW since the FDIV
bug, according to Wikipedia:

"In the aftermath of the bug and subsequent recall, there was a marked
increase in the use of formal verification of hardware floating point
operations across the semiconductor industry. Prompted by the discovery
of the bug, a technique applicable to the SRT algorithm called
"word-level model checking" was developed in 1996. Intel went on to use
formal verification extensively in the development of later CPU
architectures. In the development of the Pentium 4, symbolic trajectory
evaluation and theorem proving were used to find a number of bugs that
could have led to a similar recall incident had they gone undetected.
The first Intel microarchitecture to use formal verification as the
primary method of validation was Nehalem, developed in 2008."

(https://en.wikipedia.org/wiki/Pentium_FDIV_bug)

> Essentially no SW ever gets tested to this degree of correctness.

Certainly some SW is so tested and verified -- say, SW for nuclear
reactor control. And AIUI, formal verification of SW is increasing in
all high-reliability domains, much thanks to the slow but continuing
improvement of SW tools and computing power for eg. exhaustive model
checking.

On Tuesday, April 5, 2022 at 2:09:07 AM UTC-5, Terje Mathisen wrote:
> MitchAlsup wrote:
> > On Friday, April 1, 2022 at 5:19:21 PM UTC-5, Stefan Monnier wrote:
> >>> (There is also a difference that there is a much stronger focus on
> >>> verification, because getting something wrong is very expensive.)
> >> Tho, IIUC "verification" in the hardware world refers usually to what
> >> the software world calls "testing". In the software world,
> >> "verification" means the use of exhaustive tools such as theorem provers
> >> or model checkers.
> > <
> > Err, no. HW verification is a suite of tests upon HW carried out with the
> > hope that there is never a need for a field update (Pentium FDVI bug)
> > <
> > Essentially no SW ever gets tested to this degree of correctness.
> And we can still get effects like the RowHammer attack where a
> sufficiently non-random pattern of writes to a DRAM array can spill over
> into neighboring cells!
<
Yes, but I have a solution to this that I can't talk about just yet.
>
> I.e. it is probably impossible to detect all possible analog
> interactions of this type, using signal propagation patterns as a GHz
> scale radio transmitter, even without using DRAM for last level cache.
>
> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"