On 12/19/2022 1:18 AM, Thomas Koenig wrote:
> BGB <cr88192@gmail.com> schrieb:
>> On 12/18/2022 5:35 PM, Russell Wallace wrote:
>>> On Sunday, December 18, 2022 at 6:03:50 PM UTC, Anton Ertl wrote:
>>>> Russell Wallace <russell...@gmail.com> writes:
>>>>> I saw Fred Brooks in an interview remark that one of his few regrets about =
>>>>> the 8-bit byte was that 64-bit floating point is not quite enough for doubl=
>>>>> e precision.
>>>>
>>>> Those who made the IEEE 754 standard thought differently.
>>>
>>> Did they? Look at the resources Intel spent on getting 80-bit floating point into the 8087.
>>>
>>
>> For most use cases, 80 bit is overkill.
>>
>> Meanwhile:
>> 64-bit can do "nearly everything";
>> 32-bit is "usually sufficient";
>> 16-bit is "often sufficient", but more hit/miss.
>>
>>
>> I suspect likely Binary16 would have been far more popular had it been
>> popular 20 years earlier.
>>
>>
>> Not sure why Binary16 wasn't a thing in the 80s, given values were
>> (generally) smaller back then.
>
> For scientific or engineering computations, 32-bit floats are
> barely adequate, or they don't work at all. This is why a lot
> of scientific code is in 64-bit. 36-bit floats were much better.
>

But, the IBM PC was mostly intended for home and business users? ...

Or, I think it was like, the main PC was intended for business, PCjr for
home users, but the PCjr was a flop, people either going for the full PC
or the PC clones.

Granted, I guess it is possible Intel could have had markets for this
other than the IBM PC ?...

> And binary16 is only useful for very limited applications, like neural
> networks. Even calculating everyday geometries will get you into
> trouble.

They are usable IME for:
Neural Nets (*1);
Pixel calculations;
Audio filtering;
...

*1: Though, in my own testing here, it is necessary to fiddle with stuff
here to try to prevent the net from training itself in ways that values
will go out of range (in effect, the training algorithm also needs to
use Binary16).

For 3D geometry, they work well enough if the 3D model isn't too large
or doesn't require too much detail.

For something like a character model, no one is likely to notice.

For something like scene geometry, usually need a little more than this.
For things like the transformation matrices, one really needs full
single precision if possible.

But, with ~ 3 significant figures, they should be fairly widely
applicable to many sorts of problems (along with the common tendency of
people to express typical values as x.xx*10^y or similar).

And, also all of the people that use 3.14 as their standard value for
PI, 2.72 for E, ... And also often don't want answers that are much
longer than 2 or 3 digits.

Granted, the dynamic range for Binary16 isn't particularly large.

But, as noted, some things it wouldn't really work. For example, it
couldn't give cent-accurate values for adding up values on a typical
shopping receipt, ...

But, for many of the types of problems where one might otherwise use,
say, 13.3 fixed point or similar, Binary16 could have been be a usable
alternative.

More just a question of why Binary32 was seemingly seen as the minimum
here, in an era where aggressive cost optimization of pretty much
everything seemed like a sensible option.

Or, for x87, why it bothered with a bunch of complex operators (such as
FSIN and FCOS, ...), when presumably the FPU could have been simpler and
cheaper had they had people to do most of these in software ?...

Or, if Binary16 itself would have added cost, why not have had an
instruction that truncated the Binary32 format to 16 bits on store, and
padded it with with zeroes on load. Shouldn't have added too much
additional cost to whatever mechanism they were using to Load/Store
values to RAM.

BGB <cr88192@gmail.com> writes:
[8087]
>Granted, I guess it is possible Intel could have had markets for this
>other than the IBM PC ?...

Certainly.

Intel started the 8087 project in 1977 and the 8087 was launched in
1980, before the IBM PC (1981). And the IBM PC (and its clones) were
not an instant hit; e.g., in the early stages of the 386 project
(started 1982) the 386 was just a minor project.

Interestingly, there were earlier math coprocessors: The Am9511 (1977,
fixed point, but with, e.g., trigonometric functions) and Am9512
(1979, floating-point, compatible with draft 754; binary32 and
binary64), which Intel licensed as 8231 and 8232.

>Or, for x87, why it bothered with a bunch of complex operators (such as
>FSIN and FCOS, ...), when presumably the FPU could have been simpler and
>cheaper had they had people to do most of these in software ?...

CISC was in full swing in 1977, RISC at the time limited to the IBM
801 group.

There are also cases where a more complex instruction allows later
improved hardware implementations that are not possible in software,
although I think this kind of thinking was far more common in those
times than waranted, and it's unclear to me whether the 8087 or its
successors ever made use of that (e.g., additional precision in
intermediate results).

But thinking about it again (after reading up on the earlier
co-processors), I think the main reason was that the 8087 ran
asynchronously to the 8086/8088. So the benefit of FSIN was that the
CPU could start an FSIN operation, then do a lot of other stuff, then
deal with the result of the FSIN; by contrast, with an FSIN software
function, you call it, and the CPU is blocked until it is finished.

>Or, if Binary16 itself would have added cost, why not have had an
>instruction that truncated the Binary32 format to 16 bits on store, and
>padded it with with zeroes on load.

The 8087 converts everything into its internal 80+-bit format on
loading and back on storing (plus some optional rounding of the
mantissa to 53 or 23 bits on computations). Binary16 would certainly
have added a cost, and, at the time, provided no benefit. People were
not interesting in FP16 until a few years ago.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

Russell Wallace <russell.wallace@gmail.com> writes:
>On Sunday, December 18, 2022 at 6:03:50 PM UTC, Anton Ertl wrote:
>> Russell Wallace <russell...@gmail.com> writes:=20
>> >I saw Fred Brooks in an interview remark that one of his few regrets abo=
>ut =3D=20
>> >the 8-bit byte was that 64-bit floating point is not quite enough for do=
>ubl=3D=20
>> >e precision.=20
>>=20
>> Those who made the IEEE 754 standard thought differently.=20
>
>Did they?

According to
<https://en.wikipedia.org/wiki/IEEE-754#Basic_and_interchange_formats>:

|The binary32 and binary64 formats are the single and double formats of
|IEEE 754-1985 respectively.

>> A 10-bit data bus is 25% more expensive than an 8-bit data bus. At=20
>> the same time, with your 20/40-bit instructions, a 10-bit bus requires=20
>> 2-4 bus cycles to load an instruction, which reduces performance=20
>> significantly.=20
>
>Compared to what?

Compared to having 10-bit instructions (plus immediate operands).
There is a reason why instruction sets with many implicit registers
like the 6502 and 8086 were successful when we still had 8-bit busses
and no I-cache.

>The 2010 can add a pair of 20-bit numbers in two cycles. =

A 2010 with 10-bit instructions can take one cycle for an addition (if
you have a 20-bit ALU).

>The 6502 transistor count is much more c=
>onstrained. (For good reason; they were explicitly aimed at a minimum-cost =
>CPU for embedded applications.)

Maybe. But it was hugely successful in general-purpose computers for
the cost reason.

>I don't see a lot of room to do better with=
>in that transistor count.

I do. And actually it's a small-enough project that a dedicated
hoppyist could achieve it in reasonable time. But I am not taking it
on, proving that point is not important enough to me.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

According to Thomas Koenig <tkoenig@netcologne.de>:
>Russell Wallace <russell.wallace@gmail.com> schrieb:
>
>> I saw Fred Brooks in an interview remark that one of his few
>> regrets about the 8-bit byte was that 64-bit floating point is
>> not quite enough for double precision.
>
>No regrets about the 32-bit floating point real that was introduced?
>IBM certainly knew better, from the 704.
>
>Chosing the exponent range of real and double to coincide was not
>a great decision, either.

Indeed, but none of that was as bad as doing hex normalization, which
precluded a hidden bit, and no rounding. That lost three bits of
accuracy on each operation. They retrofitted guard digits in the
field which helped, but not enough.

It is really strange that they did all sorts of simulations but
somehow missed the key fact that leading float digits are distributed
geometrically, not linearly. It's not like it's hard to figure out.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

According to Scott Lurndal <slp53@pacbell.net>:
>MitchAlsup <MitchAlsup@aol.com> writes:
>>On Sunday, December 18, 2022 at 2:29:26 PM UTC-6, John Levine wrote:
>>> According to Russell Wallace <russell...@gmail.com>:
>>> >In that scenario, what's the best that can be done? I don't think a 32-bit RISC microprocessor can be made with the
>>> >process technology of 1974, and even if it could, I suspect it would be too expensive to succeed in the market.
>>> >
>>> >My idea is to expand the byte to 10 bits, and build a CPU with a 20-bit word size and linear address space, and a 10-bit
>>> >data bus. I'll call this architecture the 2010.
>>> Repeat after me: There Are No New Bad Ideas ...
>>>
>>> In the 1970s, BBN built the C/30 to replace the 16-bit Honeywell 316
>>> machines used in Arpanet IMPs because Honeywell stopped making 316s.
>>> The C/30 was microprogrammed to emulate a 316 so it could run the
>>> existing IMP code. BBN believed (wrongly as it turned out) that they
>>> were in the computer business so they expanded it to a larger C/70
>>> just as you proposed, with 10 bit bytes and a 20 bit address space.
>>>
>>> It was a complete failure even though it was quite fast and phyiscally
>>> worked fine. I don't know all the reasons it failed but I do recall
>>> talking to one of the people working on it who told me that porting
>>> code from 8-bit byte systems was slow and painful because there were
>>> so many implicit assumptions about the byte size.
>>>
>>> There were plenty of machines with 8-bit bytes and 16 bit addresses
>>> that used a variety of mapping and bank switching hacks to handle
>>> more memory. The PDP-11 got up to 22 bit physical addresses before
>>> it died, and the 80286 had 24 bit addresses.
>>>
>>> When IBM chose 8 bit bytes in 1964 for S/360 they really hit a
>>> sweet spot. I don't think there was ever another new architecture
>>> that didn't have power of 2 addresses other than some special
>>> purpose DSPs.
>><
>>PDP-8 was surely post 360.
>
>PDP-8 (released 1965, designed 63-64) was based on the 12-bit LINC from 1962.

The PDP-8 was a reimplemented PDP-5, which in turn was a PDP-4 cut
down from 18 bits to 12. According to Gordon Bell, the PDP-4 and -5
were influenced by the LINC. But they were different enough that DEC
later made the LINC-8 which was basically a LINC and a PDP-8 lashed
together and sharing memory, and later the much smaller and cheaper
PDP-12 which had a single CPU that could switch between PDP-8 and LINC
modes. The biggest legacy of the LINC was LINCtape which in slightly
modified form became DECtape, the block addressiable tape we all used
before there were floppy disks.

>>PDP-6 was surely post 1964.
>
>The 1963 36-bit PDP-6 followed the 18-bit PDP-1 from 1959.

The PDP-6 was nothing like the PDP-1. The PDP-1 had a single
accumulator and an instruction set that was clearly the predecessor of
the -4, the -5 and the rest of DEC's 18 and 12 bit machines. The PDP-6
had 16 registers and a large orthogonal instruction set. Legend says
that the PDP-6 was designed at the MIT Tech Model Railroading Club.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

On 12/19/2022 12:10 PM, Anton Ertl wrote:
> BGB <cr88192@gmail.com> writes:
> [8087]
>> Granted, I guess it is possible Intel could have had markets for this
>> other than the IBM PC ?...
>
> Certainly.
>
> Intel started the 8087 project in 1977 and the 8087 was launched in
> 1980, before the IBM PC (1981). And the IBM PC (and its clones) were
> not an instant hit; e.g., in the early stages of the 386 project
> (started 1982) the 386 was just a minor project.
>

OK, wasn't aware the 8087 predated the PC. Sort of thought it was a
late-stage add-on.

> Interestingly, there were earlier math coprocessors: The Am9511 (1977,
> fixed point, but with, e.g., trigonometric functions) and Am9512
> (1979, floating-point, compatible with draft 754; binary32 and
> binary64), which Intel licensed as 8231 and 8232.
>

Yeah, don't know much about them.

>> Or, for x87, why it bothered with a bunch of complex operators (such as
>> FSIN and FCOS, ...), when presumably the FPU could have been simpler and
>> cheaper had they had people to do most of these in software ?...
>
> CISC was in full swing in 1977, RISC at the time limited to the IBM
> 801 group.
>
> There are also cases where a more complex instruction allows later
> improved hardware implementations that are not possible in software,
> although I think this kind of thinking was far more common in those
> times than waranted, and it's unclear to me whether the 8087 or its
> successors ever made use of that (e.g., additional precision in
> intermediate results).
>

Dunno there.

I think the eventual result was that SSE came along, and then by x86-64
people had mostly abandoned x87 in favor of SSE, and typically doing the
math operations in software rather than using x87 ops (given SSE doesn't
have a lot of this stuff either).

If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
leave nearly everything else to software.

Did eventually re-add FDIV, in my case, which is accurate but not
particularly fast (ended up routing it though a Shift-ADD divider,
noting as it didn't seem like too huge of a jump to make it do FP divide
in addition to integer divide).

I had also re-added an FSQRT instruction, which is neither particularly
fast nor accurate. No real advantage over doing it in software.

I have noted previously that N-R seemingly is unable to converge the
last 4 bits or so of the mantissa. Seemingly when it gets to this point,
it just sorta jumps around and doesn't actually reach the answer (it
seems one would need some sub-ULP bits to converge on an exact answer here).

> But thinking about it again (after reading up on the earlier
> co-processors), I think the main reason was that the 8087 ran
> asynchronously to the 8086/8088. So the benefit of FSIN was that the
> CPU could start an FSIN operation, then do a lot of other stuff, then
> deal with the result of the FSIN; by contrast, with an FSIN software
> function, you call it, and the CPU is blocked until it is finished.
>

Probably true.

There is no async aspect in my case, as the FPU is effectively glued
onto the same logic as the integer ISA.

>> Or, if Binary16 itself would have added cost, why not have had an
>> instruction that truncated the Binary32 format to 16 bits on store, and
>> padded it with with zeroes on load.
>
> The 8087 converts everything into its internal 80+-bit format on
> loading and back on storing (plus some optional rounding of the
> mantissa to 53 or 23 bits on computations). Binary16 would certainly
> have added a cost, and, at the time, provided no benefit. People were
> not interesting in FP16 until a few years ago.
>

Probably true enough.

I had been using it in my various 3D engines, mostly in the context of
vertex arrays and similar (was supported by OpenGL via GL_HALF_FLOAT and
similar).

In my BJX2 project, its role is somewhat expanded, but it is still a
non-standard extension as far as C goes.

In my case, I use non-standard conversion rules, which can save some
cost but are a little funky. Cheaper cases don't bother with rounding
(since rounding is one of the more expensive parts of a narrowing
conversion).

Say:
F32 -> F16:
{ val[31],
(val[30]==val[29]) ||
((val[29:27]!=3'b000) &&
(val[29:27]!=3'b111)) ?
(val[30] ? 5'h1F : 5'h00) :
{ val[30], val[26:23] },
val[22:13] }

F16 -> F32:
{ val[15:14],
((val[14] || (val[14:10]==5'h00)) && !(val[14:10]==5'h1F)) ?
3:b000 : 3'b111,
val[9:0], 13'h00 }

....

Another semi-popular option is "BFloat16", or "S.E8.F7" (essentially a
truncated Binary32), but this is less well supported in my case.

For hacking something onto the x87, this could make more sense. Could be
faked in software, but on x86, this would mean wrangling the values
around in memory.

Say (assuming a GAS style syntax):
mov ax, [bx]
mov [sp+2], ax
xor ax, ax
mov [sp+2], ax
fld dword ptr [sp+0]

As first, wrote it in GAS style syntax, but then noted Intel style would
be more era appropriate...

> - anton

On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> On 12/19/2022 12:10 PM, Anton Ertl wrote:
> > BGB <cr8...@gmail.com> writes:
> > [8087]
> >> Granted, I guess it is possible Intel could have had markets for this
> >> other than the IBM PC ?...
> >
> > Certainly.
> >
> > Intel started the 8087 project in 1977 and the 8087 was launched in
> > 1980, before the IBM PC (1981). And the IBM PC (and its clones) were
> > not an instant hit; e.g., in the early stages of the 386 project
> > (started 1982) the 386 was just a minor project.
> >
> OK, wasn't aware the 8087 predated the PC. Sort of thought it was a
> late-stage add-on.
> > Interestingly, there were earlier math coprocessors: The Am9511 (1977,
> > fixed point, but with, e.g., trigonometric functions) and Am9512
> > (1979, floating-point, compatible with draft 754; binary32 and
> > binary64), which Intel licensed as 8231 and 8232.
> >
> Yeah, don't know much about them.
> >> Or, for x87, why it bothered with a bunch of complex operators (such as
> >> FSIN and FCOS, ...), when presumably the FPU could have been simpler and
> >> cheaper had they had people to do most of these in software ?...
> >
> > CISC was in full swing in 1977, RISC at the time limited to the IBM
> > 801 group.
> >
> > There are also cases where a more complex instruction allows later
> > improved hardware implementations that are not possible in software,
> > although I think this kind of thinking was far more common in those
> > times than waranted, and it's unclear to me whether the 8087 or its
> > successors ever made use of that (e.g., additional precision in
> > intermediate results).
> >
> Dunno there.
>
>
> I think the eventual result was that SSE came along, and then by x86-64
> people had mostly abandoned x87 in favor of SSE, and typically doing the
> math operations in software rather than using x87 ops (given SSE doesn't
> have a lot of this stuff either).
>
> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> leave nearly everything else to software.
>
>
> Did eventually re-add FDIV, in my case, which is accurate but not
> particularly fast (ended up routing it though a Shift-ADD divider,
> noting as it didn't seem like too huge of a jump to make it do FP divide
> in addition to integer divide).
>
> I had also re-added an FSQRT instruction, which is neither particularly
> fast nor accurate. No real advantage over doing it in software.
>
> I have noted previously that N-R seemingly is unable to converge the
> last 4 bits or so of the mantissa. Seemingly when it gets to this point,
> it just sorta jumps around and doesn't actually reach the answer (it
> seems one would need some sub-ULP bits to converge on an exact answer here).
> > But thinking about it again (after reading up on the earlier
> > co-processors), I think the main reason was that the 8087 ran
> > asynchronously to the 8086/8088. So the benefit of FSIN was that the
> > CPU could start an FSIN operation, then do a lot of other stuff, then
> > deal with the result of the FSIN; by contrast, with an FSIN software
> > function, you call it, and the CPU is blocked until it is finished.
> >
> Probably true.
>
> There is no async aspect in my case, as the FPU is effectively glued
> onto the same logic as the integer ISA.
> >> Or, if Binary16 itself would have added cost, why not have had an
> >> instruction that truncated the Binary32 format to 16 bits on store, and
> >> padded it with with zeroes on load.
> >
> > The 8087 converts everything into its internal 80+-bit format on
> > loading and back on storing (plus some optional rounding of the
> > mantissa to 53 or 23 bits on computations). Binary16 would certainly
> > have added a cost, and, at the time, provided no benefit. People were
> > not interesting in FP16 until a few years ago.
> >
> Probably true enough.
>
> I had been using it in my various 3D engines, mostly in the context of
> vertex arrays and similar (was supported by OpenGL via GL_HALF_FLOAT and
> similar).
>
> In my BJX2 project, its role is somewhat expanded, but it is still a
> non-standard extension as far as C goes.
>
>
> In my case, I use non-standard conversion rules, which can save some
> cost but are a little funky. Cheaper cases don't bother with rounding
> (since rounding is one of the more expensive parts of a narrowing
> conversion).
>
> Say:
> F32 -> F16:
> { val[31],
> (val[30]==val[29]) ||
> ((val[29:27]!=3'b000) &&
> (val[29:27]!=3'b111)) ?
> (val[30] ? 5'h1F : 5'h00) :
> { val[30], val[26:23] },
> val[22:13] }
>
> F16 -> F32:
> { val[15:14],
> ((val[14] || (val[14:10]==5'h00)) && !(val[14:10]==5'h1F)) ?
> 3:b000 : 3'b111,
> val[9:0], 13'h00 }
>
> ...
>
>
> Another semi-popular option is "BFloat16", or "S.E8.F7" (essentially a
> truncated Binary32), but this is less well supported in my case.
>
> For hacking something onto the x87, this could make more sense. Could be
> faked in software, but on x86, this would mean wrangling the values
> around in memory.
>
> Say (assuming a GAS style syntax):
> mov ax, [bx]
> mov [sp+2], ax
> xor ax, ax
> mov [sp+2], ax
> fld dword ptr [sp+0]
>
> As first, wrote it in GAS style syntax, but then noted Intel style would
> be more era appropriate...
>
>
> > - anton
>If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>leave nearly everything else to software.

Bare minimum is good if transistors are limited. Otherwise, might as well do
other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
in one instruction with one normalize and round. Might be less hardware than
separate FADD/FSUB/FMUL. I like to include a handful more instructions. Like
FCMP, FNEG, FABS, FSCALE. FSCALE works great with decimal float in numeric
to string conversions for multiplying by 10 and dividing by 10. There is a
minimum level of supported functions required for IEEE maybe worth looking
into.
I happen to be working on DFP right now for a 68k compatible. I have got 96-bit
triple precision working, now I am thinking of reducing that to 64-bit double
precision. I think it does not make a lot of sense to support lower precision
decimal float. Better to use binary floats for lower precision > 64 bits.

BGB <cr88192@gmail.com> writes:
>I think the eventual result was that SSE came along, and then by x86-64
>people had mostly abandoned x87 in favor of SSE, and typically doing the
>math operations in software rather than using x87 ops (given SSE doesn't
>have a lot of this stuff either).

They could use the 80387 instructions if they provide a benefit; they
are still there. However, I just checked this, and on a Skylake with
glibc-2.31 sin() calls __sin_fma, and I see AVX-128 instructions in
the first part of this code, so it probably does not invoke FSIN.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >leave nearly everything else to software.
> Bare minimum is good if transistors are limited. Otherwise, might as well do
> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> in one instruction with one normalize and round. Might be less hardware than
> separate FADD/FSUB/FMUL.

Compare apples to apples.
FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
But it is bigger than FPU that does either FADD or FMUL per n clocks.
Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
there exist a potential for HW savings.

> I like to include a handful more instructions. Like
> FCMP, FNEG, FABS, FSCALE.

I see no reason to not provide full set of common logicals AND, OR, XOR, also NAND or ANDN.
HW cost is close to 0, usefulness in absence of shifts and of integer add/sub is not great, but above 0.

> FSCALE works great with decimal float in numeric
> to string conversions for multiplying by 10 and dividing by 10. There is a
> minimum level of supported functions required for IEEE maybe worth looking
> into.

Well, when BGB said "If it were me", he almost certainly didn't mean decimal FP.

> I happen to be working on DFP right now for a 68k compatible. I have got 96-bit
> triple precision working, now I am thinking of reducing that to 64-bit double
> precision. I think it does not make a lot of sense to support lower precision
> decimal float. Better to use binary floats for lower precision > 64 bits.

There are little reasons to include DFP hardware at all, but if nevertheless you
decided to include DFP then it has to be IEEE-754 DFP.

On Tuesday, December 20, 2022 at 5:47:22 AM UTC-5, Michael S wrote:
> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> > On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> > >If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> > >leave nearly everything else to software.
> > Bare minimum is good if transistors are limited. Otherwise, might as well do
> > other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> > in one instruction with one normalize and round. Might be less hardware than
> > separate FADD/FSUB/FMUL.
> Compare apples to apples.
> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> there exist a potential for HW savings.
> > I like to include a handful more instructions. Like
> > FCMP, FNEG, FABS, FSCALE.
> I see no reason to not provide full set of common logicals AND, OR, XOR, also NAND or ANDN.
> HW cost is close to 0, usefulness in absence of shifts and of integer add/sub is not great, but above 0.

I have these on float-registers in Thor2023. Previously I had unified register files, so
all ops available for integers could also be used on floats. But I wanted to reduce
the size of the register spec field so now I am using separate integer and float
registers. Cannot do much about the 68k core though. I suppose some of the reserved
opcodes could be used.

> > FSCALE works great with decimal float in numeric
> > to string conversions for multiplying by 10 and dividing by 10. There is a
> > minimum level of supported functions required for IEEE maybe worth looking
> > into.
> Well, when BGB said "If it were me", he almost certainly didn't mean decimal FP.
> > I happen to be working on DFP right now for a 68k compatible. I have got 96-bit
> > triple precision working, now I am thinking of reducing that to 64-bit double
> > precision. I think it does not make a lot of sense to support lower precision
> > decimal float. Better to use binary floats for lower precision > 64 bits.
> There are little reasons to include DFP hardware at all, but if nevertheless you
> decided to include DFP then it has to be IEEE-754 DFP.

The decimal floating point uses the IEEE-754 format. I would like to include the
minimum set of operations in hardware. I am not sure about the DPD digits. I am
using Mike Cowlishaw’s encoding / decoding.

See: http://speleotrove.com/decimal/DPDecimal.html

On 12/20/2022 4:47 AM, Michael S wrote:
> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>>> leave nearly everything else to software.
>> Bare minimum is good if transistors are limited. Otherwise, might as well do
>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
>> in one instruction with one normalize and round. Might be less hardware than
>> separate FADD/FSUB/FMUL.
>
> Compare apples to apples.
> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> there exist a potential for HW savings.
>

Yeah.

Separate FADD/FSUB + FMUL units were cheaper in my case, due in large
part to the mantissa issues (FMA needing a much larger mantissa).

Also, an FMA would end up needing nearly twice the latency, so isn't too
big of an advantage in this sense over separate MUL/ADD.

I did end up with rounding-mode support on the main FPU, though my
low-precision unit (SIMD mostly) is hard-wired for Truncate. Hard-wiring
truncate is the cheapest option.

>> I like to include a handful more instructions. Like
>> FCMP, FNEG, FABS, FSCALE.
>
> I see no reason to not provide full set of common logicals AND, OR, XOR, also NAND or ANDN.
> HW cost is close to 0, usefulness in absence of shifts and of integer add/sub is not great, but above 0.
>

Yeah, I missed mentioning FCMP/FNEG/FABS, these ones are useful to have.

FCMP is basically a hacked integer compare, so not too bad. FNEG/FABS
are just twiddling the sign bit.

Format conversion ops are also nice to have, ...

Just would probably leave out FDIV and FSQRT (at least beyond cheap
approximate versions).

>> FSCALE works great with decimal float in numeric
>> to string conversions for multiplying by 10 and dividing by 10. There is a
>> minimum level of supported functions required for IEEE maybe worth looking
>> into.
>
> Well, when BGB said "If it were me", he almost certainly didn't mean decimal FP.
>

Yes, true.

Decimal FP is the extreme opposite of "cheap FPU"...

>> I happen to be working on DFP right now for a 68k compatible. I have got 96-bit
>> triple precision working, now I am thinking of reducing that to 64-bit double
>> precision. I think it does not make a lot of sense to support lower precision
>> decimal float. Better to use binary floats for lower precision > 64 bits.
>
> There are little reasons to include DFP hardware at all, but if nevertheless you
> decided to include DFP then it has to be IEEE-754 DFP.

Yeah.

I had at one point looked into trying to "borrow" the 128-bit format
used by .NET, but then decided against this and just sorta went with
Binary128 for the 128-bit floating point, as it ended up being both
faster and had more precision.

The MS format was, IIRC:
First 3 DWORDs, hold an integer value from 000000000 to 999999999.
Final DWORD, IIRC, holds a few more digits and the sign and exponent.

For a pure software implementation, almost makes more sense than either
of the IEEE formats, since it can be implemented relatively
straightforwardly with 32-bit integer operations.

If, albeit, it doesn't make the most efficient use of bits.

....

John Levine <johnl@taugh.com> writes:
>When IBM chose 8 bit bytes in 1964 for S/360 they really hit a
>sweet spot. I don't think there was ever another new architecture
>that didn't have power of 2 addresses other than some special
>purpose DSPs.

My thinking has been that the 16-bit-ness of the PDP-11, NOVA and
other 16-bit minis was due to the 4-bit bit-slice ALUs like the 74181
and the memory sizes of the time (12 address bits were too little, 20
bits more than necessary).

Many of these machines (e.g., the NOVA) are word-addressed, so 8-bit
bytes played a minor role.

And even the byte-addressed PDP-11 used little-endian byte order
(unlike IBM) and ASCII (unlike IBM), so apparently they did not feel a
need to be compatible with IBM's choices.

The IBM 1130 was too early for 4-bit bit-slice ALUs, and it was
word-addressed. Apparently it chose 16 bits due to the planned memory
size (4K-32K words).

Intel chose 8 bits for the 8008, because it was intended for a
terminal for IBM.

It's not obvious why Motorola chose 8 bits for the 6800. Was
compatibility with the 8-bit world already important? Was, e.g., 14
bits for the address already too close for comfort? Was it support
for BCD arithmetics (the 6800 has a DAA instruction)?

Maybe in the end we have to thank BCD arithmetic for the swift and
complete victory of powers-of-2 byte and word sizes. IIRC it was
also an important factor in IBM's decision to go for 8-bit bytes.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Tuesday, December 20, 2022 at 3:45:33 AM UTC-6, robf...@gmail.com wrote:
> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> > On 12/19/2022 12:10 PM, Anton Ertl wrote:
> > > BGB <cr8...@gmail.com> writes:
> > > [8087]
> > >> Granted, I guess it is possible Intel could have had markets for this
> > >> other than the IBM PC ?...
> > >
> > > Certainly.
> > >
> > > Intel started the 8087 project in 1977 and the 8087 was launched in
> > > 1980, before the IBM PC (1981). And the IBM PC (and its clones) were
> > > not an instant hit; e.g., in the early stages of the 386 project
> > > (started 1982) the 386 was just a minor project.
> > >
> > OK, wasn't aware the 8087 predated the PC. Sort of thought it was a
> > late-stage add-on.
> > > Interestingly, there were earlier math coprocessors: The Am9511 (1977,
> > > fixed point, but with, e.g., trigonometric functions) and Am9512
> > > (1979, floating-point, compatible with draft 754; binary32 and
> > > binary64), which Intel licensed as 8231 and 8232.
> > >
> > Yeah, don't know much about them.
> > >> Or, for x87, why it bothered with a bunch of complex operators (such as
> > >> FSIN and FCOS, ...), when presumably the FPU could have been simpler and
> > >> cheaper had they had people to do most of these in software ?...
> > >
> > > CISC was in full swing in 1977, RISC at the time limited to the IBM
> > > 801 group.
> > >
> > > There are also cases where a more complex instruction allows later
> > > improved hardware implementations that are not possible in software,
> > > although I think this kind of thinking was far more common in those
> > > times than waranted, and it's unclear to me whether the 8087 or its
> > > successors ever made use of that (e.g., additional precision in
> > > intermediate results).
> > >
> > Dunno there.
> >
> >
> > I think the eventual result was that SSE came along, and then by x86-64
> > people had mostly abandoned x87 in favor of SSE, and typically doing the
> > math operations in software rather than using x87 ops (given SSE doesn't
> > have a lot of this stuff either).
> >
> > If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> > leave nearly everything else to software.
> >
> >
> > Did eventually re-add FDIV, in my case, which is accurate but not
> > particularly fast (ended up routing it though a Shift-ADD divider,
> > noting as it didn't seem like too huge of a jump to make it do FP divide
> > in addition to integer divide).
> >
> > I had also re-added an FSQRT instruction, which is neither particularly
> > fast nor accurate. No real advantage over doing it in software.
> >
> > I have noted previously that N-R seemingly is unable to converge the
> > last 4 bits or so of the mantissa. Seemingly when it gets to this point,
> > it just sorta jumps around and doesn't actually reach the answer (it
> > seems one would need some sub-ULP bits to converge on an exact answer here).
> > > But thinking about it again (after reading up on the earlier
> > > co-processors), I think the main reason was that the 8087 ran
> > > asynchronously to the 8086/8088. So the benefit of FSIN was that the
> > > CPU could start an FSIN operation, then do a lot of other stuff, then
> > > deal with the result of the FSIN; by contrast, with an FSIN software
> > > function, you call it, and the CPU is blocked until it is finished.
> > >
> > Probably true.
> >
> > There is no async aspect in my case, as the FPU is effectively glued
> > onto the same logic as the integer ISA.
> > >> Or, if Binary16 itself would have added cost, why not have had an
> > >> instruction that truncated the Binary32 format to 16 bits on store, and
> > >> padded it with with zeroes on load.
> > >
> > > The 8087 converts everything into its internal 80+-bit format on
> > > loading and back on storing (plus some optional rounding of the
> > > mantissa to 53 or 23 bits on computations). Binary16 would certainly
> > > have added a cost, and, at the time, provided no benefit. People were
> > > not interesting in FP16 until a few years ago.
> > >
> > Probably true enough.
> >
> > I had been using it in my various 3D engines, mostly in the context of
> > vertex arrays and similar (was supported by OpenGL via GL_HALF_FLOAT and
> > similar).
> >
> > In my BJX2 project, its role is somewhat expanded, but it is still a
> > non-standard extension as far as C goes.
> >
> >
> > In my case, I use non-standard conversion rules, which can save some
> > cost but are a little funky. Cheaper cases don't bother with rounding
> > (since rounding is one of the more expensive parts of a narrowing
> > conversion).
> >
> > Say:
> > F32 -> F16:
> > { val[31],
> > (val[30]==val[29]) ||
> > ((val[29:27]!=3'b000) &&
> > (val[29:27]!=3'b111)) ?
> > (val[30] ? 5'h1F : 5'h00) :
> > { val[30], val[26:23] },
> > val[22:13] }
> >
> > F16 -> F32:
> > { val[15:14],
> > ((val[14] || (val[14:10]==5'h00)) && !(val[14:10]==5'h1F)) ?
> > 3:b000 : 3'b111,
> > val[9:0], 13'h00 }
> >
> > ...
> >
> >
> > Another semi-popular option is "BFloat16", or "S.E8.F7" (essentially a
> > truncated Binary32), but this is less well supported in my case.
> >
> > For hacking something onto the x87, this could make more sense. Could be
> > faked in software, but on x86, this would mean wrangling the values
> > around in memory.
> >
> > Say (assuming a GAS style syntax):
> > mov ax, [bx]
> > mov [sp+2], ax
> > xor ax, ax
> > mov [sp+2], ax
> > fld dword ptr [sp+0]
> >
> > As first, wrote it in GAS style syntax, but then noted Intel style would
> > be more era appropriate...
> >
> >
> > > - anton
> >If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >leave nearly everything else to software.
<
> Bare minimum is good if transistors are limited. Otherwise, might as well do
> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> in one instruction with one normalize and round. Might be less hardware than
> separate FADD/FSUB/FMUL.
<
It is fair to say FMAC is not much more HW but it is not fair to say it is less
HW, it is not. The augend adder needs to be 52×3+50 bits long, and the
normalize shifter is bigger.
<
> I like to include a handful more instructions. Like
> FCMP, FNEG, FABS, FSCALE. FSCALE works great with decimal float in numeric
<
With a few special cases, FCMP is an integer compare. The others are 1 cycle
calculations. Probably not wise to run them through the 3-4-5 cycle FMAC unit.
<
> to string conversions for multiplying by 10 and dividing by 10. There is a
> minimum level of supported functions required for IEEE maybe worth looking
> into.
> I happen to be working on DFP right now for a 68k compatible. I have got 96-bit
> triple precision working, now I am thinking of reducing that to 64-bit double
> precision. I think it does not make a lot of sense to support lower precision
> decimal float. Better to use binary floats for lower precision > 64 bits.

Michael S wrote:
> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>>> leave nearly everything else to software.
>> Bare minimum is good if transistors are limited. Otherwise, might as well do
>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
>> in one instruction with one normalize and round. Might be less hardware than
>> separate FADD/FSUB/FMUL.
>
> Compare apples to apples.
> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> there exist a potential for HW savings.
>
For me, a sufficient reason to include FMA is the fact that with the FMA
wide normalizer we gain the ability to handle subnormal inputs and
outputs at zero cycle cost and very marginal gate cost.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Tuesday, December 20, 2022 at 1:47:45 PM UTC-6, Terje Mathisen wrote:
> Michael S wrote:
> > On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> >> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >>> leave nearly everything else to software.
> >> Bare minimum is good if transistors are limited. Otherwise, might as well do
> >> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> >> in one instruction with one normalize and round. Might be less hardware than
> >> separate FADD/FSUB/FMUL.
> >
> > Compare apples to apples.
> > FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> > But it is bigger than FPU that does either FADD or FMUL per n clocks.
> > Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> > Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> > plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> > there exist a potential for HW savings.
> >
> For me, a sufficient reason to include FMA is the fact that with the FMA
> wide normalizer we gain the ability to handle subnormal inputs and
> outputs at zero cycle cost and very marginal gate cost.
<
For me, a sufficient reason to include FMAC is that IEEE 754 demands it.
The rest, as they say, is gravy.
>
> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

On 12/20/2022 4:10 AM, Anton Ertl wrote:
> BGB <cr88192@gmail.com> writes:
>> I think the eventual result was that SSE came along, and then by x86-64
>> people had mostly abandoned x87 in favor of SSE, and typically doing the
>> math operations in software rather than using x87 ops (given SSE doesn't
>> have a lot of this stuff either).
>
> They could use the 80387 instructions if they provide a benefit; they
> are still there. However, I just checked this, and on a Skylake with
> glibc-2.31 sin() calls __sin_fma, and I see AVX-128 instructions in
> the first part of this code, so it probably does not invoke FSIN.
>

It seems that once x86-64 came along, and code moved over to it, the use
of x87 ops was typically dropped entirely (even if one could still in
premise make use of x87 for FSIN/FCOS/etc).

> - anton

On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
> Michael S wrote:
> > On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> >> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >>> leave nearly everything else to software.
> >> Bare minimum is good if transistors are limited. Otherwise, might as well do
> >> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> >> in one instruction with one normalize and round. Might be less hardware than
> >> separate FADD/FSUB/FMUL.
> >
> > Compare apples to apples.
> > FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> > But it is bigger than FPU that does either FADD or FMUL per n clocks.
> > Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> > Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> > plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> > there exist a potential for HW savings.
> >
> For me, a sufficient reason to include FMA is the fact that with the FMA
> wide normalizer we gain the ability to handle subnormal inputs and
> outputs at zero cycle cost and very marginal gate cost.
>

If this is indeed the fact.
I am unconvinced that it is true in all possible circumstances.
There are many sensible ways to design FMA-capable FPU.

> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

Russell Wallace wrote:
[snip]
> Suppose you could go back to the early seventies and design an architecture to try to outcompete the 8080 and become the long-term basis of the personal computer industry. You get to use everything we have learned between that day and this, with two caveats:
>
> 1. You only get to bring back ideas, not equipment, so it must be technically feasible to build the first implementation with the process technology of 1974.

You might be interested in the thread:
[just fun] Time travel destination choice with (micro)architecture
knowledge
(Google Groups link:
https://groups.google.com/g/comp.arch/c/XuxpaN5fTZI/m/n4y6HkwGBAAJ

On Tuesday, December 20, 2022 at 6:17:48 PM UTC-6, Michael S wrote:
> On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
> > Michael S wrote:
> > > On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> > >> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> > >>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> > >>> leave nearly everything else to software.
> > >> Bare minimum is good if transistors are limited. Otherwise, might as well do
> > >> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> > >> in one instruction with one normalize and round. Might be less hardware than
> > >> separate FADD/FSUB/FMUL.
> > >
> > > Compare apples to apples.
> > > FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> > > But it is bigger than FPU that does either FADD or FMUL per n clocks.
> > > Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> > > Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> > > plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> > > there exist a potential for HW savings.
> > >
> > For me, a sufficient reason to include FMA is the fact that with the FMA
> > wide normalizer we gain the ability to handle subnormal inputs and
> > outputs at zero cycle cost and very marginal gate cost.
> >
> If this is indeed the fact.
> I am unconvinced that it is true in all possible circumstances.
> There are many sensible ways to design FMA-capable FPU.
<
All FMACs have to be able to produce a 106-bit product and to all to it
a 53-bit fraction such that the product is to the right of the fraction,
the fraction is to the right of the product, and that all bits are considered
when rounding is performed.
<
With such requirements, the product operands only have to avoid
producing the hidden bit when the exponents are 0 as does the augend.
At this point the proper result is produced in all circumstances.
<
One has to be able to normalize across this 106-bit fraction, but
there is a circuit trick whereby one inserts a "fake" 1 in the position
that would become the hidden bit of the denormal (should on occur)
and then simply let the normalization transpire. Presto, denorms
are properly created and handled. Overall cost less than 2% gate
count.
<
So, yes, it is indeed a fact, and is not dependent on the 5-20 ways
one has to build a FMAC unit, just that the unit is capable of IEEE
rounded result accuracy. The rest, as they say, fall out for free.
<
> > Terje
> >
> > --
> > - <Terje.Mathisen at tmsw.no>
> > "almost all programming can be viewed as an exercise in caching"

On 12/20/2022 1:47 PM, Terje Mathisen wrote:
> Michael S wrote:
>> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com
>> wrote:
>>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
>>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>>>> leave nearly everything else to software.
>>> Bare minimum is good if transistors are limited. Otherwise, might as
>>> well do
>>> other functions in hardware. FMA (fused multiply-add) does
>>> FADD/FSUB/FMUL
>>> in one instruction with one normalize and round. Might be less
>>> hardware than
>>> separate FADD/FSUB/FMUL.
>>
>> Compare apples to apples.
>> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with
>> throughput of 1 per n clocks *each*.
>> But it is bigger than FPU that does either FADD or FMUL per n clocks.
>> Renormalization after FMA is a lot more costly than after FMUL and
>> somewhat more costly than after FADD.
>> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other
>> hand, only needs 54 MS bits of result
>> plus wired or of the rest of the bits. I am not an expert in
>> multipliers HW, but it seems to me that
>> there exist a potential for HW savings.
>>
> For me, a sufficient reason to include FMA is the fact that with the FMA
> wide normalizer we gain the ability to handle subnormal inputs and
> outputs at zero cycle cost and very marginal gate cost.
>

This is one possible merit.

I started at one point looking into doing a combine FMA unit, but this
effort stalled out when I noted it would be expensive. Tried out a "less
extreme" intermediate option, but was still expensive.

Could have done it for the low-precision FPU (partly leveraging that the
DSP48 does an X*Y+Z operation already), but initially would have still
needed ~ 4 cycles, which would not work with my existing pipeline (and
not being able to pipeline it would be worse than not having FMA, and
would kinda defeat much of the gain vs the main FPU).

Say:
Main FPU:
Scalar FADD/FSUB/FMUL: 6 cycle (6C 6T)
FMAC (double rounded): 12 cycle (12C 12T)
SIMD FADD/FSUB/FMUL: 10 cycle (10C 10T)
Low Precision (at present, *):
SIMD FADD/FSUB/FMUL: 3 cycle (3C 1T)

*: Includes both Binary16 and Binary32 operations.

Though, besides the operations themselves, one are of concern for
denormals is converters, where one either needs to pay a full FADD
latency cost for format conversions, or the converters need to be fairly
expensive.

I had fudged the rules slightly in a way that allowed keeping converter
cost low, while still keeping some other useful properties.

Normal conversion between FP and integer types is still an issue though.
For general-case, needs to be routed through the FADD unit, which
requires having a mantissa larger than the largest supported integer type.

The SIMD converters use a scheme which avoids needing to use the FADD
units for this, but puts some steep restrictions on what can be converted.

Similarly, an Int<->FP converter that can only directly handle 16 or 20
bit values (and needing to fall back to emulation software for anything
bigger) would be... Kinda lame.

> Terje
>

On Tuesday, December 20, 2022 at 8:31:23 PM UTC-5, BGB wrote:
> On 12/20/2022 1:47 PM, Terje Mathisen wrote:
> > Michael S wrote:
> >> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com
> >> wrote:
> >>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >>>> leave nearly everything else to software.
> >>> Bare minimum is good if transistors are limited. Otherwise, might as
> >>> well do
> >>> other functions in hardware. FMA (fused multiply-add) does
> >>> FADD/FSUB/FMUL
> >>> in one instruction with one normalize and round. Might be less
> >>> hardware than
> >>> separate FADD/FSUB/FMUL.
> >>
> >> Compare apples to apples.
> >> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with
> >> throughput of 1 per n clocks *each*.
> >> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> >> Renormalization after FMA is a lot more costly than after FMUL and
> >> somewhat more costly than after FADD.
> >> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other
> >> hand, only needs 54 MS bits of result
> >> plus wired or of the rest of the bits. I am not an expert in
> >> multipliers HW, but it seems to me that
> >> there exist a potential for HW savings.
> >>
> > For me, a sufficient reason to include FMA is the fact that with the FMA
> > wide normalizer we gain the ability to handle subnormal inputs and
> > outputs at zero cycle cost and very marginal gate cost.
> >
> This is one possible merit.
>
>
> I started at one point looking into doing a combine FMA unit, but this
> effort stalled out when I noted it would be expensive. Tried out a "less
> extreme" intermediate option, but was still expensive.
>
> Could have done it for the low-precision FPU (partly leveraging that the
> DSP48 does an X*Y+Z operation already), but initially would have still
> needed ~ 4 cycles, which would not work with my existing pipeline (and
> not being able to pipeline it would be worse than not having FMA, and
> would kinda defeat much of the gain vs the main FPU).
>
>
> Say:
> Main FPU:
> Scalar FADD/FSUB/FMUL: 6 cycle (6C 6T)
> FMAC (double rounded): 12 cycle (12C 12T)
> SIMD FADD/FSUB/FMUL: 10 cycle (10C 10T)
> Low Precision (at present, *):
> SIMD FADD/FSUB/FMUL: 3 cycle (3C 1T)
>
> *: Includes both Binary16 and Binary32 operations.
>
>
>
> Though, besides the operations themselves, one are of concern for
> denormals is converters, where one either needs to pay a full FADD
> latency cost for format conversions, or the converters need to be fairly
> expensive.
>
> I had fudged the rules slightly in a way that allowed keeping converter
> cost low, while still keeping some other useful properties.
>
>
> Normal conversion between FP and integer types is still an issue though.
> For general-case, needs to be routed through the FADD unit, which
> requires having a mantissa larger than the largest supported integer type..
>
> The SIMD converters use a scheme which avoids needing to use the FADD
> units for this, but puts some steep restrictions on what can be converted..
>
> Similarly, an Int<->FP converter that can only directly handle 16 or 20
> bit values (and needing to fall back to emulation software for anything
> bigger) would be... Kinda lame.
>
>
> > Terje
> >

There are a lot of merits to the 10-bit byte. Somewhat of a sweet-spot for character
data and memory sizes. I have wished back programming early micros that more
programmable characters were available on-screen to allow graphics. A 40-25
screen could use 1024 character tiles.

I have done some thinking, sketched out design, in the past on using 10/11-bit bytes
because ECC can be used with single bit error correction if 16-bit wide memory were
used to support the byte. Wanting to get error correction with a standard 16-bit wide
memory leads to an odd byte size.

One of the issues with the byte size is getting stuck in a rut, of computers designed
around it. If 10-bit bytes had been adopted, then we may not have progressed as quickly
to 64-bit machines, because 20-bits would have been “good-enough” for a much longer
time. 40-bit machines may have become popular with little demand to upgrade from 40
to 80 bits. 40-bit machines would probably be adequate today. We are now likely stuck
n a 64-bit machine rut. While those who chose 10-bit bytes are using 80-bit machines.

>A 10-bit byte will be good for a few hundred characters of extended ASCII. A 20-bit
>word will be >good for Unicode, avoiding the trouble that happened OTL when the world
>tried to squeeze Unicode >into a 16-bit word.

Even 20-bits may not be enough for Unicode. Prefer to have a larger size for this. 24-bits
as I think there are a lot of Unicode characters. UTF21 format, three characters per 64-bit
word works okay but might be too restrictive in the long run.

On 12/20/2022 7:26 PM, MitchAlsup wrote:
> On Tuesday, December 20, 2022 at 6:17:48 PM UTC-6, Michael S wrote:
>> On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
>>> Michael S wrote:
>>>> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
>>>>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
>>>>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>>>>>> leave nearly everything else to software.
>>>>> Bare minimum is good if transistors are limited. Otherwise, might as well do
>>>>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
>>>>> in one instruction with one normalize and round. Might be less hardware than
>>>>> separate FADD/FSUB/FMUL.
>>>>
>>>> Compare apples to apples.
>>>> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
>>>> But it is bigger than FPU that does either FADD or FMUL per n clocks.
>>>> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
>>>> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
>>>> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
>>>> there exist a potential for HW savings.
>>>>
>>> For me, a sufficient reason to include FMA is the fact that with the FMA
>>> wide normalizer we gain the ability to handle subnormal inputs and
>>> outputs at zero cycle cost and very marginal gate cost.
>>>
>> If this is indeed the fact.
>> I am unconvinced that it is true in all possible circumstances.
>> There are many sensible ways to design FMA-capable FPU.
> <
> All FMACs have to be able to produce a 106-bit product and to all to it
> a 53-bit fraction such that the product is to the right of the fraction,
> the fraction is to the right of the product, and that all bits are considered
> when rounding is performed.
> <
> With such requirements, the product operands only have to avoid
> producing the hidden bit when the exponents are 0 as does the augend.
> At this point the proper result is produced in all circumstances.
> <
> One has to be able to normalize across this 106-bit fraction, but
> there is a circuit trick whereby one inserts a "fake" 1 in the position
> that would become the hidden bit of the denormal (should on occur)
> and then simply let the normalization transpire. Presto, denorms
> are properly created and handled. Overall cost less than 2% gate
> count.
> <
> So, yes, it is indeed a fact, and is not dependent on the 5-20 ways
> one has to build a FMAC unit, just that the unit is capable of IEEE
> rounded result accuracy. The rest, as they say, fall out for free.
> <

Yeah, just... That first part...

This is a big reason why BJX2 does not have single-rounded FMAC, nor any
immediate plan to implement it in "hardware".

There is an optional FMAC instruction, but it is double-rounded, for the
main reason that double-rounding is a lot cheaper (but, the tricks used
to implement higher-precision math via FMAC ops, will not work in this
case).

....

Otherwise, recently bought an OrangeCrab board, which I could in-theory
try porting stuff to (though, IO is a big concern; uses an Lattice ECP5
FPGA which "seems decently large on-paper"; but little idea how it
compares with an Artix or Spartan in-use).

Have been trying to think out how to wire a VGA output to the thing.

Currently looks like I might need to do it RGBI style (~ 6 IO pins), but
possibly feed it through some 220pF capacitors or similar (thinking is
that 220pF could potentially allow me to PWM the output signal, while
still having a fast enough response so as to not turn the output into a
blurry mess).

Ideally, should be able to smooth out multiple pulses within a single
pixel, while allowing a "reasonably quick" transition between adjacent
pixels (otherwise, I would be left with either 16-color for lower
capacitance, or blurred output for higher capacitance; basically as part
of a low-pass RC-filter).

Though, 68 or 100pF capacitors might also work, ideally want around a 15
to 20 MHz cutoff (say, with 2K to 10K for R, then fed into a 2n3904 (as
a driver/amplifier), which then goes to the output along with the
voltage-limiting LED, with probably a 100 ohm resistor on the NPN, ...).

Could maybe also use trimpots to keep the cutoff as adjustable.

Don't really have enough IO pins for much beyond RGBI; nor enough
clock-cycles per pixel for effective PWM/PDM ...

Unclear how many mA the IO pins can drive (might need to use a few
2n3904 transistors or similar; also to step up the H/V sync from 3.3v to
5v, though could skip if the monitor will accept 3.3v H/V sync).

Could maybe use some LEDs as voltage limiters, since if the voltage goes
too far over the 0.7v white level, the LED will dissipate the excess.

Otherwise, if I could fit a PS2 connector into the mix, that would be
good as well (would need ~ 2 or 4 more data wires, 4 for mouse +
keyboard, and source for 5v).

The board seemingly also has a connection for (I assume) a LiON/LiPO
cell. Just less IO than I initially thought as the 'A' pins are
apparently input-only. Could run off this, but would only have 3.3v.

If I want batteries and 5v, could use a NiMH pack for 5v (likely 4x 1.2v
cells), but could not use the built-in charger circuit (and would need
6v to charge the pack). Would need a 6-cell pack to run a 7805 (5V)
regulator (with ~ 9v for the charger).

Though, 6v does fall within the standard +/- 25% tolerance for TTL, so
should be OK in theory to run stuff directly off the batteries (a 4-cell
NiMH pack would be ~ 4.0 to 5.6 v depending on charge level).

Still some uncertainty about hardware mapping differences, or specifics
of using the Yosys toolchain, ...

Could be an experiment though to see if I could fit the BJX2 core onto it.

>>> Terje
>>>
>>> --
>>> - <Terje.Mathisen at tmsw.no>
>>> "almost all programming can be viewed as an exercise in caching"

Michael S wrote:
> On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
>> Michael S wrote:
>>> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
>>>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
>>>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
>>>>> leave nearly everything else to software.
>>>> Bare minimum is good if transistors are limited. Otherwise, might as well do
>>>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
>>>> in one instruction with one normalize and round. Might be less hardware than
>>>> separate FADD/FSUB/FMUL.
>>>
>>> Compare apples to apples.
>>> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
>>> But it is bigger than FPU that does either FADD or FMUL per n clocks.
>>> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
>>> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
>>> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
>>> there exist a potential for HW savings.
>>>
>> For me, a sufficient reason to include FMA is the fact that with the FMA
>> wide normalizer we gain the ability to handle subnormal inputs and
>> outputs at zero cycle cost and very marginal gate cost.
>>
>
> If this is indeed the fact.
> I am unconvinced that it is true in all possible circumstances.
> There are many sensible ways to design FMA-capable FPU.

This is an interesting statement!

Can you give at least one example that don't include a normalizer wide
enough to handle FMUL of two subnormal numbers without first normalizing
them?

Having that normalizer means that we can basically ignore normal vs
subnormal on input, and the output normalization is the same as
otherwise required for FMA with (near-)maximal cancellation, right?

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Wednesday, December 21, 2022 at 7:35:45 AM UTC-6, Terje Mathisen wrote:
> Michael S wrote:
> > On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
> >> Michael S wrote:
> >>> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> >>>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >>>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >>>>> leave nearly everything else to software.
> >>>> Bare minimum is good if transistors are limited. Otherwise, might as well do
> >>>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> >>>> in one instruction with one normalize and round. Might be less hardware than
> >>>> separate FADD/FSUB/FMUL.
> >>>
> >>> Compare apples to apples.
> >>> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> >>> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> >>> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> >>> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> >>> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> >>> there exist a potential for HW savings.
> >>>
> >> For me, a sufficient reason to include FMA is the fact that with the FMA
> >> wide normalizer we gain the ability to handle subnormal inputs and
> >> outputs at zero cycle cost and very marginal gate cost.
> >>
> >
> > If this is indeed the fact.
> > I am unconvinced that it is true in all possible circumstances.
> > There are many sensible ways to design FMA-capable FPU.
> This is an interesting statement!
>
> Can you give at least one example that don't include a normalizer wide
> enough to handle FMUL of two subnormal numbers without first normalizing
> them?
>
> Having that normalizer means that we can basically ignore normal vs
> subnormal on input, and the output normalization is the same as
> otherwise required for FMA with (near-)maximal cancellation, right?
> Terje
>
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"
|>
|>Can you give at least one example that don't include a normalizer wide
|>enough to handle FMUL of two subnormal numbers without first normalizing
|>them?
Rant on:
There is a way to keep subnormal numbers normalized, works best with round to odd
using a trailing zeros fraction/mantissa code and using the exponent to identify as subnormal.
The interpretation of this method is that subnormal numbers are inexact.
Rant off

On Wednesday, December 21, 2022 at 3:35:45 PM UTC+2, Terje Mathisen wrote:
> Michael S wrote:
> > On Tuesday, December 20, 2022 at 9:47:45 PM UTC+2, Terje Mathisen wrote:
> >> Michael S wrote:
> >>> On Tuesday, December 20, 2022 at 11:45:33 AM UTC+2, robf...@gmail.com wrote:
> >>>> On Monday, December 19, 2022 at 11:58:13 PM UTC-5, BGB wrote:
> >>>>> If it were me, I would just assume an FPU that does FADD/FSUB/FMUL and
> >>>>> leave nearly everything else to software.
> >>>> Bare minimum is good if transistors are limited. Otherwise, might as well do
> >>>> other functions in hardware. FMA (fused multiply-add) does FADD/FSUB/FMUL
> >>>> in one instruction with one normalize and round. Might be less hardware than
> >>>> separate FADD/FSUB/FMUL.
> >>>
> >>> Compare apples to apples.
> >>> FMA with throughput of 1 per n clocks is smaller that FADD + FMUL with throughput of 1 per n clocks *each*.
> >>> But it is bigger than FPU that does either FADD or FMUL per n clocks.
> >>> Renormalization after FMA is a lot more costly than after FMUL and somewhat more costly than after FADD.
> >>> Also, FMA needs full 53x53=>106 bit multiplier. FMUL, on the other hand, only needs 54 MS bits of result
> >>> plus wired or of the rest of the bits. I am not an expert in multipliers HW, but it seems to me that
> >>> there exist a potential for HW savings.
> >>>
> >> For me, a sufficient reason to include FMA is the fact that with the FMA
> >> wide normalizer we gain the ability to handle subnormal inputs and
> >> outputs at zero cycle cost and very marginal gate cost.
> >>
> >
> > If this is indeed the fact.
> > I am unconvinced that it is true in all possible circumstances.
> > There are many sensible ways to design FMA-capable FPU.
> This is an interesting statement!
>
> Can you give at least one example that don't include a normalizer wide
> enough to handle FMUL of two subnormal numbers without first normalizing
> them?
>
> Having that normalizer means that we can basically ignore normal vs
> subnormal on input, and the output normalization is the same as
> otherwise required for FMA with (near-)maximal cancellation, right?
> Terje
>
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

I can't answer your question right now.
Instead, I can give you result of the measurement on real-world hardware.
Intel Skylake (has FMA on both FP execution pipes):
1e-307 - 1.01e-307 takes ~138 clocks (latency)
That's pretty close to ~152 clocks the same operation takes on Intel Ivy Bridge
that does not have FMA.