In article <2022Aug17.101938@mips.complang.tuwien.ac.at>,
anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:

> And then you have Intel marketing, which disables AVX512 in SKUs for
> chips that actually support AVX512 (and likewise disable AVX in some
> SKUs), ensuring that most programmers will steer clear of AVX for
> many years to come and even longer for AVX512.

There are parts of Intel that don't seem to understand the difference
between firmware and applications. They behave as if machines are sold
with all their software already on them, and run that load for their
lives. This is not how the application software market actually works.

If you're an application software producer, doing an extra version for an
instruction set extension needs to give the end-user something big for it
to be cost-effective. I've experimented with AVX, and it does very little
for the software I work on. I understand why, and AVX-512 isn't going to
do any better. Improving performance at the algorithm level works better
and applies to all platforms.

John

Stefan Monnier wrote:
>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
>> with IEEE754 FP accuracy will punish you with a 400% power/area
>> penalty compared to modern 3D-optimised GPUs which, as explicitly
>
> What is the origin of this extra cost? IOW, where's the meat of the
> savings (I mean: which part of the computation of the last few bits
> costs so much)?
> Does the saving vary significantly between instructions?

I think that is pretty much bogus, Mitch have shown repeatedly that
having FMAC makes subnormal handling nearly free, as in zero cycles and
single-digit percentage gates/power. 400% is simply rubbish.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

John Dallman <jgd@cix.co.uk> schrieb:
> In article <2022Aug17.101938@mips.complang.tuwien.ac.at>,
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>> And then you have Intel marketing, which disables AVX512 in SKUs for
>> chips that actually support AVX512 (and likewise disable AVX in some
>> SKUs), ensuring that most programmers will steer clear of AVX for
>> many years to come and even longer for AVX512.
>
> There are parts of Intel that don't seem to understand the difference
> between firmware and applications. They behave as if machines are sold
> with all their software already on them, and run that load for their
> lives. This is not how the application software market actually works.

Either that, or self-compiled software.

The latter makes sense for a lot of people in scientific computing,
but those are probably not so frequent.

But even there, SIMD is only useful for some fields.

>
> If you're an application software producer, doing an extra version for an
> instruction set extension needs to give the end-user something big for it
> to be cost-effective. I've experimented with AVX, and it does very little
> for the software I work on. I understand why, and AVX-512 isn't going to
> do any better.

Which field is that, can you tell?

>Improving performance at the algorithm level works better
> and applies to all platforms.

That is always a given, but sometimes you would like to do both :-)

On 2022-08-17, John Dallman wrote:
> In article <2022Aug17.101938@mips.complang.tuwien.ac.at>,
> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>
>> And then you have Intel marketing, which disables AVX512 in SKUs for
>> chips that actually support AVX512 (and likewise disable AVX in some
>> SKUs), ensuring that most programmers will steer clear of AVX for
>> many years to come and even longer for AVX512.
>
> There are parts of Intel that don't seem to understand the difference
> between firmware and applications. They behave as if machines are sold
> with all their software already on them, and run that load for their
> lives. This is not how the application software market actually works.
>
> If you're an application software producer, doing an extra version for an
> instruction set extension needs to give the end-user something big for it
> to be cost-effective. I've experimented with AVX, and it does very little
> for the software I work on. I understand why, and AVX-512 isn't going to
> do any better. Improving performance at the algorithm level works better
> and applies to all platforms.
>
> John

I haven't played with AVX512 nor bothered to learn it (because, hey,
what's the point when there's no hardware around to run it on anyway),
but my impression is that they made some actual improvements w.r.t.
predication and such, so that it should be easier to vectorize code
than with SSE or regular AVX.

If that is true, I would really like to see narrower implementations of
AVX512, as it could potentially bring some of the benefits of vector
processing (compared to plain-ol-SIMD). E.g. 128-bit wide AVX512 (if
that's doable) would give you immunity against latency hazards up to
four clock cycles.

/Marcus

In article <tdiqmd$52v$2@newsreader4.netcologne.de>,
tkoenig@netcologne.de (Thomas Koenig) wrote:

> Either that, or self-compiled software.
>
> The latter makes sense for a lot of people in scientific computing,
> but those are probably not so frequent.
>
> But even there, SIMD is only useful for some fields.

True.

> Which field is that, can you tell?

I can speak more freely if I don't.

John

On Wednesday, August 17, 2022 at 1:35:54 AM UTC-5, Quadibloc wrote:
> On Sunday, July 24, 2022 at 4:28:47 AM UTC-6, luke.l...@gmail.com wrote:
>
> > 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
> > with IEEE754 FP accuracy will punish you with a 400% power/area
> > penalty compared to modern 3D-optimised GPUs which, as explicitly
> > spelled out and allowed in the Vulkan(tm) Spec by the Khronos
> > Group, are permitted significant accuracy reductions. Mitch
> > will (or will have already) filled you in on that.
>
> Well, if your purpose is building a device that is helpful to scientists
> wishing to perform large-scale computations on FP64 numbers, you'll
> just have to pay that penalty. After paying that penalty, though, something
> based on a GPU architecture instead of that of a conventional non-vector
> CPU is *still* vastly more powerful.
<
Those scientists were perfectly happy with CDC6600 quality floating
point. They were the ones on the side of a) "IEEE 754 containers are
file, b) leave out all that denorm and NaN stuff.
>
> And we have the NEC Aurora TSUBASA as an example of a Cray-style
> vector ISA. At a significantly higher cost, it was capable of about half
> of the FP64 FLOPS that a video card with high FP64 capabilities made
> on the same technology could provide. But because this architecture is
> more flexible, one could use it for a higher proportion of the floating-point
> calculations in typical scientific programs.
>
> That is a tradeoff that might be well worth it.
>
> John Savard

On Wednesday, August 17, 2022 at 1:39:05 AM UTC-5, Quadibloc wrote:
> On Sunday, July 24, 2022 at 4:41:19 PM UTC-6, luke.l...@gmail.com wrote:
>
> > AVX-512 *shudder*
>
> And here I was wondering why Intel hadn't been rushing to
> put AVX-512 on its consumer CPUs, so that they would
> squash AMD with vastly superior games and graphic
> performance.
<
Running 512-bit AVX goes for a dozen milliseconds before someone
has to throttle back the CPU frequency, due to the generation of heat.
>
> John Savard

On Wednesday, August 17, 2022 at 3:37:47 AM UTC-5, BGB wrote:
> On 8/17/2022 1:35 AM, Quadibloc wrote:
> > On Sunday, July 24, 2022 at 4:28:47 AM UTC-6, luke.l...@gmail.com wrote:
> >
> >> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
> >> with IEEE754 FP accuracy will punish you with a 400% power/area
> >> penalty compared to modern 3D-optimised GPUs which, as explicitly
> >> spelled out and allowed in the Vulkan(tm) Spec by the Khronos
> >> Group, are permitted significant accuracy reductions. Mitch
> >> will (or will have already) filled you in on that.
> >
> > Well, if your purpose is building a device that is helpful to scientists
> > wishing to perform large-scale computations on FP64 numbers, you'll
> > just have to pay that penalty. After paying that penalty, though, something
> > based on a GPU architecture instead of that of a conventional non-vector
> > CPU is *still* vastly more powerful.
> >
> > And we have the NEC Aurora TSUBASA as an example of a Cray-style
> > vector ISA. At a significantly higher cost, it was capable of about half
> > of the FP64 FLOPS that a video card with high FP64 capabilities made
> > on the same technology could provide. But because this architecture is
> > more flexible, one could use it for a higher proportion of the floating-point
> > calculations in typical scientific programs.
> >
> > That is a tradeoff that might be well worth it.
> >
> Pretty much, there are reasons I am mostly going for DAZ + FTZ and FPU
> designs which don't give exact 0.5 ULP rounding. These make the FPU so
> much cheaper.
>
>
> For GPU-like uses, it is possible there could be a use-case for a
> further reduced-precision Binary64 variant, say:
> S.E11.F32.Z20
<
Z20 ???
>
> This is partly because there are still a few cases where a 16-bit
> mantissa is not sufficient (and where needing to keep the main FPU
> around is also fairly expensive).
>
>
> Namely, there is the point in my rasterizer where it goes from
> single-precision to 32-bit fixed-point.
>
> On the floating-point side, it can use low precision calculations just
> fine. On the integer side, it is 16.16 fixed point. However, the
> conversion step from floating point to fixed point integer exceeds the
> accuracy requirements of both the truncated-single and full single
> precision (so, would effectively need something with a 32-bit mantissa
> to deal with this).
>
>
> This may be added as another possible sub-feature to the low-precision
> unit, namely scalar FADD/FMUL units with 32-bit mantissa, and also needs
> to support integer conversion. Mostly to serve as a cheaper stand-in for
> the main FPU.
>
>
> Though, it is possible the "GPU core" idea might be dead for now, given
> I still can't quite make stuff cheap enough.
>
>
> Well, either that, or try to cheap out on the FP->Int conversion process
> and add some additional "low cost" SIMD converters (2x FP32 <-> Packed
> Int32). Say, if one adds a bias of 1024, this is enough to get approx
> 10.6 output, which can then be shifted to give values in the 16.16
> format (but would also require doing this step via a specialized blob of
> ASM; at present this part of the rasterizer is still written in C).
>
> I guess this could potentially side-step the need for a higher precision
> unit (but would limit maximum viewport size).
>
>
> > John Savard

On 8/17/2022 10:15 AM, MitchAlsup wrote:
> On Wednesday, August 17, 2022 at 3:37:47 AM UTC-5, BGB wrote:
>> On 8/17/2022 1:35 AM, Quadibloc wrote:
>>> On Sunday, July 24, 2022 at 4:28:47 AM UTC-6, luke.l...@gmail.com wrote:
>>>
>>>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
>>>> with IEEE754 FP accuracy will punish you with a 400% power/area
>>>> penalty compared to modern 3D-optimised GPUs which, as explicitly
>>>> spelled out and allowed in the Vulkan(tm) Spec by the Khronos
>>>> Group, are permitted significant accuracy reductions. Mitch
>>>> will (or will have already) filled you in on that.
>>>
>>> Well, if your purpose is building a device that is helpful to scientists
>>> wishing to perform large-scale computations on FP64 numbers, you'll
>>> just have to pay that penalty. After paying that penalty, though, something
>>> based on a GPU architecture instead of that of a conventional non-vector
>>> CPU is *still* vastly more powerful.
>>>
>>> And we have the NEC Aurora TSUBASA as an example of a Cray-style
>>> vector ISA. At a significantly higher cost, it was capable of about half
>>> of the FP64 FLOPS that a video card with high FP64 capabilities made
>>> on the same technology could provide. But because this architecture is
>>> more flexible, one could use it for a higher proportion of the floating-point
>>> calculations in typical scientific programs.
>>>
>>> That is a tradeoff that might be well worth it.
>>>
>> Pretty much, there are reasons I am mostly going for DAZ + FTZ and FPU
>> designs which don't give exact 0.5 ULP rounding. These make the FPU so
>> much cheaper.
>>
>>
>> For GPU-like uses, it is possible there could be a use-case for a
>> further reduced-precision Binary64 variant, say:
>> S.E11.F32.Z20
> <
> Z20 ???

Z: Ignored on input, Set to Zero on output.

Basically, format compatible with Binary64, but with reduced precision,
and thus cheaper.

A 32-bit mantissa allows using 3 DSPs for FMUL.

FADD would use ~ 36 bit internally, which would be enough to handle
basic operations, and potentially deal with 32-bit integer conversion.
Here, would stick 001 on the top, and a single sub-ULP bit on the bottom
(during FADD, anything which falls below this point is simply discarded).

It is sorta like with the S.E8.F16.Z7 format.

>>
>> This is partly because there are still a few cases where a 16-bit
>> mantissa is not sufficient (and where needing to keep the main FPU
>> around is also fairly expensive).
>>
>>
>> Namely, there is the point in my rasterizer where it goes from
>> single-precision to 32-bit fixed-point.
>>
>> On the floating-point side, it can use low precision calculations just
>> fine. On the integer side, it is 16.16 fixed point. However, the
>> conversion step from floating point to fixed point integer exceeds the
>> accuracy requirements of both the truncated-single and full single
>> precision (so, would effectively need something with a 32-bit mantissa
>> to deal with this).
>>
>>
>> This may be added as another possible sub-feature to the low-precision
>> unit, namely scalar FADD/FMUL units with 32-bit mantissa, and also needs
>> to support integer conversion. Mostly to serve as a cheaper stand-in for
>> the main FPU.
>>
>>
>> Though, it is possible the "GPU core" idea might be dead for now, given
>> I still can't quite make stuff cheap enough.
>>
>>
>> Well, either that, or try to cheap out on the FP->Int conversion process
>> and add some additional "low cost" SIMD converters (2x FP32 <-> Packed
>> Int32). Say, if one adds a bias of 1024, this is enough to get approx
>> 10.6 output, which can then be shifted to give values in the 16.16
>> format (but would also require doing this step via a specialized blob of
>> ASM; at present this part of the rasterizer is still written in C).
>>
>> I guess this could potentially side-step the need for a higher precision
>> unit (but would limit maximum viewport size).
>>
>>
>>> John Savard

Marcus <m.delete@this.bitsnbites.eu> writes:
>If that is true, I would really like to see narrower implementations of
>AVX512,

The EVEX prefix supports 128, 256, and 512-bit operations through the
L and L' bits.

>as it could potentially bring some of the benefits of vector
>processing (compared to plain-ol-SIMD). E.g. 128-bit wide AVX512 (if
>that's doable) would give you immunity against latency hazards up to
>four clock cycles.

How so?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Wednesday, August 17, 2022 at 6:13:50 PM UTC+3, MitchAlsup wrote:
> On Wednesday, August 17, 2022 at 1:39:05 AM UTC-5, Quadibloc wrote:
> > On Sunday, July 24, 2022 at 4:41:19 PM UTC-6, luke.l...@gmail.com wrote:
> >
> > > AVX-512 *shudder*
> >
> > And here I was wondering why Intel hadn't been rushing to
> > put AVX-512 on its consumer CPUs, so that they would
> > squash AMD with vastly superior games and graphic
> > performance.
> <
> Running 512-bit AVX goes for a dozen milliseconds before someone
> has to throttle back the CPU frequency, due to the generation of heat.

According to what Anton told us few weeks ago, it does not apply to Rocket Lake
Probably, does not apply to Tiger Lake either, as long as only 1 or 2 cores are
crunching 512-bit stuff.
It makes sense if a big heat source is not any 512-bit operations, but
specifically FP arithmetic, i.e. FADD/FMUL/FMADD since for those operations the
max. throughput on above mentioned cores is the same for 512-bit opcodes (1 per
clock) and for 256-bit opcodes (2 per clock).

On 8/17/2022 8:32 AM, Terje Mathisen wrote:
> Stefan Monnier wrote:
>>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
>>>       with IEEE754 FP accuracy will punish you with a 400% power/area
>>>       penalty compared to modern 3D-optimised GPUs which, as explicitly
>>
>> What is the origin of this extra cost?  IOW, where's the meat of the
>> savings (I mean: which part of the computation of the last few bits
>> costs so much)?
>> Does the saving vary significantly between instructions?
>
> I think that is pretty much bogus, Mitch have shown repeatedly that
> having FMAC makes subnormal handling nearly free, as in zero cycles and
> single-digit percentage gates/power. 400% is simply rubbish.
>

Not sure whether it is anywhere near 400%, but not strictly non-zero either.

Nearly free, for the FMAC, but IME still adds a latency cost mostly in
that an FMAC would still require several additional clock cycles vs a
bare FMUL if using DAZ/FTZ, due mostly to the additional adder and
normalization steps.

For performance, FMAC would likely only win over FMUL if there were
significantly more 'A*B+C' cases than 'A*B' cases.

There is one way on using FMAC to fake higher precision math, which in
order to make use of would require a larger mantissa than a bare FADD.
Supporting this mode would add cost.

As for 0.5 ULP, this would likely require:
FMUL to produce a full-width result, rather than just the high-order bits;
FADD needs to have more sub ULP bits.

So, say, for a double-precision FMAC:
A*B, produces 108 bits of output (more expensive than ~ 56);
May need to expand further in the adder (TBD).

Mostly to deal with semantic edge cases involving catastrophic
cancellation and similar...

Without FMAC, one can mostly ignore catastrophic cancellation issue:
In cases where it would happen, there is very little below the ULP that
would need to be preserved.
But with FMAC, one may find that, suddenly, these low-order bits can
actually have a visible effect on the result.

One could reduce cost by always producing the "double rounded" output
(as-if it had been done as two separate operations), but this does
eliminate some use cases.

....

Vs, say, a cheaper version:
A*B produces ~ 56 bits (high bits only);
Can use 6 DSPs and a few extra LUT based multipliers for the rest.
A+B uses 68 bits;
Could have also been 56,
but need a few extra to deal with Int64 conversion.

Though, this is still kinda expensive, a version which uses 3 DSPs for
FMUL and a 36-bit adder would be cheaper still, but as can be noted,
would effectively cut the low 20 bits off the mantissa.

Could still be sufficient for 3D rendering tasks, but would be mostly
insufficient for "general purpose" use (code which assumes full
precision double could break in some fairly major ways).

Michael S <already5chosen@yahoo.com> writes:
>According to what Anton told us few weeks ago, it does not apply to Rocket Lake
>Probably, does not apply to Tiger Lake either, as long as only 1 or 2 cores are
>crunching 512-bit stuff.

Actually, for the Ice Lake i5-1035G4, we see an AVX512 downclock with
1 active core from 3.7GHz to 3.6GHz, but no downclock with more active
cores (2 cores are already at 3.6GHz without AVX512, 3 and 4 at
3.3GHz)
<https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html>.

I guess that the increased voltage for 3.7GHz in combination with
AVX512 can result in too much current draw for the core, so they
downclock (and lower voltage) to 3.6GHz. For the other cases the
voltage is already low enough, so no proactive downclocking is
necessary, and you can leave reactive downclocking to the power and
temperature limits. If my guess is correct, I expect that Tiger Lake
will be similar.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 8/17/2022 8:13 AM, robf...@gmail.com wrote:
>> For GPU-like uses, it is possible there could be a use-case for a
>> further reduced-precision Binary64 variant, say:
>> S.E11.F32.Z20
>
>> This is partly because there are still a few cases where a 16-bit
>> mantissa is not sufficient (and where needing to keep the main FPU
>> around is also fairly expensive).
>
> I have been thinking of going with 21-bit floats S1.E5.F15 because
> three can be fit into 64-bits and so extra vector lanes can be had
> cheaply.
>

Actually exists as a SIMD vector sub-type in BJX2.

First 3 elements are Binary16;
Last element is divided into 3x 5-bit fields, which extend the mantissa
to 15 bits.

Or, 64-bit field:
0xPPPP_ZZZZ_YYYY_XXXX;
Where PPPP is:
0zzz_zzyy_yyyx_xxxx

There is another vector type where the first 3 elements are Binary32,
and the last element extends each mantissa by 10 bits (so ~ 33 bits),
organized along a pretty much similar pattern.

An earlier form of the latter was originally developed and used in my
BGBTech2 3D engine, mostly because:
3x Binary32 was not sufficient for the world size (1024x1024 km);
The cost of passing these around, and doing Unpack/Compute/Repack on
them, was at the time faster than working directly with 3x Binary64 in
this case (a lot more time with these was spent on passing them around
rather than actually doing math with them).

That version differed though in that it was less symmetric:
Added 12 bits to X and Y, but only 8 to Z.

> I like the low precision idea. Keeping the significand under 18 bits
> means just a single DSP can be used for the multiply.
>

Yes, this was a major reason:
16-bit is padded to 18-bit by adding 01 to the top;
A single DSP is used to multiply the result.

Keeping the low order bits as zero (for both multiply and add), was
mostly because if one ignores these bits for multiply but leaves them as
non-zero in the output (because, after all, 18*16->36), but then keep
full Binary32 precision for add/subtract, tends to cause instability
issues with some calculations (such as when doing divide or square root
via Newton-Raphson).

So, it seemed like a better idea to ignore these bits entirely on input,
and force them to zero on output, across both multiply and add/subtract
in this case.

As it so happens, this format is also basically sufficient for the
projection process in OpenGL and similar (though with the main "choke
point" being when going from floating-point in the front-end to fixed
point in the back-end).

Though, I guess, "fake it using integer math" is possibly also still an
option worth looking into (and probably cheaper than having a dedicated
FMUL and FADD units effectively just to pull off Binary32 -> Fixed 16.16
conversion).

....

On Wednesday, August 17, 2022 at 12:17:04 PM UTC-5, BGB wrote:
> On 8/17/2022 8:32 AM, Terje Mathisen wrote:
> > Stefan Monnier wrote:
> >>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
> >>> with IEEE754 FP accuracy will punish you with a 400% power/area
> >>> penalty compared to modern 3D-optimised GPUs which, as explicitly
> >>
> >> What is the origin of this extra cost? IOW, where's the meat of the
> >> savings (I mean: which part of the computation of the last few bits
> >> costs so much)?
> >> Does the saving vary significantly between instructions?
> >
> > I think that is pretty much bogus, Mitch have shown repeatedly that
> > having FMAC makes subnormal handling nearly free, as in zero cycles and
> > single-digit percentage gates/power. 400% is simply rubbish.
> >
> Not sure whether it is anywhere near 400%, but not strictly non-zero either.
>
The adder for adding denorms to a 64-bit 754 FMAC* is on the order of 2%
(*) FMAC that can also perform FMUL and FADD/FSUB.
<
Which while not actually zero is close enough to not mater in the long run.
>
> Nearly free, for the FMAC, but IME still adds a latency cost mostly in
> that an FMAC would still require several additional clock cycles vs a
> bare FMUL if using DAZ/FTZ, due mostly to the additional adder and
> normalization steps.
<
1-gate of delay added to the overall FMAC latency (over the typical 4 cycles)
>
> For performance, FMAC would likely only win over FMUL if there were
> significantly more 'A*B+C' cases than 'A*B' cases.
>
Actually, even systems with an FMAC will often have a spare FADD because
{FADD/FSUB (and for those poor ISAs with separate register files) FCMP}
represent 57%-63% of FP calculations.
>
> There is one way on using FMAC to fake higher precision math, which in
> order to make use of would require a larger mantissa than a bare FADD.
> Supporting this mode would add cost.
>
That is why you do these things in the FMAC unit where you have a 168-bit adder
(typically Koogie-Stone) and a 52-bit incrementer.
>
> As for 0.5 ULP, this would likely require:
> FMUL to produce a full-width result, rather than just the high-order bits;
> FADD needs to have more sub ULP bits.
>
These are IEEE REQUIREMENTS.
>
> So, say, for a double-precision FMAC:
> A*B, produces 108 bits of output (more expensive than ~ 56);
> May need to expand further in the adder (TBD).
>
> Mostly to deal with semantic edge cases involving catastrophic
> cancellation and similar...
>
It's like Air-Bags in a car. Would you buy a car without air-bags (for street use)
today ?
>
> Without FMAC, one can mostly ignore catastrophic cancellation issue:
> In cases where it would happen, there is very little below the ULP that
> would need to be preserved.
<
It is PRECISELY the catastrophic cancellation issue which drove IEEE
to specify FMAC in the way they did.
<
> But with FMAC, one may find that, suddenly, these low-order bits can
> actually have a visible effect on the result.
>
Because they should ! Like the ABS system in you car driving downhill
in freezing rain.
<
<Apparently you just don't get the motive for doing quality work.......>

On Wednesday, August 17, 2022 at 12:17:04 PM UTC-5, BGB wrote:
> On 8/17/2022 8:32 AM, Terje Mathisen wrote:
>
>
> As for 0.5 ULP, this would likely require:
> FMUL to produce a full-width result, rather than just the high-order bits;
> FADD needs to have more sub ULP bits.
>
You keep arguing that IEEE is hard. Yes, it is, and yes it was supposed to
be--just to eliminate all the inferior ways to perform FP arithmetic, so
programmers could trust FP arithmetics across implementations.
<
And I keep beating you up on this issue.
<
If you added the caveat that "in your expressive medium (FPGA)
getting the last IEEE binary digit correct is excessively expensive,"
I would be able to avoid these <seemingly constant> beatings.

On Wednesday, August 17, 2022 at 8:33:06 PM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >According to what Anton told us few weeks ago, it does not apply to Rocket Lake
> >Probably, does not apply to Tiger Lake either, as long as only 1 or 2 cores are
> >crunching 512-bit stuff.
> Actually, for the Ice Lake i5-1035G4, we see an AVX512 downclock with
> 1 active core from 3.7GHz to 3.6GHz, but no downclock with more active
> cores (2 cores are already at 3.6GHz without AVX512, 3 and 4 at
> 3.3GHz)
> <https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html>.
>
> I guess that the increased voltage for 3.7GHz in combination with
> AVX512 can result in too much current draw for the core, so they
> downclock (and lower voltage) to 3.6GHz. For the other cases the
> voltage is already low enough, so no proactive downclocking is
> necessary, and you can leave reactive downclocking to the power and
> temperature limits. If my guess is correct, I expect that Tiger Lake
> will be similar.

An equivalent of i5-1035G4 in Tiger Lake family is i5-1155G7
that has max. frequency = 4.5 GHz.
I think, it's a too optimistic to expect that with 3 or 4 cores
running AVX512 stuff it would be downclocked by mere 9% = 4.1GHz.

Would be glad to find out that I am wrong about it.

> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 8/17/2022 2:05 PM, MitchAlsup wrote:
> On Wednesday, August 17, 2022 at 12:17:04 PM UTC-5, BGB wrote:
>> On 8/17/2022 8:32 AM, Terje Mathisen wrote:
>>> Stefan Monnier wrote:
>>>>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
>>>>> with IEEE754 FP accuracy will punish you with a 400% power/area
>>>>> penalty compared to modern 3D-optimised GPUs which, as explicitly
>>>>
>>>> What is the origin of this extra cost? IOW, where's the meat of the
>>>> savings (I mean: which part of the computation of the last few bits
>>>> costs so much)?
>>>> Does the saving vary significantly between instructions?
>>>
>>> I think that is pretty much bogus, Mitch have shown repeatedly that
>>> having FMAC makes subnormal handling nearly free, as in zero cycles and
>>> single-digit percentage gates/power. 400% is simply rubbish.
>>>
>> Not sure whether it is anywhere near 400%, but not strictly non-zero either.
>>
> The adder for adding denorms to a 64-bit 754 FMAC* is on the order of 2%
> (*) FMAC that can also perform FMUL and FADD/FSUB.
> <
> Which while not actually zero is close enough to not mater in the long run.

Bigger cost is needing to go from only producing the high-order bits of
the multiplier result (cheap case), to producing all of the bits.

It is sort of like the difference in area between a right triangle and a
square.

With just an FMUL, all these bits fall off into the void, so one doesn't
really need to see if they were never created in the first place.

Full FMAC (and also strict 0.5 ULP) can kind of throw a wrench in these
sorts of optimizations.

Free is not free if it already means one has to pay the cost for a bunch
of other features in order for a given feature to be "free".

>>
>> Nearly free, for the FMAC, but IME still adds a latency cost mostly in
>> that an FMAC would still require several additional clock cycles vs a
>> bare FMUL if using DAZ/FTZ, due mostly to the additional adder and
>> normalization steps.
> <
> 1-gate of delay added to the overall FMAC latency (over the typical 4 cycles)

The FADD part can handle subnormal numbers pretty much for free, but
that is not what I am arguing here.

It is FMUL where it is more of an issue, and bolting the units together
to allow handling this case comes at a latency cost (and needing to make
FMUL slower than it might be otherwise, is not free).

>>
>> For performance, FMAC would likely only win over FMUL if there were
>> significantly more 'A*B+C' cases than 'A*B' cases.
>>
> Actually, even systems with an FMAC will often have a spare FADD because
> {FADD/FSUB (and for those poor ISAs with separate register files) FCMP}
> represent 57%-63% of FP calculations.

OK.

Seems like one could handle FADD by skipping over the multiplier part.
But, yeah, always needing to spend the full latency is not desirable.

>>
>> There is one way on using FMAC to fake higher precision math, which in
>> order to make use of would require a larger mantissa than a bare FADD.
>> Supporting this mode would add cost.
>>
> That is why you do these things in the FMAC unit where you have a 168-bit adder
> (typically Koogie-Stone) and a 52-bit incrementer.

Granted.

Pros/Cons if one has a cheaper but faster FMAC that can't actually do
this operation (because it doesn't actually have the bits to do so).

>>
>> As for 0.5 ULP, this would likely require:
>> FMUL to produce a full-width result, rather than just the high-order bits;
>> FADD needs to have more sub ULP bits.
>>
> These are IEEE REQUIREMENTS.

Which partly I skip over, because my FPU designs aren't really IEEE
conformant anyways, nor is strict conformance necessarily a design goal...

This would turn into a game of whack-a-mole.

Nor would it actually gain enough here to make it worth the added cost.

>>
>> So, say, for a double-precision FMAC:
>> A*B, produces 108 bits of output (more expensive than ~ 56);
>> May need to expand further in the adder (TBD).
>>
>> Mostly to deal with semantic edge cases involving catastrophic
>> cancellation and similar...
>>
> It's like Air-Bags in a car. Would you buy a car without air-bags (for street use)
> today ?

In normal FADD/FSUB, the issue is effectively hidden:
The only way it could happen usually is if the mantissas have the same
exponent;
By extension, there are hardly any non-zero sub-ULP bits in this scenario.

In cases where the exponents differ, this scenario can't occur, so much
beyond a few bits, the stuff that falls off the bottom can be ignored.

For subtracting the values, one could detect whether or not the bits
that fell off the bottom were non-zero and use this to twiddle the
carry-in to the adder. But, this isn't free either (and if one has, say,
8 sub ULP bits, there is only around a 1 in 256 chance of this actually
having a visible effect on the result).

But, as for cars, I think a lot might depend on where one is living.

A relative of mine has a vehicle which is basically built out of welded
steel tubing and diamond plate, no doors or windows, no windshield, no
muffler ("straight pipe exhaust"), minimal "over the lap" style
seatbelts, etc.

Has turn signals and a license plate, so "good enough" it seems.

Not necessarily sure road safety was all that much of a design
consideration in this case.

Some of the other vehicles around are only partially painted, say with
much of the vehicle still being gray primer, ...

Well, or other scenarios one might see, say, where a car got T-boned, so
the owner removed the car-doors and similar on that side and mostly
replaced them with a piece of plywood or similar with some window holes
cut into it (say, with some clear plastic or similar stapled over the
holes), etc...

Functional airbags at that point, not necessarily a given.

>>
>> Without FMAC, one can mostly ignore catastrophic cancellation issue:
>> In cases where it would happen, there is very little below the ULP that
>> would need to be preserved.
> <
> It is PRECISELY the catastrophic cancellation issue which drove IEEE
> to specify FMAC in the way they did.
> <

But, it is neither necessarily the cheapest nor fastest way to implement
this operator.

Granted, does allow using SIMD to pull off extended precision
calculations (rather than, say, doing this using integer math or
something). This would not otherwise be possible with a cheaper version.

Making this work effectively though would require both making the FADD
part wider, and also calculating the low half of the multiplier result.

Meanwhile, integer math is easier to get right...
Is it because there is some rigidly enforced integer math standard?
Not really; The standard integer math rules happen to already mostly
align with the cheapest ways of implementing usable integer math operators.

If floating point math were defined in terms of the cheapest possible
ways of implementing it, then there wouldn't be anything to argue about.

But, I guess, maybe there would still possibility for debate, say,
between person A's system where (5.0-3.0)==2.0, and person B's whose
gives 1.999999, etc...

>> But with FMAC, one may find that, suddenly, these low-order bits can
>> actually have a visible effect on the result.
>>
> Because they should ! Like the ABS system in you car driving downhill
> in freezing rain.
> <
> <Apparently you just don't get the motive for doing quality work.......>

It is more trade-offs...

Would IEEE-754 conformance still be worthwhile, if it meant one could
*only* afford to have Single Precision?...

I don't consider this a win.

Though, this is basically what MicroBlaze does.

Sometimes, one can be like "close enough".

It is sorta similar for things like OpenGL:
* Does it implement the API?
** Well, more or less 1.3 levels (of the parts that aren't absent).
* Does it produce pixel-exact results with a reference implementation:
** No, not even close (looks almost more like the original PlayStation).

What about BC7 compressed texture support, does it look pixel-exact to
the reference version?
* Heh, no, also not even close.

What about BC1(DXT1) or BC3(DXT5)?
* Closer, but still not quite.
* It just sort of approximates these via its own format internally.

But, being like "Nope, can't afford to do this perfectly, so not going
to do it at all" is not so useful either.

And, so on, down the list...

Marcus wrote:
> On 2022-08-17, John Dallman wrote:
>> In article <2022Aug17.101938@mips.complang.tuwien.ac.at>,
>> anton@mips.complang.tuwien.ac.at (Anton Ertl) wrote:
>>
>>> And then you have Intel marketing, which disables AVX512 in SKUs for
>>> chips that actually support AVX512 (and likewise disable AVX in some
>>> SKUs), ensuring that most programmers will steer clear of AVX for
>>> many years to come and even longer for AVX512.
>>
>> There are parts of Intel that don't seem to understand the difference
>> between firmware and applications. They behave as if machines are sold
>> with all their software already on them, and run that load for their
>> lives. This is not how the application software market actually works.
>>
>> If you're an application software producer, doing an extra version for an
>> instruction set extension needs to give the end-user something big for it
>> to be cost-effective. I've experimented with AVX, and it does very little
>> for the software I work on. I understand why, and AVX-512 isn't going to
>> do any better. Improving performance at the algorithm level works better
>> and applies to all platforms.
>>
>> John
>
> I haven't played with AVX512 nor bothered to learn it (because, hey,
> what's the point when there's no hardware around to run it on anyway),
> but my impression is that they made some actual improvements w.r.t.
> predication and such, so that it should be easier to vectorize code
> than with SSE or regular AVX.
>
> If that is true, I would really like to see narrower implementations of
> AVX512, as it could potentially bring some of the benefits of vector
> processing (compared to plain-ol-SIMD). E.g. 128-bit wide AVX512 (if
> that's doable) would give you immunity against latency hazards up to
> four clock cycles.

It seems to be true, since AVX-512 at 64-byte cache line chunks have
finally reached parity with the original Larrabee design in this regard.

LRB was of course primarly designed to be able to handle GPU tasks, and
that included an easy way to mask leading/trailing parts that failed to
fill up a full cache line.

I do agree that it would have been a lot better for Intel to supply slow
512-bit support to all SKUs, then use the more power-hungry versions for
high-end cpus. I waited a year some years ago to get a (xeon) cpu which
included this, only to learn after the fact that AVX-512 actually wasn't
enabled on the mobile/portable workstation version.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Terje Mathisen wrote:
> Stefan Monnier wrote:
>>> 3) RVV or any other Cray-Style Vector ISA requiring strict
>>> compliance
>>>       with IEEE754 FP accuracy will punish you with a 400%
>>> power/area
>>>       penalty compared to modern 3D-optimised GPUs which, as
>>> explicitly
>>
>> What is the origin of this extra cost?  IOW, where's the meat of
>> the
>> savings (I mean: which part of the computation of the last few bits
>> costs so much)?
>> Does the saving vary significantly between instructions?
>
> I think that is pretty much bogus, Mitch have shown repeatedly
> that having FMAC makes subnormal handling nearly free, as in zero
> cycles and single-digit percentage gates/power. 400% is simply
> rubbish.

I am guessing that double-rounding FMADD would not provide as much
savings. Single-rounding is typically assumed, but I think the
original implementation on MIPS did double rounding (perhaps to
provide the same result as unfused FMUL and FADD).

(I kind of wonder what the costs would have been to implement
floating point multiplication as a high power-of-two bits
truncated result (i.e., ignoring carry-in from lower bits). I tend
to agree that the benefit of standardization limits different
interfaces to specialized uses, but there may have been a better
alternative. I also think that the probablistic rounding that Nick
Maclaren championed (a little) would have been interesting and
reduced or exposed some problems with floating point.)

On Sunday, August 21, 2022 at 8:46:03 PM UTC-5, Paul A. Clayton wrote:
> Terje Mathisen wrote:
> > Stefan Monnier wrote:
> >>> 3) RVV or any other Cray-Style Vector ISA requiring strict
> >>> compliance
> >>> with IEEE754 FP accuracy will punish you with a 400%
> >>> power/area
> >>> penalty compared to modern 3D-optimised GPUs which, as
> >>> explicitly
> >>
> >> What is the origin of this extra cost? IOW, where's the meat of
> >> the
> >> savings (I mean: which part of the computation of the last few bits
> >> costs so much)?
> >> Does the saving vary significantly between instructions?
> >
> > I think that is pretty much bogus, Mitch have shown repeatedly
> > that having FMAC makes subnormal handling nearly free, as in zero
> > cycles and single-digit percentage gates/power. 400% is simply
> > rubbish.
<
Who here as actually build a floating point unit at the gate level (FPGA
does not count) ??
<
> I am guessing that double-rounding FMADD would not provide as much
> savings. Single-rounding is typically assumed, but I think the
> original implementation on MIPS did double rounding (perhaps to
> provide the same result as unfused FMUL and FADD).
<
Various people have espoused several points of view recently,
including one who actually has build an FPU at the gate (and
transistor) level !
<
If you go to:: https://repositories.lib.utexas.edu/handle/2152/3082
you can get the whole story.
<
Basically, the Tree is about ½ of the FMAC unit. Since the tree is
53-bits tall, and CEIL( ln2( 53 ) ) = 8; In order to get the last bit
sort of correct, one can not chop the tree less than 61-bits.Thus
one ends up with a 53×53 triangle, and then 8 more bits with
dimensions and angles of (53 up, 8 horizontal, 45 down, and
8 at 45º) This ends up to be closer to 63% of the tree size compared
to a full 53×53 parallelogram.
<
Given that the tree was 50% and is now 50%×63%= 32%. The
unit is now 82% as big as it used to be. A savings of 18%.
<
This is a very far cry of 400%
>
> (I kind of wonder what the costs would have been to implement
> floating point multiplication as a high power-of-two bits
> truncated result (i.e., ignoring carry-in from lower bits). I tend
> to agree that the benefit of standardization limits different
> interfaces to specialized uses, but there may have been a better
> alternative. I also think that the probablistic rounding that Nick
> Maclaren championed (a little) would have been interesting and
> reduced or exposed some problems with floating point.)
<
At the time IEE754 was being done (the first version) there were
2 kinds of FP users, those who wanted fast FP (zillions) and those
who wanted correct answers (5--yes 5, maybe 6). It a quirky turn of
fate, 5 (who did not buy their own computers) won out over zillions
(who were willing to spend multi-millions per computer).
<
And now we are condemned to follow suit.

On 8/21/2022 6:45 PM, Paul A. Clayton wrote:
> Terje Mathisen wrote:
>> Stefan Monnier wrote:
>>>> 3) RVV or any other Cray-Style Vector ISA requiring strict compliance
>>>>       with IEEE754 FP accuracy will punish you with a 400% power/area
>>>>       penalty compared to modern 3D-optimised GPUs which, as explicitly
>>>
>>> What is the origin of this extra cost?  IOW, where's the meat of the
>>> savings (I mean: which part of the computation of the last few bits
>>> costs so much)?
>>> Does the saving vary significantly between instructions?
>>
>> I think that is pretty much bogus, Mitch have shown repeatedly that
>> having FMAC makes subnormal handling nearly free, as in zero cycles
>> and single-digit percentage gates/power. 400% is simply rubbish.
>
> I am guessing that double-rounding FMADD would not provide as much
> savings. Single-rounding is typically assumed, but I think the
> original implementation on MIPS did double rounding (perhaps to
> provide the same result as unfused FMUL and FADD).
>
> (I kind of wonder what the costs would have been to implement
> floating point multiplication as a high power-of-two bits
> truncated result (i.e., ignoring carry-in from lower bits). I tend
> to agree that the benefit of standardization limits different
> interfaces to specialized uses, but there may have been a better
> alternative. I also think that the probablistic rounding that Nick
> Maclaren championed (a little) would have been interesting and
> reduced or exposed some problems with floating point.)
>

I originally thought stochastic rounding was a great idea and put it in.
Kahan talked me out of it.

Ivan Godard wrote:
> On 8/21/2022 6:45 PM, Paul A. Clayton wrote:
[snip]>> alternative. I also think that the probablistic rounding
that Nick
>> Maclaren championed (a little) would have been interesting and
>> reduced or exposed some problems with floating point.)
>>
>
> I originally thought stochastic rounding was a great idea and put
> it in. Kahan talked me out of it.

I suspect Kahan was concerned with accuracy and provability while
Nick Maclaren was concerned with ordinary use where providing
jitter could either (rarely?) avoid some issues with practical
function (getting a meaningfully correct result) or make more
visible the existence of numerical issues with the computation. I
*think* Nick Maclaren considered variable results a positive
feature (i.e., FP should not be treated as integer with bit-exact
results).

Without hardware that supports such as an option, examining the
effects becomes much more difficult. Getting others to run their
code in real life conditions seems important for getting enough
data with fewer systematic 'errors'; even with a roughly equally
fast and reasonably available option, motivating such testing
seems likely to be challenging.

Paul A. Clayton wrote:
> Ivan Godard wrote:
>> On 8/21/2022 6:45 PM, Paul A. Clayton wrote:
> [snip]>> alternative. I also think that the probablistic rounding that Nick
>>> Maclaren championed (a little) would have been interesting and
>>> reduced or exposed some problems with floating point.)
>>>
>>
>> I originally thought stochastic rounding was a great idea and put it
>> in. Kahan talked me out of it.
>
> I suspect Kahan was concerned with accuracy and provability while Nick
> Maclaren was concerned with ordinary use where providing jitter could
> either (rarely?) avoid some issues with practical function (getting a
> meaningfully correct result) or make more visible the existence of
> numerical issues with the computation. I *think* Nick Maclaren
> considered variable results a positive feature (i.e., FP should not be
> treated as integer with bit-exact results).
>
> Without hardware that supports such as an option, examining the effects
> becomes much more difficult. Getting others to run their code in real
> life conditions seems important for getting enough data with fewer
> systematic 'errors'; even with a roughly equally fast and reasonably
> available option, motivating such testing seems likely to be challenging.

The best you can do these days to determine if you are close to the
endge of stability is to either rerun the calculation with fixed
rounding mode (truncate/floor/ceil) which will use the exact same time,
or you can manually truncate/jitter one or more of the bottom mantissa
bits at regular points in the calculation, but this requires source code
changes.

I can see both sides of Nick's argument:

a) It is good to teach FP programmers to never expect bit-exact
reproducibility, and instead verify error bars.

b) Having bit-reproducibility makes verification far easier.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Monday, August 22, 2022 at 3:18:19 AM UTC+1, MitchAlsup wrote:

> At the time IEE754 was being done (the first version) there were
> 2 kinds of FP users, those who wanted fast FP (zillions) and those
> who wanted correct answers (5--yes 5, maybe 6).

3D GPUs as you know are still focussed on speed (and power)
due to simple pragmatic issues that the screen resolutions simply
don't need high-accuracy to work out where a pixel goes. i.e.: if the
screen is 1280x800 you certainly don't need accuracy above
~14 bits, it is wasted silicon and power. this was why MIPS 3D ASE
ops were designed with low-accuracy.

the *influence* of 3D GPU manufacturers although they are
small in number is much higher (behind closed doors such
as the Khronos Group) and so we just don't hear about
it.

Tom Forsyth on the other hand in his talk on Larrabee mentions
that they concentrated on FP32, missed the goal of being a
commercially-viable 3D GPU Card (performance and power)
so attempted to target the scientific market instead, only to
be told that the FP64 performance sucked and consequently
they fell between two stools.

wark-wark.

damned if you do, damned if you don't.

l.