Hi!

I would like to ask a couple of simple questions in this group, related
to the IEEE 754 standard and floating-point exceptions.

(Some of you may know that I've been pondering this subject before in
other forums).

As we all know, support for floating-point exceptions comes with a cost
(sometimes significant, since it dictates many aspects of a HW
implementation for instance). If we imagine an alternative universe in
which the dominant floating-point standard did not require exceptions,
I suspect that most hardware (and language) implementations would most
likely be simpler, more energy efficient and more suitable for parallel
and pipelined floating-point operations.

My questions are:

1) What (in your opinion) are the benefits of floating-point exceptions?

2) In what situations have you had use for floating-point exceptions?

/Marcus

On 2021-07-21 15:44, Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:

To answer my own questions... In my fields which over the past 2-3
decades include 3D graphics, ray tracing, 3D polygonal mesh manipulation
algorithms, compression algorithms for triangle meshes, audio DSP, audio
synthesis, audio compression, space/satellite signal processing, image
processing and compression algorithms, AI / neural networks, image-based
eye- and head-tracking, least squares solvers, and so on, etc, I have to
say:

>
> 1) What (in your opinion) are the benefits of floating-point exceptions?
>

None

> 2) In what situations have you had use for floating-point exceptions?
>

None

In article <sd98c9$cln$1@dont-email.me>, m.delete@this.bitsnbites.eu
(Marcus) wrote:

> 1) What (in your opinion) are the benefits of floating-point
> exceptions?

They allow detection of problems closer to the site of the bad code, or
when reading the bad data. The mathematical modeller I work on has many
iterative floating-point algorithms, and predicting what they will do for
large sets of input data is beyond human capabilities.

> 2) In what situations have you had use for floating-point
> exceptions?

Finding bugs of our own, compiler bugs and bad data in bug reports.

The lack of floating-point exceptions in actual implementations of ARM64
is one of my biggest worries about its increasing popularity.

John

On 2021-07-21 15:56, John Dallman wrote:
> In article <sd98c9$cln$1@dont-email.me>, m.delete@this.bitsnbites.eu
> (Marcus) wrote:
>
>> 1) What (in your opinion) are the benefits of floating-point
>> exceptions?
>
> They allow detection of problems closer to the site of the bad code, or
> when reading the bad data. The mathematical modeller I work on has many
> iterative floating-point algorithms, and predicting what they will do for
> large sets of input data is beyond human capabilities.
>
>> 2) In what situations have you had use for floating-point
>> exceptions?
>
> Finding bugs of our own, compiler bugs and bad data in bug reports.
>
> The lack of floating-point exceptions in actual implementations of ARM64
> is one of my biggest worries about its increasing popularity.
>
> John
>

Oh, interesting. It seems that Microsoft has recognized this. Quoting
their Windows ARM64 ABI conventions [1]:

"For processor variants that do have hardware floating-point exceptions,
the Windows kernel silently catches the exceptions and implicitly
disables them in the FPCR register. This trap ensures normalized
behavior across processor variants. Otherwise, code developed on a
platform without exception support may find itself taking unexpected
exceptions when running on a platform with support."

....which of course is the natural solution. In our software we do the
same thing (even disabling subnormals) in order to normalize on the
least common denominator, so to say (as soon as one target platform
lacks a feature, it needs to be disabled on _all_ platforms).

/Marcus

[1]
https://docs.microsoft.com/en-us/cpp/build/arm64-windows-abi-conventions?view=msvc-160#floating-point-exceptions

Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:
>
> 1) What (in your opinion) are the benefits of floating-point exceptions?

Mostly theoretical

>
> 2) In what situations have you had use for floating-point exceptions?

Never.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On 2021-07-21 15:56, John Dallman wrote:
> In article <sd98c9$cln$1@dont-email.me>, m.delete@this.bitsnbites.eu
> (Marcus) wrote:
>
>> 1) What (in your opinion) are the benefits of floating-point
>> exceptions?
>
> They allow detection of problems closer to the site of the bad code, or
> when reading the bad data. The mathematical modeller I work on has many
> iterative floating-point algorithms, and predicting what they will do for
> large sets of input data is beyond human capabilities.
>
>> 2) In what situations have you had use for floating-point
>> exceptions?
>
> Finding bugs of our own, compiler bugs and bad data in bug reports.

For these use cases, would it not suffice with a compiler/software
solution similar to the sanitizers that are available in LLVM and GCC
for instance? E.g. see:

* https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
* https://dl.acm.org/doi/epdf/10.1145/3446804.3446848

I regularly enable sanitizers during test & development, and for most
purposes they tend to have an acceptable performance overhead.

> The lack of floating-point exceptions in actual implementations of ARM64
> is one of my biggest worries about its increasing popularity.
>
> John
>

In article <sd9a9h$ro6$1@dont-email.me>, m.delete@this.bitsnbites.eu
(Marcus) wrote:

> ...which of course is the natural solution.

It's the natural solution if you assume nobody will ever actually want
floating-point exceptions. A far more helpful solution would be what they
do on x86, which is to disable the exceptions by default and to provide a
CRT call to turn on the ones that you want.

> In our software we do the same thing (even disabling subnormals)
> in order to normalize on the least common denominator, so to say
> (as soon as one target platform lacks a feature, it needs to be
> disabled on _all_ platforms).

We're willing to try to make use of useful features of particular
platforms.

In our test harness, we turn on exceptions for overflow, invalid and
divide-by-zero, where we can. We leave denormal, inexact and underflow
disabled, since we want behaviour as much as possible like natural
numbers. We turn on flushing of denormals to zero where it's available.

The customers can use whatever floating-point handling they want, but we
warn them that exceptions on denormal, inexact or underflow will cause
problems.

John

It's not that the semantics of exceptions are useless, it's that the
IEEE standard's approach to them is mis-designed for modern
architectures and hence IEEE exceptions as a programming device are
effectively useless.

The definition of flags and modes in the standard makes them implicit
arguments to every instruction - which inherently linearizes the FP
instruction stream, while modern chips want to do a lot of FPs
concurrently. During my active time on the committee Kahan was still
plumping to use exceptions to support trap-value-replacement. You could
do that in Mathlab, but not in hardware. That linearization not only
breaks multi-issue and SIMD, it also precludes speculative execution of
FP ops in general - no computation overruns in loops for example.

Once you abandon value-replacement, exceptions become what the word
"exception" means in other contexts: a non-recoverable reporting
mechanism that things went off the rails somewhere. As such, as in any
debugging assist, the design as much as possible should serve to narrow
the scope of "somewhere" while not hindering computations that remain on
the rails.

In the Mill we do that by elimination the global flags and modes and
moving the event reporting into metadata attached to the result of the
failing computation. You can speculatively or concurrently do FP ops on
a Mill, and when the erroneous result finally must be used
non-speculatively (if it ever is), and you have not asked to ignore
bogus results, then the resulting fault can report fairly well where
things went bad. That really helps, not because of some numerical nicety
but because real programmers spend their lives chasing bugs.

So I use exceptions everywhere that an exception would indicate a bug.
However, even in debugging I don't use inexact; I have never used an
algorithm where I cared that rounding had happened, and in practice
inexact always gets set and so is just nuisance noise. Inexact was
intended to be a support for using FP for fixed-point calculations, but
if I had a fixed-point problem I would use DFP.

Ivan

On 7/21/2021 6:44 AM, Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:
>
> 1) What (in your opinion) are the benefits of floating-point exceptions?
>
> 2) In what situations have you had use for floating-point exceptions?
>
>
> /Marcus

In article <sd9b6k$2q4$1@dont-email.me>, m.delete@this.bitsnbites.eu
(Marcus) wrote:

> > Finding bugs of our own, compiler bugs and bad data in bug
> > reports.
>
> For these use cases, would it not suffice with a compiler/software
> solution similar to the sanitizers that are available in LLVM and
> GCC for instance? E.g. see:
>
> * https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
> * https://dl.acm.org/doi/epdf/10.1145/3446804.3446848

Possibly. An advantage of floating-point exceptions is that they work
when testing the *same* binary that will be delivered to the customer.

We've been using floating-point traps with this product since the 1980s,
and Valgrind for over a decade, and it is time to try out compiler
sanitisers. When I get a chance...

The UndefinedBehaviour sanitiser looks most useful for:

-fsanitize=alignment, since the platforms that were fussy about that are
all dead or nearly so.

-fsanitize=float-divide-by-zero and -fsanitize=integer-divide-by-zero are
a partial substitute for floating-point traps.

These also look worth a try: -fsanitize=builtin, -fsanitize=bounds,
-fsanitize=null, -fsanitize=pointer-overflow,
-fsanitize=signed-integer-overflow, and
-fsanitize=unsigned-integer-overflow.

Less useful ones are:

-fsanitize=float-cast-overflow, because to a very close approximation, we
don't convert floats or doubles to ints.

The NSan sanitiser will catch overflows, but it's not clear to me that it
will catch invalid operands. It's also going to be pretty slow, because
everything is doubles, forcing the shadow values to be 128-bit and thus
partially done in software.

> I regularly enable sanitizers during test & development, and for
> most purposes they tend to have an acceptable performance overhead.

I'm guessing you don't have a couple of hundred machines and VMs
operating distributed testing systems to get the overnight build tested
in time for a ship/don't-ship decision the following afternoon?

Sanitiser overhead is going to demand separate builds, that aren't
required to complete their testing in time for that ship/no-ship decision,
but still may be useful.

Thanks!

John

On 7/21/2021 8:44 AM, Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:
>
> 1) What (in your opinion) are the benefits of floating-point exceptions?
>
> 2) In what situations have you had use for floating-point exceptions?
>

FWIW:
This was another feature I didn't bother with in BJX2;
If you want exceptions, they would need to be implemented in the
compiler as a "trap if you see a NaN here" instruction sequence.
But, this is unlikely, given their relative level of usefulness.

The compiler can go the other direction though, as I sit around
debugging some other arguably not-very-useful features:
__int128
Some arithmetic bugs/... showed up when "actually using it".
_BitInt(n), special where n>128, ...
eg, "_BitInt(768) ib0;", create a 768 bit integer value...
Decays into a basic integer type if n<=128.
Where 'n' is required to be greater than zero.
....

But, at least stuff like this can be sometimes useful...

Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:
>
> 1) What (in your opinion) are the benefits of floating-point exceptions?
>
> 2) In what situations have you had use for floating-point exceptions?
>
>
> /Marcus

Why do you believe the HW cost of exception support
is sometimes significant? It looks to me that as long as one
triggers these at Retire/Write-Back, precise exceptions are dirt cheap.

There is an initial cost for the first exception,
and an incremental cost for each one after that.

Consider a DSP with no exceptions, no MMU so no access faults,
no privileged mode, no integer divide-by-zero etc. Nothing.

It seems to me that the initial cost of adding a precise exception to
that is 1 bit in the uOp along with the result. When the uOp reaches
write-back, WB sees the exception flag and inhibits the register write,
and diddles a wire that flushes the pipeline the same as an
indirect branch, and jams an unconditional jump IP into the fetch unit,
which jams an "I'm an exception" uOp into the pipeline,
and starts fetching from the new address.

In short, most of the logic is already present to handle
indirect branches so the initial cost is a few gates and a long wire.

The additional cost of FP exceptions is also seems small.
The FP status flags have to travel with the FP result too,
which write-back writes to the committed status flags.
The additional cost of FP exceptions is carrying the exception mask
bits with the uOp, and a AND-OR on those bits in Retire/WB.

Marcus <m.delete@this.bitsnbites.eu> writes:
>As we all know, support for floating-point exceptions comes with a cost
>(sometimes significant, since it dictates many aspects of a HW
>implementation for instance).

Unfortunately, wrt. IEEE FP "exception" has a different meaning (it's
a condition that, by default, results in setting a sticky flag) than
in computer architecture (a control-flow change). Which one do you
mean?

>If we imagine an alternative universe in
>which the dominant floating-point standard did not require exceptions,
>I suspect that most hardware (and language) implementations would most
>likely be simpler, more energy efficient and more suitable for parallel
>and pipelined floating-point operations.

Detecting IEEE exceptions is part of computation of the result; you
have to handle divide-by-zero, inexact, invalid, overflow, and
underflow conditions when computing the result. Ok, you then need to
propagate the flags such that a later instruction can read it, but
looking at the integer condition codes present in the dominant
architectures, that does not seem to be a big problem.

The dynamic rounding mode seems to be a bigger problem and has been
discussed here repeatedly.

>My questions are:
>
>1) What (in your opinion) are the benefits of floating-point exceptions?

Numerical experts use them for special purposes. I have seen and
forgotten examples that looked sensible to me.

>2) In what situations have you had use for floating-point exceptions?

None. I hardly use FP.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Wednesday, July 21, 2021 at 8:44:44 AM UTC-5, Marcus wrote:
> Hi!
>
> I would like to ask a couple of simple questions in this group, related
> to the IEEE 754 standard and floating-point exceptions.
>
> (Some of you may know that I've been pondering this subject before in
> other forums).
>
> As we all know, support for floating-point exceptions comes with a cost
> (sometimes significant, since it dictates many aspects of a HW
> implementation for instance). If we imagine an alternative universe in
> which the dominant floating-point standard did not require exceptions,
> I suspect that most hardware (and language) implementations would most
> likely be simpler, more energy efficient and more suitable for parallel
> and pipelined floating-point operations.
>
> My questions are:
>
> 1) What (in your opinion) are the benefits of floating-point exceptions?
<
There are 6 people in the world who want FP exceptions with the semantic
anywhere close to what IEEE 754 specifies. It is these same 6 people (and
18 professors) that want rounding modes anywhere close to what was
specified.
<
Something like 99% of all FP applications, turn these features off and
live with overflow->infinity, underflow->zero, and I still don't know anyone
other than Kahan and Coonen that want inexact reported. 100% of
physics programs start with data that is ALREADY inexact........
>
> 2) In what situations have you had use for floating-point exceptions?
<
None:
<
3) Is making exception detection in HW hard at all ?
<
No, absolutely not. Detecting and raising exceptions in HW adds 0.0001%
complexity to the design and construction of HW FP units {In a processor
that has precise memory exceptions and recovers from branch mispredictions.}
<
It is for this reason that all HW provides FP exceptions--the cost is minuscule
at most and nearly zero at best.
>
>
> /Marcus

On Wednesday, July 21, 2021 at 9:35:40 AM UTC-5, John Dallman wrote:
> In article <sd9a9h$ro6$1...@dont-email.me>, m.de...@this.bitsnbites.eu
> (Marcus) wrote:
>
> > ...which of course is the natural solution.
> It's the natural solution if you assume nobody will ever actually want
> floating-point exceptions. A far more helpful solution would be what they
> do on x86, which is to disable the exceptions by default and to provide a
> CRT call to turn on the ones that you want.
<
I mentioned above that FP exceptions add essentially zero cost to the FP units.
<
What I did not mention, is that My 66000 allows the thread to manage its own
rounding modes and exception recognition. {It can only damage itself!}
<
> > In our software we do the same thing (even disabling subnormals)
> > in order to normalize on the least common denominator, so to say
> > (as soon as one target platform lacks a feature, it needs to be
> > disabled on _all_ platforms).
<
Once you buy off on FMAC units, denorms add almost zero cost (around 2%
for 64-bit FMACs) it makes no sense to set/assume denorms to zero.
This is NOT and argument that denorms were the right thing to put into
754 at the beginning. But its like the steering wheel location in a car--
its too late to change it now.
<
> We're willing to try to make use of useful features of particular
> platforms.
>
> In our test harness, we turn on exceptions for overflow, invalid and
> divide-by-zero, where we can. We leave denormal, inexact and underflow
> disabled, since we want behaviour as much as possible like natural
> numbers. We turn on flushing of denormals to zero where it's available.
>
> The customers can use whatever floating-point handling they want, but we
> warn them that exceptions on denormal, inexact or underflow will cause
> problems.
>
>
> John

On Wednesday, July 21, 2021 at 10:44:22 AM UTC-5, Ivan Godard wrote:
> It's not that the semantics of exceptions are useless, it's that the
> IEEE standard's approach to them is mis-designed for modern
> architectures and hence IEEE exceptions as a programming device are
> effectively useless.
>
> The definition of flags and modes in the standard makes them implicit
> arguments to every instruction - which inherently linearizes the FP
> instruction stream, while modern chips want to do a lot of FPs
> concurrently.
<
Not really {when done right--which x86 did not do BTW}
<
As long as there is not a reader of the flags, the flags can be accumulated
with an OR gate from the writers in any exception free order.
<
As long as there is no writer to the mode bits, these can be passed en massé
to the function units. In fact since these mode bits are necessarily different
between threads on the same core, one has to pass the mode bits with each
instruction issued (at least to the stations.)
<
> During my active time on the committee Kahan was still
> plumping to use exceptions to support trap-value-replacement. You could
> do that in Mathlab, but not in hardware. That linearization not only
> breaks multi-issue and SIMD, it also precludes speculative execution of
> FP ops in general - no computation overruns in loops for example.
<
Yes, this was a step too far (way too far, essentially serializing FP)
<
However, I found ways to implement this at low cost--but the first cost to
lower is the raising of an exception has to come down from 1,000s of
instructions or cycles to 10s !! The exception control transfer has to be
efficient--within spitting distance of a subroutine call.
>
> Once you abandon value-replacement, exceptions become what the word
> "exception" means in other contexts: a non-recoverable reporting
> mechanism that things went off the rails somewhere. As such, as in any
> debugging assist, the design as much as possible should serve to narrow
> the scope of "somewhere" while not hindering computations that remain on
> the rails.
>
> In the Mill we do that by elimination the global flags and modes and
> moving the event reporting into metadata attached to the result of the
> failing computation. You can speculatively or concurrently do FP ops on
> a Mill, and when the erroneous result finally must be used
> non-speculatively (if it ever is), and you have not asked to ignore
> bogus results, then the resulting fault can report fairly well where
> things went bad. That really helps, not because of some numerical nicety
> but because real programmers spend their lives chasing bugs.
>
> So I use exceptions everywhere that an exception would indicate a bug.
> However, even in debugging I don't use inexact; I have never used an
> algorithm where I cared that rounding had happened, and in practice
> inexact always gets set and so is just nuisance noise. Inexact was
> intended to be a support for using FP for fixed-point calculations, but
> if I had a fixed-point problem I would use DFP.
<
But even when used to perform exact floating point arithmetic, the inexact
flag necessarily gets set !! by 754 mandated rules !! this is completely fubar !
<
At least My 66000 can perform these and not end up setting the inexact flag..
>

On Wednesday, July 21, 2021 at 12:19:25 PM UTC-5, EricP wrote:
> Marcus wrote:
> > Hi!
> >
> > I would like to ask a couple of simple questions in this group, related
> > to the IEEE 754 standard and floating-point exceptions.
> >
> > (Some of you may know that I've been pondering this subject before in
> > other forums).
> >
> > As we all know, support for floating-point exceptions comes with a cost
> > (sometimes significant, since it dictates many aspects of a HW
> > implementation for instance). If we imagine an alternative universe in
> > which the dominant floating-point standard did not require exceptions,
> > I suspect that most hardware (and language) implementations would most
> > likely be simpler, more energy efficient and more suitable for parallel
> > and pipelined floating-point operations.
> >
> > My questions are:
> >
> > 1) What (in your opinion) are the benefits of floating-point exceptions?
> >
> > 2) In what situations have you had use for floating-point exceptions?
> >
> >
> > /Marcus
> Why do you believe the HW cost of exception support
> is sometimes significant? It looks to me that as long as one
> triggers these at Retire/Write-Back, precise exceptions are dirt cheap.
<
Essentially zero, compared to the cost of denorm support in the 2% range
{gates, area, power}.
>
> There is an initial cost for the first exception,
> and an incremental cost for each one after that.
>
> Consider a DSP with no exceptions, no MMU so no access faults,
> no privileged mode, no integer divide-by-zero etc. Nothing.
>
> It seems to me that the initial cost of adding a precise exception to
> that is 1 bit in the uOp along with the result. When the uOp reaches
> write-back, WB sees the exception flag and inhibits the register write,
> and diddles a wire that flushes the pipeline the same as an
> indirect branch, and jams an unconditional jump IP into the fetch unit,
> which jams an "I'm an exception" uOp into the pipeline,
> and starts fetching from the new address.
>
> In short, most of the logic is already present to handle
> indirect branches so the initial cost is a few gates and a long wire.
<
Once you can flush instructions from the pipe (like a branch does)
the HW overhead in the pipe is essentially zero. The HW overhead
in the FU is essentially zero. There is a standard that mandates them
so why not just take the easy route through this feature space.
<
IEEE 754-2019 compatible :: YES in all respects.
>
> The additional cost of FP exceptions is also seems small.
> The FP status flags have to travel with the FP result too,
> which write-back writes to the committed status flags.
> The additional cost of FP exceptions is carrying the exception mask
> bits with the uOp, and a AND-OR on those bits in Retire/WB.

On 2021-07-21 17:47, John Dallman wrote:
> In article <sd9b6k$2q4$1@dont-email.me>, m.delete@this.bitsnbites.eu
> (Marcus) wrote:
>
>>> Finding bugs of our own, compiler bugs and bad data in bug
>>> reports.
>>
>> For these use cases, would it not suffice with a compiler/software
>> solution similar to the sanitizers that are available in LLVM and
>> GCC for instance? E.g. see:
>>
>> * https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
>> * https://dl.acm.org/doi/epdf/10.1145/3446804.3446848
>
> Possibly. An advantage of floating-point exceptions is that they work
> when testing the *same* binary that will be delivered to the customer.
>
> We've been using floating-point traps with this product since the 1980s,
> and Valgrind for over a decade, and it is time to try out compiler
> sanitisers. When I get a chance...

I strongly recommend it. I find that sanitizers are very useful and
accurate. It can be quite "hilarious" to run ubsan on old code bases
(lots of "this should never have worked" moments).

>
> The UndefinedBehaviour sanitiser looks most useful for:
>
> -fsanitize=alignment, since the platforms that were fussy about that are
> all dead or nearly so.
>
> -fsanitize=float-divide-by-zero and -fsanitize=integer-divide-by-zero are
> a partial substitute for floating-point traps.
>
> These also look worth a try: -fsanitize=builtin, -fsanitize=bounds,
> -fsanitize=null, -fsanitize=pointer-overflow,
> -fsanitize=signed-integer-overflow, and
> -fsanitize=unsigned-integer-overflow.
>
> Less useful ones are:
>
> -fsanitize=float-cast-overflow, because to a very close approximation, we
> don't convert floats or doubles to ints.
>
>
> The NSan sanitiser will catch overflows, but it's not clear to me that it
> will catch invalid operands. It's also going to be pretty slow, because
> everything is doubles, forcing the shadow values to be 128-bit and thus
> partially done in software.
>
>> I regularly enable sanitizers during test & development, and for
>> most purposes they tend to have an acceptable performance overhead.
>
> I'm guessing you don't have a couple of hundred machines and VMs
> operating distributed testing systems to get the overnight build tested
> in time for a ship/don't-ship decision the following afternoon?

I think it all depends on how you design your system. We have about 40
machines, each with 16-32 CPU cores (a total of about 2000 vCPU:s in
cloud terms), and they build and test about 50 different SW
configurations, in roughly 10-15 minutes (it's been a full time job the
last few years to keep that time at an acceptable level). My philosophy
has always been to test as much as possible /before/ a change goes into
mainline (a.k.a. "stable mainline" or "not rocket science" - don't break
stuff so that developers can't work at full speed).

BTW we recently activated sanitizers for all unit tests running
on those machines. Just buy more hardware ;-) (it's not quite that
simple, but the sentiment is: don't allow HW limitations to get in the
way of SW quality).

>
> Sanitiser overhead is going to demand separate builds, that aren't
> required to complete their testing in time for that ship/no-ship decision,
> but still may be useful.
>
> Thanks!
>
> John
>

On 2021-07-21 20:27, MitchAlsup wrote:
> On Wednesday, July 21, 2021 at 8:44:44 AM UTC-5, Marcus wrote:
>> Hi!
>>
>> I would like to ask a couple of simple questions in this group, related
>> to the IEEE 754 standard and floating-point exceptions.
>>
>> (Some of you may know that I've been pondering this subject before in
>> other forums).
>>
>> As we all know, support for floating-point exceptions comes with a cost
>> (sometimes significant, since it dictates many aspects of a HW
>> implementation for instance). If we imagine an alternative universe in
>> which the dominant floating-point standard did not require exceptions,
>> I suspect that most hardware (and language) implementations would most
>> likely be simpler, more energy efficient and more suitable for parallel
>> and pipelined floating-point operations.
>>
>> My questions are:
>>
>> 1) What (in your opinion) are the benefits of floating-point exceptions?
> <
> There are 6 people in the world who want FP exceptions with the semantic
> anywhere close to what IEEE 754 specifies. It is these same 6 people (and
> 18 professors) that want rounding modes anywhere close to what was
> specified.
> <
> Something like 99% of all FP applications, turn these features off and
> live with overflow->infinity, underflow->zero, and I still don't know anyone
> other than Kahan and Coonen that want inexact reported. 100% of
> physics programs start with data that is ALREADY inexact........

Thanks, that confirms my feelings.....

>>
>> 2) In what situations have you had use for floating-point exceptions?
> <
> None:
> <
> 3) Is making exception detection in HW hard at all ?
> <
> No, absolutely not. Detecting and raising exceptions in HW adds 0.0001%
> complexity to the design and construction of HW FP units {In a processor
> that has precise memory exceptions and recovers from branch mispredictions.}
> <
> It is for this reason that all HW provides FP exceptions--the cost is minuscule
> at most and nearly zero at best.

I admit that I am just a layman in this matter, but I think that there
are more dimensions to this problem, and the related costs.

For instance, the very notion of floating-point exceptions (be they
exact HW traps, inexact "something went wrong" traps, or sticky flags
that you can poll, or something else) requires that you can make some
use of them - for instance take some corrective action in software,
using means that usually depend on the programming language.

In certain classes of CPU and programming language combinations there
are straight forward ways of how to deal with this (e.g. HW traps +
language exceptions), but in other environments the answer is not as
simple.

For instance, how should an overflow exception be handled in a GPU
shader language? I'm pretty sure that no HW/language combination
supports that scenario today (and thus by definition can not be
IEEE 754 compliant), and I'm also pretty sure that the cost for adding
that support would be non-zero - not only for the GPU ALU pipeline, but
for the entire HW/driver/SW stack.

What's more - such support would most likely /get in the way/ for
programmers since even fewer than the 6 people that you mentioned
earlier would even be remotely interested in such functionality.

So the rebellion in me wants to say: "Ask not how hard it is to add
floating-point exceptions to a CPU pipeline - ask what the point is
of doing so."

/Marcus

On 2021-07-21 18:58, Anton Ertl wrote:
> Marcus <m.delete@this.bitsnbites.eu> writes:
>> As we all know, support for floating-point exceptions comes with a cost
>> (sometimes significant, since it dictates many aspects of a HW
>> implementation for instance).
>
> Unfortunately, wrt. IEEE FP "exception" has a different meaning (it's
> a condition that, by default, results in setting a sticky flag) than
> in computer architecture (a control-flow change). Which one do you
> mean?

It does not matter for answering the questions. I am aware of the
different possibilities and interpretations. For instance I think that
some TI DSP:s use sticky flags rather than HW traps.

That said I see problems with both implementations (traps vs flags), if
nothing else from a programmer's point of view.

>
>> If we imagine an alternative universe in
>> which the dominant floating-point standard did not require exceptions,
>> I suspect that most hardware (and language) implementations would most
>> likely be simpler, more energy efficient and more suitable for parallel
>> and pipelined floating-point operations.
>
> Detecting IEEE exceptions is part of computation of the result; you
> have to handle divide-by-zero, inexact, invalid, overflow, and
> underflow conditions when computing the result. Ok, you then need to
> propagate the flags such that a later instruction can read it, but
> looking at the integer condition codes present in the dominant
> architectures, that does not seem to be a big problem.
>
> The dynamic rounding mode seems to be a bigger problem and has been
> discussed here repeatedly.

I agree - that's the second grudge that I have with the IEEE 754
standard. As a programmer I have gotten nothing but problems from the
dynamic rounding modes. As a hardware developer I have just ignored the
problem (I only implement a single rounding mode).

>
>> My questions are:
>>
>> 1) What (in your opinion) are the benefits of floating-point exceptions?
>
> Numerical experts use them for special purposes. I have seen and
> forgotten examples that looked sensible to me.
>
>> 2) In what situations have you had use for floating-point exceptions?
>
> None. I hardly use FP.

Same - except I use FP all the time.

>
> - anton
>

On Wednesday, July 21, 2021 at 2:41:50 PM UTC-5, Marcus wrote:
> On 2021-07-21 20:27, MitchAlsup wrote:
> > On Wednesday, July 21, 2021 at 8:44:44 AM UTC-5, Marcus wrote:
> >> Hi!
> >>
> >> I would like to ask a couple of simple questions in this group, related
> >> to the IEEE 754 standard and floating-point exceptions.
> >>
> >> (Some of you may know that I've been pondering this subject before in
> >> other forums).
> >>
> >> As we all know, support for floating-point exceptions comes with a cost
> >> (sometimes significant, since it dictates many aspects of a HW
> >> implementation for instance). If we imagine an alternative universe in
> >> which the dominant floating-point standard did not require exceptions,
> >> I suspect that most hardware (and language) implementations would most
> >> likely be simpler, more energy efficient and more suitable for parallel
> >> and pipelined floating-point operations.
> >>
> >> My questions are:
> >>
> >> 1) What (in your opinion) are the benefits of floating-point exceptions?
> > <
> > There are 6 people in the world who want FP exceptions with the semantic
> > anywhere close to what IEEE 754 specifies. It is these same 6 people (and
> > 18 professors) that want rounding modes anywhere close to what was
> > specified.
> > <
> > Something like 99% of all FP applications, turn these features off and
> > live with overflow->infinity, underflow->zero, and I still don't know anyone
> > other than Kahan and Coonen that want inexact reported. 100% of
> > physics programs start with data that is ALREADY inexact........
> Thanks, that confirms my feelings.....
> >>
> >> 2) In what situations have you had use for floating-point exceptions?
> > <
> > None:
> > <
> > 3) Is making exception detection in HW hard at all ?
> > <
> > No, absolutely not. Detecting and raising exceptions in HW adds 0.0001%
> > complexity to the design and construction of HW FP units {In a processor
> > that has precise memory exceptions and recovers from branch mispredictions.}
> > <
> > It is for this reason that all HW provides FP exceptions--the cost is minuscule
> > at most and nearly zero at best.
> I admit that I am just a layman in this matter, but I think that there
> are more dimensions to this problem, and the related costs.
>
> For instance, the very notion of floating-point exceptions (be they
> exact HW traps, inexact "something went wrong" traps, or sticky flags
> that you can poll, or something else) requires that you can make some
> use of them - for instance take some corrective action in software,
> using means that usually depend on the programming language.
>
> In certain classes of CPU and programming language combinations there
> are straight forward ways of how to deal with this (e.g. HW traps +
> language exceptions), but in other environments the answer is not as
> simple.
>
> For instance, how should an overflow exception be handled in a GPU
> shader language?
<
When I was doing a GPU, I actually ask the question in front of the majority
of the design team:: "What, exactly, does it mean when 1,357 FP calculations
all overflow in the same clock cycle ?" What do we want the semantics to be?"
<
Nobody had an answer...........which tells you more about the problem than
of the team......
<
> I'm pretty sure that no HW/language combination
> supports that scenario today (and thus by definition can not be
> IEEE 754 compliant), and I'm also pretty sure that the cost for adding
> that support would be non-zero - not only for the GPU ALU pipeline, but
> for the entire HW/driver/SW stack.
<
In a GPU you can isolate the thread with the exception and run the rest
to completion--then rerun the thread in isolation--you can get away with
this because of the embarrassingly large amounts of parallelism.
<
This, however, fails in GPGPU applications, so it provides no insight
en the large.
>
> What's more - such support would most likely /get in the way/ for
> programmers since even fewer than the 6 people that you mentioned
> earlier would even be remotely interested in such functionality.
>
> So the rebellion in me wants to say: "Ask not how hard it is to add
> floating-point exceptions to a CPU pipeline - ask what the point is
> of doing so."
<
The thing is that memory has exceptions, DIV has exceptions that are
already present, DECODE may have exceptions, Stores may have late
exceptions; and once the pipeline has been configured to deal with
memory and and the others, the infrastructure is already present.
<
So I ask:: "Why not" ???
<
It is only when you can get rid of memory exceptions that you can get
rid of the pipeline infrastructure. {On the other hand this is already a
"solved problem" in computer design, so it is not perceived as even
hard, just work.
>
> /Marcus

> For instance, the very notion of floating-point exceptions (be they
> exact HW traps, inexact "something went wrong" traps, or sticky flags
> that you can poll, or something else) requires that you can make some
> use of them - for instance take some corrective action in software,
> using means that usually depend on the programming language.

I always assumed (for no concrete reason) that the main users of those
flags are very special-purpose code such as the implementation of the
`sin` or `atan` functions, or the implementation of 128bit floats
primitives on top of 64bit float hardware ops, ...

Stefan

On Wednesday, July 21, 2021 at 11:15:48 PM UTC+3, Stefan Monnier wrote:
> > For instance, the very notion of floating-point exceptions (be they
> > exact HW traps, inexact "something went wrong" traps, or sticky flags
> > that you can poll, or something else) requires that you can make some
> > use of them - for instance take some corrective action in software,
> > using means that usually depend on the programming language.
> I always assumed (for no concrete reason) that the main users of those
> flags are very special-purpose code such as the implementation of the
> `sin` or `atan` functions, or the implementation of 128bit floats
> primitives on top of 64bit float hardware ops, ...
>
>
> Stefan

I very certain that exceptions do not help sin or atan.
I am only like 95% certain that exceptions don't help your second case,
but I am not aware of environments that do anything like that in practice.

On Wednesday, July 21, 2021 at 3:15:48 PM UTC-5, Stefan Monnier wrote:
> > For instance, the very notion of floating-point exceptions (be they
> > exact HW traps, inexact "something went wrong" traps, or sticky flags
> > that you can poll, or something else) requires that you can make some
> > use of them - for instance take some corrective action in software,
> > using means that usually depend on the programming language.
> I always assumed (for no concrete reason) that the main users of those
> flags are very special-purpose code such as the implementation of the
> `sin` or `atan` functions, or the implementation of 128bit floats
> primitives on top of 64bit float hardware ops, ...
<
Things like ATAN do the following--to avoid exceptions !!
<
double ATAN2( double y, double x )
{ // IEEE 754-2019 quality ATAN2
// deal with NANs
if( ISNAN( x ) ) return x;
if( ISNAN( y ) ) return y;
// deal with infinities
if( x == +∞ && |y|== +∞ ) return copysign( π/4, y );
if( x == +∞ ) return copysign( 0.0, y );
if( x == -∞ && |y|== +∞ ) return copysign( 3π/4, y );
if( x == -∞ ) return copysign( π, y );
if( |y|== +∞ ) return copysign( π/2, y );
// deal with signed zeros
if( x == 0.0 && y != 0.0 ) return copysign( π/2, y );
if( x >=+0.0 && y == 0.0 ) return copysign( 0.0, y );
if( x <=-0.0 && y == 0.0 ) return copysign( π, y );
// calculate ATAN2 high performance style
if( x > 0.0 )
{
if( y < 0.0 && |y| < |x| ) return - π/2 - ATAN( x / y );
if( y < 0.0 && |y| > |x| ) return + ATAN( y / x );
if( y > 0.0 && |y| < |x| ) return + ATAN( y / x );
if( y > 0.0 && |y| > |x| ) return + π/2 - ATAN( x / y );
}
if( x < 0.0 )
{
if( y < 0.0 && |y| > |x| ) return + π/2 + ATAN( x / y );
if( y < 0.0 && |y| > |x| ) return + π - ATAN( y / x );
if( y > 0.0 && |y| < |x| ) return + π - ATAN( y / x );
if( y > 0.0 && |y| > |x| ) return +3π/2 + ATAN( x / y );
}
}

>
>
> Stefan

On Wednesday, July 21, 2021 at 4:44:22 PM UTC+1, Ivan Godard wrote:

> The definition of flags and modes in the standard makes them implicit
> arguments to every instruction - which inherently linearizes the FP
> instruction stream, while modern chips want to do a lot of FPs
> concurrently.

i solved this with data-dependent fail-first. the same trick
shouuld be possible with VVM on an OoO engine by relying on
"Precise Shadowing"

> During my active time on the committee Kahan was still
> plumping to use exceptions to support trap-value-replacement. You could
> do that in Mathlab, but not in hardware. That linearization not only
> breaks multi-issue and SIMD, it also precludes speculative execution of
> FP ops in general - no computation overruns in loops for example.

not quite true. it breaks "systems which don't properly do Shadowing"
and, yes, SIMD is inherently and by design incapable of linear Shadowing.

by "Shadowing" i am referring (in Mitch's Scoreboard 2nd Chapter) to the
possibility of some operation causing irreversible damage, so what is done
is: the instruction *and all downstream instructions* are "Shadowed",
i.e. *all* are prevented and prohibited from writing (to memory, to regfiles)
until such time as it has been determined that the opportunity for "damage"
is 100% determined.

if "no" then the "Shadow" is dropped, and the instruction *and all downstream*
instructions - whether they be multi-issue or single-issue - are permitted
to write.

if "yes" (e.g. an FP exception) then the *instructions are cancelled*, that one
*and* all down-stream ones.

where that all goes to s**t is SIMD. SIMD inherently and by design is
completely incapable of even recognising the concept that some random
number of its elements had an exception.

which makes SIMD worse than useless, it makes it a massive hindrance.

where this can be "fixed" is if you have something called "Data-Dependent
Fail-First". like LOAD/STORE ffirst, you allow speculative execution in
a linear and sequential fashion. anything that could raise an exception
TRUNCATES the execution - automatically - to the point where it *would not*
have occurred.

this can also even be done on Predicated SIMD, by creating an automatic
(implicit) predicate mask, made up of the bits which did *not* fail, looking
for the first bit in that (implicit) mask, throwing away everything beyond
that point and ANDing it with the "real" Predicate mask on the SIMD
back-end ALUs.
> In the Mill we do that by elimination the global flags and modes and
> moving the event reporting into metadata attached to the result of the
> failing computation. You can speculatively or concurrently do FP ops on
> a Mill, and when the erroneous result finally must be used
> non-speculatively (if it ever is), and you have not asked to ignore
> bogus results, then the resulting fault can report fairly well where
> things went bad. That really helps, not because of some numerical nicety
> but because real programmers spend their lives chasing bugs.

i suspect that the Mill, which has those "result invalid" tags, could also
do the same trick, spotting when the exception occurred and invalidating
all downstream results from that point onwards.

all that would then be needed would be a way to report back to the
program exactly what that point was, and the program may either run
a TRAP or the programmer may explicitly check for that point...

.... but what did *not* need to happen was that the parallel execution
of a batch of operations gets poisoned by explicit error checking,
drastically slowing down inner loops.
l.

On 7/21/2021 6:46 AM, Marcus wrote:
> On 2021-07-21 15:44, Marcus wrote:
>> Hi!
>>
>> I would like to ask a couple of simple questions in this group, related
>> to the IEEE 754 standard and floating-point exceptions.
>>
>> (Some of you may know that I've been pondering this subject before in
>> other forums).
>>
>> As we all know, support for floating-point exceptions comes with a cost
>> (sometimes significant, since it dictates many aspects of a HW
>> implementation for instance). If we imagine an alternative universe in
>> which the dominant floating-point standard did not require exceptions,
>> I suspect that most hardware (and language) implementations would most
>> likely be simpler, more energy efficient and more suitable for parallel
>> and pipelined floating-point operations.
>>
>> My questions are:
>
> To answer my own questions... In my fields which over the past 2-3
> decades include 3D graphics, ray tracing, 3D polygonal mesh manipulation
> algorithms, compression algorithms for triangle meshes, audio DSP, audio
> synthesis, audio compression, space/satellite signal processing, image
> processing and compression algorithms, AI / neural networks, image-based
> eye- and head-tracking, least squares solvers, and so on, etc, I have to
> say:

Without in any way deprecating your experience, which is much greater
than mine, it seems to be oriented toward the kinds of situations where
errors are "contained" either in space or time. E.g. if an audio DSP
messes up, it probably won't be noticed by the vast majority of users,
and in any event, its effect will be gone in a small fraction of a second.

Contrast this with the types of applications where errors might
propagate or expand either in space or time, and thus cause noticeable
problems. I am thinking of perhaps FE modeling, or physics simulations.
John Dallman could probably provide more and better examples.

Thus your response might be biased toward the "they don't matter" side
of the question.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)