Is there any consensus among CPU producers (or programmers, for that
matter) as to what rounding modes to have as defaults?

The potential rounding options I can think of are:

round down
round up
round towards zero
round away from zero
round nearest

Further, there are variants of the latter where a decision has to be
made as to what to do when two valid values are equally near. That leads
to six 'round nearest' options:

round nearest down
round nearest up
round nearest towards zero
round nearest away from zero
round nearest even
round nearest odd

meaning that a value should be rounded to the nearest acceptable value
but if a true value is exactly midway between two such values it should
be rounded as stated.

So I make it ten potential rounding modes altoegether. Any more?

As a guess, perhaps the most useful would be to use

round down for integer
round nearest even for floating point

but what are the defaults for various CPUs?

--
James Harris

On Sunday, June 13, 2021 at 6:15:05 AM UTC-4, James Harris wrote:
> Is there any consensus among CPU producers (or programmers, for that
> matter) as to what rounding modes to have as defaults?
>
> The potential rounding options I can think of are:
>
> round down
> round up
> round towards zero
> round away from zero
> round nearest
>
> Further, there are variants of the latter where a decision has to be
> made as to what to do when two valid values are equally near. That leads
> to six 'round nearest' options:
>
> round nearest down
> round nearest up
> round nearest towards zero
> round nearest away from zero
> round nearest even
> round nearest odd
>
> meaning that a value should be rounded to the nearest acceptable value
> but if a true value is exactly midway between two such values it should
> be rounded as stated.
>
> So I make it ten potential rounding modes altoegether. Any more?

One could generalize the rounding modes by declaring an offset to
be added. The offset could be fixed (in terms of relative magnitude,
probably with some consideration for denormals), sign-matching, or
sign-inverting.

One could then imagine a random offset with a linear or quasi-guassian
distribution. (I do not know if quasi-guassian rounding would be
useful, but it seems interesting and 'fitting' for approximation with
multiple partially independent error factors.)

The nature of the random number generation would also be considered
with reproducability (including detecting hardware errors) and regularity
(distance from quantum-style randomness).

I am not well-informed about numerical analysis, applications of
floating point, and other factors related to rounding modes, but the
above seems a comment worth posting until those with some expertise
respond.

On 6/13/2021 6:47 AM, Paul A. Clayton wrote:
> On Sunday, June 13, 2021 at 6:15:05 AM UTC-4, James Harris wrote:
>> Is there any consensus among CPU producers (or programmers, for that
>> matter) as to what rounding modes to have as defaults?
>>
>> The potential rounding options I can think of are:
>>
>> round down
>> round up
>> round towards zero
>> round away from zero
>> round nearest
>>
>> Further, there are variants of the latter where a decision has to be
>> made as to what to do when two valid values are equally near. That leads
>> to six 'round nearest' options:
>>
>> round nearest down
>> round nearest up
>> round nearest towards zero
>> round nearest away from zero
>> round nearest even
>> round nearest odd
>>
>> meaning that a value should be rounded to the nearest acceptable value
>> but if a true value is exactly midway between two such values it should
>> be rounded as stated.
>>
>> So I make it ten potential rounding modes altoegether. Any more?
>
> One could generalize the rounding modes by declaring an offset to
> be added. The offset could be fixed (in terms of relative magnitude,
> probably with some consideration for denormals), sign-matching, or
> sign-inverting.
>
> One could then imagine a random offset with a linear or quasi-guassian
> distribution. (I do not know if quasi-guassian rounding would be
> useful, but it seems interesting and 'fitting' for approximation with
> multiple partially independent error factors.)
>
> The nature of the random number generation would also be considered
> with reproducability (including detecting hardware errors) and regularity
> (distance from quantum-style randomness).
>
> I am not well-informed about numerical analysis, applications of
> floating point, and other factors related to rounding modes, but the
> above seems a comment worth posting until those with some expertise
> respond.
>

This is stochastic rounding
(https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which has
been proposed many times, and I put it in the initial Mill. Then I
joined the 754 committee and they explained that it was a bad idea, and
I took it out.

On 6/13/2021 5:15 AM, James Harris wrote:
> Is there any consensus among CPU producers (or programmers, for that
> matter) as to what rounding modes to have as defaults?
>
> The potential rounding options I can think of are:
>
>   round down
>   round up
>   round towards zero
>   round away from zero
>   round nearest
>

In my CPU core, at present FPU ops are hard-wired to round-to-nearest
(for Double ops), with a modestly small probability of an incorrectly
rounded result. Probably for 99% of code it does not matter.

For FP SIMD ops (single or half precision), generally round-towards-zero
(truncate) is used, since this allows for cheaper / lower-latency format
converters (combined with denormal-as-zero semantics).

One possible merit of round towards zero (over round towards nearest):
Hard wired round towards zero is potentially cheaper;
Exactly reproducible when implemented with partial precision (*);
...

With the drawback:
Typically, computations will tend to drift towards zero.

*: Approximate round-to-nearest is not strictly reproducible, since
rounding may depend on things like number of bits which exist sub-ULP,
or if the rounding imposes a limit in terms of carry propagation (it may
not always be practical to round if doing so would result in a large
carry propagation).

Usual practice is to assume that FP results are not strictly
reproducible between systems or implementations, which avoids a lot of
issues here.

Some values are checked against fixed values for boot time sanity
checks, but these tend to be cases where my CPU core, main PC, and
theoretical value, tend to all agree with each other. These will
generally trigger if rounding behavior breaks in ways more significant
than the existing set of compromises.

A few game consoles had FPUs set to round towards zero, which can
apparently lead to slight deviations when emulated on systems with the
FPU set to "round to nearest". Generally effects "low%" speedruns or
similar (where one may aim to beat the game by doing as little as
possible, which frequently involves things like standing in one spot for
hours and waiting for rounding errors or similar to slowly push them
through a wall or similar).

For float to integer conversion, the general rule tends to be "round
toward negative infinity". However, this tends to have a more obvious
effect on the results.

There is also a potential difference when it comes to overflow behavior.
The more traditional behavior for overflow is to either clamp the value
to the integer range, or produce a special sentinel value (typically
0x80000000 or similar).

For many applications though, it may be more useful to produce a value
which follows modular wraparound semantics (as with integer wraparound).

Some ISA's have ended up with separate conversion ops for these cases.

In BJX2, currently, the integer conversion op produces a 64-bit integer
value. For narrower types, the compiler can generate code to can clamp
it themselves (or use a sign or zero extension operation if this is the
desired behavior).

The current implementation is neither saturate nor wrap, in that from
the way it is implemented, the result will more just "go wonky" if it
goes too far out of the 64-bit integer range (though, will behave as
wrap within this range). Effectively, the converter aligns the mantissa
by adding the FP value against a "gigantic zero" (larger than the
largest possible 64-bit integer), but the mantissa may become misaligned
if the FP value is larger than this "zero" (result is then "undefined").

In practice it doesn't seem like too much of an issue though. Most cases
which involve FP<->Integer conversion are also within the range where
the FP value can represent discrete integer values (which is a lot
smaller than the "giant zero" limit).

> Further, there are variants of the latter where a decision has to be
> made as to what to do when two valid values are equally near. That leads
> to six 'round nearest' options:
>
>   round nearest down
>   round nearest up
>   round nearest towards zero
>   round nearest away from zero
>   round nearest even
>   round nearest odd
>
> meaning that a value should be rounded to the nearest acceptable value
> but if a true value is exactly midway between two such values it should
> be rounded as stated.
>
> So I make it ten potential rounding modes altoegether. Any more?
>
> As a guess, perhaps the most useful would be to use
>
>   round down           for integer
>   round nearest even   for floating point
>
> but what are the defaults for various CPUs?
>
>

If the CPU were to implement IEEE rounding modes, then presumably the
default is:
Whatever the C library prefers;
Whatever happens to be 0 in whatever control-register is used to control
this;
....

On Sunday, June 13, 2021 at 5:15:05 AM UTC-5, James Harris wrote:
> Is there any consensus among CPU producers (or programmers, for that
> matter) as to what rounding modes to have as defaults?
>
> The potential rounding options I can think of are:
>
> round down
> round up
> round towards zero
> round away from zero
> round nearest
<
Round nearest even is the IEEE std.
<
Round nearest even has the unique property that if the result is ½ way
between the last representable value in a binade and the first representable
value in the higher binade, that one chooses the first of the next binade to
loose the lease amount of significance.
<
>
> Further, there are variants of the latter where a decision has to be
> made as to what to do when two valid values are equally near. That leads
> to six 'round nearest' options:
>
> round nearest down
> round nearest up
> round nearest towards zero
> round nearest away from zero
> round nearest even
> round nearest odd
<
These are not round nearest, but directed roundings.
>
> meaning that a value should be rounded to the nearest acceptable value
> but if a true value is exactly midway between two such values it should
> be rounded as stated.
>
> So I make it ten potential rounding modes altoegether. Any more?
>
> As a guess, perhaps the most useful would be to use
>
> round down for integer
> round nearest even for floating point
>
> but what are the defaults for various CPUs?
>
Round nearest even is the RM most (?ALL?) CPU default to.
>
> --
> James Harris

James Harris <james.harris.1@gmail.com> writes:

> Is there any consensus among CPU producers (or programmers, for that
> matter) as to what rounding modes to have as defaults?
>
> The potential rounding options I can think of are:
>
> round down
> round up
> round towards zero
> round away from zero
> round nearest
>
> Further, there are variants of the latter where a decision has to be
> made as to what to do when two valid values are equally near. That
> leads to six 'round nearest' options:
>
> round nearest down
> round nearest up
> round nearest towards zero
> round nearest away from zero
> round nearest even
> round nearest odd
>
> meaning that a value should be rounded to the nearest acceptable value
> but if a true value is exactly midway between two such values it
> should be rounded as stated.
>
> So I make it ten potential rounding modes altoegether. Any more?

I wouldn't mind a random rounding option where .1 has a 10% chance of
rounding up.

In article <sa5b66$fld$1@dont-email.me>, ivan@millcomputing.com (Ivan
Godard) wrote:

> This is stochastic rounding
> (https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which
> has been proposed many times, and I put it in the initial Mill.
> Then I joined the 754 committee and they explained that it was a
> bad idea, and I took it out.

Unless I'm failing to understand something, stochastic rounding seems to
mean that calculations involving rounding aren't reliably repeatable.
This would make regression-testing software after changes a nightmare.

John

On Sunday, June 13, 2021 at 4:59:41 PM UTC-5, John Dallman wrote:
> In article <sa5b66$fld$1...@dont-email.me>, iv...@millcomputing.com (Ivan
> Godard) wrote:
>
> > This is stochastic rounding
> > (https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which
> > has been proposed many times, and I put it in the initial Mill.
> > Then I joined the 754 committee and they explained that it was a
> > bad idea, and I took it out.
> Unless I'm failing to understand something, stochastic rounding seems to
> mean that calculations involving rounding aren't reliably repeatable.
> This would make regression-testing software after changes a nightmare.
<
But it makes error analysis surveys easy !!
>
> John

On 13.06.2021 18:19, Ivan Godard wrote:
> On 6/13/2021 6:47 AM, Paul A. Clayton wrote:
>
> This is stochastic rounding
> (https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which has
> been proposed many times, and I put it in the initial Mill. Then I
> joined the 754 committee and they explained that it was a bad idea, and
> I took it out.
May I ask you to outline shortly why it is a bad idea according to the
754 TC (expect for non-deterministically repeatable results)?

As far as I understand is the strength of stochastic rounding, that
rounding errors will outweigh each other in computing chains.

TIA
Bernd

On Sunday, June 13, 2021 at 5:19:36 PM UTC-5, Bernd Linsel wrote:
> On 13.06.2021 18:19, Ivan Godard wrote:
> > On 6/13/2021 6:47 AM, Paul A. Clayton wrote:
> >
> > This is stochastic rounding
> > (https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which has
> > been proposed many times, and I put it in the initial Mill. Then I
> > joined the 754 committee and they explained that it was a bad idea, and
> > I took it out.
> May I ask you to outline shortly why it is a bad idea according to the
> 754 TC (expect for non-deterministically repeatable results)?
>
> As far as I understand is the strength of stochastic rounding, that
> rounding errors will outweigh each other in computing chains.
<
Stochastic is great for those situations where the numbers being manipulated
were derived by measurements (and include random errors therefrom).
<
Stochastic is terrible for those situations where the numbers are exact (pi,
e, ln2(e),....) or for those situations where Kahan is running test cases on your
implementation.
>
> TIA
> Bernd

MitchAlsup wrote:
> On Sunday, June 13, 2021 at 4:59:41 PM UTC-5, John Dallman wrote:
>> In article <sa5b66$fld$1...@dont-email.me>, iv...@millcomputing.com (Ivan
>> Godard) wrote:
>>
>>> This is stochastic rounding
>>> (https://en.wikipedia.org/wiki/Rounding#Stochastic_rounding), which
>>> has been proposed many times, and I put it in the initial Mill.
>>> Then I joined the 754 committee and they explained that it was a
>>> bad idea, and I took it out.
>> Unless I'm failing to understand something, stochastic rounding seems to
>> mean that calculations involving rounding aren't reliably repeatable.
>> This would make regression-testing software after changes a nightmare.
> <
> But it makes error analysis surveys easy !!

It means that all verification work needs to be Monte Carlo, i.e. run a
bunch of times and verify that all the different results can be due to
rounding and that the final result has the expected distribution in the
lowest bits.

I.e. a lot more work than just a simple 8-byte compare.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

In article <sa73mk$1ogn$1@gioia.aioe.org>, terje.mathisen@tmsw.no (Terje
Mathisen) wrote:

> It means that all verification work needs to be Monte Carlo, i.e.
> run a bunch of times and verify that all the different results can
> be due to rounding and that the final result has the expected
> distribution in the lowest bits.
>
> I.e. a lot more work than just a simple 8-byte compare.

Yes ... for some things, this is clearly impractical.

John

On 13/06/2021 18:56, MitchAlsup wrote:
> On Sunday, June 13, 2021 at 5:15:05 AM UTC-5, James Harris wrote:

>> Is there any consensus among CPU producers (or programmers, for that
>> matter) as to what rounding modes to have as defaults?

....

> Round nearest even is the IEEE std.

....

> Round nearest even is the RM most (?ALL?) CPU default to.

Thanks, that's helpful to know. I take it you mean for floating point.

What about integers? Intel apparently chose for IDIV signed division:

Nonintegral quotients are truncated toward 0

Anyone know if it's the same for other processor architectures?

--
James Harris

On 13/06/2021 17:25, BGB wrote:
> On 6/13/2021 5:15 AM, James Harris wrote:

>> Is there any consensus among CPU producers (or programmers, for that
>> matter) as to what rounding modes to have as defaults?

....

> For float to integer conversion, the general rule tends to be "round
> toward negative infinity". However, this tends to have a more obvious
> effect on the results.

Round towards negative infinity is, at least, easy for programmers to
deal with by adding 0.5.

--
James Harris

On Monday, June 14, 2021 at 6:51:49 AM UTC-5, James Harris wrote:
> On 13/06/2021 18:56, MitchAlsup wrote:
> > On Sunday, June 13, 2021 at 5:15:05 AM UTC-5, James Harris wrote:
>
> >> Is there any consensus among CPU producers (or programmers, for that
> >> matter) as to what rounding modes to have as defaults?
> ...
> > Round nearest even is the IEEE std.
> ...
> > Round nearest even is the RM most (?ALL?) CPU default to.
> Thanks, that's helpful to know. I take it you mean for floating point.
>
> What about integers? Intel apparently chose for IDIV signed division:
>
> Nonintegral quotients are truncated toward 0
<
This makes MOD and REM straightforward.
>
> Anyone know if it's the same for other processor architectures?
<
The vast majority but not all.
>
>
> --
> James Harris

On 6/14/2021 6:51 AM, James Harris wrote:
> On 13/06/2021 18:56, MitchAlsup wrote:
>> On Sunday, June 13, 2021 at 5:15:05 AM UTC-5, James Harris wrote:
>
>>> Is there any consensus among CPU producers (or programmers, for that
>>> matter) as to what rounding modes to have as defaults?
>
> ...
>
>> Round nearest even is the IEEE std.
>
> ...
>
>> Round nearest even is the RM most (?ALL?) CPU default to.
>
> Thanks, that's helpful to know. I take it you mean for floating point.
>
> What about integers? Intel apparently chose for IDIV signed division:
>
>   Nonintegral quotients are truncated toward 0
>
> Anyone know if it's the same for other processor architectures?
>

Truncate toward zero is pretty much standard.
Similar to how modulo may be negative for negative values, ...

Not every architecture has hardware divide though.

For BJX2, I went the same direction as ARM / A32 and used a divider
implemented in software. I could in premise add special accelerator ops
(or potentially even a hardware divide instruction), but in common use,
divide is rare enough that the use of a pure software form is acceptable.

I can also note that on a few past PCs of mine (with a Phenom II and
Piledriver), the hardware divide instruction wasn't much faster than
doing the same basic divide loop in software.

Likewise, a few other options (such as using a lookup table of
reciprocals, which may then be shifted to accommodate a larger range),
can be somewhat faster (though, my own attempts at getting this approach
to be integer-exact had been unsuccessful, *).

It is possible to use this approach, followed by calculating the
remainder (R), and adjusting the result if (R>=Q), but this isn't free.

*: One might also expect this to be true of dividing by multiplying by a
constant reciprocal, but typically it is possible to fudge the
reciprocal upwards slightly such that the results come out correct.

For signed divide by reciprocal one may also need to conditionally add a
bias (based on the sign of the input value) such that the result rounds
towards zero (rather than towards negative infinity).

One generally wants to use constant-reciprocal when possible, as it
turns things like "x/1000" into an integer multiply and shift
(reasonably fast), vs using a C runtime call and divider loop (slow).

....