Re: Round to Nearest at Binade

mitchalsup@aol.com (MitchAlsup1) writes:

In a message over on Unum NG, there was a statement which got
me thinking about whether Round to Nearest is properly defined
in IEEE 754 (all the way back to 1985) in a theoretic sense
not in a practical sense.
>
Consider a calculation which produces in a result prior to
rounding with a fraction:: 1.1111111...11111 and 1 as the
½ ULP bit--properly ½ way between nearest representable
points except for the binade switching points. (ignore sticky}
>
Since rounding up increments the exponent, the closest number
above the result is 2 units of ½ ULP above

No.

Let's consider a (before rounding) 4-bit mantissa (plus hidden bit),
and a 3-bit mantissa after rounding, and the full-length unrounded
number in binary is

x = 1.1111 e0

The closest number above is (before normalization)

y = 10.0000 e0

and after normalization and with the correct number of mantissa bits:

y = 1.000 e1

Nothing is lost in this normalization.  The closest representable
number below x is

z = 1.111 e0

Both y and z are representable in FP with a 3 bit mantissa (plus
hidden bit), and both are 0.0001 e0 away from x.  So when you round
from 1.1111 e0 to a 3-bit mantissa with round-to-nearest-or-even, the
effect you claim does not exist.  And thanks to the or-even rule, the
rounded number is y.

However, there is one case where the effect probably exists. Consider

1.1111 eM

where M is the maximum finite exponent for our FP numbers.  Do you
round up towards infinity, or down to the maximum finite FP number
with 3-bit mantissa?  I have not tried, but I expect rounding to
infinity in this case.

If you rounded to the maximum finite FP number, at what intermediate
result would you stop doing that and round to infinity?

while the closest
number below is only 1 unit of ½ ULP below. A span of 3 units
of ½ ULP. The midpoint of a span of 3×½ is 3/4 not 1/2.

Unums may have this problem, but IEEE FP numbers don't.

I know this ship sailed <gosh> 40 years ago, and I am not trying
to alter it, I am trying to wrap my head around the numerics of
it only.

Maybe hanging around in a Unum NG is not good for you:-)

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
  Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>