Q69333: How to Work Around Floating-Point Accuracy/Comparison Problems

Article: Q69333
Product(s): See article
Version(s): 1.00 1.01 1.02 2.00 2.01 3.00 4.00 4.00b 4.50
Operating System(s): MS-DOS
Keyword(s): ENDUSER | B_BasicCom B_GWBasicI | mspl13_basic
Last Modified: 14-FEB-1991

To reliably test whether two floating-point variables or expressions
are equal (using IEEE format or MBF), you must subtract the two
variables being compared and test whether their difference is less
than a value chosen at the limits of significance for single or double
precision. NO OTHER TEST FOR EQUALITY WILL BE RELIABLE. The following
formulas reliably test whether X and Y are equal:

1. For single precision, you must test whether the difference of X and Y
   is less than the value 7 significant digits smaller than X or Y.
   Divide X or Y by 10^7 to find the comparison value. For example:

      IF ABS(X! - Y!) <= (X! / 10^7) THEN PRINT "Equal within 7 digits"

2. For double precision, you must test whether the difference of X and Y
   is less than the value 15 significant digits smaller than X or Y.
   Divide X or Y by 10^15 to find the comparison value. For example:

      IF ABS(X# - Y#) <= (X# / 10^15) THEN PRINT "Equal within 15 digits"

The IEEE floating-point format is found in Microsoft QuickBASIC
versions 3.00 (QB87.EXE coprocessor version only), 4.00, 4.00b, and
4.50 for MS-DOS; in Microsoft BASIC Compiler versions 6.00 and 6.00b
for MS-DOS and MS OS/2; and in Microsoft BASIC Professional
Development System (PDS) versions 7.00 and 7.10 for MS-DOS and MS
OS/2.

MBF (Microsoft Binary Format) is found in Microsoft QuickBASIC
versions 3.00 (QB.EXE non-coprocessor version only), 2.01, 2.00, 1.01,
and 1.00 for MS-DOS; and in Microsoft GW-BASIC Interpreter versions
3.20, 3.22, and 3.23 for MS-DOS.

   NOTE: Significant digits in a calculated number can be lost due to
   the following: multiple calculations, especially addition of
   numbers far apart in value, or subtraction of numbers similar in
   value. When a number results from multiple calculations, you may
   need to change your test for equality to use fewer significant
   digits to reflect the mathematical loss of significant digits. If
   your test of significance uses too many significant digits, you may
   fail to discover that numbers compared for equality are actually
   equal within the possible limit of accuracy.

In the above BASIC versions that use IEEE floating-point format,
intermediate calculations are performed in an internal 64-bit
temporary register, which has more bits of accuracy than are stored in
single- or double-precision variables. This often results in an IF
statement saying that the intermediate calculation is not equal to the
expression being compared, as in the following example:

   X = 25
   Y = 60.1
   IF 1502.5 = (X * Y) THEN PRINT "equal"

Running the above code will NOT print "equal". In contrast, the
following method using a placeholder variable will print "equal", but
is still NOT a reliable technique as a test for equality:

   Z = 25 * 60.1
   IF 1502.5 = Z THEN PRINT "equal"

Note that explicit numeric type casts (! for single precision, # for
double precision) will affect the precision in which calculations are
stored and printed. Whichever type casting you perform, you may still
see unexpected rounding results:

   PRINT 69.82! + 1    'single precision, prints 70.82
   PRINT 69.82# + 1    'double precision, prints 70.81999999999999

For an exact decimal (base 10) numeric representation, such as for
calculations of dollars and cents, you should use the CURRENCY (@)
data type found in BASIC PDS 7.00/7.10. The CURRENCY data type exactly
stores up to 19 digits, with 4 digits after the decimal place.

Reference:

Both the IEEE and MBF standards attempt to balance accuracy and
precision with numeric range and speed. Accuracy measures how many
significant bits of precision are not lost in calculations. Precision
refers to the number of bits in the mantissa, which determines how
many decimal digits can be represented.

Both IEEE format and MBF store numbers of the form 1.x to the power of
y (where x and y are base 2 numbers; x is the mantissa, and y is the
exponent).

MBF single precision has 24 bits of mantissa, and double precision has
56 bits of mantissa. All MBF calculations are performed within just 24
or 56 bits.

IEEE single precision has 24 bits of mantissa, and double precision
has 53 bits of mantissa. However, all single and double precision IEEE
calculations in QuickBASIC 3.00/4.x, BASIC Compiler 6.00/6.00b, and
BASIC PDS 7.00/7.10 are performed in a 64-bit temporary register for
great accuracy. As a result, IEEE calculations are more accurate than
MBF calculations, despite MBF's ability to represent more bits in
double precision.

Most numbers in decimal (base 10) notation do NOT have an exact
representation in the binary (base 2) floating-point storage format
used in single- and double-precision data types. Both IEEE format and
MBF cannot exactly represent (and must round off) all numbers that are
not of the form 1.x to the power of y (where x and y are base 2
numbers). The numbers that can be exactly represented are spread out
over a very wide range. A high density of representable numbers is
near 1.0 and -1.0, but fewer and fewer representable numbers occur as
the numbers go towards 0 or infinity.

The above limitations often cause BASIC to return floating-point
results different than you might expect, as explained in many separate
articles found with the following query:

   floating and point and format and QuickBASIC

THE INFORMATION PROVIDED IN THE MICROSOFT KNOWLEDGE BASE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. MICROSOFT DISCLAIMS ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL MICROSOFT CORPORATION OR ITS SUPPLIERS BE LIABLE FOR ANY DAMAGES WHATSOEVER INCLUDING DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF MICROSOFT CORPORATION OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME STATES DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES SO THE FOREGOING LIMITATION MAY NOT APPLY.

Copyright Microsoft Corporation 1986-2002.