Lingo Dictionary > D-F > floatPrecision

 

floatPrecision

Syntax

the floatPrecision

Description

Movie property; rounds off the display of floating-point numbers to the number of decimal places specified. The value of floatPrecision must be an integer. The maximum value is 15 significant digits; the default value is 4.

The floatPrecision property determines only the number of digits used to display floating-point numbers; it does not change the number of digits used to perform calculations.

If floatPrecision is a number from 1 to 15, floating-point numbers display that number of digits after the decimal point. Trailing zeros remain.

If floatPrecision is zero, floating-point numbers are rounded to the nearest integer. No decimal points appear.

If floatPrecision is a negative number, floating-point numbers are rounded to the absolute value for the number of decimal places. Trailing zeros are dropped.

This property can be tested and set.

Example

This statement rounds off the square root of 3.0 to three decimal places:

the floatPrecision = 3
x = sqrt(3.0)
put x
-- 1.732

Example

This statement rounds off the square root of 3.0 to eight decimal places:

the floatPrecision = 8
put x
-- 1.73205081