AnsweredAssumed Answered

On Mac OX Lion, SetPrecision() returns only 8 digits of precision after the decimal

Question asked by disabled_HeliosOne on Dec 10, 2011
Latest reply on Dec 11, 2011 by psijmons

Hello everyone,

 

I have FileMaker Pro 11 Advanced installled on OS X Lion.

 

In the module 5 (calculation fonctions) of the FM Pro Series, it is stated that numeric operations are calculated up to 16 digits of precision. If I need more precision (up to 400), I can use SetPrecision().

 

When I tried this:

 

SetPrecision ( Pi ; 20 )

 

1) If I set the calculation result to "Number", it will only return 8 digits of precision after the decimal point. (3.14159265)

 

2) If I set the calculation result to "Text", then and only then will I have 20 digits of precision. (3.14159265358979323846)

 

I'm certain there is something I don't understand about the concept.

 

Thanks in advance for any help.

 

Regards,

 

Mario Marceau

Outcomes