10 Replies Latest reply on Dec 11, 2011 1:13 PM by psijmons

    On Mac OX Lion, SetPrecision() returns only 8 digits of precision after the decimal

      Hello everyone,

       

      I have FileMaker Pro 11 Advanced installled on OS X Lion.

       

      In the module 5 (calculation fonctions) of the FM Pro Series, it is stated that numeric operations are calculated up to 16 digits of precision. If I need more precision (up to 400), I can use SetPrecision().

       

      When I tried this:

       

      SetPrecision ( Pi ; 20 )

       

      1) If I set the calculation result to "Number", it will only return 8 digits of precision after the decimal point. (3.14159265)

       

      2) If I set the calculation result to "Text", then and only then will I have 20 digits of precision. (3.14159265358979323846)

       

      I'm certain there is something I don't understand about the concept.

       

      Thanks in advance for any help.

       

      Regards,

       

      Mario Marceau