1 2 Previous Next 17 Replies Latest reply on Dec 16, 2016 8:58 AM by jbante

    WRONG MATH : Multiplying by 100 change the result of decimal numbers, loss of precision

    Vincent_L

      Summary

      WRONG MATH : Multiplying by 100 change the result of decimal numbers, loss of precision

      Product

      FileMaker Pro

      Version

      13.0v3 / 11.0v4

      Operating system version

      Mac OS X 10.9.3 / Window 7

      Description of the issue

      Here's a math operation :

      1-1,975/6,5833333333333333 = ,7

      Now let's multiply that result by 100, so we should get 70 :

      (1-1,975/6,5833333333333333) *100 = 69,9999999999999998

      So we went from a round result to one with lots of decimals.

      So now let's do a Floor on this result :

      Floor( (1-1,975/6,5833333333333333) *100) = 69

      This is very BAD. So we got 70 instead of 69

      Doing the same operation in Mac OS X Alfred calculator gives correct result. Perhaps the *100 is correct but the first result is false

      Test ran on Mac OS X.9.3, FMPA 13.0v3

      *** IMPORTANT UPDATE ***
      THIS OCCURS IF DECIMAL SEPARATOR IS COMMA

      1-1.975/6.5833333333333333)*10 is ok (of course you've to set your OS with . as decimal separator).

      So the bug a appears on Languages that use COMMA as decimal separators

      Steps to reproduce the problem

      Do the above math operation

      Expected result

      70

      Actual result

      69,9999999999999998

      Configuration information

      FMPA 13.0v3 / 11.0v4 (didn't try 12)
      Mac OS X 10.9.3 / Windows Seven
      OS decial separator is a COMMA not dot

      Workaround

      None

      Capture_d%E2%80%99%C3%A9cran_2014-05-21_%C3%A0_10.37.54.png

        1 2 Previous Next