
1. Re: Scientific notation dilemma
philmodjunk Dec 21, 2014 10:42 AM (in response to calexmac)1) If this value is in a field the field type determines whether this is a number or text. If this is a calculation field, the result type setting specifies whether the value is stored as number or text.
If you set a variable to this value, it's a bit less obvious as it is the type of calculation that returns the type used to store the value. At times, it is necessary to use GetAsNumber ( $Variable ) to be sure that FileMaker treats the value as a number.
But in all cases, the format of the number does not determine the data type. A given value such as your scientific notation value will display in either the default setting you see now or in a data format that you choose for it. The data format option selected in the inspector does not change how the value is stored, only in how it is displayed.
2) 1.233e 10, IS greater than zero as this is not a negative number. The minus sign specifies a negative exponent not a negative mantissa. 1.233e 10, on the other hand, would be less than zero. Note the minus sign at the beginning of this number used to make the mantissa negative.
3) Filemaker stores all floating point values the same way not matter how they are displayed via different display formats and you would use them in calculations in exactly the same format.

2. Re: Scientific notation dilemma
calexmac Dec 21, 2014 11:58 AM (in response to calexmac)Thanks for your answer.
Re 1: All of the calculations are occurring in variables and this is perhaps where I am running aground. I would expect the data type to be independent of the representation. I still don't understand the behaviour of GetAsNumber when used on variable with a tiny number in it... I have always presumed it means that If I set a variable say $vTest to GetAsNumber, the results of GetAsNumber ( $vTest ) will always be treated as a number, not the default text type that a variable value is expected to evaluate to.
In that case, if I set a variable say $vTest to 0.0000000000002, then why is the calculation GetAsNumber ( $vTest ) * GetAsNumber ( $vTest ) displaying (via the show custom dialog) as 0 for me?
Re 2: my mistake on the comparison! Duh! Too many solo hours trying to get to the bottom of a problem!
thanks

3. Re: Scientific notation dilemma
Jade Dec 21, 2014 2:01 PM (in response to calexmac)You are exceeding FMP's default precision with that calculation ( 2e13 * 2e13 = 4e26). Hence it rounds (or truncates) the result to 0.
Use the SetPrecision function { e.g. SetPrecision( 2e13 * 2e13 ; 30 ) } to display the correct answer.

4. Re: Scientific notation dilemma
philmodjunk Dec 21, 2014 1:59 PM (in response to calexmac)if I set a variable say $vTest to 0.0000000000002, then why is the calculation GetAsNumber ( $vTest ) * GetAsNumber ( $vTest ) displaying (via the show custom dialog) as 0 for me?
Just as a calculator has a limited number of digits that it can work with and when a value exceeds that value, it rounds the value, so does computer software such as FileMaker. I would assume that a value of
0.00000000000000000000000004
Exceeds that limit and is rounded to zero. I have not researched that limit with FileMaker and if this value does not appear to exceed that limit, you may want to report this as a possible bug over in Report An Issue.