The expression, 12/7/2009, evaluates as 12 divided by 7 divided by 2009 as the / operator is Filemaker's division operator.
Thus you are comparing
0.00085330299367133613027092370049065 to 0.00003316749585406301824212271973466
in your first example which will evaluate to true.
Dates are really whole numbers counting the number of days elapsed since an early arbitrary date.
Thus, you should use the date function on both sides of the comparison operators in your expression.