Or is a logical operator. If either the expression preceding it or the expression following it is true ( a numeric value other than zero evaluates as "true"), then the entire expression is True. Thus, the value 1 is returned by your calculation when you remove the comment slashes (//)
Use a Case or If function such as:
IF ( Vendor do not pay::VN_Donotpay ; 0 ; Round ( ((IN_LineItemPayableTotal / IN_LineItemInvoiceTotal) / -.01)+100;2) )
But keep in mind that I am guessing as to what the expected results of this expression should be.