This function works the same in 11 as it does in 5.5 so we need to spot what's different. Are all fields of type number?
The case function would produce a simpler expression:
case ( Cheque # = 0 and Second Cheque # = 0 and Third Cheque # = 0 ; Empty ;
Second Cheque # = 0 and Third Cheque # = 0 ; Cheque # ;
Third Cheque # = 0 ; Second Cheque # ;
Third Cheque # )
Case branches on the first true result so you don't have to repeatedly test the value of all three fields. (This is also true for your original expression.)
Your if function as written should produce the same results, however, so the problem lies elsewhere...
Yes, the check number fields we set as text and the Calculation field was set as number. I changed the check number field to number and then the formula works. Thanks again for your help!