assuming that your check box field has a single check box and is of type text:
If ( CheckboxField = "yes" ; quantity * cost = Total )
Should work. If your checkboxfield has multiple check boxes then you have to use a more sophisticated check to see if "yes" is one of the multiple options selected.
Note also that a check box format can hide other data present in the field and this can keep your CheckboxField = "yes" expression from being true.