Such is the standard, expected behavior for a group of check boxes--not only in FileMaker but in most other applications.
If you want to limit the user to a single option, first change to a radio button format instead of check boxes.
Do you know of a tutorial somewhere for setting up Radio button sets?
It's the same as a value list for a check box set. You just select the radio button format on the inspector's data tab instead of check boxes.