I should also mention, that when I Preview in FileMaker, the baselines also disappear. So I can see them in Browse Mode, but in Preview Mode they disappear.
I do have all text fields set for a Text Baseline of "Solid" "Black" and "0.25pt"
Nevermind, I figured it out. There is a checkbox in Layout Setup / General that says "Delineate fields on current record only" and it was checked. Didn't know what that meant, so I unchecked it, and now my baselines are displaying and printing.