You could adapt this suggestion of PhilModJunk's:
Try the following as an auto-entered calculated value with the "do not replace existing value..." option cleared:
Let (ph = GetasNumber(Phonenumber);
TextFormatRemove(Case (Length(ph) = 7; Left (ph; 3) & "-" & right(ph;4);
Length(ph) = 10; left(ph; 3) & "-" & middle (ph; 4; 3) & "-" Right (ph; 4);
PhoneNumber is the name of your text field (If you are using a recent version of FMP, you can replace it with "Self") to which you've set up this auto-enter option.
GetasNumber strips out any non numeric characters that get entered.
TextFormatRemove eliminates any undesirable text styles that might otherwise get pasted into the text field along with the number.
If the number of digits in your phone number isn't 7 or 10, the expression will simply return the original text, but with any pasted text styles removed.
I copied it from this thread (for example). FM help also has suggestions on how to format Zip codes and phone numbers, for example, that can be adapted.
Or use the Middle function to grab the first 2 characters, then insert a colon, then the 3rd and 4th, insert a colon, for example
Speaking as someone who also has developed in Access, FileMaker doesn't have the same input masking feature--but you can get fairly similar results with an auto-enter calculation such as the examples Sorbsbuster has provided.