In previous versions of FileMaker, I've seen this once or twice where a layout seems to get "stuck" on a specific page size and you can't seem to change it to a different size for correct printing.
Try creating a new layout. It doesn't have to be fancy, but create it from scratch--don't copy and paste layout objects from the problem layout, and try printing to an envelope paper size from it.
If this works, then your layout may be damaged. You can replace that layout with a new one or you might try advanced recovery options to see if that corrects the issue:
"Copy File Blocks As-Is"
"Delete cached settings"
While I usually recommend against using a recovered copy of a file, this is an except given the very limited changes it will make to your file.
Thanks for your advise and you were close....
After much profanity and other expletives I’ve discovered what the offending component was. For some reason my two scripts that I was using to print the two separate envelope layouts (one script was a modified duplicate of the other) were poisoned. I could not see any logical reason for either of these scripts to fail.
What I ended up doing in the end was creating a new blank Filemaker file and wrote out the script again, including the page setup info. It has all the same script steps as the scripts that didn’t work. I then tested this in the new file and it worked ok. Then I created a new script in the main file and copied the script steps into that script. This works just fine now.
The only thought that is different is that I did all this while I was at work where the computer would have been able to talk to the printer, should it wanted to, whereas I setup the initial scripts at home and the computer wouldn’t have been able to contact the printer. Not that I’d imagine that would matter.