Thank you for your post.
I am using FileMaker Pro 10 on an iMac with OS X 10.5.7. I am able to use the Send Mail script step using SMTP Server without a problem. Two of the more important settings is getting the correct name of the SMTP server and the port number. If the SMTP Server requires authentication, then be sure to enter the correct User Name and Password. I would contact the system administrator for all the details on what is needed to send emails via the SMTP Server.