I also noticed this recently. I went and checked and it's a change in behaviour from FileMaker Pro 11.0v4 on Mac OS X 10.7.4. In FileMaker Pro 11.0v4 you could leave the Message field empty and a new email would be created ready for the user to enter their message body and send the email.
I'm testing this with Outlook for Mac 2011 and have found that if I don't have a Subject and Message it will just create a new email and put it into the Drafts folder. However if I have something in both the Subject and Message fields it will create a new email ready for the user to compose. You can test this quite easily as Dennis notes by creating a button on a layout.
I also use Outlook for Mac 2011 and I have found that whether or not there is a Subject specified, the absence of anything in the Message will defeat the procedure.
What I expect is that Perform Without Dialog will put the message in the Outbox immediately and send it the next time Outlook does a send/receive.
But if the Perform Without Dialog option is not checked the message should either go into the drafts folder or present itself as an open window for the user to do any necessary editing.
Can't say I have a preference for the Drafts folder or the open window, but either would be acceptable in lieu of what we have now.