I dont'know if it you already found this post but maybe it cans help here : http://dbaspot.com/filemaker/409768-send-mail-script-step-crashes-filemaker.html
Moreover it was confirmed here : http://fmforums.com/forum/topic/60795-send-mail-script-step-crashes-fm9/
Ok i know it is not so recent and versions changed but...
Anyways, for issues / bugs i suggest you to report an issue here : http://forums.filemaker.com/hives/1eea103f05/summary then click New Issue Report link.
I think when you tried to catch error with Script debugger it cannot return an error because it crashed during step execution, right ?
Good luck and sorry to cannot really help you at this time...
PS : Sorry for bad english