You may want to investigate:
* creating a script trigger for OnLastWindowClose (File > File Options > Script Triggers tab)
* Kiosk mode (http://www.filemaker.com/help/html/fmpa_tools.24.20.html)
* Runtime (http://www.filemaker.com/help/html/fmpa_tools.24.12.html)
depending on your needs.
You could also replace any entire pull-down menu with your own hand-built custom menu rather than trying to-assign one of the standard commands.
When I do this, I also put a "logout" button onscreen to give users an easy way to pick exiting the application/files, and I can program any script I want to it.
I try to reserve any closing routine to the OnLastWindowClose trigger, as Debi indicated.
You could replace any entire pull-down menu except the "Tools" menu in FileMaker Pro Advanced and the application menu (named the same as the running application) on Mac — which contains the Mac equivalent of the "Exit" option. The features Debi mentioned are much more robust approaches than custom menus.