The signing tool should work on any application. Actually, the application must be fully assembled for the signing to work and is supposed not to change afterwards. The source code plays no role here as the verification tool cannot check it anyway. I haven't seen a FileMaker-specific tutorial, but otherwise there's plenty, e.g. this one.
Also, the database itself is packaged with the runtime app somewhere in the resources. This means it needs to be somehow excluded from the signed files. I'm not sure if this is possible.
It's rather straight forward. Here's the overview:
After you have your signing ID's from Apple, you can sign the runtime application (assuming it's in your Applications folder) using the Terminal command:
codesign -s "Developer ID Application: your_name" /Applications/"your_app_name.app"
However, there is a easier way to circumvent this and the $99 fee from Apple. Tell your users to hold down the control key when they install your runtime.