do you have any plugins installed that you can move out of the folder and try again ?
I just launched my 11 advanced on Sierra 10.12.6 beta without problems.
You could use activity monitor to sample the process and see which functions it's spending time inside.
FileMaker 11 works fine here on macOS 10.12.5.
Yes, I've tried that. Also tried launching when logged in as a different user.
I've given up. Not important enough as long as I can do the occasional work in Windows.