I posted in the Apple developer forums about this but have yet to get a single reply. Hoping someone here will have a clue. I'm unfamiliar enough with this forum that I'm not sure this is even the right place to post this. To save time I'm just reposting a copy of what I've already written up on the Apple forum.
A few days ago I had a MacBook Pro running El Capitan developer beta 6, upon which I had FileMaker Pro 11 Advanced running successfully for weeks. Now, I have a new MacBook Pro, also running beta 6, but FileMaker Pro 11 absolutely refuses to run after installing the legacy Java SE 6 that it requires.
I have tried all of the solutions that have previously been discussed in all Java threads I found in this part of the forum. Seems like the only thing I did differently a few weeks ago was to forcibly install the older Apple Java SE 6 package from the recovery partition, while this time I used the updated Apple Java SE 6 package that happily installs on El Capitan. But that hasn't worked, and neither did trying to force-install the older package. FMP11 just keeps crashing immediately after the splash screen appears. Removing all traces of any Java VMs just gets me back to the "You need to install Java SE 6 to run this application." And so on.
It was working pretty much perfectly before, same version of El Capitan. Now it doesn't, and I can't figure out why. I've tried installing the newer JREs and JDKs, made sure that the Info.plist contains "CommandLine", "JNI" and "BundledApp" tags, etc.
FMP10 works fine. FMP12 works fine. It's only FMP11 that's crashing. It's weird, annoying and going to be a major issue if I can't find a solution.
Any suggestions are appreciated.
Well, in case anyone else encounters this issue, I seem to have accidentally found a workaround. Apparently this has something to do with FileMaker's Instant Web Publishing component. If another version of FileMaker is already running and has taken over the FileMaker network sharing port (5003), then FMP11 will complain with the usual error dialog about IWP like this:
"FileMaker Pro cannot start Instant Web Publishing because FileMaker Server or another instance of FileMaker Pro is running on this machine."
... then it will successfully start and be functional. If the other FileMaker instance quits while FMP11 is running then FMP11 will keep running fine but crash on exit with the same kind of error as the startup crash. If the other instance is still running then there is no crash on exit.
The workaround requires that you run the other FileMaker Pro version and at least open the "Open Remote..." dialog which causes the application to bind to port 5003. It's not necessary to actually open a remote file, just attempt to connect to any valid remote FileMaker server process.
This seems to work with FMP12 and FMP13 but not with FMP10 for some reason. I don't have any other versions available to test. I'm guessing FileMaker Server or any other process that binds port 5003 might also cause FMP11 to avoid starting its Instant Web Publishing component and thus bypass this startup crash. But it's weird that FMP10 doesn't stop the crash even if it's actively connected to a remote database on port 5003.
This remains a vexing and bizarre problem that didn't exist a week ago on an idfentical version of El Capitan.