As a first thought, have you installed any other web services on the server, or are you using remote access software such as LogMeIn or TeamViewer? I've sometimes had a problem with them starting up first and grabbing ports, blocking the admin console from connecting. It can take a server re-start to sort this out.
To be honest, I've often found a clean install is the quickest fix for these problems—if you have a machine to spare (even a reasonably new desktop should be able to handle the loads for a while).
You can prepare the temporary machine first (on a disjointed subnet, to avoid licence clashes, if necessary), then make a backup of the live server, take it down, transfer the files, swap the IP address of the temp server and bring it online.
If you're well-prepared, you will only have a few minutes down time. Then the new machine can handle the load (perhaps slightly slower) for a day or so as you rebuild and test the original server. If you upgrade the OS, make sure your are install Java 7 from the start, as upgrading from 6 can be painful with live databases. Then reverse the process and put the users back to the original server.
The total downtime for the whole process will only be a few minutes, plus a day or so of perhaps slightly slower database responses.
Thanks for taking the time to respond to this. I'm hopeful that there is a simple solution other than rebuilding the server. I will keep that option open as a last resort. I don't really have a second machine that I can put online as a server. So I could reinstall on the main machine and just put up with the downtime.
Sorry I can't offer you easier advice. But the Java console is a bit temperamental!
A reinstall of Filemaker Server 11 resolved this issue. We reinstalled the fms11patch listed in the first post. Interestingly if we used the Filemaker uninstall and then install again all of our databases come back up - even though we couldn't launch the admin console at the very end.
Yes - temperamental.
Thanks for sharing briefly description of MAC OS X 10.6.8 FIlemaker server 11 (126.96.36.1990) for filemaking.
other option is file sharing in MAC . that is also very helpful in sharing from one PC to another.