First check to see if any scheduled backups fail to complete. There's a bug that FileMaker still hasn't gotten a handle on where a scheduled backup never changes status back to "ok". One of the symptoms is that you can't close the file.
If that's the case, you may need to cross your fingers and restart your server. (This force quit might damage a database file so don't do this unless you've ruled out all other possible options.
This command line may close the files:
Open the command line window, navigate to the program files directory where you find the database server application and then execute:
Thanks. What I did was check every option and then I unistalled Java and FMserver and did a complete re-install. I am almost positive it was due to a java update. Works fine now.
I'd monitor it frequently. One of my servers stalls on a scheduled backup and goes "deaf" once every three months or so. It never affects use of the system so I just have to wait till close of business to restart everything once it happens.