FileMaker Server 13 is unstable after scheduled backups
Operating system version
Description of the issue
We are having ongoing problems with the reliability/stability of FileMaker Server 13.0v2 running on a brand new Mac Pro machine running OS X 10.9.2.
Several times per month, the FileMaker Server 13 daemon/service will freeze/crash/hang in the background, which kicks all the users out of all the hosted databases. The users are never able to log back into any databases until a restart of the Mac Pro, because nothing shows up in the Open Remote dialog box until after a restart of the Mac Pro.
The FileMaker Server 13 Admin Console interface completely freezes as well, showing us incorrect & outdated information. The FileMaker Server 13 Admin Console always shows that "everything is just fine" with FileMaker Server… it shows that all the users are connected and all the databases are open & running… even though this is not true.
In reality, the daemon/service has kicked all the users out of the system and has prevented anybody from re-logging into the databases again. None of the databases are open & running, and no users are connected. However, the database service might still be running in the background (as you will see below), even though it is inaccessible to the users.
None of these freezes/crashes/hangs are visibly listed or recorded in FileMaker's log files, BUT we can tell by the times listed in the log files that these freezes/crashes/hangs always seem to happen right after a scheduled backup successfully completes.
We are not doing incremental backups. We are just doing normal regular backups. We have verification enabled in our scheduled backup, and the databases always pass the verification step.
For example, on our most recent crash/freeze/hang, the FileMaker Server log lists these things in the seconds leading up to the crash/freeze/hang:
- 2:00:00 PM Schedule "Local 2pm" running.
- 2:00:00 PM Starting to back up 1 database(s) to folder "filemac:/Macintosh HD/Library/FileMaker Server/Data/Backups/Local 2pm_2014-07-11_1400".
- 2:00:04 PM Starting to backup database: Keslow2...
- 2:00:11 PM Backup attempting to pause 1 database(s)...
- 2:00:12 PM Backup attempting to pause 1 database(s)… (Why is this listed twice?? Is this part of the problem??)
- 2:00:12 PM Backup of Keslow2 completed.
- 2:00:15 PM Deleted backup folder "filemac:/Macintosh HD/Library/FileMaker Server/Data/Backups/Local 2pm_2014-07-04_1400
Right around this time (between 2:00 PM and 2:05 PM) is when all the clients got kicked out of the databases. There are NO FURTHER LOG ENTRIES showing that any of this is happening. The next log entry appears at the time when we restarted the Mac Pro:
- 2:31:00 PM Stopping FileMaker Server processes
- 2:31:01 PM Stopping Database Server process
- 2:31:01 PM Stopping FileMaker Server
- 2:31:01 PM Stopping FileMaker Database Engine
Then, after a restart of the Mac Pro, everything is back to normal again.
This backup is scheduled to run every single day, so please note that we can go several days or weeks with successful backups… it's only a few times per month that the scheduled backup actually kicks people out of the system.
Also, our users are constantly running scripts that utilize the new "Perform Script on Server" script step. My feeling is that these freezes/crashes/hangs might be happening whenever a user is trying to run a "Perform Script on Server" script step at the exact same time that a scheduled backup is taking place, but I'm not 100% sure about that theory.