Our FileMaker server has crashed three times so far, and it's not obvious what our next step should be.
We're on a Windows Server 2008 R2 64-bit virtual machine, and the crash log says:
Faulting application name: fmserver.exe, version: 126.96.36.199, time stamp: 0x4b6b3007
Faulting module name: ntdll.dll, version: 6.1.7600.16915, time stamp: 0x4ec49d10
Exception code: 0xc0000005
Fault offset: 0x00066c9e
Faulting process id: 0x1bc
Faulting application start time: 0x01cd27840dc880fa
Faulting application path: C:Program Files (x86)FileMakerFileMaker ServerDatabase Server mserver.exe
Faulting module path: C:WindowsSysWOW64 tdll.dll
Report Id: 64e26cb5-999a-11e1-9667-005056a10056
It crashed Monday and Tuesday last week at around midday (when it would have seen a fair amount of activity), and tonight (Wednesday) at around quarter to six.
In the two cases we've been able to look at close to crash time, the process monitor seems to indicate that there's a large spike in CPU activity around the time of the crash.
The admin console seemed to be unaffected, and in the most recent crash, we were able to start the server process again from the console. Last Monday and this Wednesday, all the databases appeared to recover fully; during the crash last Tuesday, one frequently written-to file would not open, and we had to revert to an earlier version. (Because it is a file that basically keeps state/settings, rather than one that collects data, very little was lost.)
The virtual machine is simulating a 4-processor 2.53GHz box, with 12 GB ram and a 100GB disk (which has 27.3 GB free). There are 85 db hosted, and the total size of the Databases folder is 6.69GB. We backup at 2am, and keep four days worth of backups locally (just in case the process that copies the backups to a filer fails over a long weekend; that's handled separately, at about 3am).
We've turned off web publishing on this machine. We limit the number of connections to 75, and typically have 40 to 50 users during the day; we disconnect idle users after 3 hours, and allow external server accounts. Many of the databases have shadow tables connecting to tables on our MySQL servers.
On the off-chance that it would help, last Tuesday we rebooted the machine, installing all outstanding Windows updates. We also rebuilt the 28 most active databases -- or rather, we cloned them, and copied their data into the clones -- in case there was some sort of weird corrupt file thing going on.
Um -- is there anything else useful that I could say that would help diagnose the problem? Or somewhere else that I should be asking this question?