5 Replies Latest reply on Jul 31, 2010 7:05 PM by hschlossberg

    Server Admin Console freezes then crashes

    kdoronzio

      Summary

      Server Admin Console freezes then crashes

      Product

      FileMaker Server

      Version

      FMSA 11

      Operating system version

      Windows 2003 Server R2

      Description of the issue

      This issue was reported on the old forums, but I can't find it here. We've had this same problem on FMS10, and was hoping it would go away on FMS11. But it hasn't.

      The symptoms are:
      After FileMaker Server is running for about 3 weeks, the admin console stops updating its information, and shows non-current information. It seems to occur during a scheduled backup, as one of the schedules displays "running".

      This screen remains in this "altered" state for about a week, then a message comes up that the admin console can no longer be accessed.

      During this time, the Command Line Interface works OK, and shows current information. After the admin console can no longer be accessed, the CLI no longer works either.

      Also during the time the console appears frozen, backups continue to work properly, but email notifications are not sent out. After the week is up, backups no longer occur.

      Clients can continue accessing files throughout. Even after the console is completely inaccessible.

      When we were using FMS10, we were able to run the "catalina" restart script, which got the console running again for the week, but still crashed after the week was up. In FMS11, we had the same experience with the CLI "restart adminserver" command. It got the console back when it appeared frozen, the server still crashed after the week was up.

      At that point, rebooting the entire server seems to be the only sure-fire way to get back control of the server.

      We have also checked for corruption in our databases using both FileMaker's recover command, consistency check, and FMDiff. There were small problems that FMDiff reported, that FM Recover did not. Recovering the file fixed the errors in FMDiff, but did not seem to change the problem with FileMaker Server.

      Configuration information

      FM Server Advanced  11.0.1.99
      Windows Server 2003 R2 SP2
      Java 1.6.0_20
      IBM Xseries_3400
      Intel Xeon 1.6GHz (x2)
      2 GB RAM

      Workaround

      We reboot the machine at night. We can check that no clients are logged on by first running a "get(usercount)" on the main files.

        • 1. Re: Server Admin Console freezes then crashes
          philmodjunk

          Here is the link to the Issue you were looking for and couldn't find (I think, there are actually several on this issue).

          FMS 10 scheduled back up starts but never completes. Winserver 2008.

          • 2. Re: Server Admin Console freezes then crashes
            TSGal

            kend:

            Thank you for your post.

            This "deaf" issue was supposedly fixed in FileMaker Server 11.

            Can you tell me more about the scheduled backup?  That is, what options do you have selected?

            Please update to FileMaker Server 11v2 and let me know if you notice any change.  I know this happens after some period of time, so I don't expect an immediate reply.

            Please continue to keep me posted.

            TSGal
            FileMaker, Inc.

            • 3. Re: Server Admin Console freezes then crashes
              kdoronzio

              Good morning.

              It looks like our server acted right on schedule again. The symptom is slightly different in that it doesn't show "running" next to any schedule.

              The symptoms right now are:

              • shows wrong number of users.
              • backups are still being performed, but date on admin is wrong
              • I don't receive emails
              • can still administer the server to the command line, CLI shows the correct # of users

              To answer your questions: After the last post, I upgraded to 11v2. And I separated my backups to see if I could isolate a particular file. So daily, I back up most of our files. Weekly, I backup our huge audit log file. I also added a couple of maintenance scripts - one that runs a FileMaker Script that finds and replaces on several fields (each morning). And one that runs the fmsadminrestartadminserver each week.

              As far as I can tell, it probably went deaf during a Daily backup, scheduled for 7/26. The last emails I received was for Daily 7/25 pm, and one for 7/26 am for the find & replace script.

              The Daily backup is set for 11pm, with verification, no clone, email me upon completion, keep 7 backups. Set to backup 1 folder which has everything but our large audit log file.

              Interestingly, on the schedules page,  the Daily shows successful for 7/27 pm (but not 7/28 pm). And the utility scripts shows success for 7/28 am, but not today, 7/29 am.

              I will not attempt a restart today, but probably tonight, after the users are off.

              Also a note, hopefully not related as this part hasn't occurred as part of past occurences of this problem: the web ui has been having problems around this time - the php pages aren't doing what they should, jumping to the wrong page, and we got a error from the server that the web component stopped responding (but the site is still up).

              I hope this helps, and isn't too much information. I feel like the next step is rebuilding the server from the OS on up. But that's a big job that requires approval and IT involvement.

              Any other thoughts?

              • 4. Re: Server Admin Console freezes then crashes
                TSGal

                kend:

                Thank you for the additional information.

                This was a symptom that occurred in FileMaker Server 10 and was supposedly fixed, but it appears this is also happening in FileMaker Server 11.

                If you have the time, please do rebuild the server installing from the original OS.  Then install FileMaker Server.  In the mean time, I would like to get a copy of your Server log files.  Check your Inbox (top of this page) for instructions where to send the files.

                TSGal
                FileMaker, Inc.

                • 5. Re: Server Admin Console freezes then crashes
                  hschlossberg

                  I have found the admin console to be much more stable since upgrading to 11v2.  Keep in mind, too, that there are server command lines you can run to force an admin server (the piece that communicates between the database server and the admin console) restart.  I keep a batch file on my server's desktop so I can run this when needed; it works about half the time, which is enough to save having to do at least some reboots.