9 Replies Latest reply on May 20, 2014 12:41 PM by TSGal

    FM Server Unexpectedly Stops and Won't Refresh Screen

    scottworld

      Summary

      FM Server Unexpectedly Stops and Won't Refresh Screen

      Product

      FileMaker Server

      Version

      13.0.2.295

      Operating system version

      OS X 10.9.2

      Description of the issue

      Hi there,

      We are having a double problem with FM Server 13.0.2.295:

      1. We are having a recurring issue with FileMaker Server 13.0.2.295, where it will unexpectedly turn itself off, and therefore forcefully eject all the users out of the databases that they are connected to. We then have to manually launch the Admin Console via Safari and press the "On" switch to turn FileMaker Server back on again.

      2. This problem is compounded by the fact that after we turn FileMaker Server back on again, the Admin Console doesn't refresh its windows at all. It still shows "ghosted information" from before it turned itself off. For example, please take a look at the attached screenshot. FileMaker Server is currently showing us that 10 users are connected, but there are actually ZERO USERS CONNECTED, because it just force ejected all of them out of the system. There is nothing that we can do on our end to get FileMaker Server to refresh this window. Even if we log out and log back in again, it STILL shows this inaccurate information.

      Problem #2 sometimes feels like a bigger problem than problem #1, because it isn't giving us accurate information about our FileMaker Server software. It seems like the only way that we can fix problem #2 is by completely restarting the FileMaker Server machine altogether. That is the only way to get the Admin Console to refresh its windows with current & accurate information.

      Thanks,
      Scott

      Screen_Shot_2014-05-17_at_12.09.35_AM.png

        • 1. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
          TSGal

               scotty321:

               Thank you for your post.

               With problem #1, what are the last few listings in the Event.log file before FileMaker Server "turns itself off"?  This may provide a clue to why the server is turning itself off.

               With problem #2, after a restart showing the incorrect listing of clients, are guests still able to connect to the server?  If so, do the clients show up with a second instance in the Admin Console?  Or is the Admin Console totally unresponsive?  Have you tried manually disconnecting the users after the restart?  That is, click the Envelope icon on the right side and select "Disconnect All Clients".

               TSGal
               FileMaker, Inc.

          • 2. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
            scottworld

                 Hi TSGal,

                 Thanks for your help. 

                 Hi TSGal,

                 Regarding problem #2: After toggling FileMaker Server "on" again from within the Admin Console (but prior to a complete restart of the entire machine), all the guests are able to successfully connect to the server again. However, the Admin Console is totally frozen in time from a previous time, possibly up to 24 hours prior. So when the clients reconnect successfully, they do NOT show up as a second instance in the Admin Console, because the Admin Console is NOT updating with any new information… the Admin Console is only showing outdated & frozen information that was frozen in time from many hours ago, possibly up to 24 hours ago. The Admin Console is also unresponsive to any choices that you may make. You can switch around between the different screens, but it won't respond to any choices that you make with your mouse. For example, you can try choosing "Disconnect All Clients", but nothing happens. Another example: you can click on a client's name and try to perform actions on it, but nothing happens. You can try to run a schedule, but nothing happens.

                 Regarding problem #1, below are the last few log listings (from newest to oldest) before FileMaker turned itself off. The very first (newest) listing is when I was forcefully kicked out of the database because FileMaker Server turned itself off. Everything newer than that, which I didn't include, is just users opening the database again. Leading up to FileMaker Server turning itself off (i.e. everything earlier in the list), you can see that there were quite a few scripts being run on the server (using the new "Perform Script On Server" functionality), and there was a successful backup performed as well. So everything looked pretty good, right up until the point when I got disconnected. HOWEVER, the "frozen Admin Console" reflects a state in time that was many hours prior. In other words, in the screenshot above, you'll notice people that are supposedly connected to the database, but they had actually logged out up to 24 hours earlier.  

                 Sat May 17 22:13:58 PDT 2014 Server Events Warning Client "admin (Scott’s MacBook Pro) [66.68.191.40]" no longer responding; connection closed. (51)

                 Sat May 17 22:03:49 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" closing a connection.

                 Sat May 17 22:03:49 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" closing database "Keslow2" as "kuhn".

                 Sat May 17 22:01:55 PDT 2014 Server Events Information Schedule "Local 10pm" completed.

                 Sat May 17 22:01:55 PDT 2014 Server Events Information Schedule "Local 10pm" scheduled for 05/18/2014 10:00 PM.

                 Sat May 17 22:01:55 PDT 2014 Server Events Information Consistency check succeeded on backup of database "Keslow2".

                 Sat May 17 22:01:05 PDT 2014 Server Access Information Client "Watto" opening a connection from "Watto’s MacBook Pro (23.241.255.226)" using "Pro 13.0v3 [fmapp]".

                 Sat May 17 22:01:05 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" opening database "Keslow2" as "kuhn".

                 Sat May 17 22:00:11 PDT 2014 Server Events Information Deleted backup folder "filemac:/Macintosh HD/Library/FileMaker Server/Data/Backups/Local 10pm_2014-05-10_2200/".

                 Sat May 17 22:00:09 PDT 2014 Server Events Information Backup of Keslow2 completed.

                 Sat May 17 22:00:09 PDT 2014 Server Events Information Backup attempting to pause up to 1 database(s)...

                 Sat May 17 22:00:03 PDT 2014 Server Events Information Starting to back up database: Keslow2...

                 Sat May 17 22:00:00 PDT 2014 Server Events Information Starting to back up 1 database(s) to folder "filemac:/Macintosh HD/Library/FileMaker Server/Data/Backups/Local 10pm_2014-05-17_2200/".

                 Sat May 17 22:00:00 PDT 2014 Server Events Information Schedule "Local 10pm" running.

                 Sat May 17 21:50:37 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:50:37 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" closing a connection.

                 Sat May 17 21:49:42 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 33 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:49:42 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 33 (FileMaker Script)" closing a connection.

                 Sat May 17 21:49:42 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 33" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:49:42 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 33" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:49:42 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 33 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:49:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 32 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:49:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 32 (FileMaker Script)" closing a connection.

                 Sat May 17 21:49:01 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 32" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:49:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 32 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:49:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 32" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:48:28 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 31 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:48:28 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 31 (FileMaker Script)" closing a connection.

                 Sat May 17 21:48:28 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 31" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:48:27 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 31" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:48:27 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 31 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:48:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 30 (FileMaker Script)" closing a connection.

                 Sat May 17 21:48:01 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 30 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:48:01 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 30" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:48:00 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 30 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:48:00 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 30" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:47:29 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 29 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:47:29 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 29 (FileMaker Script)" closing a connection.

                 Sat May 17 21:47:29 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 29" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:47:29 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 29" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:47:29 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 29 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:46:55 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 28 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:46:55 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 28 (FileMaker Script)" closing a connection.

                 Sat May 17 21:46:55 PDT 2014 Server Events Information Schedule "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 28" scripting error (3) at "Keslow2 : Batch Unreserve Quote Line Items 2: Unreserve LIne Items : Freeze Window".

                 Sat May 17 21:46:54 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 28" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:46:54 PDT 2014 Server Access Information Client "USER-INITIATED: Batch Unreserve (All/Extra) Quote Line Items 1b (ON SERVER) - kuhn 28 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:39:58 PDT 2014 Server Access Information Client "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27 (FileMaker Script)" closing database "Keslow2" as "kuhn".

                 Sat May 17 21:39:58 PDT 2014 Server Access Information Client "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27 (FileMaker Script)" closing a connection.

                 Sat May 17 21:39:58 PDT 2014 Server Events Information Schedule "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" scripting error (101) at "Keslow2 : Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) : Go to Record/Request/Page".

                 Sat May 17 21:39:53 PDT 2014 Server Events Information Schedule "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" scripting error (3) at "Keslow2 : Batch Reserve All Quote Line Items 4: Reservation post-checks : Freeze Window".

                 Sat May 17 21:39:53 PDT 2014 Server Events Information Schedule "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" scripting error (3) at "Keslow2 : Batch Reserve All Quote Line Items 3: Perform Actual Reservation for this line item : Freeze Window".

                 Sat May 17 21:39:53 PDT 2014 Server Events Information Schedule "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" scripting error (3) at "Keslow2 : Batch Reserve All Quote Line Items 2: Reservation pre-checks : Freeze Window".

                 Sat May 17 21:39:53 PDT 2014 Server Events Information Schedule "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" scripting error (3) at "Keslow2 : Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) : Freeze Window".

                 Sat May 17 21:39:53 PDT 2014 Server Access Information Client "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27 (FileMaker Script)" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:39:53 PDT 2014 Server Access Information Client "Batch Reserve All Quote Line Items 1b: Go to related records and loop (ON SERVER) - kuhn 27" opening a connection from "FileMaker Script" using "Server 13.0v2 [fmapp]".

                 Sat May 17 21:20:11 PDT 2014 Server Access Information Client "Watto (Watto’s MacBook Pro) [23.241.255.226]" opening database "Keslow2" as "kuhn".

                 Sat May 17 21:20:11 PDT 2014 Server Access Information Client "Watto" opening a connection from "Watto’s MacBook Pro (23.241.255.226)" using "Pro 13.0v3 [fmapp]".

                  
                 Thanks,
                 Scott
            • 3. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
              TSGal

                   scotty321:

                   Thank you for the additional information.

                   Unfortunately, the displayed log does not provide enough information.  I do see scripts being run, users opening and closing connections, backups, etc.  Is there a specific time where you know when everyone was booted off?  This may help narrow it down further.

                   TSGal
                   FileMaker, Inc.

              • 4. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                scottworld

                     Yes, the boot off time was: Sat May 17 22:13:58 PDT 2014

                Is there a way for me to get you more detailed logs? I pulled this log from the Server Admin Console, on the "Log Viewer" screen.

                • 5. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                  TSGal

                       scotty321:

                       A system log from the client would reveal more about the disconnect.  This will determine if the connection was lost, if you timed out, or something else.

                       TSGal
                       FileMaker, Inc.

                  • 6. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                    scottworld

                         Sure. Could you please give me instructions on how to find that log on the client computer?

                         Thanks,

                         Scott 

                    • 7. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                      TSGal

                           scotty321:

                           The system logs are found in the hidden private folder.  A good source for finding and viewing the system.log file can be found at:

                      http://pondini.org/OSX/Logs.html

                           Be sure to get the system.log file that corresponds to the times when disconnected from the server.

                           TSGal
                           FileMaker, Inc.

                      • 8. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                        scottworld

                             Thanks, TSGal. I was able to find the system.log file. It looks like leading up to the crash, there are about 20 errors in the log that look very similar to this one line:

                             May 17 22:00:11 Turbo.local mds[84]: (/)(Error) IndexPath in oid_t _oidParentForOid(SI_PersistentIDStore_t, oid_t, const MDCredential *):stat succeeded, getattrlist returned error  2 for 1117590 (/.vol/16777217/1117590 = /Library/FileMaker Server/Data/Backups/Local 10pm_2014-05-10_2200/Databases/RC_Data_FMS/Keslow2/Files/Keslow2/Purchase Orders Attachments/2-523-46131.jpg)

                              
                        • 9. Re: FM Server Unexpectedly Stops and Won't Refresh Screen
                          TSGal

                               scotty321:

                               If possible, I would like a copy of that system log.  This may help narrow down the issue further.  Check your Inbox at the top of this page for instructions where to send the file.

                               TSGal
                               FileMaker, Inc.