13 Replies Latest reply on Mar 18, 2013 8:24 AM by philmodjunk

    Server crashed

    EP

      Title

      Server crashed

      Post

           My Mac mini server rebooted itself over the weekend.  I have FMS set to NOT auto start.  How can I tell if my database file has been corrupted?  Would my daily backup with "verify integrity" selected detect this? Thanks

            

        • 1. Re: Server crashed
          davidanders

          http://help.filemaker.com/app/answers/detail/a_id/6603/~/filemaker-server-consistency-check---when-its-run-and-why

          FileMaker Server Consistency Check - When it's Run and Why
               When does FileMaker Server perform a consistency check?

          • 2. Re: Server crashed
            EP

                 Thanks.  I have not rreceived an email notification so I'm assuming that everything is okay.  The next backup with "verify backup integrity" runs tonight.  If I don't get an email notification, then I guess all is well.

            • 3. Re: Server crashed
              philmodjunk

                   Odds are very good that a "no email" means that your file is ok, but there are no absolute guarantees.

                   You can take a post crash copy of the file down off the server and run a recover on the file for a more in depth check. There is also a 3rd party utility, FmDiff, that claims to be able to spot issues that recover misses.

                   Things to keep in mind about Recover:

                   While Recover almost always detects and fully corrects any problems with your file...

                     
              1.           The recovered copy may behave differently even if recover reports "no problems found".
              2.      
              3.           Recover does not detect all problems
              4.      
              5.           Recover doesn't always fix all problems correctly
              6.      
              7.           Best Practice is to never put a recovered copy back into regular use or development. Instead, replace the damaged file with an undamaged back up copy if this is at all possible. You may have to save a clone of the back up copy and import all data from your recovered copy to get a working copy with the most up to date information possible.

                    

                   And here's a knowledgebase article that you may find useful: What to do when your file is corrupt (KB5421).

              • 4. Re: Server crashed
                EP

                     I recovered the file and I also have the recover.log file, but not sure how to interpret it.  I got this message from recovery.  I know it is not much, but does this message tell us anything?  

                     P.S. If I recover the file over and over again, I still get the same message (I guess Recovery never truly fixes it?)

                • 5. Re: Server crashed
                  philmodjunk

                       Please read my notes on Recover that I posted previously.

                       Recover does fully fix problems found nearly all the time. But there are no guarantees.

                       The message shown is telling you that while you may be able to import the data from this copy into an undamaged copy of this file, it is not safe to put this recovered copy back into regular use as it found some major problems during the recover process. If you check what I said in those notes, it really isn't 100% to use any recovered file and you are better off replacing a damaged file with an undamaged backup than to use the recovered copy if that is an option for you.

                       Just to make life even more of a challenge, if this is FileMaker 10 or 11, a bug in the recover code can produce this message when your file is not at all damaged. This only happens if a layout has a grouped set of objects with different alignment checked, but if you are using those versions of FileMaker, you may want to take a copy of your file, go to each layout, enter layout mode, select all objects and ungroup them--even though this destroys buttons and then do a recover on this copy and see if you get the same error message. If you do, your file is probably not damaged.

                       You can sometimes open the log file and find out which parts of your file had a problem. In that case and if you don't have a backup copy that recovers cleanly, you may be able to selectively delete portions of your file until it recovers with out issue. Then you rebuild the parts you had to remove. This can be a major chore, but is less work than starting over from scratch.

                  • 6. Re: Server crashed
                    philmodjunk

                         Forgot to include this information on this recover bug:

                         For More Information see:     FMPA 10: group graphic + align = corrupt file?

                         This is one of many acknowledged bugs that can be found in the Known Bug List thread here in the Report an Issue section of the forum.

                         It can also be downloaded as a database file from:    https://www.dropbox.com/s/jt09b82i0xijbu3/FMP%20Bugs.zip

                    • 7. Re: Server crashed
                      EP

                      You can sometimes open the log file and find out which parts of your file had a problem. In that case and if you don't have a backup copy that recovers cleanly, you may be able to selectively delete portions of your file until it recovers with out issue. Then you rebuild the parts you had to remove. This can be a major chore, but is less work than starting over from scratch.

                      Thanks Phil, I guess I have to go this route.  I learned the hard way.  I had "Autostart" checked in my server settings and apparently this file has been corrupt for a couple months now (Server must have suffered a crash/reboot).  All backups seem to recover the same way, even the clones.  Aside from unchecking the "autostart" option and making sure I keep a clone of each updated version of the file, what is the best way to interpret the recover.log to try to find the corrupted part of my file?
                      Thanks
                      • 8. Re: Server crashed
                        philmodjunk

                             It's pretty cryptic. And each time you run a recover, Filemaker appends those log entries to the current log file so sometimes it helps to trash the log file and run a new recover just to make sure that you are reading through entries from a single recover operation.

                             Look for keywords that show a problem was found. I can recall that "changed' was one such key word. If that is used in conjunction with a layout or table, you can try deleting that layout and table to see if the file now recovers cleanly.

                             A really "brute force" method that we used to use before we had a recover log was to use a divide and conquer method:

                             Delete half of the tables and layouts in the file. Recover that copy. If it still has an issue, delete half of what is left. If it recovers cleanly, start gain and delete the other half of your database structure. Repeat until you've narrowed it down as much as you can. Make your recovers on clones of these files to save time as otherwise The recover can chew up a lot of time rebuidling indexes. This :"test one half at a time" method can narrow down the issue to (hopefully) just a small portion of all the tables/layouts/scripts in your file and then you can go back to your original copy, delete just those portions, do one last recover to make sure and then carefully rebuild the deleted portions.

                        • 9. Re: Server crashed
                          EP

                               I read the recovery.log and found the 2 modified items:

                                

                                         "...........
                                         2013-03-05 19:40:09.986 -0500 File Test1.fmp12 0  Recovering: field 'PatientPhone' (39)
                                         2013-03-05 19:40:09.986 -0500 File Test1.fmp12 0  Recovering: field 'POA' (41)
                                         2013-03-05 19:40:09.986 -0500 File Test1.fmp12 0  Recovering: field '_fkClientID' (44)
                                         2013-03-05 19:40:09.987 -0500 File Test1.fmp12 0  Recovering: field 'FollowUpCallStatusNonMod' (45)
                                         2013-03-05 19:40:09.987 -0500 File Test1.fmp12 8486  Reset field formatting
                                         2013-03-05 19:40:09.987 -0500 File Test1.fmp12 8477  Calculation modified
                                         2013-03-05 19:40:09.988 -0500 File Test1.fmp12 8476    This item changed
                                         2013-03-05 19:40:09.989 -0500 File Test1.fmp12 0  Recovering: field 'FollowUpNotesNonMod' (47)
                                         2013-03-05 19:40:09.990 -0500 File Test1.fmp12 8486  Reset field formatting
                                         2013-03-05 19:40:09.990 -0500 File Test1.fmp12 8477  Calculation modified
                                         2013-03-05 19:40:09.991 -0500 File Test1.fmp12 8476    This item changed
                                         2013-03-05 19:40:09.991 -0500 File Test1.fmp12 0  Recovering: field 'FollowUpCallDate' (48)
                                         2013-03-05 19:40:09.992 -0500 File Test1.fmp12 0  Recovering: field 'gSearchField' (54)
                                         2013-03-05 19:40:09.992 -0500 File Test1.fmp12 0  Recovering: field 'CaseNotes' (58)
                                         ........."
                                          
                                     
                                
                               Could I be in luck and it is just 2 fields that are corrupt?  Looking at the sample I posted would you agree that the log is referring to: Recovering: field 'FollowUpCallStatusNonMod' (45) and Recovering: field 'FollowUpNotesNonMod' (47)  as the problem fields?

                                

                          • 10. Re: Server crashed
                            EP

                                 Success! I deleted those 2 fields and ran a successful recovery (on a test file of course).  Now the trick is replacing those fields.  One is a text field Calc (stored) and the other is unstored.  It seems like it would be a major task to find out where I used these field throughout the entire databse, and since one of them works as an auto-enter calc (text field), I might screw up my data.  Any tips?

                            • 11. Re: Server crashed
                              philmodjunk

                                   If you had FileMaker Advanced, you could produce a database design report and do a text search of that report to find each layout, script, calcculation, relationship that uses that field.

                              • 12. Re: Server crashed
                                EP

                                     Ok, I was able to successfully pick through the design report.  Seems the file is not corrupt, rather there were 2 calc fields that went haywire because the table and field they referenced were missing (probably calc fields that were set up before a dev overhauled the file).  I deleted the 2 fields and no more errors upon recovery.  The file is working well.  I do notice 2 table occurrences that appeared from the recovery procedure.  Can I delete them?

                                • 13. Re: Server crashed
                                  philmodjunk

                                       I suggest testing this out on a copy. Open a back up copy, delete them and put the file through it's paces, run a design report and search for the keywords "missing" and "unknown"...

                                       Chances are that it's safe to delete them.