7 Replies Latest reply on Nov 25, 2011 4:40 AM by gdurniak

    Consistency Checks

    RossHurley

      Hi all,

       

      What do events in the Recover Log such as "Page 16485 marked free but not in free list" mean? I'm investigating mysteriously disappearing records in one file of a multi-file setup for one client. I've ruled out all the obvious suspects: overwritten by an older file, cascade deletes, deliberate acts, etc and so far have not found the cause. But I did find that the Recover Log in the folder where all the files are kept (it's a peer-to-peer setup - and that's a story too long to go into here) reports most files in the system getting consistency checks every few days. Most pass the check OK but a couple get this 'marked free but not in free list' note. The problem file seems to get get this every time. I've compared the current file with an older (and presumably good) copy of the file with FMDiff and it reports no differences or problems.

       

      I've gone back to the client to see if the date/times of these checks mean anything - manual backups are done every few days and I don't know if the dates/times align - but I'd like to know what these notes mean, and how serious they are.

       

      Thanks.

       

      Ross Hurley

      Adelaide, Australia

        • 1. Re: Consistency Checks

          Hi  Ross,

           

          I cannot enlighten you on a meaning for the message, but do wonder if the alternative process, of saving a compacted copy, might be more insightful. From what I've heard Jon Thatcher say at various DevCon, it is obvious that the saving of a compacted copy puts the file through a lengthy housekeeping operation as it cleans up the block structure. Others I've heard say anecdotally that it is an excellent way of maintaining a 'healthy' file.

           

          It would certianly be worth a try in my opinion.

           

          FMDiff is only going to look at the file structures, not at the status of data within the various tables.

           

          Let us know what you discover.

           

          Cheers,

           

          John

          Hamilton, NZ

          1 of 1 people found this helpful
          • 2. Re: Consistency Checks
            Mike_Mitchell

            Ross -

             

            The fact that you're seeing consistency checks every few days is a red flag. It indicates the files were improperly closed, and FileMaker is having to double-check them. This is potentially problematic, and might be the reason you're seeing missing records. Here's the scenario:

             

            The first person comes into the office in the morning and opens the database. His coworker comes in and opens the database as a guest of person #1. Person #1 then says, "I need to go to the dentist" and tries to close the database, but person #2 has gone to lunch. Person #2 also has a record open with edits - hence, the record is locked. Person #1 says, "$##$%%^@#", and forces person #2 out of the database so he can get to his appointment. You've now lost person #2's edits, and possibly damaged the database.

             

            Other issues can contribute to this, too; network or power blips, that sort of thing. This is why we don't recommend peer-to-peer, unless you set up one computer to be the host all the time with a UPS protecting it. That way, it keeps people from arguing over who owns the file.

             

            And I second John's suggestion of saving a compacted copy. This should be done on a routine basis, from once a week to every day, depending on how much record turnover the database has.

             

            If any of this is old news to you, I apologize.

             

            Mike

            1 of 1 people found this helpful
            • 3. Re: Consistency Checks
              Stephen Huston

              Ross,

               

              This is almost certainly the result of peer-to-peer (p2p) sharing. Consistency checks may be triggered any time a p2p client has been disconnected by the closing of the p2p host.

               

              In p2p, whoever is the host can inadvertently disconnect a user who may be creating, editing, or even deleting a record if the host quits FMP or has it quit/crash/close on them. This can happen if any potential host's disk is ever allowed to go to sleep or hibernate, or they simply logout/shutdown and allow apps to close as a result, or their computer goes to sleep at any level.

               

              I have seen this type of thing with p2p sharing so often that I simply expect this problem to  arise in in all p2p situations.

               

              There is also an issue with manually saving a copy in p2p while any record is being edited or if there are uncommitted new records  being created by a p2p client.

               

              FMP supports p2p, but not well, not reliably, and not safely.

               

              I don't work for FileMaker, but I urge you to move to a FMServer solution. GYou may need to run Recover and test the new files against old copies for changes/missing stuff before serving them properly. Good luck with the files.

              • 4. Re: Consistency Checks
                RossHurley

                Thanks to all who responded. 25 years in this game doesn’t equate to being full bottle on all areas of FMP. Winfried’s previous answer tells me what I want to know. His under-the-hood knowledge of FileMaker is always a valuable resource.

                 

                The client has come back to me – the dates/times of the consistency checks marries up with when the manual backups were done. The person doing them said that she only closes all the guests but leaves the files open on the host PC when she copies them to the flash disc. I’ve told her to fully exit FMP in future. Next time I go in I’ll see if the consistency checks have stopped.

                 

                I do have a follow-up question for John Wolff. I did import all the data from the problem file into a clone. Does a clone get all the ‘housekeeping’ that compacting a file does? Maybe I should have compacted it too.

                 

                Mike, your description of forcing peer-to-peer guests out was interesting.  But how can you force them out other than switching off the host PC with FMP still running? Just attempting to close the files brings up the list of guests logged in and you can’t exit FMP until they’ve all got out, which the client always does. I can’t see how this is ‘improperly closing’ the files.

                 

                Stephen might have provided the answer however. Inadvertent disconnection can occur if the host PC goes to sleep, hibernates or is shut down while guests are logged on. I’ll get the client to set the host PC not to sleep or hibernate.

                 

                On the question of peer-to-peer, this has been an ongoing saga with hospital management for years. The department I’m dealing with has now taken the bull by the horns and will buy a PC from their own grant finds and just get the hospital IT staff to install it. I’ll put FMS on it and, for the first time in the 15 years I’ve been supporting them, they’ll at last be on a secure network setup. It will also help their surgeons travelling overseas with the new iPads. They’ll also be able to do nightly ‘calls home’ from hotel rooms to update their travelling copies.

                 

                In the meantime I’ll do as one of the helpful respondees suggested and do a compaction of all their data files on a regular basis. It’s a separation model setup but a conversion from FMP6 (it dates back to at least FMP3 when I first became involved with them) and I’ve left their original 30-odd single-table files as separate data files. I always compact the interface file when I upload an update but I’ve never done this routinely to the data files.

                 

                Thanks again for all the help.

                 

                Ross

                Adelaide, Australia

                • 5. Re: Consistency Checks

                  Hi Ross,

                   

                  I cannot recall now whether the clone+import is the same as saving a compacted copy. The compacted copy scavenges the empty space within each block so the copy invariably will have a smaller file size. Importing the data afresh into the clone would most likely have the same overall effect.

                   

                  A comparison of the two file sizes would be one way to find out. If they are comparable the two processes must be virtually equivalent. We'll leave it to someone more knowledgeable to provide a definitive answer!

                   

                  I'm glad to see that you resolved the issue and sold the client on getting FMS installed. Well Done! They will surely wish they had made that decision years ago.

                   

                  Regards,

                   

                  John

                  • 6. Re: Consistency Checks
                    Mike_Mitchell

                    John -

                     

                    Yes, there is a difference between compacting and cloning / importing. A clone reworks several items in the internal database structure that compacting doesn't; it's a more thorough cleanup.

                     

                    Ross -

                     

                    Yes, I was referring to the dreaded "three finger fix" for forcing a client out. Sorry I wasn't clearer.

                     

                    And yes, the correct answer is definitely FMS. Congrats on making that happen.

                     

                    Mike

                    • 7. Re: Consistency Checks
                      gdurniak

                      Save As Compacted is considered the safer "fix" (if it works) and should always be tried before a full Recover. At DevCon 2008, FileMaker's Clay Maeckel explained that Save As Compacted rebuilds the file's block index, and so can fix consistency problems ( it does not re-index fields ).

                       

                      see here:

                      http://www.fileshoppe.com/recover.htm

                       

                      greg