9 Replies Latest reply on Jan 22, 2017 9:10 AM by gdurniak

    File won't pass recovery cleanly but functions, laying out a plan

    ericjlindholm

      I started my DB in FM11 and have been building it and my skills ever since then.  I run my entire audio/video rental and production company off of it and It is the most valuable asset to my business.

       

      I planned on one day starting a new DB that will be much better than my current.

       

      As it turns out my file is terminally ill.  full recovery appears to be impossible and all my back ups from over 2 years ago are also producing the same result when recovered.  I think i narrowed it down to 2 tables with key errors.  I replaced those tables and it appears to be working but I don't want to risk staying in that file.  Clones, copies and backups all fail the same way.

       

      I have done what i can to patch it up so my business can keep running but I feel like my only option is to build something new and fast.

       

      I plan on creating new file from scratch but importing all my records from my first file when it is complete.

       

      This is going to be a HUGE project.  I am trying to figure out if i can do it in modules.

       

      I could create modules in the New DB and have the file reference the tables in the new DB while i am working.  I know that it is not a best practice but has anyone heard of a functioning but corrupt DB damaging another file while connected as an external data source?

       

      I know that throwing my users into and entirely new DB will cause lots of little clean up that could cripple us for a week for to so i am really hoping to do this slowly. 

       

      any input would be appreciated.  thanks

        • 1. Re: File won't pass recovery cleanly but functions, laying out a plan
          wimdecorte

          Files typically get corrupted because of something in the deployment; I would scrutinize the deployment to make sure it is completely to best practices before deploying any of the new files.

           

          In addition to the recovery: can you run an XML DDR on the files?  Does that error out at some point?

          • 2. Re: File won't pass recovery cleanly but functions, laying out a plan
            gdurniak

            "Recover" is not well documented, so "problems" found can be unnecessarily scary

             

            I have collected info here:   http://www.fileshoppe.com/recover.htm

             

            The key is the Recover Log.  You need to see what the "Problems" actually are

             

            ( You can send me the log, if not sure )

             

            If you can remove, or replace objects,  and get the "Problems" to disappear,  then the file is probably okay

             

            Are there any other symptoms ?

             

            I have a few files in production,  that have "problems", that are minor,  and just don't matter

             

            greg

             

            PS

            if one file in a multi-file system is "corrupt", it can affect performance of the entire solution. I have seen this happen

             

             

            > I think i narrowed it down to 2 tables with key errors.  I replaced those tables and it appears to be working but I don't want to risk staying in that file.

            1 of 1 people found this helpful
            • 3. Re: File won't pass recovery cleanly but functions, laying out a plan
              ericjlindholm

              I can run a full xml DDR that does not error out.   Sorry I am not positive what you mean by deployment.  I assume it means making sure my server machine is set up correctly.  power failover as well as the right versions of java and the like? 

               

              I have a great server now and properly deployed server now.  unfortunately in the past, I did not. Based on checking my backups from when i first moved the file over, it was giving the same errors then.

               

              I will post my symptoms in a follow up post.

              • 4. Re: File won't pass recovery cleanly but functions, laying out a plan
                ericjlindholm

                I get the usual 8495 problems error.   then i get

                 

                Schema: scanned fields and tables; some problems were found...

                fields created to match record data: 0

                field values deleted due to invalid ID or repetition: 0

                records deleted due to invalid ID: 0

                Structure: scanned; 2 item(s) modified

                 

                there are no other errors in the log file.  I was praying that i could perform surgery on the file.  I started by deleted EVERYTHING from a copy to see if it would go away.  down to the privilege sets and custom functions. nothing worked. 

                 

                then i tried copying the tables to a new DB and receiving that, it failed the same way

                when I tried copying table by table, i narrowed it down to 2 fields that gave me the error "Resetting invalid table key"

                 

                I attempted to complete delete those tables and their TOs in my file and that still does not fix the issue.  and that was hours of work but worth a shot based on my trouble shooting.

                 

                symptoms:

                     -exiting layout mode does not work.  edits are kept but i have to close the window and reopen it to resume normal operation. 

                     -random quits which were not really a concern but these things are finally adding up.

                     -gernerally poor performance which i blamed on a few calcs that were verifying data on a few related records.  I recently made them auto enter fields that just get updated with the child rec is updated and that really highlighted the laggy/skippy scrolling in list view.

                     - most recently, at times, selecting a rec in list view makes the field you are editing actually change the record below it.   deleting a record will also delete the record below where the button was clicked.  this can happen to the next record in a table even if it is not in the found set!

                 

                This has happened across many macs running different versions of the latest OS and server 15.0.1, 15.0.2 and now 15.0.3

                 

                also, running a server class windows PC with windows server 12 as the os. 

                • 5. Re: File won't pass recovery cleanly but functions, laying out a plan
                  gdurniak

                  Not quite clear.  Did removing the Tables remove the "Problems", but not the symptoms ?

                   

                  Just out of curiosity, send me the full Recover Log  ( info@fileshoppe.com )

                   

                  If  "2 item(s) modified",  you need to find those items,  in the Log  ( search for "Changed" )

                   

                  greg

                   

                  PS

                  Worth a try:  Save the file as "compacted",  export all data to .MER files,  then import back into a Clone

                   

                  > I attempted to complete delete those tables and their TOs in my file and that still does not fix the issue

                  • 6. Re: File won't pass recovery cleanly but functions, laying out a plan
                    ericjlindholm

                    Removing and recreating had no effect.  same symptoms. 

                    ah ha! searching "changed" produced  the following

                     

                    Item count changed from 62 to 63 in: library catalog 'User'

                    Rebuilding library catalog 'User' order list: quick reference

                    Rebuilding library catalog 'User' order list: type

                     

                     

                    Recovering: library catalog

                    Item count changed from 64 to 65 in: library catalog

                    Rebuilding library catalog order list: quick reference

                    Rebuilding library catalog order list: type

                     

                    the second one is strange because its not identifying a table, I wonder if it thinks there is a blank table. 


                    I am going to run a fresh recovery log of my current file and see if i get the same errors.  will email shortly.  thanks so so much for your responses.

                    • 7. Re: File won't pass recovery cleanly but functions, laying out a plan
                      gdurniak

                      Since you are up to FM 15,  it might pay to call Tech Support

                       

                      They may not fix your file,  but any clarification you get would help the rest of us

                       

                      "Libraries" contain the items stored in container fields such as graphics, or Word documents. There is also a Library that contains any graphics stored on layouts ( and even graphics no longer in use )

                       

                      "Catalogs" are internal lists of things,  layouts, field ID's, etc.  If a Catalog gets scrambled,  all bets are off

                       

                      If Recover did "fix" the Catalog,  for two libraries,  that could make a difference

                       

                      greg

                       

                      > Recovering: library catalog

                      • 8. Re: File won't pass recovery cleanly but functions, laying out a plan
                        wimdecorte

                        ericjlindholm wrote:

                         

                        Sorry I am not positive what you mean by deployment. I assume it means making sure my server machine is set up correctly. power failover as well as the right versions of java and the like?

                         

                        It's the "set up correctly" that needs to be scrutinized.  Things like indexing, OS-level file sharing, AV and external backup software,... basically anything that could touch the FM files directly where it should not.

                        Starting and stopping procedures, FMS auto-hosting settings, event log monitoring to get early warnings,...

                        • 9. Re: File won't pass recovery cleanly but functions, laying out a plan
                          gdurniak

                          There are plenty of "Help" articles on Server Setup

                           

                          e.g.   http://help.filemaker.com/app/answers/detail/a_id/14165/~/performance-management-best-practices-on-the-filemaker-platform

                           

                          But first,  you need to get your file working

                           

                          greg

                           

                          PS

                          Corruption is fickle, and can happen even with the best of practices

                           

                          > I am not positive what you mean by deployment