7 Replies Latest reply on Dec 20, 2012 7:32 AM by philmodjunk

    Field Mapping Lost Upon Backup

    JeanAnneMayhall

      Title

      Field Mapping Lost Upon Backup

      Post

           INFO:  We have a dedicated machine where FM Pro v11 is hosted and acts as a file server.  This machine is backed up offsite each night.  We do not close the FM files. The back up system has ghosting, a Raid set up, etc.

           At one of the FM stations in our office (NOT the host), a large file is imported every morning from an outside source.  The Field Map is set up on that computer ...the one handling the import. 

           Yet, when the host is backed up each night, the host then retains the field mapping and the machine that set up the mapping does not.  Can you tell me how and if this is solvable. 

           Note:  We prefer not to do daily work on the machine acting as the host.  But we seem unable to predictably retain the field mapping when it is set up on the remote machine.  It works for a few days and then we lose the mapping?  Yet mapping while never set up at the host, is still retained at the host after backup.

           Suggestions?  Thanks

           Jean Anne 

            

        • 1. Re: Field Mapping Lost Upon Backup
          philmodjunk

               This machine is backed up offsite each night.  We do not close the FM files.

               This is nto a good idea. Backup copies made of open FM files will trigger an automatic consistency check the first time that they are opened if you are lucky. If you are unlucky the backed up copy will be damaged and unusable. Only closed files should be backed up by your back up software. FileMaker server back up schedules and FileMaker Pro host applications can both be set up to create a local closed backup file that your backup software can then safely back up to a remote source.

               For you import mapping, this sounds like a typical case where changes to system settings from a client instance of the database are not retained. I would recommend that you set up a script for doing your data import. Not only can you preserve the correct field mapping inside the Import Records script step, it will be faster and less prone to error. Such a script need not specifically refer to the name or location of the file from which you will import data if that is an issue.

               If the file of data to be imported always has the same name, you also may want to investigate whether setting up a recurring import script will work for you. With that setup, you can drop a new copy of the file into a specific folder and FileMaker will automatically import the data--replacing the data originally in the table that is the target of that import.

          • 2. Re: Field Mapping Lost Upon Backup
            JeanAnneMayhall

                  Phil,

                 Thank you so very much for your note above regarding losing data/mapping during open-file backup.   I have one more question.  I was told my a techie at FM, over the phone, that if we use FM SERVER instead of using FM Pro as a pseudo-server, we can back up an OPEN file.  Is that true? Or do you just always recommend closing the file before backing up?

                 Jean Anne

            • 3. Re: Field Mapping Lost Upon Backup
              philmodjunk

                   FileMaker Server has a number of value added features that make for better file hosting, but you can back up an open file from FileMaker Pro as well as from Server. The methods are different and it's easier to do from server but you can do Save A copy as... from the hosting computer and thus generate a backup copy of your file while it is still open and hosted by FileMaker Pro.

                   And a script on FileMaker Pro can save such a copy on a schedule--but it's a more complicated option and FileMaker Pro can't "verify" the copy like server can.

                   With either method, never, ever, set up 3rd party backup software to make backups of your open files. Make a local backup copy from FileMaker Pro or Server first, then have your backup software make a backup of this closed backup file.

              • 4. Re: Field Mapping Lost Upon Backup
                JeanAnneMayhall

                     Again, thank you so much for your direct answers.  I and my small business have been using FM for over 15 years.  Our company depends upon it, so I am glad to know I should never Backup open files via a third party. I have been doing so for about two weeks and STOPPED today after I read your post.   An outside company came to my office, viewed FM in action, and told me it would be safe back up open files, and my gut said otherwise.  Glad I sought you out.

                     I will use Save A Copy As until I get the scripts done.  THANK YOU!

                     Jean Anne

                • 5. Re: Field Mapping Lost Upon Backup
                  philmodjunk

                       There's a bit on a FileMaker file that is set to mark it as a file that has been properly closed. When FileMaker crashes, is force quit or when a copy of the open file is made by some other application, this bit isn't set and the first time this file is opened, Filemaker makes a consistency check that makes a quick scan for damage to the file. On large files, this consistency check can take quite a few minutes to complete. This is the BEST possible result when you back up that open file using backup software.

                       Unlike other appliations such as MS Word, FileMaker regularly writes data to the hard drive as long as it is open. A backup copy made of the file that has the misfortune to copy it in midwrite can result in a file that is corrupted. Sometimes such file corruption is obvious--the file immediately crashes or hangs. Other times it can be latent and no obvious issue is noticed until considerable time has passed. Though rare, such latent damage has been known to be invisible until either the operating system or FileMaker is updated.

                  • 6. Re: Field Mapping Lost Upon Backup
                    JeanAnneMayhall

                         Ouch.  So given that we have used this 3rd party open-file backup for a about three weeks....Now I am worried that we might have a latent corruption issue.  We have lots of FM files.  Some are large. Over the many years with FM, I have experienced only a couple of crashes and/or data losses and it is a crushing and office-stopping situation.

                         So....here hopefully are my last questions:

                         1)  Will a RECOVER recognize and deal with any corruption?  Do you recommend this?  Or should I clone (empty) the files and import?  My fear there is that the data I am importing MIGHT be corrupt? Or if you have any other ideas, lay them on me!

                         2)  Lastly, to save time, is there a script somewhere that is ready-made for backing up in FM Pro 11. (Recognizing we will soon be switching over to 12 on all computers, will all scripts translate in 12 ...I know 12 is a big step as was fm7.)

                         Thank you once again.

                    • 7. Re: Field Mapping Lost Upon Backup
                      philmodjunk

                           The damage can only occur, if it occurs, to the backup copy. The original cannot be affected by this. If you have restored from backups made this way, keep in mind that the odds are very good that your files are just fine.

                           1) Recover almost always finds and almost always fixes any problems. But there are no guarantees and I can quote you case histories where recover did not detect a problem yet the file became unstable.

                           Things to keep in mind about Recover:

                           While Recover almost always detects and fully corrects any problems with your file...

                             
                      1.           The recovered copy may behave differently even if recover reports "no problems found".
                      2.      
                      3.           Recover does not detect all problems
                      4.      
                      5.           Recover doesn't always fix all problems correctly
                      6.      
                      7.           Best Practice is to never put a recovered copy back into regular use or development. Instead, replace the damaged file with an undamaged back up copy if this is at all possible. You may have to save a clone of the back up copy and import all data from your recovered copy to get a working copy with the most up to date information possible.

                           And here's a knowledgebase article that you may find useful: What to do when your file is corrupt (KB5421).

                           2) It depends on what you mean by "ready made". A server back up schedule is quite easy to set up and most third party backup programs can be configured to back up the contents of the folder containing such backups instead of the open files.