7 Replies Latest reply on Jan 5, 2017 11:07 PM by CarstenLevin

    Progressive backups - copying off server while in progress

    smarty

      Hello all,

      I am running fm server 15 on windows 2012 r2. Progressive backups are set to at the default 5 minute interval to a local drive.

      I want to copy the progressive backup folder every say 30 minutes to a network drive to have a recent backup to restore from in the event I lose the FM server.

      Is this recommended practice? Could copying while a backup is in progress corrupt the backup files, either locally or on the network drive? I don't want to take multiple full backups during the day as it impacts responsiveness so hoped this would be an alternative.

      Regards,

      Todd

        • 1. Re: Progressive backups - copying off server while in progress
          coherentkris

          do you have evidence that "multiple full backups during the day as it impacts responsiveness"

          • 2. Re: Progressive backups - copying off server while in progress
            smarty

            Anecdotal evidence from many users only but compelling all the same. Our databases are 7GB in size and a full backup takes 50 minutes to complete.

             

             

             

            Todd Bennett

            Senior Systems Engineer

             

             

             

                       

             

             

            DDI +64 9 442 0626    |   M +64 21 570 016

            T 0800 4 SMARTPAY (0800 476 278)    |   F 0800 353 300

            E todd.bennett@smartpay.co.nz   |   W www.smartpay.co.nz

            Smartpay Limited | 205-209 Wairau Road, Wairau Valley, Auckland 0627 | PO Box 100 490, North Shore, Auckland 0745

             

               

             

                  

             

             

             

            This email and any attachments may be confidential. If received in error please delete the message and inform us by return email. Further, the internet contains many uncertainties and Smartpay cannot guarantee that what you receive is what we intended to send. Smartpay has taken great effort to verify the accuracy of this document but assumes no responsibility for any technical inaccuracies or typographical errors. If you have any doubts about the authenticity of an email from Smartpay, please contact us immediately. However, if you are amazed by our low prices, your eyes do not deceive you. New Zealand's best EFTPOS prices are what we do. Please maintain an up to date virus scanner and think before you print. Thank you.

            • 3. Re: Progressive backups - copying off server while in progress
              wimdecorte

              It shouldn't take 50 minutes; do you have 'verification' turned on for all backups?  If so turn it off and your backups should drop to a few minutes.

              Is this one file or multiple files?  If it is one file you should consider breaking the solution up into multiple files; it does wonders for backup efficiency if you have some data that does not change between backup intervals.

              1 of 1 people found this helpful
              • 4. Re: Progressive backups - copying off server while in progress
                wimdecorte

                smarty wrote:

                 

                I want to copy the progressive backup folder every say 30 minutes to a network drive to have a recent backup to restore from in the event I lose the FM server.

                Is this recommended practice? Could copying while a backup is in progress corrupt the backup files, either locally or on the network drive?

                 

                Much depends on how you try to do the copying.  If you just blindly that that folder then yes: you are going to end up with broken / incomplete files and you will probably interfere with the progressive backup routine itself.

                 

                The issue is that while you can set the interval for the progressives you have no control over when they will run.  Progressives start when the db engine starts.   So you don't know when they run exactly.

                Unlike regular backup schedules where you know exactly what time they execute.

                 

                If you make your copy routine smart you can check the date/time stamp of the progressive sets.  And you can check for a folder named "inProgress" which indicates that FMS is busy doing a progressive backup.

                Keep in mind that this is not foolproof you could for instance not see an "in progress" folder and start to copy but FMS may kick in a progressive while you are copying.  if you are trying to copy the oldest backup set you may run into FMS trying to delete that folder.

                 

                Copying 7GB worth of files is going to take a while so I think you'll always run into the FMS progressive routine.  With the short interval of 5 minutes you have you would need to be extremely careful, do a lot of checking and trapping and use a very very fast disk and network i/o to minimize the copying time.

                 

                If I had to do this, I would store the folder name of the last progressive that I copied, and at runtime of your backup script check to see if there is a newer one, and then check to only take the newer of the two sets that FMS keeps so that you will have the most time before FMS tries to delete it.

                 

                Coming back to my original point: you have no control over when progressives happen.  You can calculate the run times from looking at the progressive backup folder names for the time stamp and read from the FMS prefs what the progressive interval is.  But again: a lot of work to do in your backup script.

                 

                One thing to consider is to use the Primary/Standby feature that is built into FMS.  That will give you a backup set away from your computer, without any of these risks.

                1 of 1 people found this helpful
                • 5. Re: Progressive backups - copying off server while in progress
                  smarty

                  Yes, verification is turned on for all. Perhaps I should turn it off for all except the end of day backup. I will see how much time is saved.

                  It is multiple files with 3 or 4 larger than 1 gb. Thanks for suggestions.

                  • 6. Re: Progressive backups - copying off server while in progress
                    smarty

                    Okay, I am officially scared away from copying while progressive backups are running. Thanks. Standby server may be overkill for us but will consider this as well as 2 hourly backups to network drive without verification during the working day.

                    Regards,

                    Todd

                    • 7. Re: Progressive backups - copying off server while in progress
                      CarstenLevin

                      Hi Smarty,

                       

                      Just read this now ..

                      2 hourly backups to network drive without verification during the working day.

                      You can not directly do a fms backup to a network drive. It was possible in very early versions of FileMaker Server but FileMaker then blocked it. For good reasons ... backups to non local/not directly connected drives is always risky.

                      Start by doing the backup to a local folder, best to another physical disk rather than the one you are using for your live files. And then copy by using a system/OS script. You can call this after the backup.

                       

                      Best regards

                       

                      Carsten