7 Replies Latest reply on Oct 27, 2013 3:08 PM by stephensexton

    Backup Strategies


      It is common knowledge that you need to backup your data unless your information is not important. So I am going to assume your FileMaker data is important.


      You can have a small multi-user office running peer-to-peer sharing with FileMaker Pro. But if you do, you will need to periodically quit the application running the FileMaker solution to back up the data and then restart it. I've seen people even do fancy OS level scripts to automate this. But the biggest danger with using FileMaker Pro, particularly as muti-user, is that if the application dies, it invariable causes corruption to the data. FileMaker Pro has a lot of User Interface code that often interacts with other programs, making it much more prone to a crash. And backups of corrupt data just propogates the problem, it doesn't take care of it.


      FileMaker Server (FMS) exists for the purpose of serving files only. There is no User Interface, making it a less complex software tool and less likely to crash. Also, it runs as a service and not as an application, therefore making it much more stable. Basically, this means it runs whether you are logged in as a user or not.


      While FMS does many things beyond FileMaker Pro, one of the biggest advantages are backups that occur while the databse is running. You do not have to close the files to make backups. This really is a big deal in the database world and FileMaker had to do a lot to technically pause the database to create backups.


      FMS automatically installs with a schedule script to run a daily backup of all files and keep the backups for 1 week. I recommend changing this. I set a daily (evening) backup to run daily and keep 4 backups. Then I set another backup to run once a week and keep 4 backups. And then I have a third schedule to keep a backup every 30 days and I keep 4 backs up that. That way I have periodic backups up to 4 x 30 or 120 days ago.


      As many of you know, it is often recommended to run your OS on one drive and your data on another drive (preferably a fast RAID or SSD). Additionally, it is recommend the backup data be on a different drive than the live data drive. While a third drive system is preferred, I often just do the backups on the OS drive from the live data drive. Separating the live data from the backups is a good idea in case the drive dies, your live data and backups are not on the same drive.


      FMS 12 came with a new type of backup called Progressive Backups where FileMaker basically makes frequent backups that only backup the differences between the last save, usually done once every few minutes (e.g., 5 minutes). This is great if you want to restore to a backup right up to the time before a crash or other problem (e.g., you accidentally deleted that every so important table <grin>). There is one caveat. If you just deleted a bunch of records and do a restore from a progressive backup past that point, the deleted records will not show up.


      Lastly, there is the off site backup strategy. Traditionally, this is the office copy you make once a week and take home and swap out with the previous weeks copy so you always have an off site copy. But the last couple of years a lot of off site copying is now being done over the internet, or as all the marketing hype says, copying "to the Cloud". This could be copying to another company location that has a server, but more often, people are looking at third party places to backup to. 360 Works has a FileMaker Server solution called SafetyNet that is designed to work with FMS. It is pretty slick in that it works as a service and just happens in the background. It copies your data to the Amazon S3 cloud and you don't pay for the software, you pay for the storage. The disadvantage to SafetyNet is that it was designed before Remote Container storage and it currently does not backup your remote container files. Also, it has a limit of about 6 TB and won't store anything bigger. 360 Works has told me version 2 is coming out soon that is supposed to take care of these issues and I hope it comes out quickly. An alternative is to run one of the general file backup programs and there are a lot of them. The one I use most often is Arq. Its benefit is it works with more files than just FileMaker. Also, it gives you the option to save to the more readily available Amazon S3 Cloud or alternatively, to the less available (delays up to 4 hours to copy back) Amazon Glacier storage, which is much cheaper. The big disadvantage is that Arq is an application and not a service. This means you have to be logged in as a User with the application running for it to work. That is a real bummer for a server. I'm just hoping the new version of SafetyNet comes out soon and it does what all Arq does, only as a service. Or maybe a new version of Arq will come out as a service. Does anyone know any other such Cloud backup solutions running as a service, particularly ones that support Mac Servers since most of my servers are Macs?


      One last important comment: FileMaker recommends that your backups do not touch or copy live FileMaker data files. Only make copies from the backups folder. Copying live data often results in corrupting the original data. This is also true of virus scans. This is an issue in the FileMaker world, and not SQL databases, because FileMaker databases are available to Apple's Find and Windows Explorer. SQL databases are hidden from the native file navigation tools. Not scanning and virus scanning SQL databases is just important for them as for FileMaker, but they handle things differently by hiding the files. The disadvantage is that it takes special software tools to do their backups whereas FileMaker can have backups from many different software tools.


      Let me know if any of you have some other backup strategy suggestions.

        • 1. Re: Backup Strategies

          . Does anyone know any other such Cloud backup solutions running as a service, particularly ones that support Mac Servers since most of my servers are Macs?


          Hi Taylor,


          Following the recommendations of Joe Kissel of TidBITS, I've used CrashPlan for my off-site backup service for the critical files from our Macs. CrashPlan now has some servers in Australia which eases the bandwidth somewhat for those of us in the South Pacific.


          I zip the FMS Hourly backup forlder and move it to Hourly, Daily and Weekly folders on my Archives HD using Rob Russel's Shell Script scheduled at 5 minutes after the hour for the Hourly Script. The Daily and Weekly scripts run once each day at 10 and 15 minutes after the 10 pm FMS backup. The Weekly folder only has the Daily Backups from the latest week and is the folder that I have CrashPlan watch, along with our Bookkeeping and important image files.


          We've found the CrashPlan service to be both affordable and reliable and is thus an alternative to the Amazon S3 you've described. It's only downside for us is the time it would take to restore a large number of files.


          I hope this answers your question.



          • 2. Re: Backup Strategies

            I've not used CrashPlan, but it is another good example of a "Cloud" backup plan.  It would be interesting if there were a comparison between these different Cloud backups comparing price, speed, etc.  Since I primarily use SafetyNet, I can speak for them.  It is stored on the Amazon S3 cloud.  They cost 99 cents per month, $1.00 per gigabyte per month stored, and $1.00 per gigabyte transferred. 


            I've moved away from doing frequent backups like Hourly backups because it makes the database go very slow during the backups and tends to make users unhappy.  Instead I run the progressive backups and nobody seems to notice the difference.  You might consider giving progressive backups a try as an alternative to doing hourly full backups. 

            • 3. Re: Backup Strategies

              Hi Taylor,


              Under FMS V11 we did notice the the server was unresponsive for some 30-40 seconds each hour and was one of the factors that motivated our conversion to V12. That pause is no longer seen under FMS 12 and thus we have no good reason to forego the hourly backups.


              I'm paying about US$120 per year for CrashPlan which is holding over 600 GB of data from four Macs, most of it is large image files of the quilts we've photographed. It would take us several weeks to restore it all, should that ever be required.





              • 4. Re: Backup Strategies



                I use Wuala by LaCie for my off-site backups. It does a synchronization between FileMaker Server Backups folder and the cloud.

                Pros: First 5 GB are for free.

                Cons: it is an application and not a service.

                • 5. Re: Backup Strategies

                  Hi John,


                  I am also using Crashplan on my server and with some clients.  Crashplan has an other free option, you can use an other machine in your network to write your backup to, beside the cloud.  If you need to restore multi gigs of data from a machine in the network will be much quicker then from the cloud.


                  Hope that helps,


                  Best regards,


                  Ruben van den Boogaard

                  Infomatics Software


                  • 6. Re: Backup Strategies

                    For folks working on a non-hosted file locally on their desktops, they can create a backup schedule without having to close the file periodically using the Local Backup module. This is not a substitute for FMS backups for an application with several simultaneous users, but a convenience tool for single users. Friends don't let friends do peer-to-peer hosting.

                    • 7. Re: Backup Strategies

                      CrashPlan has worked great for some of my clients, and badly for others... I've had 3 business recently that have experienced repeated crashes of "CrashPlan" whereby CrashPlan's technicians have been unable to remedy the situation, and my clients have gone back to rotating a couple of external drives for their offsite backup solution.  That being said, when CrashPlan works, its great.  There is the option to create mutliple backup sets (e.g. the FMS "Backups" folder as one set) and to prevent backing up open files in the advanced settings.  You can send the backups to multiple destinations (cloud, other computer, external drive) and customise the frequency of the backups.  I'd love to know if anyone else has experienced similar problems with CrashPlan and has been able to resolve it...  because I haven't been able to find an economical alternative as yet.