6 Replies Latest reply on Apr 14, 2014 7:31 PM by perren

    Backups to NAS/SAN via iSCSCI

    taylorsharpe

      FileMaker's technical advice has been to not serve databases up over networks and they enforce this with the Admin Console path validations. However, there are times that it would be nice to do backups to a NAS or SAN network storage and I just thought it was not possible. So in the past I did things like have a batch file that copies anything in the FileMaker backups folder over to the network storage or used a file manipulation plugin to perform such functions.

       

      I thoroughly understand why FileMaker does not want live connections going out over a less stable network topology. But backups sure would be nice. This week I was introduced to using iSCSI on two Mac Servers (both Mavericks) to mount a network storage volume on the server as if it was a serially attached hard drive and it works great. In this case we were using Synology storage systems and the globalSAN iSCSI Intiator (http://www.studionetworksolutions.com/globalsan-iscsi-initiator/) which is an $89 driver. You could use it to run live databases too, but I agree with FileMaker that this would not be recommended. But for backups, iSCSI is a great way to backup to network storage! Speeds can rival the performance of Fibre Channel and cost a lot less. We measured speeds in the 300 MB/s range which was great.

       

      Now it isn't near the speed of a Promise Pegasus on Thunderbolt II RAID, but many enterprise level clients have a SAN and getting backups over on the SAN usually means to me, as the database developer, a way of passing on responsibilities to another IT group in the same way that I like to set up Active Directory so I don't have to manage passwords on FileMaker and leave that up to some IT help desk group. Not having to worry about the backups and having them on a large network storage that someone elses manages minimizes my dealing with storage so I can focus on what I like to do, FileMaker development.

       

      Well, one more thing learned out in the field of FileMaker developing.

        • 1. Re: Backups to NAS/SAN via iSCSCI
          wimdecorte

          A couple of things:

           

          SANs have always been supported for live files and backups.  Obviously you'd have to take a long and hard look at the SAN infrastructure and implementation but a good dual fibre-channel SAN setup is certainly not a problem.  In fact, most virtual machine deployments would use something like this.

           

          backups to NASs and network shares (same thing really) used to work for a while and they might still work now (haven't checked in a long time) but I fail to see why you would use that:

          - it's always going to be much slower than a backup to the local disk

          - it's infinitely more vulnerable than a backup to the local disk

           

          So the aproach that you metioned in the start of your post is still the preferred method: in the absence of a SAN: backup to the local disk and then use an OS-level script to copy or move the files to a network destionation.  This way:

          - FMS can release the files quickly

          - the backup will always be there, even if the network location is not available at the time

           

           

          Let's not confuse SANs and NASes though...

          • 2. Re: Backups to NAS/SAN via iSCSCI
            taylorsharpe

            I clearly am not a storage expert, but this is what understand (and may need corrected).  SAN's usually use a Fibre Channel connection which is really a type of serial connection I guess and that is why they have always worked. 

             

            The NAS usually uses ethernet and TCP/IP to connect, which is not as stable of a network and databases are easily corrupted when a live connection fails. 

             

            The Synology is a cheap version of a SAN which is really a NAS with SAN functionality built in (a lot more utility control of the storage).  So in our case, we are wanting to make use of inexpensive network storage over ethernet. 

             

            Most NASs are are not as poweful as a SAN, but the Synology storage devices are very robust NASs with functionality commonly found in a SAN.  We are wanting to store on the Synology which is traditionally not recommended by FileMaker. 

             

            While using ethernet is not the recommended way to do things, this is a very controlled setting where the FileMaker Server, the Synology storage and the ethernet switch are all rackmounted very close to each other in a controlled server room.  There are no multiple hops for the ethernet and all of the devices share a common power and backup power supply.  Additionally, unlike most NAS's, the Synology storage is very robust and we tested speeds up to 300 MB/s, which is faster than a lot of single hard drives. 

             

            Basically, Wimdecorte is pointing out that this setup is not recommended by FileMaker Inc. and has valid issues to work out.  But I think there can be a situation where you have something like the Synology storage already in place storing other corporate data and you can have FileMaker make use of it.  Obviously you need to be very careful in such a setup to make sure the network is not multi-hop, the devices are close to each other, the network cabling is excellent, and the storage device, switch and computer are all on backup power supplies.  In this situation, this can be a solution that will save you having to buy an additional external storage device for your FileMaker Server if there is already a NAS device available. 

             

            Alternatives are to have a purchase a cheap external USB hard drive.  But that is not redundant like a RAID and is slower than Synology NAS devices.  Or you can purchase a Pegasus RAID or other such speedy RAID, but that costs a lot and you get to set up RAID and maintain it.  If you're in a corporate environment and a Synology NAS already exists, you can run batch files or scripts with file manipulation functions to move backups to the Synology over ethernet.  But it is just easier to copy directly there using iSCSI. 

             

            I agree with Wimdecorte that a NAS is not a SAN and ethernet is not as stable as a serial connection, but some NASs are close in funcitonality and speed.  And sometimes those NASs are available to you and under the right circumstances, can be an alternative to getting the backups to a more stable and redundant storage system. 

             

            There are a lot of caveats here, but iSCSI can be something to discuss in the right circumstances. 

            • 3. Re: Backups to NAS/SAN via iSCSCI
              ch0c0halic

              Have you considered they may not be valid? We have a backup verification policy that says check at least one backup file each quarter.

               

              Have a recent backup restored to you and compare it to the original backup. Comparing check sum values is usually good enough then at least open the file to make sure it works.

              • 4. Re: Backups to NAS/SAN via iSCSCI
                taylorsharpe

                Yes, I have opened backups.  Just for testing purposes, I even ran one of the databases live on the iSCSI even though that clearly is not the right thing.  So I know it is working well. 

                 

                But I will comment that ch0c0halic's recommendation is a good one because I have had people tell me backups are occurring and whent they need them, they find out the data is not there or is corrupted, etc.  I do like the validate option on backups to confirm the backups are good.  Just another way to look for issues. 

                 

                And of course there are all the other good backup rules such as having off site backups, and a plan for recovering from a disaster, etc. 

                 

                We plan to watch this setup closely for the next few months since it is new to me and I was wary of it at first.  But the company IT guy really pushed trying it so that all the company backups could be on the same storage device. 

                 

                I'll report any pitfalls. 

                 

                PS:  I am also doing backups on a cheap local USB drive that I plugged in to this new Mac Pro that I didn't tell the IT guy about.  I'm sure he'll see it in there, but it is my safety blanket. 

                • 5. Re: Backups to NAS/SAN via iSCSCI
                  PointInSpace

                  I don't see any reason why you couldn't use iSCSI for your backups.  I think FileMaker is overly paranoid with their recommendations sometimes, to the point I wonder if they know what they're talking about.

                   

                  Note that the big difference between iSCSI and Fibre Channel is that, since iSCSI runs over Ethernet a lot of the networking is done by the server's processor, while Fibre Channel does the bulk of the heavy lifting in the controller chips.  This is why we chose Fibre Channel for use with our VM SAN storage.  But for backups, I don't see this as being as big a consideration.

                   

                  - John

                  • 6. Re: Backups to NAS/SAN via iSCSCI
                    perren

                    I totally agree with John.

                     

                    A system level script schedule to shovel an existing native FMS backup to a remote volume, be it to an iSCSI, SAN, FTP, AFP, NFS (on and on and on...) endpoint is perfectly viable.

                     

                    One way to lessen the potential for corruption, or implement an early warning system is to wrap up the solution files in a disk image of some sort before transferring.

                     

                    Since you have a Mac server it's not too big a deal to create an encrypted sparse image file of the solution files with hdiutil and then shovel the image to the remote endpoint. At least when you try to crack open the image file it'll let you know if it's corrupted out or not without having to test file all the FM files therein.

                     

                    I'm guessing it's totally possible on the other OS, but I don't do Windows and am out of touch with fancy tools on that side...heh.

                     

                    Just don't try to use network volumes for the native FM backups (as preached already), that's just painful, slow and ultimately playing with fire.

                     

                    Let me know backchannel if you want some sample code snips on how to automate hdiutil in a system scheduled script.

                     

                    --Perren