1 2 Previous Next 26 Replies Latest reply on Jan 1, 2016 2:07 PM by bigtom

    Ideal Mac configuration for FMS: to RAID or not to RAID..?

    abinning

      I'm currently re-writing our Filemaker solution and am trying to work out the best new Mac hardware to host it, looking for a boost in performance now and as much 'future-proofing' as is realistic, as well as some advice on the ideal Server configuration to ensure everything runs as efficiently as possible...

       

      Historically I wrote the original solution ~9 years ago and for the past couple of years it's been hosted on a Mac Mini (quad-core i7 + SSD). There are currently less than 10 users, although the new solution needs to provide access for our mobile team of 5/6 iOS users plus I'm toying with leveraging the potential of Webdirect and/or CWP...

       

      The database file is currently ~2GB although only 200MB or so of that is that actual database with the rest being stored images, however the plan is to store more content in the new solution (as externalised Containers).

       

      So as not to disturb the existing solution I'm using my creative workstation as a test server – this is a 12-core 2.7GHz Mac Pro (with 64GB RAM and the dual D700 6GB graphics cards) and I'm finding it's very snappy despite also being my graphics and video workstation. I do have a Pegasus R6 RAID, but the testing database is just stored in the default location within the System folder.

       

      My question really is, given the relatively small database size (although this will grow with increased container storage) what is the best hardware solution, database configuration and backup strategy..? From a hardware perspective I'm leaning towards an 8-core Mac Pro with 64GB RAM and 1TB internal flash-drive, rather than the 12-core rather or an iMac. Sensible? I also wondered what the best server setup would be: should I partition the internal storage into 2x 512GB volumes and either (a) put the database on the non-system volume; (b) put just the external container data on the second volume; or (c) just leave the drive as a single volume and store the database (and externalised container data) in the default location...? There is of course (d) which is use an external drive...

       

      I'm also thinking about backups. I'm familiar with the Pegasus 2 R6 Thunderbolt RAID so was considering a smaller R4 (RAID 10 configuration) for externally-stored daily backups, but perhaps a standard Thunderbolt SSD drive would suffice..? We do also want to have a cloud based backup, ideally stored in Dropbox... can FMS reliably write a backup to the locally synced Dropbox folder, or is the best strategy here to use a system script of some sort to copy the locally stored backup (that FMS creates daily) to the cloud..?

       

      Sorry for all the questions in one post, and thanks in advance for any recommendations and advice!

        • 1. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
          colins

          In my time - and Filemaker's - I have tried assorted computers, Raid configurations and back-up schemes, but have for the past few years settled on the following, which has run trouble free. Completely trouble free. Any restarts have been my choice - not forced on me by a crash or any other symptom.

           

          First, here's what I have. 71 databases totalling 13.5GB. Two of the databases used to be larger, but I removed the embedded thumbnail images from the container fields and swapped to reference only, bringing each down to under 3GB. Also, for external users, I swapped the use of container fields for web image serving and display using webviewer. So, I have internal users and external users. I do not use Instant Web Publishing (WebDirect), just FMP clients.

           

          In addition the server provides all the companies web services (a commercial business to business site) using FMS's built in xml coupled with lasso server 8.5.5 & 8.6. And it's fast. Very fast. Compared to the old FMSAs 8, 9, 10, 11 it flies. I continue to use 12, but have fully tested 14 on my reserve FM server and web development server with the same results. I operate on the basis that if it ain't bust don't fix it. And right now I don't need the extras provided by 14. But I will soon. And I'm prepared.

           

          My server is a Mac Pro 5,1, 2 x 2.93 quad core with 32GB of RAM. OS X 10.6.8 (Snow Leopard) FMSA12

          My reserve server is a Mac Pro 5,1, 2.66 12 core with 32GB of RAM. OS X 10.10 (Yosemite) FMS14

           

          I find no speed difference between the two. In fact, I have tested FMSA12 on an old Mac Pro 1,1, 2 x 2 dual core with 4GB of RAM running 10.6.8 and still it runs fast enough to make little or no difference to a development site I worked on a few years ago.

           

          I use SSDs. One for the System AND Filemaker databases. One for my scheduled back-ups. One as a mirror of the first SSD - i.e. System and Filemaker databases. In fact, SSDs were the best thing I ever bought for FMS(A). They're not massively expensive these days - in fact they're quite cheap. I just have 3 x 256GB - unmatched. A Samsung Evo and a couple of OWCs.

           

          As I say, this works and has worked for me over the past years. Honestly, it's trouble free. Filemaker clients and web clients 24/7 and 365 days per year. Our entire business is predicated on Filemaker and Lasso. From registered users to baskets to Orders to downloads and billing. Oh yes and royalty payments to our image providers - we're a photo archive. The only thing outside Filemaker is our accounts package - Xero. And I set up exporting routines to allow batches of invoices created in FMP to be imported into Xero.

           

          I await the cries of - you can't do that!!! Sorry, but I can and have.

           

          CS

          • 2. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
            wimdecorte

            abinning wrote:

            backup strategy..?

            Lots of different questions here so I'm splitting them apart.

             

            To determine your best backup strategy there are two questions:

            - how much data are you willing to lose?  This will decide the restore point (how old is the backup you are going to restore from)

            - how long can the business continue without the solution?  This will decide the restore time (how fast do you need to be back up and running)

             

            The answers to these will determine how to set up backups.

             

            As an overarching rule, make sure that you backups in more than one place.

            • 3. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
              wimdecorte

              abinning wrote:

              is the best strategy here to use a system script of some sort to copy the locally stored backup (that FMS creates daily) to the cloud..?

               

               

              This.  Don't let FMS write directly to the dropbox folder; you have no control over dropbox or when it syncs.

              • 4. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                wimdecorte

                abinning wrote:

                From a hardware perspective I'm leaning towards an 8-core Mac Pro with 64GB RAM and 1TB internal flash-drive, rather than the 12-core rather or an iMac. Sensible?

                 

                Don't decide this in a vacuum.  Base it on the current FMS usage stats (they are turned on, right?) and extrapolate as necessary for the projected growth in users and complexity.

                 

                If you are serious about your WebDirect plans then be prepared to invest now in a beefier machine or buy a second machine.

                • 5. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                  colins

                  I use something like this, except it isn't the cloud - it's an off-site drive with everything necessary to rebuild the business. And that includes nightly FM databases. Disaster recovery.

                   

                  I have found the progressive back-ups to be reliable. But I have never used them in a production environment - just in a test.

                  • 6. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                    colins

                    Just a further thought. I much prefer a Mac Pro to an iMac. An iMac is a desktop computer with fans/cooling to match that environment. A always prefer a chunky Mac Pro for serious serving. And generally, headless, with all other extraneous services turned off. Especially Spotlight.

                    • 7. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                      abinning

                      Thanks for the input you two... much appreciated

                       

                      Backups: currently I only do these overnight and weekly. I've not deployed any progressive backups (yet) but am thinking about adding one (to external SSD?) on top of the existing daily and weekly scheduled backups (to internal drive). This, plus also doing a Dropbox copy (of the overnight scheduled backup) should give us enough belts and braces to suit our needs. Any pointers on the best script to use for the Dropbox copy wimdecorte..?

                       

                      Hardware: it'll definitely be a Mac Pro and I guess the 8-core machine is just the happy medium... however it would be good to know under what circumstances you would opt for the 12-core? I shall perhaps need to get my head around those usage stats. I'm more serious about getting to grips with CWP rather than Webdirect, but fully appreciate the potential need for a beefier or second machine. The quad-core i7 Mac Mini might well be useful in this regard as it will be surplus to requirements once the new system goes live.


                      What about the question of partitioning the internal flash drive..? In the past I have come across the recommendation (in non-FMPro circles) to have the system/applications on one drive but the data/working files on another separate partition or physical drive. Does this advice ring true for FM Server as well, or is it better to stick with the default data location buried in the System folder..? I had figured that in this instance I would store the externalised container content on either a separate partition or fast external RAID, but perhaps this is pointless...

                      • 8. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                        colins

                        My external back-up copies are FMS back-ups transferred from my FM server to another share on our network and thence out. I simply use an applescript paired with Timbuktu file exchange via a secure connection - it is to a remote computer I own and manage. I haven't used drop box so cannot comment.

                         

                        The 8-core is fine for my current purposes (OSX 10.6.8 + FMSA12 with XML CWP). I'll move to the 12-core as and when I upgrade everything to FMS14 + 10.10 Yosemite. I'm loathe to go to 10.11 at this stage. That would require more testing before taking the plunge.

                         

                        Since I use smaller drives I see no need to resort to partitioning. Mac Pros have 4 sata slots - plenty for my purposes. Worth mentioning that I have never found sata SSDs to have any speed issues in my set-up. I work on the basis that if a partitioned drive fails, you lose everything. Whereas a separate drive is immune from such failings - notwithstanding the possibility of simultaneous failures or corruptions. In my experience a single drive running the system and filemaker databases has not been a problem - this includes the progressive back-ups run every 5 minutes, which require sufficient space for two full copies of all databases... although I would be very interested to hear the thoughts of others on this subject.

                         

                        My local FMS back-ups are written to a separate internal drive (an SSD). I keep a weeks worth (7 sets) in addition to my offsite back-ups, as well as one clean historic set should I ever experience a major corruption. With the final addition of my mirrored System/FMS drive. Sounds a lot, but I concur - belt and braces.

                        • 9. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                          abinning

                          Thanks @colins

                           

                          As far as I can tell, transferring one of the FMS scheduled backups to Dropbox would be the same as what you do just now as it's just a case of copying the backup to the local Dropbox folder via the Finder... in all my years using Macs, I've never scripted a repeating schedule for this sort of thing though.

                           

                          The new Mac Pros of course don't have the luxury of internal SATA slots (although I can live with that given the whisper quite enclosure!!) so all backup and additional storage would need to be external. I do think I'm leaning towards just storing all database files in the default location on the internal flash-based drive though as the speed is incredibly fast, and do all local FMS backups to an external SSD (or RAID).

                           

                          On the question of RAIDs are any particular configurations better for database use? I had read somewhere (can't remember where) that the Pegasus 2 R6 I have is optimised for video/media work...

                          • 10. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                            colins

                            I use cron/crontab for such scripts. Basically, I create the script and use cron or crontab to launch it and run the operation - in this case copying my FM database files to a networked location. A quick google search returned plenty of examples of using crontab and, separately, scripting moving files to Dropbox.

                             

                            I'm sure that a Thunderbolt SSD will be plenty fast enough to use as a device for regular FMS back-ups using the scheduler.

                             

                            I cannot help you with RAID drives. I imagine FM would recommend fast read and write speeds, data protection etc. I have RAIDs for image storage, but have never used one with FMS. However, you currently have a smallish database, which whilst I appreciate it will grow, suggests you could possibly consider going with a solution that will get you through the next few years. By then technology might well offer you other storage options.

                            • 11. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                              johnnyb

                              Wim is right that you should make these decisions based on measured and projected resource utilization. For the database server, RAM is more important than cores, and IOps is more important than throughput. For WebDirect, both cores and RAM are relatively important. But with Apple hardware, you only have a few choices, so don't try to get too surgical. Just get plenty. What are you trying to decide here?

                              • 12. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                                wimdecorte

                                johnnyb wrote:

                                 

                                RAM is more important than cores

                                 

                                While I like people that agree with me, I have to disagree on this statement...

                                 

                                RAM is almost never an issue and processing power is pretty much always an issue.  But these are generalizations so look at the metrics first, then decide on what potential bottleneck to spend your money.

                                • 13. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                                  johnnyb

                                  Topic for another time, perhaps. I'd be curious how FileMaker Server is different from other database applications in this regard.

                                  • 14. Re: Ideal Mac configuration for FMS: to RAID or not to RAID..?
                                    wimdecorte

                                    What regard specifically?

                                     

                                    Of the four typical bottlenecks (memory, processing power, disk i/o, network throughput) the ones that cause the most performance issues with FM deployments are processing power and disk i/o.  At least in my experience for the deployments that I get called on to remedy.

                                     

                                    It's never a silver bullet though but a complex set of circumstances.

                                    1 2 Previous Next