9 Replies Latest reply on Jan 26, 2015 1:32 PM by jcull

    Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID

    taylorsharpe

      I posted the following a while back and we had a number of good discussions. The tests I had done on my client's machine had been done remotely and their computer center. I got to go to the facility and see the machine this past week and realized something I hadn't reported on previously. On the Mac Pro with Apple RAID card running RAID 5, the drives are eneterprise level SAS drives whereas the drives used on the Pegasus R6 were only inexpensive and slower SATA drives. This makes the 20% speed increase by the Pegasus even more impressive since it was using slower/cheaper drives.

       

       

      ========================

       

       

       

      I have a client with a reasonably large database (2/3's TB) that would like it to run faster. They are running on a Xeon Nehalem Mac Pro (MacPro4,1) with Apple RAID Card, RAID 5 over 4 drives. This is the Mac Pro made between March 2009 and July 2010 and it is running Snow Leopard 10.6.8 on FM! S 11.0v3, 16 GB RAM.

       

      I just recently purchased a Mac Mini wi th Lion on it, 4 GB RAM, and a Promise Pegasus RAID with 4x1TB drives over a Thunderbolt connection and also RAID 5.

       

      Technically, the Apple RAID reads/writes in the up to 250 MB/s range and the Pegasus reads/writes up in the range of 500-750 MB/s. So I was really hoping for a much faster database on this new Mac Mini system.

       

      I have a script from the database using MANY relationships and calculations, none of which are imports.

       

      I used exact copies of the database and ran them on servers using a schedule script, so no client was directly connected.

       

      Results:

      Mac Pro: 26 seconds, Mac Mini 21 seconds.

      Mac Pro: 24 seconds, Mac Mini 22 seconds.

      Mac Pro: 24 seconds, Mac Mini 21 seconds.

      Mac Pro: 25 seconds, Mac Mini 22 seconds.

       

      So the Mac Mini is always faster, sometimes up to 20% faster than the Mac Pro with RAID. This is an OK improvement and we're going to go with it, but I have to admit, I had hoped for a MUCH bigger improvement.

       

      The big plus is that the Mac Mini Server with Pegasus was about $2000, making this a MUCH cheaper solution than the Mac Pro which had about $6000 worth of hardware.

       

      We also tested this on client machines running FMP 11 over ethernet and were getting in the 20% improvement, but nothing more.

       

      The next test I will want to do is compare this against SSDs which are much better at random data calls.

        • 1. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
          fmpvince

          Hi Taylor,

           

          Thanks for sharing this excellent test.

           

          In my experience, for my solution, FMP is much more CPU boudn than storage speed bound. But I use a ramdisk (for storing the fp7 files) for quickest possible data access + volume booted on SSD. But I didn't notice a lot of difference.

           

          Pay attention to the fact that today's CPUs are much more powerfull than previous one, so your test is also affected by the difference of CPU speed. Moreover Filemaker clientcan use only one core, server can use two. So the extra core of the Mac Pro don't count.

           

          What if you run your test on the mac mini just on teh internal HD ?

          • 2. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
            taylorsharpe

            I agree the Mac Mini i7 processor helps, but the Mac Mini also only has 1/4 the RAM of the Mac Pro (4MB vs 16MB) and the Mac Pros Xeon processors are not exactly consumer grade. 

             

            If FileMaker has multiple tasks going (two users running different scripts, etc.), then FileMaker can send that to different cpus for faster processing.  However, any given single task or script cannot share multiple cores.  So FileMaker is not really fully optimized for threading tasks to more than one core at least as of version 11.  One reason FileMaker is often limited by CPU is that the scripts it runs are not natively compiled.  I imagine if they were, they would be a lost faster.  I know FileMaker is working to keep things simple, but there is always the push to have it perform better even if it makes things a little more difficult.  I know they are juggling the balance between those two desires.  But I want it all and now!  <grin>.   Anyway, I do have hopes that FileMaker 12 will be 64 bit and will have more array functions and a few more layout features (and maybe even PostgreSQL ESS now that Apple is not including MySQL in the OS).  But we'll just have to wait and see.  One thing that I have noticed is that during big imports, the cpu levels are low and the disk speeds are slow too... so it makes me wonder what is the bottle neck if its not cpu or disk speed. 

             

            I'll have to give the Mac Mini internal disk a speed test comparison compared to Pegasus and report that here this weekend sometime.  It will be a slightly different test since the Pegasus is now formated as RAID 1+0.  But it will be a good comparison to make and thanks for the suggestion!

            • 3. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
              fmpvince

              One thing that I have noticed is that during big imports, the cpu levels are low and the disk speeds are slow too... so it makes me wonder what is the bottle neck if its not cpu or disk speed.

               

              Were you in client - server mode for this test, if so I guess the bottleneck is "message" passing between the client and the server.

               

              I tested that server + client is 50% less effecient than standalone (on the same machine with setrpe ip 127.0.0.1 so no dns issue).

               

              And of course if there's network involved that's the network latency.

              • 4. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                taylorsharpe

                All of my speed tests were done on an FMPA client that was on the same machine as the FMS to rule out networking issues.  I realize that running it directly on a non-shared database is a little faster, but due to some Active Directory validations, it is a pain to do off-network.  Plus all of my databases run on an FMS and that is what I am interested in the performance. 

                 

                When I made a similar comparison a long time ago, I was only seeing about 15% improvement and am surprised to hear you found a 50% increase.  My guess is that the network is more of an issue in your test. 

                 

                Most of my networks have no more than 2 switch hop and all switches are gigabit because improving network performance with good switches and quality ethernet have been a high priority for me.  But for those who do not know, FileMaker, like many other databases, can be severly hampered by poor or noisy networks with cheap switches/routers.  Oh yeah, absolutely NO hubs (unless you nasty undercover guys are trying to "listen" to everything on a portion of a network). 

                 

                Anyway, it does make it interesting if there was some standard test script to run and compare all these various situations to figure out which ones are most significantly in improving performance. 

                • 5. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                  fmpvince

                  whey i saw 50% improvement using standalone vs client-server, There was no network involved (everything running on same machine). I guess the difference between you and I, lies in the script we used, mine was very heavy on unstored calcs and imports.

                   

                  Yes we would need a standard bench fp7 file.

                  • 6. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                    taylorsharpe

                    Some standard scripts to test various functions would be interesting.  Have you ever wanted a list of all FileMaker functions and their relative speed sayin miliseconds?  I know unstored calcs get complciated due to relationships and probably have to be evaluated under each circumstance.  But it would be nice to know which functions carry a much higher time penalty.  And it would be nice if there were some standard speed tests on various platforms so we could make recommendations to our clients on what is worth spending more money on (e.g., cpu or hard drive or network, etc.). 

                    • 7. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                      jcull

                      Any data you have found with the CURRENT Mac Pro vs the mac mini server i7?

                      • 8. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                        taylorsharpe

                        Yes, I just upgraded an Mac Mini quad core i7 with 16 Gigs of RAM and a Pegasus Thunderbolt RAID to a Mac Pro 6 cpu machine with 32 gigs of RAM.  The performance was noticeable and more than I expected.  I'm not sure if its the networking or the memory working better.  I know the PCIe flash memory if faster, but not a lot faster because the Pegasus RAID was pretty fast.  I wish I had done some side-by-side testing, but it was a client's machine and they just needed it to get going. 

                        • 9. Re: Speed Test:  Mac Mini w/ Thunderbolt vs. Mac Pro with Apple RAID
                          jcull

                          Apple's new Mac Pro kicks HDDs and SSDs to the curb in favor of PCI-e flash | ZDNet

                           

                          PCIe, or peripheral component interconnect express, is a high-speed serial expansion card format that uses a point-to-point architecture. PCIe-based flash has far better performance than more typical SATA- or SAS-connected solid-state drives (SSD) or flash because of the direct connection to peripherals. There's no translation layer required.

                           

                          http://www.zdnet.com/article/apples-new-mac-pro-kicks-hdds-and-ssds-to-the-curb-in-favor-of-pci-e-flash/

                           

                           

                          PCI Express flash storage.

                          This storage is not just fast, it blows away SSDs. PCI Express flash storage is up to 2.5 times faster than the fastest SATA-based solid-state drive, and a whopping 10 times faster than a 7200-rpm a SATA hard drive.

                          To put this into hard numbers, a SATA hard drive offers about 110MBs, while a SATA flash drive can top 550MB/s. Compare this to PCI Express flash which can manage 1,250MB/s.

                          That's an incredible level of data throughput.

                          The PCIe-based flash storage drive is clocking 2.5 times faster then a solid state drive - is this a enough of a reason to consider the Mac Pro?