We’re thrilled that FileMaker v16 has been announced today. This version has transformed the user experience on our RemoteApp cloud servers when accessing via Microsoft Remote Desktop for Macs due to the replacement of MDI with SDI. Also great to be receiving confirmation of previous bugs having been fixed.
We’ve been waiting for the official release to publish some tests carried out to compare results of virtual servers of different specifications on the performance of our systems. The majority of tests were carried out using FileMaker Pro 16 and FileMaker Server 16, although we did do some comparative tests using FMP v15 connected to FMS v16 and FMP/FMS v15.
The servers tested were AWS EC2 Windows 2012 R2 instances, the smaller server set to 4Gb RAM, 2vCPU and 80Gb SSD storage (t2.medium) and the larger set to 256Gb RAM, 64vCPU and 80Gb SSD storage (m4.16large).
Our conclusions were:
The configuration of a virtual server has virtually no impact on speed of FileMaker completing a single routine and in some cases the 4Gb/2vCPU server completed the test marginally quicker than the 256Gb/64vCPU. The same conclusion was reached when we carried out a few tests excluding ExecuteSQL.
The sizing of a server should be based around the number of people accessing it rather than the complexity of the solution.
There was no identifiable speed improvement between FileMaker v15 and FileMaker v16. Combinations between the different FileMaker Servers and FileMaker Pro produced almost identical results, with either versions of FileMaker Pro occasionally outperforming each other.
Running ExecuteSQL calculations for the first time after launch and then again during the same session produced results where the subsequent tests provided a variation in speeds of 230% to 3600% faster than the time taken for the first run (for example a test initially taking 00:01:48 would then take 00:00:03). We haven’t had a chance to compare the data sets since getting these results but assume this is down to the structure of the SQL used in each test. However, this did prove that ExecuteSQL will always run significantly slower on first run, regardless of server specification.
For anyone using XenApp/RemoteApp or perhaps Remote Desktop, a disconnected user session will retain the faster ExecuteSQL speeds until the user session is logged out (or perhaps when the FMS server disconnects the user?). We’ll try to understand this better when we get some time to follow this up, we’re not sure whether this is FileMaker Pro or FileMaker Server dependent, possibly FMP caching?).
All FMP tests were from our streaming servers, therefore all processing took place within the cloud (rather than from a typical Internet connection). There would have been a small amount of latency between our infrastructure supplier (Hyve - UK) and AWS (Ireland). We know that similar tests within our own cloud infrastructure would have been faster (due to previous tests where we were comparing AWS, Rackspace and our supplier’s internal infrastructure). Running FileMaker locally will often negate any internal infrastructure speed advantage, as the slowest part of the connection is usually from the user to the Internet.
The tests were not scientifically carried out, we had preset scripts to run routines on one of our insurance systems with a start and end time set to report the result. The majority of tests did include ExecuteSQL, so our results will not necessarily reflect all aspects of FileMaker’s performance. 5 separate tests were run twice: immediately after FileMaker Pro was launched and then again within the same session (our main focus was with ExecuteSQL). A total of 61 separate runs of these tests were carried out, with repeats to validate the results.
The above has allowed us to be more confident when sizing servers for our clients and I hope it is of interest.