1 2 Previous Next 16 Replies Latest reply on Dec 22, 2016 11:25 AM by NickLightbody

    PSOS Performance comparing FMS14 and FMS15

    jerrysalem

      I have a system being hosted using FMS14.  I also have a duplicate database hosted for testing/exporting using FMS15. PSOS is much slower in my hands with FMS15.

      Both servers are physical boxes, with similar (maybe identical) specs (Cores, RAM HD Space, HD Space available)

      I also have a Server Side script that creates a found set, exports data to excel and emails it to a user.  (This is based on https://www.skeletonkey.com/restoring-filemaker-clients-found-set-within-server-side-script/).  The exports have a couple of related fields, but no un stored calculations.

       

      Using FMS14 this has been working just fine.  Using FMS15 it has been significantly slower.

       

      As a test, I exported 13,000 records from both FMS14 and FMS15.

      Using FMS14, it takes about 10 Seconds to recreate the found set, then 70 Seconds to create the export.

      Using FMS15, it takes about 20 Seconds to recreate the (same) found set, then 175 Seconds to export the data.

       

      In another series of tests, this time exporting 30,000 records (more realistic in my scenario) I found;

      Using FMS14, it takes about 25 Seconds to recreate the found set, then 168 Seconds to create the export.  (2.5x longer to find/2x longer to export 3x records)

      Using FMS15, it takes about 90 Seconds to recreate the (same) found set, then 825 Seconds to export the data. 9x longer to find/11x longer to export 3x records)

       

      In the last series of tests, this time exporting 50,000 records I found;

      Using FMS14, it takes about 59 Seconds to recreate the found set, then 262 Seconds to create the export.  (6x longer to find/4x longer to export 5x records)

      Using FMS15, it takes about 160 Seconds to recreate the (same) found set, then 1700 Seconds to export the data.(16x to find/24x to export 5x records)

       

      Any ideas?  Anyone see similar results with PSOS on FMS15?

      Again, the machines are identical, the databases are identical and the scripts are identical.  the only difference is FMS14 vs. FMS15.  I am also letting another user pull data using ODBC.  That has also extremely slow using FMS15, but that is a discussion for another thread.  I am not using WebD on this server.

      In general the FMS15 database performs find using FMP Clients.  But the FMS15 functions are not very impressive.  I am petrified to move my production database to FMS15, and even considering moving the test server back to FMS14.  Help!

       

      TIA

      Jerry

        • 1. Re: PSOS Performance comparing FMS14 and FMS15
          mcrip

          I don't know if this applies at all, but you might try playing with the cache settings on the FMS 15.  See Nick Lightbody's report on performance comparisons between server 14 and 15

          Re: Understanding and how to tune FileMaker Server

           

          You could also try his DsBenchmark software to fine tune your settings for the 15 server to find the sweet spot on performance. 

          1 of 1 people found this helpful
          • 2. Re: PSOS Performance comparing FMS14 and FMS15
            jerrysalem

            Both machines have lots of RAM.  I dialed the ram down on the test machine from 170G down to 44G it didn't seem to make a difference.

             

            Jerry

            • 3. Re: PSOS Performance comparing FMS14 and FMS15
              mcrip

              Did you change the amount of memory available to the (?virtual?) machine?  Or change the amount of memory cache available in FileMaker Server?  My understanding, which is probably a bit limited, from comments NickLightbody has made in both the community and some of the videos he's done, is that  FMS15 actually performs better with less memory dedicated to cache.  Perhaps he'll jump on and clarify that.

              1 of 1 people found this helpful
              • 4. Re: PSOS Performance comparing FMS14 and FMS15
                jerrysalem

                its a physical machine.

                I changed the cache down, it didn't seem to effect anything.

                • 5. Re: PSOS Performance comparing FMS14 and FMS15
                  NickLightbody

                  Jerry,

                   

                  Here is my current advice on deploying FileMaker Server. You are welcome to come back on specifics once you have thought this through.

                   

                  It does look like you may need to look hard at point 5 below.

                   

                  To properly compare FMS14 & 15 you need to swop the deployments between the boxes otherwise you may find you have a false comparison.

                   

                  GENERAL Advice on deployment factors

                   

                  Do use FMS15 which offers significant performance improvements over FMS14.

                   

                  1. IO: The biggest factor as you know is to deploy with the fastest possible io - so bus speed, RAM speed and disk read-write speed is the first thing to get right - hence SSD or better storage is optimal. I generally always use mirrored RAID myself on the precautionary principle, but provided you set up FMS incremental backups efficiently the priority for mirrored RAID requirement clearly reduces, so there may now be a case for faster striped RAID? The key thing to bear in mind is that FMS makes many small read and writes so optimisation for that behaviour is required.

                   

                  2. STORAGE: the amount of disk storage required is clearly based on the size of the db and planned backups and then modified by the amount of disk space required by the OS for use as temporary memory. The outer sectors of rotating media are much faster than then inner sectors, one reason why disks which are more than half empty are always faster, but much better to use SSD or better so rotational factors will not be relevant.

                   

                  3. POWER: Clearly the faster the box / v.boxes the better.

                   

                  4. CPUs: I have seen conflicting views on the efficiency with which FMS manages multiple CPUs. I believe that FMS can treat one vCPU as 2 "virtual" vCPUs, but I am not current sure of the constraints on this behaviour. It is certainly the case that each recent version of FMS has become progressively more efficient but I can currently give no definitive advice on the extent to which many vCPUs would be of benefit. My advice is to start with say 4 and then if possible track the load and performance and experiment with more and observe the difference. I assume here that the virtualisation being used means that the deployment can be conveniently adjusted in this manner?

                   

                  5. RAM: Giving the box as much RAM as possible is always a very good idea, but, it is also very important not to give FMS any more cache than is strictly required. It is a very common fault that folk confuse FMS cache with RAM available to FMS. Increasing the FMS cache setting reduces the amount of RAM available for the OS and every application running on the box including FMS. Hence as noted above watch the stats and optimise the FMS cache setting so that the cache hit rate is 99% and then increase it a little so that it is just 100%.

                   

                  SPECIFICALLY: the following factors may result in the preceeding general advice being varied:

                   

                  SIZE of dbase: a very large db will require more FMS cache, the old rule about watching FMS stats and adjusting the FMS cache to run with a cache hit rate at 99% still holds true, since it remains essential to use no more FMS cache than is actually required.

                   

                  NUMBER of USERS: the more users the more separate threads and the more CPUs required to run those threads in parallel.

                   

                  MODE of connection: If WD is to be used, and in my experience it is far far better than some folk suggest, then you do require sufficient RAM for all those WD client sessions on FMS. With FMS14 each session required about 250mb RAM (not FMS cache!) - with FMS15 that number is I think smaller but I have not yet remeasured it myself.

                   

                  The EFFICIENCY of the SOLUTION: this probably makes the biggest difference of all. A well designed fast solution will run so very much faster in all respects. If the solution is legacy, ie it was designed for any version of FileMaker pre 14, or prehistoric as in pre 12, then taking steps to improve how it handles the Appearance Layer, ie using styles and themes effectively will make a significant difference. Older solutions often have wide tables and taking steps to reduce table width will pay dividends as will avoiding permitting the current user layout to be based on a table with many records wherever possible. As you know there are many ways in which a FileMaker solution designed a while ago can run very slowly, purely because of a design which is far sub-optimal in a modern context.

                   

                  I hope that you find this helpful. If so then do let me have a note saying whatever you are happy to say about the benefit you have derived from my advice.

                   

                  I shall be very interested to hear your own observations on what you have found works for you?

                   

                  Cheers, Nick

                  4 of 4 people found this helpful
                  • 6. Re: PSOS Performance comparing FMS14 and FMS15
                    NickLightbody

                    Ok, sure, please see my comments below.

                    Cheers, Nick

                    • 7. Re: PSOS Performance comparing FMS14 and FMS15
                      user19752

                      Your "create found set" is very slow on FM14S also, it seems finding on unstored calculation or unindexed field.

                      • 8. Re: PSOS Performance comparing FMS14 and FMS15
                        jerrysalem

                        Nick,

                         

                        Thanks for chiming in.

                        I investigated all these things before posting here.

                        For these tasks, our instance of FMS15 is slower.

                        I have shown this to be independent of the solution, there must be something in the deployment or Hardware.

                         

                         

                        Jerry

                        • 9. Re: PSOS Performance comparing FMS14 and FMS15
                          jerrysalem

                          user19752

                           

                          the find is over a stored number field.  If you haven't read threw the Demo file, it takes a SnapShot link and deconstructs it.  The server does multiple finds based on the RecordID of the table.  Depending on the number of records and their order, when it recreates the found set it can be a large number of find requests.

                           

                          In my case, my table has about 3.5 million records.  The RecordID field is a stored number. 

                           

                          My main concern is that the finds are so slow on FMS15, it is not usable anymore by the end users.

                           

                          Jerry

                          • 10. Re: PSOS Performance comparing FMS14 and FMS15
                            abdenterprises

                            Jerry,

                             

                            I have nearly identical FM Servers (HW Specs), one 14 and one 15.

                             

                            Would it help you if ran your find on my servers to try and reproduce this issue?

                             

                            We might also be able to compare my FMS 15 results with yours.

                             

                            Alan

                            1 of 1 people found this helpful
                            • 11. Re: PSOS Performance comparing FMS14 and FMS15
                              jerrysalem

                              That would be very neighborly!

                               

                              Download the demo file from skeleton key web site in the OP.

                              Put it on each server

                              Make a found set of 13k records (just to be consistent)

                              on our FMS 14 server It takes 0.015 seconds to create the found set

                              on our FMS15 server it takes 0.215 seconds to create the found set.

                               

                              (I wish I had done this on our FMS15 server while it still had FMS14 on i)

                               

                              TIA

                               

                              Jerry

                              • 12. Re: PSOS Performance comparing FMS14 and FMS15
                                carlsson

                                Guys, did you ever perform this test?

                                • 13. Re: PSOS Performance comparing FMS14 and FMS15
                                  jerrysalem

                                  carlsson

                                   

                                  Alas no.  I haven't gotten two IDENTICAL servers one running 14 and one running FMS15 to do a real head to head test.

                                   

                                  I tried downgrading one of my FMS15 servers to 14 but hit some roadblocks reinstalling FM14 that I couldn't over come. 

                                   

                                  Personally I am in a little bit of quandary over this. 

                                  In my experience, I found a degradation in performance of both Performing multiple finds, and ODBC connectivity (that wasn't overcome when applying the 15.2 update) on my secondary machine.

                                   

                                  I haven't been able to duplicate it on other machines, and anecdotally most people have reported better or similar performance.

                                   

                                  So I am very hesitant to update my production FMS14 machine.

                                   

                                  Jerry

                                  • 14. Re: PSOS Performance comparing FMS14 and FMS15
                                    jerrysalem

                                    Last night I just updated our production server from FMS14 to FMS 15.  So I can compare the same hardware/OS.

                                    To remind you, I am comparing finding all odd records in the PSoS Found Set Demo file using FMS14 and FMS15.

                                     

                                    To find all odd records using the server side script under FMS14 took about 388 Seconds.

                                    Doing the exact same server side script under FMS 15 took about 413 seconds.

                                     

                                    Both sets of numbers were gathered on the same hardware, with the same OS.

                                    The server setups are the same (same cache settings).

                                    Both servers were under similar loads at the time they were run.

                                     

                                    Doing multiple finds in the server side scripts was about 9% slower in FMS15 vs FMS14

                                    1 of 1 people found this helpful
                                    1 2 Previous Next