7 Replies Latest reply on Sep 4, 2015 10:42 AM by wimdecorte

    Poor Performance - FM12 on Hyper-V

    ShaunS

      We have a reliable and quick FMS14 serving 108 databases to just under 100 users with no problems.

       

      Spec - HP ML350p G8 - 2 x E5-2620 2Ghz Xeon - Windows Server 2008 R2 64bit - 32Gb RAM - 2 x 146Gb 15k 6G SAS in Raid 1 for the OS and 4 x 300Gb 15k SAS in Raid 10 for the databases.

       

      A month ago we moved to a brand new server running Hyper-V.

       

      Spec - HP ML350p G9 - 2 x E5-2650 2.3Ghz Xeon - Windows Server 2008 R2 on Hyper-V - 32Gb RAM - 8 x 300Gb 15k 12G SAS in Raid 10

       

      But performance was very poor so we had to revert to the original server.

       

      After researching the problem the only thing we could put it down to was the disks being formatted as VHD. So we changed the disks to VHDX and initial tests were fine with 3 or 4 connections.  So last night we rolled out the new server again, and again we have performance issues.

       

      Any help would be greatly appreciated.

        • 1. Re: Poor Performance - FM12 on Hyper-V
          wimdecorte

          Define "performance problems".

          The FMS stats log (hopefully you have it turned on) gives you numbers on all 4 traditional bottlenecks: processing, disk i/o, memory, network throughput.   One, some or all of them are causing this but you need the numbers to know which one(s).

           

          Compare your stats logs from the old server to the new one and it should be clear which of those numbers are different under load.

          Don't get fooled by the CPU numbers that you can collect from Windows, those are not relevant.  Only the FMS gathered stats are relevant for now.

           

          Now, you can't really compare the specs of the two machines; the old machine was a dedicated physical box running only FMS, right?

          The new box is the virtual host, that's not what FMS runs on.  FMS runs in a virtual instance so we'd need the specs of what has been assigned to that instance and compare that to the old box.  For all we know they've only given you 1 virtual core (of the 20 or so that the physical host has available), or given you 1GB of RAM,...

          • 2. Re: Poor Performance - FM12 on Hyper-V
            Markus Schneider

            we got one quite 'big' solution under v12 running - although under VMWare - fast! We don't have V12 solutions u der HyperV - but there are some under V11 and they perform fine

             

            What about the network? What about tasks when the data is in the RAM of the client? (RAID 1 versus RAID 10...)

            • 3. Re: Poor Performance - FM12 on Hyper-V
              ShaunS

              Lots of delay in switching between files, whirly wheels or egg timers, summarising fields dialogues, all of which weren't there yesterday.

               

              We have been watching the stats but we don't log usually as things have been running fine.  It appears the wait time and elapsed times are much higher than we would usually expect.

               

              The comparison between the machines is pretty fair.  Both machines are / were dedicated to running FMS.  So all resources are available to the virtual machine.  Both have all 32Gb allocated to them, the original server had 24 cores available whereas the new server has 20 cores but they are faster processors and the drop in performance we are seeing would not be reasonable even if they were identical processors.

               

              We took the decision to go Hyper-V as we know we need to upgrade to Server 2012 to support FMS14 soon.  But we have an IWP setup that needs rewriting to work using Webdirect.  With this in mind the plan was to make the changes for Webdirect then create a second Hyper-V install on the same server running 2012 and FMS14.  That would make it a simple case of closing the first server and opening the new one as the databases would reside on the same machine.  If we have to go to a physical install of 2008 on the new server then when we come to update to FMS14 we will have to take the server down, wipe it and install both 2012 and FMS14.  That is a lot of downtime and gives you no escape plan should something go wrong. 

              • 4. Re: Poor Performance - FM12 on Hyper-V
                ShaunS

                Network seems fine.  Very fast ping replies (although not a great measure of network speed).  Plus we have moved 300Gb of data between the servers when moving the databases and speed was very good.

                 

                Both machines have 4 teamed gigabit nics and are showing minor throughput in both task manager and on the switch which is very much the same as before.

                 

                RAID versions are the same for both machines.

                • 5. Re: Poor Performance - FM12 on Hyper-V
                  wimdecorte

                  ShaunS wrote:

                   

                   

                  We have been watching the stats but we don't log usually as things have been running fine.

                   

                  Until they don't and that's when you need that baseline to compare the normal to the abnormal.  So turn it on and leave it on, it will help tremendously now and in the future.

                   

                  ShaunS wrote:

                   

                  It appears the wait time and elapsed times are much higher than we would usually expect.

                   

                  Without seeing the full stats over a longer period of time - this seems to be an indication of of processing power bottleneck.

                  But we'd need to know the "i/o time per call" too.

                   

                  ShaunS wrote:

                   

                   

                  That would make it a simple case of closing the first server and opening the new one as the databases would reside on the same machine.  If we have to go to a physical install of 2008 on the new server then when we come to update to FMS14 we will have to take the server down, wipe it and install both 2012 and FMS14.  That is a lot of downtime and gives you no escape plan should something go wrong.

                   

                  That is a bit of a weird reasoning...  but I'll follow.  But realize you are introducing a level of abstraction and a lot of moving parts for very little benefit, only one short-lived benefit in fact.

                   

                  ShaunS wrote:

                  the original server had 24 cores available whereas the new server has 20 cores but they are faster processors and the drop in performance we are seeing would not be reasonable even if they were identical processors.

                   

                  I'd have to look it up but I thought the spec for your old processors was 6 cores per processor, so 12 real cores, 24 with Hyper Threading.  The new one would have 10 real cores per processors so 20 real ones and 40 HT ones.  Can you confirm that?

                   

                  The long and short is: FM plays extremely well in a virtual world.  So it looks like something is misconfigured but without a good baseline of performance data to compare to we're just shooting in the dark.

                  • 6. Re: Poor Performance - FM12 on Hyper-V
                    ShaunS

                    ShaunS wrote:

                     

                     

                    We have been watching the stats but we don't log usually as things have been running fine.

                     

                    Until they don't and that's when you need that baseline to compare the normal to the abnormal.  So turn it on and leave it on, it will help tremendously now and in the future.

                     

                    Agreed.

                     

                    ShaunS wrote:

                     

                    It appears the wait time and elapsed times are much higher than we would usually expect.

                     

                    Without seeing the full stats over a longer period of time - this seems to be an indication of of processing power bottleneck.

                    But we'd need to know the "i/o time per call" too.

                     

                    Capture.JPG ! !

                     

                    ShaunS wrote:

                     

                     

                    That would make it a simple case of closing the first server and opening the new one as the databases would reside on the same machine.  If we have to go to a physical install of 2008 on the new server then when we come to update to FMS14 we will have to take the server down, wipe it and install both 2012 and FMS14.  That is a lot of downtime and gives you no escape plan should something go wrong.

                     

                    That is a bit of a weird reasoning...  but I'll follow.  But realize you are introducing a level of abstraction and a lot of moving parts for very little benefit, only one short-lived benefit in fact.

                     

                    Not really.  The options when it comes to upgrading a physical installation are either a:) Having new hardware to setup prior to the switch over (in this case £6000+ of server) or b:) you perform the installation on the existing box.  In this case FM14 wont run on Windows Server 2008 so it would be both an OS and FMS install.  The minimum downtime for a full OS install and FMS install has to be around a day.  Plus you run the risk of something not running correctly (like last nights switch) in which case you have no server to go back to.  That could mean another days downtime while you reinstall 2008 etc.

                     

                    ShaunS wrote:

                    the original server had 24 cores available whereas the new server has 20 cores but they are faster processors and the drop in performance we are seeing would not be reasonable even if they were identical processors.

                     

                    I'd have to look it up but I thought the spec for your old processors was 6 cores per processor, so 12 real cores, 24 with Hyper Threading.  The new one would have 10 real cores per processors so 20 real ones and 40 HT ones.  Can you confirm that?

                     

                    The original processors have 6 cores per processor.  The new sever has 8 cores per processor so 32HT cores.  We have dedicated 20 to this Hyper-V installation.  The remaining are dedicated to a 2012 install carrying FMS13 that we were using to test Webdirect.  But this install is powered down completely so is not impacting on the performance.  We could give the FMS12 install everything but it should not be necessary and would also negate the point of going Hyper-V as we would not be able to have a second server running 2012 to pre-configure.

                     

                    FMS is stopped on the new server as we have moved back to the old one but.....

                     

                    Capture 2.JPG

                    • 7. Re: Poor Performance - FM12 on Hyper-V
                      wimdecorte

                      ShaunS wrote:

                       

                      In this case FM14 wont run on Windows Server 2008 so it would be both an OS and FMS install. 

                       

                       

                      FMS runs on Windows 2008 R2 like you have:

                      System Requirements for FileMaker Server 14 | FileMaker