12 Replies Latest reply on Mar 21, 2016 10:38 AM by DaveNicely

    FileMaker Server 14 memory usage

    DaveNicely

      We're trying to track down the source of the WPE quitting in our CWP / PHP deployment.


      I took a screen shot of memory usage after the WPE had quit.


      Is it unusual that 1) there are two java processes for fmserver and 2) one of the java / fmserver processes is using the most memory?

       

      Single machine deployment

      Configuration:

      OSX 10.11.2 (clean install)

      Mac Pro (mid 2012)

      3.2 GHz Quad-Core Intel Xeon

      12 GB memory

      FileMaker Server 14.0.4

      Java 8 update 66

      SuperContainer 2.93

       

      PHP Publishing enabled

      XML Publishing enabled

      FM WebDirect disabled

        • 1. Re: FileMaker Server 14 memory usage
          BowdenData

          Check out this article/posting about tuning FM server. In particular, I have had a lot of success in tuning the FM server cache settings (in the Admin console) when running on a Mac server. Instead of just giving the FM server cache the maximum allowed amount, I reduced this figure to better match what it needed. Crashes just about went away all together. I also recommend a restart from time to time (every "x" amount of time) of the entire server machine that is hosting the FM server.

           

          Doug

           

          http://www.deskspace.com/understanding_filemaker_server_performance_introduction.html

          • 2. Re: FileMaker Server 14 memory usage
            wimdecorte

            Completely agreed with Doug.  I see a lot of deployments where people just max out the FMS cache setting and it does more harm than good.  Set it just higher than required based on the load, but nothing more.

            • 3. Re: FileMaker Server 14 memory usage
              DaveNicely

              Thanks Doug. I've got a strong cup of coffee in hand and my reading glasses on.

              • 4. Re: FileMaker Server 14 memory usage
                DaveNicely

                Done with reading and doubt I understand it all just yet.

                 

                I'm looking at the FileMaker Server 14 Server Statistics and seeing my Cache Hit % at an Average of 100, Low 98 and Peak 100.

                 

                The Database Cache maximum allow size is 11059 MB, and since I have Web Publishing enabled the database cache should be set at no more than 50% the maximum. The RAM Reserved for the Database Cache (MB) is currently set to 4096.

                 

                So if I'm understanding the performance tuning, I can bring down the cache size as long as I'm still get an average of 100% correct?

                • 5. Re: FileMaker Server 14 memory usage
                  Vaughan

                  wimdecorte wrote:

                   

                  Completely agreed with Doug.  I see a lot of deployments where people just max out the FMS cache setting and it does more harm than good.  Set it just higher than required based on the load, but nothing more.

                   

                  Maxing out the cache has been best-practice recommendation from FMI itself.

                  A common misconception regarding caching is that one can tune the cache for best performance by reducing the cache size to some optimal number. In reality, setting the cache to the maximum makes that amount available to FileMaker Server when it needs it. In low usage situations the cache is unused. There is no performance loss associated with a large cache setting, especially when compared to the performance loss associated with too small a setting.

                   

                  • 6. Re: FileMaker Server 14 memory usage
                    BowdenData

                    Not all situations will be the same, but in my case, reducing the cache was the correct way to go. Clients tend to complain a bit more when the server is not working at all as compared to the chance of a small performance decrease.

                     

                    The environment I had in this instance was as follows:

                     

                    2 Mac Mini's. 16GB RAM. Both older models with the I7 Quad core CPU. Plenty of disk space. One unit had a 256GB SSD and the other one had a 1TB 7200rpm spindle drive. Both machines hosted a handful of databases, all of which were lightly to moderately used. All clients except one were either FM Pro or FMGo. One used WebDirect very lightly and sporadically.


                    FM Server recommended a dB cache of 8GB (1/2 the total RAM) on each machine which is what we set them to originally. After running for 7-10 days, we observed (through activity monitor) that both machines had exhausted all free RAM and were heavily paging to disk.  FM performance slowed down some, but not dramatically. The FM server cache hit remained in 95-100% range.


                    What did slow down dramatically were things like FM server backups, OS level tasks that copied the FM server backups to network volumes, connecting remotely via Apple Remote Desktop, etc. Basically, everything outside of FM Server itself ground to a halt and then the FM Server process would hang or even crash. Note that the mini with the SSD fared better than the one with the rotating drive which makes total sense. Paging to a SSD is going to be much better than to a rotating drive.


                    After reading Nick's article, we unscientifically began reducing the FM server cache from 8GB downwards in 1GB increments. We settled on 5GB cache from what I can remember. The number of memory page outs to disk dropped to almost nothing even after 2-4 weeks of uptime and most importantly, the hanging/crashing stopped as well. The FM server stat's were still okay with the smaller setting.  All OS level tasks and remoting into the server were acceptable performance-wise.


                    HTH.


                    Doug

                    • 7. Re: FileMaker Server 14 memory usage
                      wimdecorte

                      Vaughan wrote:

                       

                      Maxing out the cache has been best-practice recommendation from FMI itself.

                      A common misconception regarding caching is that one can tune the cache for best performance by reducing the cache size to some optimal number. In reality, setting the cache to the maximum makes that amount available to FileMaker Server when it needs it. In low usage situations the cache is unused. There is no performance loss associated with a large cache setting, especially when compared to the performance loss associated with too small a setting.

                       

                       

                      That has been their recommendation.  But we're not talking so much about performance here but stability.  I've seen enough instances now where NOT following this recommendation but just keeping the cache to where it stays effective reduces instability.

                      • 8. Re: FileMaker Server 14 memory usage
                        DaveNicely

                        Yesterday I monitored the server via the Activity Monitor Memory tab and noticed one of the java/fmserver processes crept from 940+MB in the morning to over 4.3 GB by 10pm. At some point in the evening before I checked at 10pm, the WPE had gone down. I adjusted the database cache down from 4096 to 3072 (of a possible 11059) as we were getting 100% on the cache hits, restarted the server and verified the database and WPE were accessible.

                         

                        At 7am this morning all looked good. The memory on the java/fmserver process was at 943MB and the WPE was up and running.

                         

                        At 9:30am the client messaged that things were running slow. I remotely checked the server and memory allocations (via Activity Monitor) looked okay with the java/fmserver process at 940+MB but found that the WPE was down (via FM Admin Console). Since an auction was beginning in 30 minutes, we opted to increase the database cache back to 4096 and to restart the server. So far things are stable as they conduct the live auction.

                         

                        One thing I don't understand, and don't know if I should be concerned about, is the apparent memory creeping on the java/fmserver process. Is the normal behavior that the memory allocation will increase and decrease over time as web clients come and go? As far as I can tell, the memory allocation doesn't appear to decrease only increase over time.

                        • 9. Re: FileMaker Server 14 memory usage
                          wimdecorte

                          No need to restart the whole server is WPE is down, you can just use the command line:

                           

                          fmsadmin START WPE

                           

                           

                          We have had deployments where we use a scheduled OS task to run this command every 5 minutes just to make sure it was restarted whenever it went down.

                          • 10. Re: FileMaker Server 14 memory usage
                            DaveNicely

                            I've used the command line in the past, but when I use it in this case the WPE is not coming back online in a reasonable amount of time (5 minutes) and there's typically 15 hung fmphp clients.

                             

                            We have a timeout in our php command that closes the session after three minutes of inactivity so it's odd that there so many hung clients. Something is not happy, just haven't figured it out yet.

                            • 11. Re: FileMaker Server 14 memory usage
                              DaveNicely

                              Also, thanks for the tip on the scheduled OS task. Researching how to set that up now.

                              • 12. Re: FileMaker Server 14 memory usage
                                DaveNicely

                                Still having issues with this beast.

                                 

                                Did a "Monday morning clean restart" to get the week off to a good start around 6:30 am.

                                 

                                At 9:10 am, the client messaged that the web site was down.

                                 

                                Checked the FM Admin Console and WPE was off and there were 15 CWP connections.

                                 

                                Started the WPE via the FM Admin Console. Logged out and quit the FM Admin Console.

                                 

                                Then attempted to stop the WPE via the Command Line and go the following results:

                                fmsadmin stop wpe

                                really stop wpe? (y, n) y

                                Error: 10007 (Requested object does not exist)

                                fmsadmin start wpe

                                Error: 10006 (Service already running)

                                 

                                Launched FM Admin Console and WPE was off and still had 15 CWP connections.

                                Decided to close the databases and restart the server

                                 

                                Restart of server cleared WPE

                                 

                                Database cache max is 11059, RAM setting at time of today's crash was 4608.

                                 

                                Found notes on possible issue with 'launched':

                                I've had some situtations where launchd (an OS component that is responsible for launching FMS) gets confused, and needs the FMS plist to be reloaded. This is fixed the issue for me:

                                -- sudo launchctl unload /Library/LaunchDaemons/com.filemaker.fms.plist

                                -- sudo launchctl load /Library/LaunchDaemons/com.filemaker.fms.plist

                                 

                                Besides /Library/FileMaker Server/Logs/Event.log are there other logs that I should be monitoring to track down the offending event?

                                 

                                 

                                Single machine deployment

                                Configuration:

                                OSX 10.11.2 (clean install)

                                Mac Pro (mid 2012)

                                3.2 GHz Quad-Core Intel Xeon

                                12 GB memory

                                FileMaker Server 14.0.4

                                Java 8 update 66

                                SuperContainer 2.93

                                 

                                PHP Publishing enabled

                                XML Publishing enabled

                                FM WebDirect disabled