14 Replies Latest reply on May 4, 2016 7:35 AM by Smef

    Memory usage for XML publishing

    Smef

      Hello,

       

      It seems that CWP and the XML gateway are having issues releasing memory after a query has been made. I'm POSTing 3-5 MB of text data, and with each post I can see the memory usage jump up in the activity monitor on Windows Server 2012 R2. There's a very clear step-up of memory usage with each post, and the memory usage never decreases until the WPE has been turned off. Memory usage goes back to normal when it starts up again.

       

      Any suggestions on improving this? The RAM gets used up pretty quickly, and having to manually stop and start it over and over again isn't feasible.

       

      FMS 14.0.4a Windows Server 2012 R2

        • 1. Re: Memory usage for XML publishing
          beverly

          David, I do not know your entire web site and methods used.

           

          Keep in mind that web servers are optimized for serving FILES. If there are parts of your data that do not change frequently and can be off loaded as includes (ftp once, use many times by reference), then you are not pushing from the database with so much data. This type of offloading may be things like common value-lists.

           

          beverly

          • 2. Re: Memory usage for XML publishing
            Smef

            This issue appears when using a POST to insert data into FileMaker records, not when pulling data from the server. Also, this is uploading data through the XML gateway, not a PHP solution or with the PHP api, so there are no includes here or files to reference. It's just a URL such as http://...../fmi/xml/fmresultset.xml?paramsHere with a POST body which has all the &-field=data values.

            • 3. Re: Memory usage for XML publishing
              beverly

              ok, David, thank you for more information. let's see if the TS folks have any insight.

              beverly

              • 4. Re: Memory usage for XML publishing
                TSGal

                Smef:

                 

                Thank you for your posts.

                 

                This definitely sounds like an issue.  I would like to get sample files for testing purposes.  Specifically, a database file and the file used that makes the POST call.  This will help me narrow down the issue.  I have sent you a private message with instructions where to send the file.

                 

                TSGal

                FileMaker, Inc.

                • 5. Re: Memory usage for XML publishing
                  TSGal

                  Smef:

                   

                  I received your files.  Thank you.

                   

                  Everything has been sent to Development and Testing for further review.  I will let you know when I receive any feedback.

                   

                  TSGal

                  FileMaker, Inc.

                  • 6. Re: Memory usage for XML publishing
                    TSGal

                    Smef:

                     

                    Testing has been unable to replicate the issue.  Do you know what process is not releasing memory?  Please give us the exact steps to reproduce the issue.  Something is missing from your original steps.

                     

                    Since you are on Windows, run perfmon and send in the logs.  This may help narrow down the cause.

                     

                    TSGal

                    FileMaker, Inc.

                    • 7. Re: Memory usage for XML publishing
                      Smef

                      I think the steps were correct. I've tried posting that large data I sent to you a few different ways, and each way has the same issue. The posting application gets a response from the web server and the connection completes successfully (from the client side, at least).

                       

                      I think it's the Java process of the WPE which is not releasing the memory.

                       

                      What logs from perfmon are you interested in? Should I make something through a data collector?

                       

                      We can do screen sharing if you would like so that I can show you this happening and you can see the server and client directly.

                       

                      Thanks for your help.

                      • 8. Re: Memory usage for XML publishing
                        TSGal

                        Smef:

                         

                        Send all perfmon logs.  This will show everything that is happening on the server.

                         

                        TSGal

                        FileMaker, Inc.

                        • 9. Re: Memory usage for XML publishing
                          Smef

                          Perfmon doesn't log anything unless you specifically set up data collectors and run them. Is there something in particular you want to see?

                          • 10. Re: Memory usage for XML publishing
                            TSGal

                            Smef:

                             

                            File has been received.  Thank you.

                             

                            Development and Testing informed me the memory is controlled by Java.  The starting and maximum size can be configured in catalina.bat file.  Since you are using Windows, the catalina.bat file is located in:

                            Web Publishing/publishing-engine/jwpc-tomcat/bin

                             

                            Look for the line

                            set JAVA_OPTS=....

                             

                            Try setting xms and xmx values to 1024M.

                             

                            For more information about configuring JVMheap settings, see:

                            http://crunchify.com/how-to-change-jvm-heap-setting-xms-xmx-of-tomcat/

                             

                            Continue to keep me updated with any progress.

                             

                            TSGal

                            FileMaker, Inc.

                            • 11. Re: Memory usage for XML publishing
                              Smef

                              I lowered the memory from 2gb to 1gb as you recommended, but that doesn't seem to have made a difference. To be clear, the issue isn't that the Java process is running out of memory or using too much memory while it's doing things, but that it isn't releasing the memory after it has processed a POST. There's a step-up in memory usage after each POST which never goes back down again.

                              • 12. Re: Memory usage for XML publishing
                                Smef

                                Ive now noticed this on a totally separate database and server. The WPE is running into the same issue where it isn't releasing memory.

                                • 13. Re: Memory usage for XML publishing
                                  TSGal

                                  Smef:

                                   

                                  Development and Testing says that when memory is allocated to JVM for say a data buffer, it does not get released back to the OS once the application is done with it, but released within the JVM for re-use.  When JVM exits, the memory will be released back to the OS.  The way to control the maximum memory size for JVM is by editing the lines in the catalina.bat file in my previous reply.

                                   

                                  TSGal

                                  FileMaker, Inc.

                                  • 14. Re: Memory usage for XML publishing
                                    Smef

                                    Thanks for this information. This makes sense.