14 Replies Latest reply on Feb 21, 2011 2:12 PM by TSGal

    Administrator "fmsadmin" no longer responding; connection closed

    user13578

      Summary

      Administrator "fmsadmin" no longer responding; connection closed

      Description of the issue

        FileMaker Product(s) involved:FileMaker Server 10.1 Operating System(s) involved:PC + II7 + Java 6..7 Detailed description of the issue:In the past two weeks, the server continues to experience an error in communication approximately one hour after restart concerning the connection of the fmsadmin. This prevents the use of the port 16000 for reaching the server administration screen on the local host. Many tickets are found in this filemaker forum database which are related but all issues seem to have differing causes, implying different issues such as a backup routine is running concurrently or a java version upgrade to 10/11 when only support 7 or a FMSv9 certificate s/w update. Plus, many people seem to resort to a re-install of a FMS but in my case we cannot re-install at this stage with production run 6 days a week. I've isolated this one instance by cross-relating the event.log with the java catalina log to find an exact point of failure. 2009-09-03 02:02:46.460 +0100    Information    518     FMS10    Administrator connected: "fmsadmin" (). 2009-09-03 03:09:42.528 +0100    Warning    628     FMS10    Administrator "fmsadmin" no longer responding; connection closed.   03-Sep-2009 03:09:50 org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildrenSEVERE: Exception invoking periodic operation:java.lang.OutOfMemoryError: Java heap space    at java.io.WinNTFileSystem.list(Native Method)    at java.io.File.list(Unknown Source)    at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)    at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1190)    at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:292)    at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)    at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1305)    at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1569)    at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1578)    at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1558)    at java.lang.Thread.run(Unknown Source) Exact steps to reproduce the issue:1.  Restart FM2.  Load databases within successfully all within 20 minutes3.  Some one hour later find the error with catalina and the mis-commuication No users at 2am, no xslt publishing, no web publishing Expected Result:fmsadmin remains connected and the server admin console is able to workActual Result:So, no server admin console nor the 16000 port works Exact text of any error message(s) that appeared:See above for catalina messageAny additional configuration information/troubleshooting that is relevant to the issue:1.  All users have access, xslt publishing works, xml publishing works,2.  I have no problem accessing these files via Instant Web Publishing 

        • 1. Re: Administrator "fmsadmin" no longer responding; connection closed
          user13578
            

          Additionally... a recurring java thread is executed with sporadic failures....

           

          03-Sep-2009 03:11:58 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:12:00 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:14:20 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:16:51 org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler processConnection
          SEVERE: Error reading request, ignored
          java.lang.OutOfMemoryError: Java heap space
          03-Sep-2009 03:17:26 org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor processChildren
          SEVERE: Exception invoking periodic operation:
          java.lang.OutOfMemoryError: Java heap space
          03-Sep-2009 03:18:27 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:18:29 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:19:42 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:19:44 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:20:55 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected
          03-Sep-2009 03:20:56 org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme
          INFO: basic authentication scheme selected











          • 2. Re: Administrator "fmsadmin" no longer responding; connection closed
            user13578
              

            FM Server Stats Log

             

            2009-09-03 03:06:52.419 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:07:07.397 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:07:22.421 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:07:42.506 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:07:57.499 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:08:12.492 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:08:27.485 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:08:42.478 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:08:57.471 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:09:12.464 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:09:42.466 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:09:42.466 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:09:57.443 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:10:12.436 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:10:27.430 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
            2009-09-03 03:10:42.423 +0100    0    0    100    0    0    44    0    0    0    0    0    0    0
             







            • 3. Re: Administrator "fmsadmin" no longer responding; connection closed
              user13578
                

              Yet, through the course of a normal working day with +10 users, over 300 xslt calls performing system tasks and publishing to a web server from over 52 databases. No other issues exist, no performance issues.

               

              But, the 16000 port call results in the following blank statement, since the fmsadmin has disconnected with no recovery. A restart is not possible till 2am this morning with the numerous clients working off the server using FM-Client along with 3 people using the web publishing version.

               

              16000

               

              HTTP Status 500 -
              FileMaker Server has encountered an error:
              The server encountered an internal error () that prevented it from fulfilling this request.

              If the Server Launch Page is not available, visit the FileMaker Website for more assistance.
              FileMaker Server 10

               

               

               



              • 4. Re: Administrator "fmsadmin" no longer responding; connection closed
                user13578
                  

                I've also registered an issue in a related thread noting the issue when xslt publishing goes into an odd system overload... when prescribing incorrect calls or sending far too many calls. That issue has *NOT* recurred since enacting a very very careful process of only loading assured correct clean xslt calls to the server.

                 

                Therefore, the fmsadmin dropping prior to any xslt calls or logins by clients or use of any web publishing methods... appears to be outside of the issue whereby xslt publishing subsequently goes awry when called inappropriately in the course of a busy day.

                 

                 

                • 5. Re: Administrator "fmsadmin" no longer responding; connection closed
                  MartinBrändle
                    
                  There is now FileMaker Server 10v2 available, please update to see if the error still remains.


                  ekoss wrote:

                   

                   

                  FileMaker Product(s) involved:

                  FileMaker Server 10.1

                   

                   



                  • 6. Re: Administrator "fmsadmin" no longer responding; connection closed
                    sluggosqueen

                    I am having this issue also with IWP users.  It started after an upgrade from FMSAdvanced 10.2 to 11.  I removed 11 and reinstalled 10.1 yesterday.  Connection closed still happening.  I just updated to 10.2 to see if that would correct the issue.  User inhouse with clients(FMP10) connect fine during the day and seem to keep fmsadmin connected.  It seems to be the evening when the clients are no longer in uses that the connection stops responding.  Sometimes I just have to restart the filemaker service, other times it takes a re boot.  IWP works fine during the day when the clients are working.

                    • 7. Re: Administrator "fmsadmin" no longer responding; connection closed
                      hschlossberg

                      In 11, there is a way via command line "fmsadmin restart adminserver".  I'm hoping such a patch will be released for FMS 10, though I somehow doubt it.  Even though this feature is included in 11, it doesn't work half the time.  Nor does the adminserver automatically restart like it's supposed to when it loses communication between the server and the admin console.  We just downgraded a client's FM Server from 11 back to 10 after about a week because of this and some more serious problems and all of my clients are now prohibited from upgrading to either FM Pro 11 or FM Server 11.

                      • 8. Re: Administrator "fmsadmin" no longer responding; connection closed
                        mrvodka

                        Hey Howard.

                         

                        I havent been over on TechTalk in a few weeks so I have missed out. Can you further elaborate on the additional server issues?

                        • 9. Re: Administrator "fmsadmin" no longer responding; connection closed
                          hschlossberg

                          I am not a TechNet member, so I'm not sure what has been discussed there.  But in our one-week trial in a real-world environment, we had more crashes and slowdowns than we ever had in 10.  We have 20 clients accessing via FM Pro, and then another 20-30 at a time (usually 1-5 active requests simultaneously) using a web interface (via FM's XML API).  We had moved to 11 primarily in hopes of reducing backup of XML traffic, since 11 boasts an "all-new" XML engine.  What we got instead were constant crashes of the web publishing core, nightly scripts where the script schedule completed but the script user was still logged in as a client and couldn't be disconnected, slowdowns to the point of unusability by either FM or web clients (presumably because of the WPC problems and backlogs of XML requests).  10 has its occasional problems with the XML engine going deaf, but not like in 9 or 11.  And while 10 has its problems with the Admin Console and sometimes its fmsadmin command line going deaf, 11 seems to be no better despite its claims.  

                          • 10. Re: Administrator "fmsadmin" no longer responding; connection closed
                            TSGal

                            All:

                             

                            The Admin Console going "deaf" is an ongoing problem that we (FileMaker, Inc.) have not fixed, because we still don't know the cause.  In FileMaker Server 10, one would need to restart the server again.  In FileMaker Server 11, we gave the ability to restart the Admin Console without having to reboot the server.

                             

                            howards:

                            I would like more information about your FileMaker Server 11 setup.  In particular, do you notice more crashes when accessed via IWP?  XML?  general client traffic?  Server OS?I'd like to make sure your issues get reported.  (On a side note, I have replied to your private message from two days ago).

                             

                            TSGal

                            FileMaker, Inc.

                            • 11. Re: Administrator "fmsadmin" no longer responding; connection closed
                              hschlossberg

                               


                              TSGal wrote:

                              I would like more information about your FileMaker Server 11 setup.  In particular, do you notice more crashes when accessed via IWP?  XML?  general client traffic?  Server OS?I'd like to make sure your issues get reported.  (On a side note, I have replied to your private message from two days ago).


                               

                               

                              I only happened to be using IWP that one day in order to register a server-side plugin.  It is XML that we normally use and which has proven to be troublesome, where an XML call will 'hang' and then all following XML calls hang.  These are general WPC hangs, NOT admin console hangs, and my SE, BAM, FMI engineering and others are all aware of our issues.  Unfortunately, so is my client.

                               

                              As for the Admin Console -- my experience in 11 is that anything having to do with server side scripts or plugins will hang it.  My FMS 11 experience, however, was only less than a week before we had to uninstall 11 and go back to 10.  

                               

                              There is a joke within the FileMaker developer community that if you want to build an app that is sure to crash at least once daily, then build it in Java.


                              • 12. Re: Administrator "fmsadmin" no longer responding; connection closed
                                yakimax

                                We are having the same problem. We had this in server version 9 and in version 10.  The Event Viewer says "no longer responding connection closed 51"

                                 

                                The users get an error message and the database has closed down.

                                 

                                I have replaced all routing boxes and switches, thinking it was a network problem.

                                 

                                Would like to get this resolved.

                                 

                                • 13. Re: Administrator "fmsadmin" no longer responding; connection closed
                                  user16545

                                  Filemaker: when can we expect a solution for this issue ?

                                  • 14. Re: Administrator "fmsadmin" no longer responding; connection closed
                                    TSGal

                                    Han Roest:

                                    Thank you for your post.

                                    This problem is still unresolved.  Testing and Development continue to research this.

                                    TSGal
                                    FileMaker, Inc.