1 2 Previous Next 17 Replies Latest reply on Jun 4, 2013 12:30 AM by Hans

    Server Access Slowness & "Coffee Cups"

    user10625

      Hello all,

       

       

      FMS 12, FMP 12 (v3-4)

      Windows 7 - clients

      Windows Server 2008 R2 SP 1

      64-bit machine with quad-core processor, etc.

      24GB RAM --- I think at leat 10GB allocated to FMS.

      Hard drive is 1 TB, in RAID10 configuration.

      appr. 30 users during working hours.

       

       

      A couple of weeks ago, users on this system suddenly started seeing a lot of "coffee cups" (for a few seconds each), random "processing find" boxes, and so on. These happen in all different places/modules/areas in the system, not localized to one particular bit of functionality.

       

      The database file(s) have their own inherent slowness issues to be sure, which we've been working on, but this was a sudden change in speed issues, above and beyond what people were used to, which we are pretty sure is not due to unstored calcs, complex graph, or any of the other usual suspects. And again, it doesn't seem to be particular to any one part of the system--happens on all different layouts, tables, etc.

       

      As noted above, the server is tricked out with tons of RAM (I think I allocated 16GB or so, for about 7GB of database), has a good quality RAID10 hard drive with 1TB of space, it's 64-bit, etc. etc.

       

      I recently had some users (including myself) log into a desktop (via terminal server) on the server itself, and then use the database by running FMP there (through Open Remote, of course--no bad behavior!). In those cases, they didn't see a single coffee cup or random find processing. I think this indicates that the issue does not lie in the solution, or on the server, itself, but in the LAN between users and server.

       

      (Also, for 1 day, the users wrote down the times when they saw the "Coffee cups" or what-not. There seemed to be no correspondence between those times. . i.e. people did not all experience slow-downs at 1:14 PM, or whenever. Everyone had different reports.)

       

      So . . .does anybody know of any problems with the LAN that one should check up on in such cases? Or any other ideas of things to look into?

       

      Thanks,

      Christopher Bailey

      Baytaflow

      Boston, MA

      Certified FM 12, 11, 10, 7

        • 1. Re: Server Access Slowness & "Coffee Cups"
          mikebeargie

          Is this a LAN or a WAN?

           

          If the users are outside the building/local network, it's a WAN, in which case the application speed can vary wildly with their internet connection speed in terms of loading data down off the server. This could also be the case if they are on the local network, but using a public IP (WAN IP) to connect to the server, as the data is passing out of the building and back.

           

          If it's truly a LAN (inside the local network, referring to a local host IP address), then I'd consider some LAN troubleshooting.

          • 2. Re: Server Access Slowness & "Coffee Cups"
            user10625

            If it's truly a LAN (inside the local network,

            > referring to a local host IP address),

            >  then I'd consider some LAN troubleshooting.

             

            Yes . . . so what are some ideas for  LAN troubleshooting?

             

            C

             


            • 3. Re: Server Access Slowness & "Coffee Cups"
              mikebeargie

              I can think of a few things to do, it sounds like you already confirmed that the server itself was fast, so I agree it's an issue somewhere on your lan.

               

              I would try:

               

              1) Put a utility like wireshark on the server to see if it's maxing it's in/out connection. Maybe someone on the network is blasting packets and is causing an issue.

               

              2) Check your routing and switching hardware to see if something needs rebooted/upgraded, or is suffering from major lag.

               

              3) Check a TRACERT to view the path from a client to server, see if there's any unexpected "hops" it's making on the way to/from, and check all hardware along the way.

               

              4) Check what other services the server are performing that are suspect in network performance (firewall or antivirus on the filemaker packets that could slow them down).

               

              5) Think if any changes were made in the database (schema, amount of records, etc..) You could test against this if you had an old copy of the database you re-hosted and tested the speed on.

               

              6) Has anything changed recently on the server that is suspect (IE java updates seem to always do a number on filemaker). Is filemaker server edition up to date? Has it been rebooted recently?

               

              That's what I can think of to start.

              1 of 1 people found this helpful
              • 4. Re: Server Access Slowness & "Coffee Cups"
                mikebeargie
                1 of 1 people found this helpful
                • 5. Re: Server Access Slowness & "Coffee Cups"
                  user10625

                  Thanks for the ideas!

                   

                   

                  Christopher Bailey

                  Baytaflow

                  • 6. Re: Server Access Slowness & "Coffee Cups"
                    timwhisenant

                    Hi Christopher,

                     

                    If it is your LAN, how is the Domain Controller? Is it seeing peak load issues? I have also seen switches go bad, is there any other network software taking up bandwidth on the network?

                     

                     

                     

                    Just an idea that occurred to me,

                     

                    Tim

                    • 7. Re: Server Access Slowness & "Coffee Cups"
                      wimdecorte

                      To add to this excellent list:

                       

                      1) check VOIP: is it running on the same network?  If so it could be pushing other network traffic down the priority ladder which would hurt FM traffic more than other types of traffic

                       

                      2) along the same thought: any users streaming video (netflixh, hulu, amazon prime,...)?  Any surveillance cameras that are streaming video on the same network?

                      • 8. Re: Server Access Slowness & "Coffee Cups"
                        mikebeargie

                        Ah yes, forgot about VOIP. It's a terrible thing to have on the same network as a mission critical app.

                         

                        We've actually isolated the phones to their own router and subnet in the past for one of our clients to "tame" the VOIP traffic.

                        • 9. Re: Server Access Slowness & "Coffee Cups"
                          BruceHerbach

                          HI,

                           

                          Although you are seeing the slow down on the clients,  so are thinking lan,  the fact that this seems to have come on very quickly may indicate other possible issues.  You might take a look at Performance Monitor and see what the disk is doing.  Also consider having the IT admin check the disks to be sure no problems are surfacing there.

                           

                          See if the slow downs correspond to Database backups run by FMS. 

                           

                          Check with IT and see if they added any new services to the server that are hogging CPU cycles and interferring with FMS.  

                           

                          Ask about Anti Virus software,  have they added a new version to the FMS server?  DId they remember to exclude the FMS Data directory.  FMS doesn't like it when other software tries to access the databases it is sharing.  Besides slow downs this can lead to Database corruption.

                           

                          Along the same lines,  have they added any services to the office recently that could be hogging Lan bandwidth.

                           

                          Bruce

                          • 10. Re: Server Access Slowness & "Coffee Cups"
                            user10625

                            Hi Bruce,

                             

                            Most of the things you mention I have already investigated or ruled out. When people are logged into the server itself, they see no issues, so that rules out disk problems.  The timing of the slowness does not correspond to any backups.  We have also investigated Anti-virus software, and that is all disabled (at least the real-time, live scanning stuff that is usually bad news). 

                             

                            I am looking into the LAN bandwidth question . .  .

                             

                            thanks

                            Chris Bailey

                            • 11. Re: Server Access Slowness & "Coffee Cups"
                              mikebeargie

                              I agree with Chris, he originally posted his configuration is RAID10, I am assuming that's 1&0, which is the best config I know for running a database server drive off of.

                               

                              There were some other things Bruce posted though that rang true with what I was reiterating. If this was not a problem before and is now a problem, everything I know points towards something is to be expected somewhere on your network. Could be an infected computer blasting packets, faulty router or switch, or a dozen other things.

                               

                              Packet sniffing should help isolate if there is something flooding your network.

                              • 12. Re: Server Access Slowness & "Coffee Cups"
                                usbc

                                Stepping outside a bit, you don't make any mention of the building. A few years back I had a customer report a dramatic slowdown. After much testing it boiled down to an adjacent business had installed a new electrical system to accommodate the four new rooftop air conditioners. They had added a new mechanical room on the opposite side of the wall where all my client's Ethernet cables came down from the ceiling. EMI (ElectroMagnetic Interference). We had to relocate and shield our bundle. Obviously, this may have nothing to do with your situation.

                                • 13. Re: Server Access Slowness & "Coffee Cups"
                                  kibiz

                                  So, I am curious whether, you ever found the cause of the issue, after all the troubleshooting ideas that were offered.  I also have had similar issues recently, that I was not able to pinpoint the cause of the network slowness.  The solution would work fine, when run on a local computer, run from FileMaker Pro on the same machine as the Server was on, and other networks.  This client also had an X-Serve hosting FileMaker Server 11.   When the solution was run from one of their workstations, it would get FileMaker not responding message when trying to enter an order.  It would take 5 times as long to create and process an order on their network, than on other networks.   In looking at the Performance on the Activity monitor, it would show huge spikes at various times when data entry started.  This performance activity and time would vary though from computer to computer on the network.

                                   

                                  Has anyone found evidence, that anti-virus software, spyware, security video system software and/or a network backup program, can interfere with performance of FileMaker Server, and even cause data loss or corruption? 

                                  • 14. Re: Server Access Slowness & "Coffee Cups"
                                    doug_hogg

                                    Try restarting any switches. They are usually reliable, but that can cause them to be overlooked. On more than one occasion, I have seen switches become "confused" and bog down a network, and a restart handled them.

                                    1 2 Previous Next