4 Replies Latest reply on Oct 14, 2009 12:02 AM by FluffyBear

    Web deployment recommendations

    r3media

      Title

      Web deployment recommendations

      Your post

      Good day everyone,

       

      I am in the process of revamping a client's web site and would like some recommendations on how to proceed.

       

      Here are the paticulars

      The client operates an annual conference with roughly 5,000 attendees a year.

      They have been using Filemaker for a long time and have developed a decent FM database over the years. They are currently running FM Server 9 on the same host as their website (Windows 2003 server). The database has been for internal use mainly, up until last year when I integrated some of it to the web.

      Contacts in DB: 55,000+

      Companies in DB: 35,000+
      Moderator/Panelists in DB: 5,700 total
      Yearly delegate/members: 4-5,000
      Panels per year: 150

      (I don't need access to all of the records above, but this is where I will be pulling the info, filtered by year)

       

      What I would like to do is integrate FM with the website. I have a couple of concerns/questions on how I do this.

      My biggest concern is speed and performance. The site gets about 4,000,000 hits per month at it's peak

      Last year I connected the database (online registration, panelists information, an online members area etc) using ODBC DSN connection and ASP. Big mistake! People were trying to purchase tickets, log in to the member's area, find out who is speaking where all at once. The poor performance of the SequeLink driver and using ASP shut down the website during the whole conference.

       

      So, this year I was considering scrapping FM and moving everything to a web-based system and using PHP and mySQL. But, before I do that I wanted to visit the FM solution one more time. I just did a test with the PHP site wizard with the Moderator DB (147 fields, 5700 records). It was pulling back 25 records in about 16 seconds. I scaled it back and now have 100 records loading in about 10 seconds. This seems like a lot of time to me especially when there is no load on the server at all right now.

       

      http://www.cmw.net/cmw2010/fmtest/test/moderators.php(load time at the bottom)

       

      So...

       

      What would you recommend?
      FM with PHP AP? (still have concerns about performance)
      FM with Fx.php
      Just exporting the data I need for the web and use MySQLwith PHP

      I see FM has MySQL support. Would this help?

       

      Any recommendation would be greatly appreciated. I have to nail down a solution in the next couple of days.

       

      Thanks

       

      Randy

      R3 Media and Design



        • 1. Re: Web deployment recommendations
          FluffyBear
            

          Your tables seem fairly small, it should have much faster performance than that.  We were using some big databases, one was at 10GB+ with several tables each having 7-10M records and I could usually get about 200-500 records out per second using the XML CWP system.

           

          Check what it is that the web site is connected to when it try to load data.  Make a custom layout for the web site to access with only the fields that is absolutely needed.  Anything that is calculation or linked to another table / database as a lookup field will massively massively slow you down.  If you don't need the field, don't access it.

           

          On the extreme scale, I've seen anywhere from 1.5-3 seconds per record to 500-750 records per second.  FM is a little picky and you have to work with it a bit to get the optimum possible performance out.

           

          And looking at your test page, why are you displaying the empty rows?  Maybe refine the search criteria so it doesn't return unnecessary rows?

           

          • 2. Re: Web deployment recommendations
            r3media
              

            Thanks for the response FluffyBear.

             

            I am happy to hear that FM is successfully pulling back 200-500 records per second with some of your development - it is very encouraging to hear.

             

            The current server is in a co-location and is VERY suspect. We are upgrading the hardware in the next week or two. In the meantime I just downloaded the trial version of FM server to do some testing locally. If I see a dramatic performance increase I will at least know one source of the problem.

             

            As far as the test page - I basically just used the PHP assistant and removed some of the unwanted columns, just to test it as "out of the box". I will take your advice regarding creating custom layouts for the web as well as removing unnecessary calculation fields.

             

            If I can pull 300 records per second I would be very happy. I think testing this locally will give me a better look at the capabilities.

             

            Other responses are welcome.

             

            Thanks again, feeling better already.

             

            R

            • 3. Re: Web deployment recommendations
              r3media
                

              FluffyBear - FYI,

               

              I installed FM server on one of our local machines, loaded just the moderators DB, created a "webtest" layout and got 1000 records back in about 2-3 seconds (locally) and 4-5 second through our LAN.

              Huge improvement. Thanks!

               

              But, when I opened the rest of the FM databases (47 total)  the times increased 2.5 times. Seems the extra load on the FM server slows it down quite a bit. I suspect that this will reduce with an increase in hardware and processing power. Plus there are some databases that do not need to be hosted.

               

              Now I am wondering hom much this will affect the performance, considering at any given time we could have 10 FM clients connected to the server, plus PHP web clients.

              And will using the 2-machine install make an improvement.

               

              I am still testing out everything right now, so if anyone that has went through these motions before has any pointers or suggestions, I would appreciate it.

               

              Thanks,

              • 4. Re: Web deployment recommendations
                FluffyBear
                  

                At your load level, the extra RAM & CPU power does not necessarily help.  We have been running FM for a very long time,  and even though sometimes 1 user doing a huge search can get our CPU to hit near 300+%,  I suspect a lot of it is actually IO Block and IO Wait where the CPU is running spinlock.  We use Apple XServe with 2 Dual Core.  Unfortunately the setup was not my own and we use RAID 5.   We have about 7GB ofRAM on the server,  FM does not appear to be really capable of using large amount of RAM in an affective manner with a cache.  A dual machine deployment sound nice but might not necessarily help if your current machine is well enough to handle thing.   If you have like a pair of Pentium 4s, then it'll help.  But beyond a good Core 2 Duo system, I'm not sure if the scaling is worth it compared to the network lag and overhead generated by FM.

                 

                The biggest determinant of your load time for CWP will be in the layout structure.  Did your web test layout had any link at all to another database once you opened them up?  If there is even 1 calculation or look up field that referenced any of the other DB,  you will see performance degradation.