13 Replies Latest reply on Oct 6, 2012 9:21 AM by Vincent_L

    Speed and multithreading

    duaneburghard

      FM Staff,

       

      I realize that you spend a lot more time fielding questions and comments about what people dislike about FM than what they like, so let me start by saying that I've been programming with FM for 18 years now and I'm not going anywhere, it's by far the best platform (in terms of the combination of ease of use and capability) for the vast bulk of what I do. That said, there are two issues that MUST be VERY serious and systemic problems or they would have been fixed by now (because I KNOW I'm one of many people experiencing these two issues) and I'd like you to discuss why these are systemic issues and what plans if any you do have to address them along with a timeline to solution.

       

       

      The first issue is speed over the Internet. Nearly all of my solutions are, to use today's metaphor du jour, cloud based apps (databases hosted on an FM Server in the midwest, used by businesses (most of them mine) all over the country. The speed of *basic* actions in FileMaker is, and always has been, wildly inconsistent and often times simply not functional. In some cases, this is because FM Server is sending *far* more data than is necessary downstream to the client and making the client do the work that the server should be doing (e.g. calculation fields ... this has gotten somewhat better over the years but is still nowhere near where I think it should be). What is FileMaker doing to increase the amount of work that FM Server is doing and making sure that it's not "pushing" more bits downstream than is necessary?

       

      The second issue is somewhat related, multi-threading. It's pretty weird to many of us to be sitting here in 2012 and not have a version of FileMaker capable of doing more than one thing at a time. The lag we often endure while waiting for FileMaker to catch up to often simple requests would be made a LOT better in many cases if we could simply let it chew on a given task while we move to another window and do something else. Is there ANY plan on the boards to make FM capable of handling multiple tasks simultaneously and can you tell us when?

        • 1. Re: Speed and multithreading
          eric

          That should probably have been separate posts.

           

          But, yes,

          FileMaker should be multi-threaded so that users and developers can continue to work while long tasks complete.

           

          As many applications as our business has for FileMaker, it becomes a handicap when FileMaker gets bogged down with a large report, import, or other task. I imagine that at least each window should have its own independent thread, similar to web browsers like Internet Explorer. If one IE page is slow loading, we can simply open another window and continue browsing elsewhere until the other page finishes. FileMaker should be as operationally flexible.

          • 2. Re: Speed and multithreading
            Vincent_L

            I completly agree about those,

            For me if filemaker don't address those speed issues I'd be forced to abandonned the platform. I hope that FM 13 focus is on speed, speed, and speed.

            • 3. Re: Speed and multithreading
              vince.menanno

              Eric,

               

              In one meeting we talked about the idea of having clients run some scripts in thier own threads ... what you are saying about windows also sounds interesting. Say you konw that a script that you run to process thousands of records. What if you had some way to choose where you run it ... Or maybe its simply defined by the develooper and the user doesn't get to choose where it runs.

               

              Run on client ( within FileMaker )

              Run on client ( in its own process )

              Run on server

               

              All these scripts would be running as if you were the person that were logged in. But the benefit with option 2 and 3 is that once the script is fired off then you could countinue to use the database and some sort of indeterminate progress dialog could tell you that the script is still running. And if we had some native progress dialogs that we could use then an actual progress dialog could show you where things are at.

              • 4. Re: Speed and multithreading
                eric

                It seems that this idea of threading per window wouldn't be totally unprecedented.

                OnTimer scripts are per window, as well.

                • 5. Re: Speed and multithreading
                  danny

                  One work around that we  implement is to install a copy of FileMaker Pro and FileMaker Pro Advanced on our development machines. That way, when one gets sucked into a task, we can switch to the other.

                   

                  Of course, this is not ideal. I love almost everything about FileMaker, but "cloud" performance and lack of multithreading are major drawbacks that seriously affect usability and especially scalability. Simply having a thread for each window would be more than sufficient, but at the very least, it would be nice if each database had it's own thread.

                  • 6. Re: Speed and multithreading
                    eric

                    We do the same for our developers, but not the users (yet).

                    It's embarrassing to have to this, though.

                    Especially when you have several databases, and each database looks and acts like a separate application—until one gets tied up, then all your FileMaker applications are tied up. What happens if your whole office is running almost entirely on FileMaker apps and one gets tied up?

                    • 7. Re: Speed and multithreading
                      duaneburghard

                      I have a similarly silly workaround. I have a MacMini set up at the data center where our FM Server is, and when I need to get more than one thing done at once, I simply launch both FileMaker and Apple Remote Desktop (running FileMaker on the Mini).

                       

                       

                      I agree with Eric though, this is embarrassing. In fact, I once met an engineer who told me that he thought it was just ridiculous that after all these years FM still couldn't do this.

                       

                      Regarding speed, I agree with Vincent, focusing on speed as opposed to interface or other things would be a far more important reason for me to want to upgrade.

                      • 8. Re: Speed and multithreading
                        taylorsharpe

                        Technically, FileMaker is multi-threaded.  So if several of us run a script at the same time, each script can be multi-threaded out to different cpu's for processing.  What FileMaker does not do is allow any given thread to be shared among multiple processors and that kind of technology is very difficult.  But it sure would be nice if FileMaker could do it in the same way that video processing is frequently spread across many processors for one process.  And there is the potential to do as suggested above where each window could be assigned a thread and run independently of other windows so that you could be doing some data entry in one window while a long script runs in another window.  This does bring up an interesting topic that lots of people overlook is that you can set up FileMaker to run batch jobs and if you have big scripts, submit the script to a batch process that a schedule script looks for frequently and then runs it in the background on the server without interupting what you are doing. 

                        • 9. Re: Speed and multithreading
                          jrenfrew

                          But this is also about handing off, in the middle of the thing you are doing right now, not just scheduled tasks. There are lots of processes where I want to use the results that a server might be better calculating, and that is everything from calc fields to creating a PDF from a found set which is then manipulated, but I want to send now, not layer.

                           

                          So big thumbs up for intelligent get it done multithreading

                          • 10. Re: Speed and multithreading
                            mbraendle

                            We had this discussion already, see https://fmdev.filemaker.com/message/87159

                             

                            Multithreading isn't so easy to implement in databases, because of race conditions, record locking etc. There are many cases, especially in FM scripting, where a process step depends on the outcome of the previous steps, so this can't be parallelized. Remember also that DB transactions have to be ACID compliant.

                            • 11. Re: Speed and multithreading
                              simple

                              My work around is to move more and more application to PHP for my customers. With FM 12 Server, PHP is fast and can handle 200 connections.  Also I use as much server schedule script as possible. For huge data , I have a server schedule script to dump them into MySQL using the ESS for reporting and analysis purpose. Work for me

                               

                              As a result,  I am buying less and less FM licenses for my customers

                              • 12. Re: Speed and multithreading
                                Vincent_L

                                taylorsharpe a écrit:

                                 

                                Technically, FileMaker is multi-threaded.  So if several of us run a script at the same time, each script can be multi-threaded out to different cpu's for processing. 

                                 

                                Hum, I'd really like to see evidences of this because this is not at all what's going on for me. I just did a test I created a 10000000 looping script, duplicated it 3 times, and run as server side processing all of them simulatenously. Guess what ? only a total of 100% CPU usage. Only on thread. So that's not at all what we could expect, and not looking in sync with you description unfortunately.

                                • 13. Re: Speed and multithreading
                                  Vincent_L

                                  My work around is to move more and more application to PHP for my customers. With FM 12 Server, PHP is fast and can handle 200 connections.  Also I use as much server schedule script as possible. For huge data , I have a server schedule script to dump them into MySQL using the ESS for reporting and analysis purpose. Work for me

                                   

                                  As a result,  I am buying less and less FM licenses for my customers

                                   

                                  As I said, lack of speed focus at FM Inc is driving us away, and this cost to FM Inc, I hope they'd wake up as far as speed is concerned, because though I'd dispise it, I'll certainly be forced out the plattform because of this lack of speed.