1 2 3 Previous Next 38 Replies Latest reply on Jul 17, 2016 4:36 AM by fmpdude

    Is there a plugin for FileMaker to perform multiple regression?

    A2analytics

      Does anyone know of an extension or plugin for FileMaker to perform multiple linear regression?

        • 1. Re: Is there a plugin for FileMaker to perform multiple regression?
          fmpdude

          The way I handled statistical needs was to gather up the data in FM and pass it via URL to a REST web service I wrote. Using known algorithms for linear (in my case) regression, I do the calculations in the web service and then make another call from FM to get the result.

           

          This approach is free and you have total control over the code, but it's not plug and play like third-party tool (if one exists for stat).

           

          HOPE THIS HELPS.

          • 2. Re: Is there a plugin for FileMaker to perform multiple regression?
            jbante

            There are a number of plugins that let you access execution engines for other languages, such as ScriptMaster (Java/Groovy) and bBox (Python, Perl, others). You could then access linear regression tools available for those languages.

             

            You could also just implement the calculations in FileMaker, which is my personal preference. Keeping everything in FileMaker makes things easier to maintain and more portable.

            • 3. Re: Is there a plugin for FileMaker to perform multiple regression?
              fmpdude

              In my statistics use-case in point, my remote web service + FileMaker export of the data takes about 5 seconds to complete. Here's the break down:

               

              1. Web Service Stat routines run time in compiled code: 0.025 seconds.  (FAST!!!!!)

               

              2. FileMaker script to do required table scan to bundle up the data and complete the INSERT FROM URL POST: > 4 seconds.

               

              In Java, for example, there are free (for personal or commercial use) stat libraries you just import into your project. Then, import your FileMaker data (either manually or via a FM script of some type like INSERT FROM URL), and call the stat methods to give you the results you need.

               

              -------

               

              A stat plug-in for FM would be a great alternative, if it exists, since that would be compiled code and everything would stay in FM. Best of both worlds except that would most likely not be free (but a plug-in would take no coding skills, either, which may be a good thing). And, "Free" is relative since it takes some time to externalize non plug-in library code, but having said that, if you're going to write code you still have to write it ... somewhere.

               

              I initially tried to write the stat routines all in FM, but got so frustrated by the script work space I quickly gave up using that. Script work space is fine for short stuff, but not, IMHO, a couple hundred lines of complex code. Just takes too long.

               

              My two cents. YMMV.

              1 of 1 people found this helpful
              • 4. Re: Is there a plugin for FileMaker to perform multiple regression?
                user27087

                What fmpdude said, or similarly if you have javascript that accepts input in a simplified manner (i.e., that FileMaker can send as a simple string) and you can parse a resulting string it can be done entirely within FMP. You can paste the code as text onto your layout (off to the side) and use it within a Web Viewer. The JavaScript needs to be modified to return the result using a call back to the FMP program, but it performs some relatively complex stuff in about a second.

                 

                My need was different (Simplex vs Stats) but along the same lines as yours. I looked for a few hours to find javascript for the Simplex method that fit my needs, but once found it was far easier to latch onto working code than to reinvent the wheel. I knew that creating a custom function in FM was not going to be worth my time.

                • 5. Re: Is there a plugin for FileMaker to perform multiple regression?
                  fmpdude

                  Simplex method? Do they still use that?

                   

                  Take me back to my Industrial Engineering courses, which I loved! Queuing theory, Markov chains, etc. So fun!!!

                  • 6. Re: Is there a plugin for FileMaker to perform multiple regression?
                    user27087

                    "Simplex method? Do they still use that?"

                     

                    Pretty much all routing problems are solved with Simplex or a variation, like shipping or transportation. Used since 1947, and very underutilized from what I have seen.

                     

                    Many large companies could be taking advantage of Simplex in some manner, but getting Simplex tied to a database is what makes it powerful and useful. Some managers know about Excel Solver, but then they try setting up (and accurately entering) a 15 x 15 matrix of numbers to get one answer when there can be many or 100s each day. Too difficult and time consuming. Tie Simplex to a database and entry issues are solved.

                    • 7. Re: Is there a plugin for FileMaker to perform multiple regression?
                      fmpdude

                      So cool.

                       

                      I loved that stuff.

                      • 8. Re: Is there a plugin for FileMaker to perform multiple regression?
                        jbante

                        fmpdude wrote:

                         

                        I initially tried to write the stat routines all in FM, but got so frustrated by the script work space I quickly gave up using that. Script work space is fine for short stuff, but not, IMHO, a couple hundred lines of complex code. Just takes too long.

                        Numerical computing is difficult and takes a long time to get right no matter what language it's written in. FileMaker's disadvantage is not the programming environment; it's that no one else has already written the hard parts for it yet. I believe there are good reasons for that to change.

                        • 9. Re: Is there a plugin for FileMaker to perform multiple regression?
                          fmpdude

                          I like FileMaker a lot so let me be clear about that up front.

                           

                          However, I would not call FileMaker's programing environment "professional" by competing standards. And, I do NOT mean that as a slam against FM. FMI, for whatever reason, has chosen the product as it is now.

                           

                          Unlike a professional programming environment, in FMP there is no refactoring (at all), it's clumsy at best with the silly pop-up fill in as you go of the "Script Workspace". You can't resize fonts, window panes, etc. If you compare that with a professional programming IDE (.NET or Java, for example), I think you'd be shocked at the difference.

                           

                          Also, in FMP, as I understand it, scripts are not compiled so they run very slowly compared to compiled code. That's hugely important especially with numerical computing. I can post some benchmarks I've done to back this up.

                           

                          FMP definitely has its place in my arsenal, but for serious programming projects with lots of code that must run quickly, well, not so much. For me, FMP is excellent with "CRM" type apps - screens with customer information and not a lot of code. And, for projects that can take advantage of its strengths which are many.

                           

                          Again, just my two cents based on six years with FMP (others have much more experience with it).

                          • 10. Re: Is there a plugin for FileMaker to perform multiple regression?
                            FileKraft

                            you can do a lot in custom functions - it is FM's functional programming paradigm.

                            i used it successfully for a lot of stochastic and combinatorics in the past.

                             

                            i agree though that there is no IDE for it and you are on your own trying to debug recursive CFs - still so a lot of fun and if you understand functional programming with all its benefits you can built a nice framework there.

                             

                            • 11. Re: Is there a plugin for FileMaker to perform multiple regression?
                              fmpdude

                              When you said "Functional Programming", that makes me think of Java 8 as in Declarative programming, lambda calculus, and the like. As you know, when functional programming, you don't actually specify the algorithm code. It's more like "SQL" where you specify "what" needs to be done, but now "how" (as in imperative coding).

                               

                              I get what you're saying though and yes, my partial list above also missed that there is no way to debug CFs. The modal nature of getting to the Data Viewer (got to close everything you need it for first) is another pain. Can't resize its fonts or move window panes there, either.

                               

                              I use FMP for its strengths. Fortunately, it has many.

                              • 12. Re: Is there a plugin for FileMaker to perform multiple regression?
                                FileKraft

                                the let function combined with evaluate is kind of a "lambda-lite function".

                                the custom functions as the name implies are functional programming and nothing else - the only side effects there are  happening via let function if used to declare $vars or $$vars.

                                 

                                it follows the 2 main computational patterns of substitution and recursion to express your algorithms.

                                 

                                it even allows call-by-reference if the argument passed is evaluated with the geFieldName function.

                                 

                                limitations to consider are the stack size for non-tail recursive calls 10.000 or plain tail-recursive calls 50.000

                                • 13. Re: Is there a plugin for FileMaker to perform multiple regression?
                                  jbante

                                  To say that FileMaker has "refactoring", and by implication that programming tools for other environments do, tells me that you mean something very different from "refactoring" than what I understand it to mean. I refactor FileMaker code every day. It's something the developer does, not the environment. FileMaker doesn't have things like linters, I'll grant you, but that's not refactoring. FileMaker does have some catching up to do compared to other IDEs, but resizing fonts and window panes are not significant game changing features to distinguish professional from casual programming tools. Script Workspace pop-up windows and such are comparable to the code completion features of other IDEs; I concede using them efficiently has an uncomfortable learning curve, which is also perfectly comparable to other IDEs. Version control and smoother deployment options might be better examples, but that has nothing to do with FileMaker's value for numerical computing. Better tools for in-memory data structures like multi-dimensional arrays and trees would make things much easier, but it is possible to make these structures in FileMaker anyway. It isn't always as pretty as we might like, but it can work well. If it were ever completely pretty, software developers (in any tool set) wouldn't have jobs anymore.

                                   

                                  We are not privy to FileMaker's implementation details, but I agree that it's fair to lump it together with interpreted rather than compiled languages. A lot of cutting edge numerical computation is done in interpreted languages. Many fields of research use interpreted languages almost exclusively. Statisticians are usually not writing C++. When you study software for math and statistics, they don't teach you C++; they teach you Python and one or another field-specific tool (with an interpreted language) like R or Mathematica. I agree that FileMaker has a consistent performance disadvantage, but the magnitude of that disadvantage is often not enough to make a substantial difference to my clients' operations. It's also not as if FileMaker doesn't care about performance. They obviously do, within the constraints of their other goals. As it relates to numerical computing, why else would they have gone with fixed-point rather than floating-point arithmetic? Math has one of the smaller performance disadvantages in the FileMaker platform.

                                  • 14. Re: Is there a plugin for FileMaker to perform multiple regression?
                                    fmpdude

                                    Agree with you. I was talking about built-in refactoring tools like search and replace in scripts.

                                     

                                    Just for grins, here's some benchmarking I just did:

                                     

                                    I just did a simple loop in FMPA 14 in a script for 10,000,000 iterations. Each loop iteration simply incremented a variable by 1.


                                    FMPA took 11 minutes and a few seconds to compete the loop.

                                     

                                    ---------

                                     

                                    I did the exact same program in Java.

                                     

                                    It took 3 ms. (0.003 seconds).

                                     

                                    ---

                                     

                                    Neither program wrote anything out to the screen until complete.

                                     

                                    -------------

                                     

                                    Also, I also found that a SQL query against 500,000 records took over 5 seconds in FMP with no apparent way to speed it up. Simple query. The same query using a Java program to do a RegEx search of a disk text file took 0.2 seconds. Using a b-tree index in MySQL took that query to 0.1 seconds.

                                     

                                    So, for me and numerical computing, I'd either look for a plug-in or exchange FMP data with another platform for analysis.


                                    Can you confirm these findings?

                                     

                                    ---

                                     

                                    Where I do find FM very fast is with its native Finds. Love that! Super quick.


                                    I love FMP for the forms it helps me quickly build and the data I can quickly manage.

                                     

                                    FMPA is a very nice program.

                                    1 2 3 Previous Next