1 2 3 Previous Next 40 Replies Latest reply on Aug 12, 2017 7:56 AM by beverly

    Limitations of using the Data Separation Model in FileMaker 13?

    jgalt

      I am about to start rewriting a large complex solution and over the past few days I have been experimenting with some of the new features in FileMaker 13. Before I start my rewrite I am trying to get a clear understanding of the limitations that I might run into using the data separation model (specifically any limitations when using FileMaker 13).

       

      I am asking "specifically about limitations using FileMaker 13" because it seems like there are some new features in FileMaker 13 that could change what previously could not be done in the relationship graph.

       

      For example, one major thing that I discovered today is that I can now join tables in the relationship graph using calculation fields. I don't work with FileMaker enough to keep up with all of the new changes but this appears to be something that I could not do in FileMaker 12 and the possibilities of this new capability seem significant.

       

      I understand the advantages and the level of complexity that implementing the data separation model can add to a solution (and I am really not asking about that) what I am trying to get a clear understand of are specifically the things you give up (or cannot do) if you go down the path of using the data separation model.

       

      Your feedback is greatly appreciated.

       

      Thanks.

        • 1. Re: Limitations of using the Data Separation Model in FileMaker 13?
          BruceRobertson

          It has always been possible to use calc fields in the relationship graph. There is no change.

          It has also, always, been the case that the fields used for the "right side" of the graph do need to be indexed fields.

          • 2. Re: Limitations of using the Data Separation Model in FileMaker 13?
            jgalt

            BruceRobertson wrote:

             

            It has always been possible to use calc fields in the relationship graph. There is no change.

            It has also, always, been the case that the fields used for the "right side" of the graph do need to be indexed fields.

             

            Bruce, I must not be remembering it correctly. I thought in the past when I would try to do something like this (see snapshot below) that it would bark at me and tell me that I could not do it. I assume my current test file is working because the fields that are being referenced in the calculation are indexed.

            16153241181_33d56eec06_o.png

            • 3. Re: Limitations of using the Data Separation Model in FileMaker 13?
              wimdecorte

              Relationships in FM are always bidirectional.  The key field in the context of where you are needs to be indexed for the relationship to work, the "foreign key" does not.

               

              In your example you can be on a layout based on 'CONTACTS" and as long as the UUID in "CONTACTS found_set" is indexed or indexable your relationship will work.

              However, using the same relationship, if you are on a layout based on "CONTACTS found_set" and "calc_uuids_of_current_found_set" is an unstored calc then the relationship back to "CONTACTS" will not work.

               

              That has not changed since FM7 came out and we've had these bidirectional relationships.

              • 4. Re: Limitations of using the Data Separation Model in FileMaker 13?
                jgalt

                I was initially fooled into thinking that it did not behave the same in FM12 because my test layout was using an FM13 theme. When I saw the blank fields in FM12 I incorrectly came to the conclusion that the relationship graph was behaving differently. Ugh!

                 

                So it's still not totally clear to me what functionality you give up (if any) by implementing the data separation model. It just seems like a good idea to me to try and keep the data separate from the GUI so I can update the GUI at my leisure without disrupting users.

                 

                I am planing to keep both the GUI and the data files on FileMaker Server 13, not on the user's computer or mobile device.

                • 5. Re: Limitations of using the Data Separation Model in FileMaker 13?
                  RalphLearmont

                  Perhaps you meant the other way round? 

                   

                  Doesn't indexing need to be on the foreign side (or destination side) for a relationship to work when finding matching values?  (With the exception of cartesian joins where it doesn't matter)

                  wimdecorte wrote:

                   

                  Relationships in FM are always bidirectional.  The key field in the context of where you are needs to be indexed for the relationship to work, the "foreign key" does not.

                  • 6. Re: Limitations of using the Data Separation Model in FileMaker 13?
                    jormond

                    jgalt,

                    In my experience, in using both Data Separation and Single File...it comes down to the solutions and the needs of the user.

                     

                    There are not many limitations as far as functionality using Data Separation. Most the pain-points with data separation have to do with the developer. For instance, if you need to make a change to the data fields, you can't edit that directly from the UI file. You must go into the data file to make the schema change. Also, at least for me, you run into a few situations more often where you need to force the relationship to refresh to pull back the edited data, etc.

                     

                    Outside of that, I find few issues with using Data Separation. Though I don't use it all the time. And nothing that I have run into that is 13 specific.

                    • 7. Re: Limitations of using the Data Separation Model in FileMaker 13?
                      Garryt

                      We've been using the Data Separation model for more than 20 years and I would not change..

                       

                      I find that it is particularly beneficial in large applications, where you can become very modular with the data and still have one file as the "front end". If most of the "work" is performed in the front end, there are fewer issues related to having to re-import data after a change or addition to the front end.

                       

                      In my experience, I have also found that there is less likelihood of database corruption, although I cannot offer any evidence that this is a factual statement.

                       

                      I would say that it is slightly more complicated to build an application using this methodology as there are many other elements to consider. Security in particular can be more complex as it needs to be taken care of throughout the multiple databases.

                       

                      As an example, if you want to export a data set from the "main menu" file, you would need to ensure the use has permissions to export in the source data file that the TOC refers to..

                       

                      On a personal note, I feel the benefits far outweigh the disadvantages.

                       

                      I would not consider this method for a small solution as I think the scales tip the other way...

                      • 8. Re: Limitations of using the Data Separation Model in FileMaker 13?
                        Mike_Mitchell

                        You still have some disruption to users, since you have to shut down the interface file and replace it with the new one. But it's less disruptive since there's (in theory) no data migration.

                         

                        However, in practice, it often doesn't work out that way. Since your schema are in the data file, you'll have to update that more often than you might think. So you end up doing a migration on most major upgrades anyway.

                         

                        Another thing to watch out for is live updates. Although I recommend against it, some people like to make updates to schema in their live solutions. If you do this in the Separation Model, you have to be careful about your next major upgrade (when you replace both files). The internal pointers from the interface file to the data file will likely not match, and you will likely have to repoint your layout objects to the fields in the new version when you do the migration.

                         

                        For minor changes, Separation Model is nice. You just have to decide if the headaches (managing security across multiple files is one) are worth the benefits. And that will largely be situation-specific.

                         

                        Mike

                        • 9. Re: Limitations of using the Data Separation Model in FileMaker 13?
                          Garryt

                          Another situation where the DSM really comes into its own, is where you use SQL rather than FileMaker relationships.

                           

                          When making changes, you need enter the schema less often and adding new fields to another database in the set is easier to deal with. You need to be aware though, that if you change the names of fields, this is not dealt with in scripts that utilize SQL in the same way as it is using native FileMaker field references.

                          • 10. Re: Limitations of using the Data Separation Model in FileMaker 13?
                            wimdecorte

                            Yes, I got that reversed in my explanation, thanks for calling it out!

                            • 11. Re: Limitations of using the Data Separation Model in FileMaker 13?
                              DavidZakary

                              Things I find annoying about the separation model....

                               

                              Muscle memory - I keep hitting the keyboard shortcut for Manage Database and I'm in the UI file rather than the data file. Workaround - a script to select the data file and then open Manage Database. Of course, you're likely not on the table you need. If you want - get fancy, test to see what table you're on and then select the datafile, the correct layout and then open Manage Database.

                               

                              Duplicating Schema - ExecuteSQL can minimize this but you need to have the relationship graph (or at least parts of it) in both files for business logic on one side and display purposes on the other. A pain to maintain.

                               

                              Custom Functions - need to be where they are executed.

                               

                              Security - previously mentioned by others

                               

                              Most of the annoyances are on the development side. The big benefit - being able to swap a UI file has never really been that big a benefit to me. It is to others. Most of the time I find that if I'm changing something on the UI side, I've probably had to change something on the data side as well.

                               

                              ExecuteSQL can make things easier in some respects - fewer relationships. But you need to know what you're doing with SQL queries which can be a pain to debug. You're also going to have to get pretty handy with parsing returned data and the virtual list technique. As Josh stated - it really depends on the system. Some will benefit from it, others will not.

                              • 12. Re: Limitations of using the Data Separation Model in FileMaker 13?
                                EdwardMcPikeJr

                                I used the Separation Model for the largest solution I deployed.  I have one Interface file and numerous data files organized by function - Core (projects/contacts), Financials (transactions), Timecards, Admin, Logs, Archives, etc.  I even separated a file for ESS, as we have a lot of connections to Oracle (PeopleSoft) that are easier to handle if they are isolated.  That file runs most of the server-side scripts that update data to/from PeopleSoft.  Separating the data files from the interface has made things much easier.

                                 

                                As others have said, updates have been easier at times, if it's just the interface file, but often there has been a lot of schema to update on the data side, making it more complex.  However, I built in data update scripts that made this fairly easy.  Just had to check the import mapping on each time to make sure newly added fields didn't disrupt the mapping.

                                 

                                If you're going to have many different files, I highly recommend using Active Directory for External Authentication.  When I first built the solution (FileMaker 8 or so), the client was not using AD.  I had to build a huge (huge in the background, not for the user) User Admin interface that allowed the database administrator to add/remove users.  When doing so, the user would first be added/activated/removed/deactivated from the Interface file, but the script would then have to call on its "sister" script in each of the data files as well.  What was nice was it was fairly seamless to the end user.  They just clicked a button when ready to activate/deactivate and it was done in all of the files.

                                 

                                Once the client moved to AD, security was MUCH easier to handle.

                                 

                                Good luck with your solution rewrite!

                                • 13. Re: Limitations of using the Data Separation Model in FileMaker 13?
                                  easyaspi

                                  What you give up using TSM:

                                   

                                  Running a script with Full Access in the interface has no effect, you must go to the data file and run it from there if you want the script to run with full access.

                                   

                                  There are some cases in which related values will not refresh, but I can't remember specifically what they are.

                                   

                                  Other than those two things, I can't think of anything you cannot do with TSM.

                                  The question of whether it is an appropriate choice is another matter.

                                   

                                  And, as other have pointed out, there are no changes in how FM 13 handles relationships.

                                   

                                  Mark

                                  • 14. Re: Limitations of using the Data Separation Model in FileMaker 13?

                                    You can create a data separated file at any time so it is not an either or question. I did this before the Internet and any support list. Start with a new file and link it to a non-separated file. Create numerous separated files.

                                     

                                    I Found it quite convenient to design a file for the owner and one for the bookkeeper and one for the warehouse All of which connected to the main file on the server.

                                     

                                    The Big Advantage...I didn't have to worry about shared passwords being used secretly. I could open the file on the server or the separate one on a computer.

                                     

                                    FIlemaker finally added the checkbox to prevent ANYONE from doing this.

                                    1 2 3 Previous Next