1 2 3 Previous Next 77 Replies Latest reply on Feb 14, 2014 2:54 AM by NickLightbody

    To use Data Separation Model or Not

    arthursc

      I have had an interesting conversation where we questioned the use of data separation.

       

      So my question is thus;

       

      Should I be considering data separation?

       

      My application will always be stand alone. Almost certainly updates will include changes to the UI and NOT the Schema.

       

      Why would I want to use DS model rather than single file model.

       

      Updates will always involve export and import of data, and I agree if it was just a UI update then DS maybe easier.

       

      So what would you consider using and why?

       

      Regards

      ArthursC

       

      Message was edited by: arthursc

        • 1. Re: To use Data Separation Model or Not
          beverly

          ArthursC,

          It depends on:

                

          • how often changes are made (to UI or data files)

                

          • how many records to import/export

                

          • are your relationship key independent of auto-enter serial (which can munge with repeated export/import - if you're not careful

                

          • does this "standalone" mean use of run-time?

           

          Beverly

          • 2. Re: To use Data Separation Model or Not
            ChrisG

            Hi ArthursC

             

            I use the data separation model on all my projects now. It is a bit more work to build but I find it has many advantages. It is also possible to build the data side of your solution with almost no scripts or calculation fields. I say almost because the only scripts I use in the data side are startup, shutdown and accounts scripts. With FM 11 you can do so much with conditional formatting and the use of merge variables to make your interface look good. Previously a lot of calculation fields were used for display purposes only.

             

            The reason I use this model is: I design different interfaces. There is one for the desktop, there is one for ipad, another one for iphone. Sometimes you want to design one for IWP. Then there are management interfaces. In my experience, once you have built a nice solution for a client they invariably get more and more ideas about data and reports they can extract from the system. This is a good thing, it is one of the main reasons we have DB's. What happens in practice is that you start to build a whole lot of features (bloat ware) which are only used rarely by management or accountants etc. For this relatively "light" usage I do not want to pollute the operational system. Another great benifit of this approach is that I never have to take the operational system down when they request more features. Data Separation is not such a big deal. We all did something like it pre FM 7.

             

            Cheers

            Chris

            • 3. Re: To use Data Separation Model or Not
              RayCologon

              Hi ArthursC,

               

              I think Beverly has summed up a number of the issues well.

               

              One of the chief benefits of data separation is that if updates involve changes to UI much more frequently than to schema, you gain some speed and simplicity for a proportion of updates. But there are some costs that offset those benefits.

               

              In your case, since you've indicated that updates will generally involve schema changes - and therefore a migration of data, the benefits of data separation will be few and most likely outweighed by the various (relatively minor) inconveniences.

               

              A further consideration is that it is a relatively straightforward matter to move from a unified file architecture to a separated one (but not quite so easy to go in the other direction) and you are probably better off commencing development with a single file solution, and only switching to a data separation if (at some point down the track) you see clear benefits of doing so.

               

              Regards,

              Ray

              ------------------------------------------------

              R J Cologon, Ph.D.

              FileMaker Certified Developer

              Author, FileMaker Pro 10 Bible

              NightWing Enterprises, Melbourne, Australia

              http://www.nightwingenterprises.com

              ------------------------------------------------

              • 4. Re: To use Data Separation Model or Not
                LyndsayHowarth

                Ray Cologon wrote:

                 

                A further consideration is that it is a relatively straightforward matter to move from a unified file architecture to a separated one (but not quite so easy to go in the other direction) and you are probably better off commencing development with a single file solution, and only switching to a data separation if (at some point down the track) you see clear benefits of doing so.

                 

                This is really good advice, Ray. Too many hours spend re-integrating pre-7 systems teaches you that in a big way!

                 

                I like data-separation when it is done well. The difficulty is keeping calculation and relationships to a minimum in the data file. With redundancy built in to the data file you can swap the interface file and maybe rename a field and 'bob's your uncle' you have another function with no need for any import or export. Tricky naming of local and remote tables and associated layouts works wonders...

                 

                Done badly... data-separation can become a mess very easily.

                 

                - Lyndsay

                • 5. Re: To use Data Separation Model or Not
                  beverly

                  Amen Lyndsay!

                   

                  I've not had to "re-integrate", so don't know what that entails.

                   

                  I also like the separation, as does ChrisG, because I can have different interfaces. Mostly for a browser-based (with PHP or other web application) interface - I let the web do the joins (relationships), calculations, queries (scripts) and other necessary actions. If I can keep the data "near pure", it's an advantage over the web.

                   

                  Beverly

                  • 6. Re: To use Data Separation Model or Not
                    arthursc

                    Hi Ray,

                     

                    Bit of a typo, meant to say Few schema updates. more UI.

                    • 7. Re: To use Data Separation Model or Not
                      Mike_Mitchell

                      Arthur -

                       

                      I was all agog about the separation model when 7 first came out, and, since I was in the process of upgrading a large pre-7 application to 7 at the time, considered it an ideal opportunity to use the model. What I found was that the separation model is not a one-size-fits-all solution to everything. In the particular case where I was working, it ended up not saving me much - if anything - because the solution I was working with was organized around releases, every one of which involved schema changes. Hence, I ended up taking the entire system down every time there was an upgrade for data migration anyway. Not much in the way of savings.

                       

                      That said, this was during the version 7 days. Much has changed since then. We now have Conditional Formatting, merge variables, Script Triggers ... lots of ways to insert logic into the solution without involving the schema. This is by design. I have similar experiences to Chris - there always seems to be that manager who want some obscure report added ("can I have a report of all the records that were entered by employees over 6' tall wearing purple pants while standing on one foot under a full moon?"), and, in the version 7 days, these often involved schema changes. They may not now.

                       

                      All that to say: If you're willing to design carefully to push chunks of your business logic to the interface layer and avoid embedding things in the schema, separation model may work well for you. Otherwise, it's probably more trouble than it's worth. IHMO.

                       

                      Mike

                      • 8. Re: To use Data Separation Model or Not
                        RayCologon

                        arthursc wrote:

                         

                        Hi Ray,

                         

                        Bit of a typo, meant to say Few schema updates. more UI.

                         

                        Okay, gotcha.

                         

                        In that case, you need to decide how onerous the data migration exercise is (and how frequently updates are likely to occur), as compared to the moderate additional compexity and effort of building and maintaining in a separated solution.

                         

                        If the amount of data is (or will become) quite large, and you expect UI-only updates to be relatively frequent, then the advantages of separation may outweigh any disadvantages.

                         

                        It's a judgement call. As I said, you can start with a unified model and make the decision to separate later, but if you can see that going for separation will offer significant benefits, there's no harm at all in opting for that from the outset.

                         

                        Regards,

                        Ray

                        ------------------------------------------------

                        R J Cologon, Ph.D.

                        FileMaker Certified Developer

                        Author, FileMaker Pro 10 Bible

                        NightWing Enterprises, Melbourne, Australia

                        http://www.nightwingenterprises.com

                        ------------------------------------------------

                        • 9. Re: To use Data Separation Model or Not
                          KylePutzier

                          I use a hybrid model. Like most, I like the convenience of a single table, but hate the time spent updating. Many of my tables never accumulate that much data, so I frequently leave the data with the UI. For tables that grow records quickly (e.g. log files), the data is separated from the UI. I also compartmentalize my tables. Tables belonging to similar UI functions stay together in a common file. I'm usually only updating one part of the database at a time anyway. I usually end up with around 4...6 files for a moderately complex database. Works for me.

                           

                          Kyle

                          • 10. Re: To use Data Separation Model or Not
                            Stephen Huston

                            Hi ArthursC,

                             

                            I use Data Separation on all stand-alones.

                             

                            Planning for a few data schema changes is pretty straightforward. I build in some extra fields of each type (number, text, date) into the schema which are named "xtraText1, xtraText2, ect" for later use if needed. Those names make them easy to find when needed. Those "future version fields" simply don't appear until I update the User Interface to incorporate them when needed.

                             

                            All fields in the data file are non-calcs, with any calc-results set via scripting done by the interface file. That way even fields which appear to be data calcs can be updated via the interface change.

                             

                            Stephen Huston

                            • 11. Re: To use Data Separation Model or Not
                              jormond

                              +1 for Data Separation.  With where FileMaker is at with v11, I really don't see any drawbacks to going Data Separation.  Except one, it involves a little for thought to handle user Accounts.  But not all that difficult.

                               

                              Admittedly, there is a slight change in thought process that takes some getting used to.  But after working with a separate data table for a few weeks, it becomes second nature.  Plus, when you then go to work on a hosted solution and you need to limit the amount of data moving between the server and the client, you will already have the experience working with Data Separation that makes that an easier transistion.

                               

                              Having a 'data-only' data file can have speed benefits also when working over a WAN, and even a LAN.  I'm not against single file solutions, but I rarely work with standalone solutions anymore ( run-time or not ).

                              • 12. Re: To use Data Separation Model or Not

                                I too prefer Separation.  The first time that you have to do a full data migration on 75 tables with over a million records, when you simply created a report, is when you will wish that your data was separated.

                                 

                                Others have provided many good reasons for separation; I'll add one more - backups.  We keep all backups from the Data file (back through time) but there is no need to store incremental backups of the UI.  Master copies by version are kept by the Developers elsewhere.

                                 

                                Separation does not mean moving between the two files (like we had to do in versions < 7) because, with Data file referenced in UI, all layouts and scripts are handled in the UI (you never need to go to the other file at all).  The only thing I do not like about Separation is lack of refresh (in a few situations) without a nudge but that is very small price.

                                • 13. Re: To use Data Separation Model or Not
                                  arthursc

                                  I would just like to say how pleased I am that so many of you have responded.

                                   

                                  I am eager to add some additional questions and thoughts, but the wife is nagging me to watch a film with her, so I'll catch up later..

                                   

                                  Regards

                                  Colin

                                  • 14. Re: To use Data Separation Model or Not
                                    Mike_Mitchell

                                    Happy wife, happy life ...   

                                    1 2 3 Previous Next