1 2 3 4 Previous Next 52 Replies Latest reply on Mar 5, 2015 12:21 PM by mbk28

    Data Separation. How many people use it and why?


      I am taking a serious look at making this the standard way for me to develop most solutions from now as the relationships will not change much if ever, but the UI will change frequently in the future as things like iOS change and general design improvements occur. This should also allow me to easily adapt standard data structures to unique Layouts for different uses/clients.


      I have seen some people say it is a bad idea, but they never really have a solid reason for it.


      So far as I can tell if I do not mess with the relationships and do not change the file names this should work just fine. The data file may have tables and fields added in the future but nothing changed or removed. I am assuming this works well with ODBC just the same as it does in one file.


      One of my main solutions is very image heavy and I was considering moving all the images into a third file. Is there any drawback to 3-4 file solution? Any advantage?


      How does data separation affect speed of the solution?

        • 1. Re: Data Separation. How many people use it and why?


          we never started with that, since most of our updates include new fields in the tables. Plus perhaps new content in preference fields. This destroys the overall idea of the separation model.

          • 2. Re: Data Separation. How many people use it and why?

            Before Filemaker 7 I have worked on solutions with 20 databases/tables (at that time only one table per database).  Those solution where also using the separation model.  The most work I think is programming all the finds a user can think of.  In a direct model he can just do a find and do whatever he wants.  Those solutions where still fast, only opening takes a little longer.


            In current solutions I do use the separation, although most of the time the tables are all in a single database file.  All data tables are prefixed with data_ and the other tables are interface/utility tables.  Either using relationships or virtual tables.


            Todd Geist has some videos on the subject you might want to see.


            FileMaker Master Detail 2.0 Video - geist interactive




            With these techniques it's easy to use the data seperation model.  Just add to table-occurrences to external data tables.


            Once opened I don't get the impression there is much speed difference.  Speed comes more from not doing certain things that bog down filemaker search for the Design for Performance... Design: Performance


            Happy Coding!

            1 of 1 people found this helpful
            • 3. Re: Data Separation. How many people use it and why?

              The downsides are managing two relationship graphs ( you still need a graph on the data side for auto-enter calcs) and managing user accounts (if you use FM's own authentication, you have to manage privileges in both).


              Having images in the data file slows backups, but shouldn't really affect performance. I always use external storage though, and not a file itself because of the backup issue. And often use SuperContainer.


              I don't know about OBDC.


              I've only use data separation for "converted" solutions with a "rollout". Where I was taking over for someone else and the clients wanted to keep using the old solution. So I would make a new interface file using a whole new second relationship graph in the data file and shut down layouts in the original file as I replaced them.


              But I would certainly consider it if I were in your situation. I would also consider it if I were giving access to "external parties" say a solution used by an organization that wanted to open some of their data to the clients via iOS. That seems a prudent security measure.


              I've heard reports that data sep doesn't play well with Web Direct, but I haven't done bother in the same solution so I don't know.

              • 4. Re: Data Separation. How many people use it and why?


                - currently is a partial separation.

                - my dream is total separation, to perform maintenance (total) of applications offline -> just would send files (updates), goodbye to sending files (bidirectional) (risk of desynchronization data), no more visits to the customer (by maintenances).


                The total separation It ought to be accompanied by script steps:

                - Create Table

                - Create Occurrence

                - Create Field

                - Create Relationship

                - Update Field Calculation

                - remove Table

                - remove Occurrence

                - remove Field






                • 5. Re: Data Separation. How many people use it and why?

                  what do you mean by ODBC? if you are SHARING with FMS and using ODBC/JDBC to access the data, then separation is entirely SQL-like! I don't rely on scripts or relationships or calcs when using xDBC. Your FM interface is not what you want when making queries for sharing data whether with xDBC or XML, for the most part. You want the raw data as much a possilble.


                  I see this as the advantage of a separation model in FM: you get the best of both worlds - FMS/FMP for networked from the "interface" file(s) and pure data for connections to the rest of the world from the "data" file(s).


                  Yes, there are caveats, but you learn to work around them.

                  • 6. Re: Data Separation. How many people use it and why?

                    I would caution against getting carried away with "standard."


                    Consider the problems you face with each project. Among those problems, does data separation offer an applicable solution? For the next project you plan, are there natural benefits to separating different user workflows from the source of the data? It sounds like you're building individual apps to solve different types of problems for your users, in which case, yeah go for it. But if you find you're developing a tool that serves relatively stable needs for a small set of users, you might deliver more value in an integrated solution than in a data-separated model that is more complicated to manage and maintain, particularly if you end up with just one "UI file" and just one data file, in which case they may as well be one and the same.


                    We use data separation where I am to compartmentalize different users, workflows, and developers. Each set of tools has different constraints and thus fits naturally into separate files. For example, our warehouses use lightweight iPad-oriented layouts and themes, a PIN-based user authentication scheme, and specific tables nobody else really uses, while our tech service team uses large, data-dense layouts, LDAP-authenticated users with single-sign-on, and tables of their own. I assign developers to these projects separately. Work proceeds in each of these areas without disturbing the others. And they all refer back to a central database of customer and production data. It all works well because there is a natural taxonomy to the workflows and tools we provide to the user.


                    But we have a few problems that emerge. Fragmentation comes to mind, with different files being at different versions of a theme. The different habits of different developers also show up clearly in the different layouts of the relationship graphs. There is some slippage in coding conventions between the different tools. They're becoming their own little worlds. It's fine with me, as long as the tools work, but if I were ever to want to bring things over from one file to another, it's going to be a lot of work. We've already merged two separate files into one. I'm not sure "merged" is the right word to use. More like re-implemented. With no way to import layouts in FileMaker, the amount of work required was about the same as starting from scratch.


                    The point is, remember to let the work call for the tools. Data separation is going to be more advantageous more often than not for a lot of business-scale projects, but remember to consider the advantages of integration as well. Think about what you're trying to do, and let the goals define the parameters.

                    • 7. Re: Data Separation. How many people use it and why?

                      Hi bigtom


                      I think you should try the separation model. One of two things will happen: 1) you'll hate it. 2) You won't .


                      After a decade using separation I finally hated it enough to never come back, so your mileage may vary...


                      just my two cents of mexican pesos.

                      • 8. Re: Data Separation. How many people use it and why?

                        Before v7 every solution was "total separation", if you had more than

                        one table you had to have a separate file.


                        I like working because it means that you don't have a monolithic data

                        source attached to the UI. In fact, you don't need to have a single data

                        source, you can have as many different data sources as you like.


                        It's a great way to modularise your solutions. You can provide different

                        UI files to different departments. Some of these department files can

                        contain unique tables which belong to them. Other data can come from the

                        shared data files. You can mix and match to suit your needs.


                        I've found it is a fantastic when we have been upgrading or rewriting

                        existing solutions. Using the data separation model the developer can

                        build the UI against a reference copy of the data. When it's ready, you

                        only need to modify the external data sources and the UI can be loaded

                        and used for beta testing and user acceptance testing without the risks

                        associated with tearing down the original to load the new.



                        • 9. Re: Data Separation. How many people use it and why?

                          Thanks. I love Mexico by the way. I have spent a lot of time in Baja. Definitely a retirement destination for me.

                          • 10. Re: Data Separation. How many people use it and why?

                            I appreciate all the input. It is what I was looking for. I will have a go at putting together a test solution together. The thing that seems confusing at the moment is that certain scripting needs to be in certain files. I am sure I will wrap my head around it once I get into it.

                            • 11. Re: Data Separation. How many people use it and why?

                              What scripting? In the data file? Nope.

                              • 12. Re: Data Separation. How many people use it and why?

                                Your mileage (kilometers?) may vary! I work in "total separation" with web sites. The UI is the web application and browser working on the raw data. Malcom is wrong that separate files per table pre v7 means separation. Any "separation model/method" should be total separation of data and UI reqardless of the number of files and/or tables. But Malcom is correct that you can have multiple UI files based on needs, pointing to the same DATA file(s).


                                Bruce asked, "What scripting?" you shouldn't need the *same* scripting in the separation of UI and data, but you may have some scripting that is *different* in the separation of UI and data. When importing/exporting I may have these scripts in the DATA file only, for example, but not in both locations.


                                Mostly, I strive to have NO UI in the data file(s).

                                • 13. Re: Data Separation. How many people use it and why?

                                  I have scripts in the data file.  They prepare stuff for export and import.  And the possibility to update a solution with only last years records for example.

                                  • 14. Re: Data Separation. How many people use it and why?
                                    Benjamin Fehr



                                    I recommend: build up relationships in GUI-File similar to those in the data file for all relations needed for scripts.

                                    Rebuild the scripts from data file in the GUI file.


                                    There's a trick I use to perform script on a "hidden" layout when needed. Just set parameters for "New Window" for coordinates to negative values at least for one of them. For example:

                                    - height= 5px

                                    - width = 5px

                                    - top = 0

                                    - left = -50px

                                    1 2 3 4 Previous Next