9 Replies Latest reply on Apr 28, 2016 10:51 AM by CICT

    Context in a data seperation model

    mchancevet@gmail.com

      Hello,

      i'm wondering, in a data separation model , tables and fields are defined in the data file.

      Business rules and scripts and layouts are defined in the UI file.

      How are contextual field calculations that refer to specific TO's handled or defined.

      Do you need an intermediary table in the UI  file to place all the contextual fields in and relate this to the TO of the Data table?

       

      Thanks,

      Morgan

        • 1. Re: Context in a data seperation model
          Mike_Mitchell

          Remember that each file has its own Relationships Graph. Calculations in fields are contextualized based on the Graph for the file in which the table appears.

           

          If you have layout objects that use calculations (e.g., Conditional Formatting, Hide Object), then those are contextualized based on ... the file in which they appear. (I'm sensing a pattern here.)  

           

          Basically, calculations leverage the Graph for the file in which they are resolving. You don't need any additional files.

           

          HTH

           

          Mike

          • 2. Re: Context in a data seperation model
            mchancevet@gmail.com

            Hi Mike,

            Thanks for the reply.

            I think my thinking might be a bit fuzzy on this.

            I'm contemplating Data Separation as I anticipate a need to provide an updatable framework in  an ongoing 24/7 production system. New data inputs are entered directly to the core Data file via a PHP file interacting with a website.

            SO even if there is no UI, the system can receive order data without interruption and a UI update can be uploaded as needed.

            At least that's my theory.

             

            The following is not an issue yet (early dev stage). But I'd like to get my head around a solution before I create the problem.

             

            Almost all of the 'context' in the system will be defined in the graph (anchor-bouy or connector -selector etc) of the UI file. The tables in core 'Data' file will be external TOs in this file.

            Layout objects, script and functions in the UI will be fine as these can reference TOs in the UI. However, the core 'Data' fields are only defined in the 'Data' file and these have no reference to the UI file within the 'Data' file. I can't define calculation fields in an external file's Tables from within the UI which is where the context is derived. And when I define them within 'Data' file, TOs of the same table in UI file are not visible or selectable in the 'Evaluate this calculation from the context of.......' list of the calculation options dialogue within the 'manage database' tool.

             

            SO I'm thinking an intermediate table in UI related to the TO of the external 'Data' table and containing any 'context essential' calculation fields. - Potentially a weak link in an update scenario? as these will be disengaged while UI off-line I suppose.

             

            Cheers,

            Morgan

            • 3. Re: Context in a data seperation model
              Mike_Mitchell

              mchancevet@gmail.com wrote:

               

              Almost all of the 'context' in the system will be defined in the graph (anchor-bouy or connector -selector etc) of the UI file. The tables in core 'Data' file will be external TOs in this file.

               

              Yes, that's the normal arrangement.

               

              However, the core 'Data' fields are only defined in the 'Data' file and these have no reference to the UI file within the 'Data' file.

               

              Yes, this is also normal.

               

              I can't define calculation fields in an external file's Tables from within the UI ...

               

              Correct.

               

              ...which is where the context is derived.

               

              No. The context for the calculation in a field definition is the context of that file's Graph.

               

              And when I define them within 'Data' file, TOs of the same table in UI file are not visible or selectable in the 'Evaluate this calculation from the context of.......' list of the calculation options dialogue within the 'manage database' tool.

               

              Exactly. Therefore, each file must have its own Graph, including all necessary relationships for it to function - which would include any TOs / relationships you need for the calculations.

               

              You cannot leverage the Graph of the UI file from the data file. You have to duplicate whatever level of setup you need for the data file to work. No file can address the Graph in another file.

               

              SO I'm thinking an intermediate table in UI related to the TO of the external 'Data' table and containing any 'context essential' calculation fields. - Potentially a weak link in an update scenario? as these will be disengaged while UI off-line I suppose.

               

               

              Unnecessary, and a very bad idea from a performance and update standpoint. Just build your data file Graph to support what you need to do in that file.

              • 4. Re: Context in a data seperation model
                BruceRobertson

                And generally the data file should have NO external file references. UI files point to the data file but it knows nothing about external files.

                • 5. Re: Context in a data seperation model
                  mchancevet@gmail.com

                  Thanks Mike,

                  Very good and valuable advice. I'm glad I asked BEFORE I started building it.

                  I'll build all the contextual machinery I need in the data file.

                  This has been a productive discussion for me.

                  Morgan

                  • 6. Re: Context in a data seperation model
                    CICT

                    Hi Morgan

                     

                    We're in a similar situation with systems used 24x7 and have been using the data separation model for some years to allow a 'drop in' update (also allows the 'roll back' should it be needed).

                     

                    Any field or table additions do mean adding to the live data files, but that isn't usually a problem, with copies of these being returned to the development server for further development of the UI file. On the rare occasion we do slip in an additional field simultaneously on both live and development data files, we have a FieldID check in the UI field to ensure these match before continuing development.

                     

                    Although we may use calculations within a table, or use an auto enter data/auto calc, we try to ensure as little business intelligence goes into the data files as possible. Cross table relationships on the data files are avoided as much as is possible.


                    Therefore on a classic orders with order line items type system we would have all totalling of values carried out by OnObjectModify, OnObjectSave, etc. script triggers. So entering a quantity or amending a unit rate would trigger a script that sets the line total field using the context of the UI file. This script may in turn call a subscript that updates the actual totals for the whole order, tax, discount, etc, again using the context of Orders2OrderLineItems context from the UI file.

                     

                    This does create other problems, and is one of the reasons we've been requesting an OnRecordExit trigger for a while, and other scripted routines can end up calling these script triggers, but currently this can be overcome using global values to define whether a script runs or not (until we get a 'bypass script triggers' script step, hopefully sometime in the future).

                     

                    Another advantage of this approach against traditional calculation fields is that you can setup 2-way calculations for the users, such as allowing a discount of 10%, calculating this value and deducting it across the totals, or enter £50 discount and calculating the percentage and again calculating this across the totals. This provides a very flexible system.

                     

                    There is more work to this approach, but it is nearer to a true separation model than having to maintain relationships within the data files. We have built one solution, as a proof of concept, where it will run with FileMaker files as external data sources or SQL files as external data sources - recreating a subsummary equivalent was an interesting challenge and not one I'd recommend for large data sets. However, in this case, all business intelligence was in the UI file only.

                     

                    I hope this provides some food for thought.

                     

                    Andy

                    • 7. Re: Context in a data seperation model
                      mchancevet@gmail.com

                      HI Andy,

                      Thanks for your thoughts on this.

                      AT this point I'm up against it time wise so I'll probably stick to a minimalist set of logic in the Data file. I hope you get your way re script triggers. It is excruciating that you can't bypass them explicitly in scripts.

                      Could you elaborate on your FieldID check if you have time?

                      Morgan

                      • 8. Re: Context in a data seperation model
                        coherentkris

                        Minimal schema / logic in the data file is one of the main ideas around the sep model.

                        • 9. Re: Context in a data seperation model
                          CICT

                          Hi Morgan

                           

                          The field ID check is just a layout (from any context) in the UI file that uses the FieldIDs ( Get (FileName) ; Get (LayoutName)) function, usually triggered via a button script and displayed in a show custom dialogue box. If we do have to add a field to both the live and the development versions of the data files, we can drag this field into the layout on both UI files and run the script, which will confirm whether the field IDs are the same.

                           

                          This avoids any nasty surprises later when you upgrade the live system only to find that scripts in the updated UI are not recognising the new field(s) as field creation got out of sync. As mentioned, we normally upgrade the data files outside of working hours and transfer them to the development version to continue updating the UI file, but there are times..........!

                           

                          All the best

                          Andy