12 Replies Latest reply on Feb 3, 2017 10:09 AM by cplinta@accelse.com

    Using Data Separation Model and am having Deployment Issues

    cplinta@accelse.com

      I have FM Pro 15 Advanced installed locally.

      I started to develop an app in a single database file.

      At some point i decided to adopt the data separation model.

      I now have two databases, one containing data (DATA),

      the other contains the layouts, scripts and such (CODE).

       

      My development process entails making changes to CODE locally,

      and then pushing those changes out to the server running FM Server 15

      by doing a file level copy/paste, replacing the remote CODE database file

      with my local CODE database file.

      I also have FM Pro 15 Advanced installed on that server.

       

      After splitting out the data into the DATA database file, i have added new

      fields to existing tables with no issues.  I've made a point to

      add the fields locally to DATA and to the remote (live) DATA database

      at the same time to avoid any issues.

       

      Here is the problem I'm now seeing...

      I added a new table to both databases (the table is named Logs).

      I made code changes to my local CODE database (a new script and layout).

      and when i pushed those changes out to the server, by coping and pasting the

      CODE database file, the new layout and script had the fields either changed

      to different fields in the same table or field refs were marked as missing.

       

      Thinking i created something out of order, I deleted the Log tables in

      both DATA databases and re-created them from scratch to ensure all

      fields were created the same and in the same order.

      I am seeing different but similar results.

       

      I then decided to delete the Log table from the DATA database being served.

      I copied my local DATA database to the server desktop.

      i opened this dev DATA database on the server and "copied" the Log table (Manage->Database->Tables->Copy).

      i opened the production DATA  database on the remote server and pasted the table into it (Manage->Database->Tables->Paste).

      Now, When i opened the CODE database on the server, all fields that referenced

      the Log table on the new layout and script show as missing.

      I also tried "Import" and see the same result (Manage->Database->Tables->Import)

       

      In the CODE database on the remote server, i did this

          File->Manage->Database

          Relationship tab

          and when i clicked on the Logs table reference, it had NO table selected.

          Once i selected the Logs table, all was well, the new script and layout now have

              the fields properly mapped/defined

         

      But, this is unacceptable because i dont want to have to remember to

      take this step every time i push out code changes...

      It also has me leery of adding new tables going forward.

       

      Its been years since i used Filemaker... I picked up a project recently to use it again,

      so im not up to date on the current approaches/tools to use.

       

      Thanks in advance for your help.

       

      Chuck

        • 1. Re: Using Data Separation Model and am having Deployment Issues
          TSPigeon

          cplinta@accelse.com:

           

          Thank you for your post!

           

          It seems the process you use for updating your databases is breaking a relationship or requiring the relationship to be setup between the newly uploaded file and the other existing file. I believe developers on this site could off some good advice on this.

           

          I'm going to move this thread from the FileMaker Community Feedback Space to the Discussions Space where others might find your post easier and offer further advice.

           

          TSPigeon

          FileMaker, Inc.

          • 2. Re: Using Data Separation Model and am having Deployment Issues
            philmodjunk

            FileMaker accesses data via internal IDs for table occurrences and fields, not names (though there are a few exceptions). Thus, defining new fields in two different copies of your data files might not produce fields with the same internal IDs Even though they have the same name.

            • 3. Re: Using Data Separation Model and am having Deployment Issues
              philmodjunk

              Try hosting your new CODE file on the server so that it accesses the same DATA file as does your regular users. When you need to change field/table definitions, take Data off the server briefly, make your changes and put it back up. Then update your new CODE file to reference the new stuff.

               

              It's possible to host the new copy of CODE so that you can open it while it remains hidden from your users.

              2 of 2 people found this helpful
              • 4. Re: Using Data Separation Model and am having Deployment Issues
                ESM

                Hi:

                 

                Having successfully managed a separation model solution for many years which was distributed across about fifty different machines, I will tell you what I worked out as an absolutely reliable method.

                 

                When I wanted to add new tables or fields to my solution, I always and only added those new tables and fields to the main server's data file. I also manually incremented a "version" field in that file's "system" table so the file itself, due to it's change in schema, became a new "version" of that file. This "data file version" was always exposed in the UI of the solution as a double-check.

                 

                As I did this, I meticulously logged each and every change to my data file, all recorded as having been made for this "version", in my schema change log file.

                 

                Then, I ran a backup of the new version of the data file on my main server and copied that backed up data file to my development server. Shut down my files on the development server,  replaces the existing data file (which is now an "old" version of the data file) on my development server with the backed up, new version copy I just pulled from my main server, and started the files back up on my development server.

                 

                All new development work was then done on the "UI" file on the development server, appropriately versioned in the UI file as well when it was ready for production (just like I versioned the data file when coming the other direction), and then the "UI" file was moved to the Production server using a similar methodology. The version of the UI file was, again, exposed in the UI as a double-check.

                 

                100% reliable and repeatable. I did this, quite literally, hundreds of times over many years. This process completely avoids the issue where the internal ids get mixed up between the different files yielding the problems you describe.

                 

                It takes structure and discipline but I recommend you use this method as it works 100%.

                1 of 1 people found this helpful
                • 5. Re: Using Data Separation Model and am having Deployment Issues
                  ESM

                  Ah yes, and by the way, I also recommend that you always develop on a server rather than locally. Setup a development server with a backup schedule set to backup at an interval that is the maximum amount of work that you are willing to lose. If the server ever crashes, go back to the last known good backup of your files and recreate the lost work. This is not possible when you develop locally in FileMaker .

                   

                  You should never, ever use a FileMaker file that has been improperly closed. File corruption after a "recovery" can lurk in these files and cause you enormous problems down the road. Not worth the risk.

                  • 6. Re: Using Data Separation Model and am having Deployment Issues
                    philmodjunk

                    This is not possible when you develop locally in FileMaker .

                     

                    Actually it is. You can set up an OnTimerScript with Save a Copy to save timestamped copies of your file while you develop in it. This sometimes causes a minor inconvenience when you have the script debugger up and the ONTimerScript pops up in it, but it's pretty minor.

                    • 7. Re: Using Data Separation Model and am having Deployment Issues
                      cplinta@accelse.com

                      ESM,

                      Thanks for the info... Im at the beginning of my development project, and by trial and error, i've come to the same conclusion that you recommended.  Thanks for your feedback as it verifys my approach as a sound one.  I didn't think about the backups, and your version ideas... great suggestions...

                      As i mentioned before it been years since i developed in filemaker... i was hoping the DB corruption issues had been solved and your comment in the next post does give me pause though...

                      • 8. Re: Using Data Separation Model and am having Deployment Issues
                        beverly

                        I work this way. The "interface" file is hosted for the advantages here. Phil's point of "hiding" this file can be used as well. I just have the Interface file hosted for those "in office" and the latest backup copy of it downloaded by the users working remotely. (Remember that all data is in other files on the server at all times.)

                         

                        This has worked well since FM10. You can have MORE than one reference to the data files from this Interface. A return-delimited list can be:

                        1) the local path (file:___) as the server interface file sees the data files relative to it

                        2) the remote path (fmnet:___) as the remote interface file would see the data files hosted

                         

                        If I have major work on the Interface file, I will revise the latest backup (no data, remember) on my local desktop and pointing to the remote data. I can then replace the previous version with no hiccups when all done. I like to take the server down to replace the remote Interface file, but that's me.

                        beverly

                        • 9. Re: Using Data Separation Model and am having Deployment Issues
                          BruceRobertson

                          "i was hoping the DB corruption issues had been solved"

                          What does that mean? AFAIK, corruption is almost always a result of bad practices: using Dropbox, directly opening a served file, etc.

                          • 10. Re: Using Data Separation Model and am having Deployment Issues
                            cplinta@accelse.com

                            Can you point me to info on "bad practices"?

                            im esp interested in know more about what u mean by "directly opening a served file... "

                            but need to be aware of other such "bad" things

                            • 11. Re: Using Data Separation Model and am having Deployment Issues
                              ESM

                              "directly opening a served file... "

                               

                              This means opening FileMaker Pro/FileMaker Pro Advanced locally on a server, selecting to "Open..." rather than "Open Remote..." and then browsing to the directory where the FileMaker files are actively/already being served on that server and attempting to open them. The files are already open by FileMaker Server and then you open them with the FileMaker client software.

                               

                              A corollary to this would be to mount a shared drive where the hosted files exist and then opening them with a local copy of FileMaker Pro directly on that machine... again through "Open..." rather than "Open Remote..." while they are simultaneously being served by FileMaker Server. This is a double bad practice as you should never "fileshare" volumes that host FileMaker files that are being served.

                               

                              The rule with files open in FileMaker Server is "never let any other software touch them besides FileMaker Server"... this includes either of the FileMaker client versions (FMPro/FMPro Advanced), virus scanning software, backup software, any sort of file sharing mechanism (system level, dropbox, box, etc.), etc.

                               

                              So for backups, always backup the backups created with FileMaker Server, not the actual served files.

                              • 12. Re: Using Data Separation Model and am having Deployment Issues
                                cplinta@accelse.com

                                ok, that explains my confusion... i didn't think you could do that.

                                so i tried it... and I see the following error message, which shows

                                you cant do that.