1 2 3 Previous Next 69 Replies Latest reply on May 17, 2016 2:39 AM by beverly

    How do people preserve data content when developing?

    douglerner

      There doesn't seem to be any good way of separating the data from the schema (layouts, scripts, connections, etc.). So how on earth do people keep things straight to prevent overwriting data changes made by the end customer?

       

      For example, in our situation, we have:

       

      1. My client's database hosted at FMPHost.

      2. A developer working on bug fixes, changes and enhancements on his dev server elsewhere.

       

      So far he's been making changes, letting me download the new database at different stages for testing, and I've been replacing the database at FMPHost with new database versions from him.

       

      But now we are at a point where my client wants to actually start using the database, and adding new records,  and/or changing existing records. Yet development of new features, and bug fixes to existing features are ongoing. And since people have new ideas all the time, adjustments will be ongoing for the foreseeable future.

       

      How do people deal with that? If my client adds new records what's a best practice to make sure the developer can add his updates without me losing the changes in records in the database.

       

      Thanks,

      doug

        • 1. Re: How do people preserve data content when developing?
          bigtom

          Basic data separation with two files helps. I have worked on databases only during off hours of client use. You should have some import plan in place for any new records added.

          • 2. Re: How do people preserve data content when developing?
            wimdecorte

            bigtom wrote:

             

            Basic data separation with two files helps.

             

            Only to the point where you don't have to make changes to the data files.

            • 3. Re: How do people preserve data content when developing?
              wimdecorte

              In essence you have two choices:

               

              - replace their production copy with your dev copy and import all the data

              - or redo all the changes from your dev environment to their live environment

               

              We do both depending on how many changes there are.  Both approaches carry risks.

              • 4. Re: How do people preserve data content when developing?
                schamblee

                I prefer the replace production then import data because testing has been done on the file.  If you make changes on a live database then the end user does the testing.   In either case I would make extra backup before making any changes.  

                • 5. Re: How do people preserve data content when developing?
                  douglerner

                  Replace production and import the data again is not really feasible. It requires too much keeping track of imported data files. And also ignores the fact that people might edit records between versions rather than just importing them.

                   

                  It certainly seems like FM is awkward for development in this regard. Hmm...

                   

                  doug

                  • 6. Re: How do people preserve data content when developing?
                    wimdecorte

                    douglerner wrote:

                     

                    Replace production and import the data again is not really feasible. It requires too much keeping track of imported data files. And also ignores the fact that people might edit records between versions rather than just importing them.

                     

                    It certainly seems like FM is awkward for development in this regard. Hmm...

                     

                    doug

                     

                    I don't follow your reasoning about it not being feasible to import data.  It's a fairly standard approach.  What data files would you need to keep track of?

                    Every go-live would obviously have a deployment plan that has a checklist of all files/tables that need to be import, with accompanying checks to very the record count and serial #s after import.  None of this is rocket science.  It's just something that needs to be done carefully.

                    And there are tools to help you (RefreshFM for instance).

                     

                    Redoing all the code changes works too but has to be done equally diligently. And here too there are tools that can help you diff the code-base (fmDiff, BaseElements, Inspector Pro,...).

                     

                    If you don't make may table/field changes then perhaps the data separation model will help you out.

                    • 7. Re: How do people preserve data content when developing?
                      DavidJondreau

                      It's actually very feasible and used by many professional developers in complicated settings. The issues that need to be dealt with are: it can be complicated to set up, you need to keep track of privilege account changes, and the system needs to be "down" during the import (no editing).

                      • 8. Re: How do people preserve data content when developing?
                        schamblee

                        I would not upgrade the database while users are using the system.   Imports can be scripted so there is no keeping track of data files and the update process would be much faster.   Your import script can be tested on a test files before installing on the live server.  

                         

                        So if you make changes to a live database and you modify a script and then a user run the script you are modifying what happens then?   Users would need to be out of the system while modifying a live database, well if you are modifying scripts.  Modification to current layouts, I could see being done on a live database, but in most cases I would not. 

                        • 9. Re: How do people preserve data content when developing?
                          alquimby

                          If you are using FMPHost's FileMaker Server hosting, your developer should be able work on the schema aspects of your file directly without touching the data. That is one of the many advantages to using Filemaker Server to host databases. (Of course, he should preferably work on/in the file when none of you are accessing it.)

                           

                          Al Quimby

                          • 10. Re: How do people preserve data content when developing?
                            wimdecorte

                            schamblee wrote:

                             

                            So if you make changes to a live database and you modify a script and then a user run the script you are modifying what happens then?   

                             

                            Making changes to a live system carries a boatload of potential issues.  Those can all be accounted for by diligent error trapping and handling (see all errors in the 300 category: FileMaker Pro error codes)

                            But it does take a LOT of development effort to catch and handle them.  Most developers don't; which makes live changes extremely dangerous.

                            • 11. Re: How do people preserve data content when developing?
                              schamblee

                              Yes, I agree and that was the point I was trying to make.  I would rather have a polished import system which also take effort.  I add error trapping but I doubt it would account for someone running the script while it is being modified. 

                              .

                              • 12. Re: How do people preserve data content when developing?
                                wimdecorte

                                schamblee wrote:

                                 

                                I add error trapping but I doubt it would account for someone running the script while it is being modified. 

                                .

                                 

                                The point is that you pick a date and time to do this that *nobody* is in the system.  That's an accepted way of doing things aka "maintenance window".

                                 

                                I mentioned a deployment plan earlier.  One of the steps in it is to notify users of what the downtime is and let them know they can't use the system during that time.

                                When that time comes you change the setting on FMS to only only your connection so that nobody can work on the system until you are done; or you shut down FMS altogether.

                                • 13. Re: How do people preserve data content when developing?
                                  wimdecorte

                                  douglerner wrote:

                                   

                                  It certainly seems like FM is awkward for development in this regard. Hmm...

                                   

                                   

                                  Let me turn the question around; in an ideal world: how do you see this going?

                                  • 14. Re: How do people preserve data content when developing?
                                    schamblee

                                       You stated

                                     

                                    Making changes to a live system carries a boatload of potential issues.  Those can all be accounted for by diligent error trapping and handling

                                    then you stated

                                    The point is that you pick a date and time to do this that *nobody* is in the system.  

                                     

                                    Did you read any of my post?    You repeat what I have already stated.

                                    I said I prefer to replace the database then import data into the new databases over modifying a live database.  I pointed out one of the flaws of modifying a live database and then you stated that could be handled by error trapping then you go back and say that no one should be in the database.  My definition of a live databases is a database that is being used.  

                                    1 2 3 Previous Next