6 Replies Latest reply on Aug 23, 2015 9:09 AM by quirkycrone

    Methods to update a DB containing externally stored container on a Server


      Hello everyone,


      I want to open a thread here to discuss the best methods we each use to update databases on FMS which use Container Fields stored externally.


      When we want to create a new version of a "live" database on the server, we take the copy of the file, do our development locally, but what is the best method to push that newly developped database to the server AND integrate the data of the "live database" in this new version.


      One version we wanted to avoid was to :

      - on the "live" database put back all the container fields to "local storage"

      - download ALL the database and its container (either locally on the server or to another machine)

      - do the importation from database.fmp12 to database_NEW.fmp12

      - upload database_NEW.fmp12 (after renaming it to database.fmp12)

      - put back all the container fields to "external storage"


      We found it kind of nasty.



      The database being stored on the server, one solution we use is to :

      - upload the new database on the server (it is empty for the moment) under the name database_NEW.fmp12

      - run a script (on a FileMaker Pro running on the server) doing the importation table per table of all the data from database.fmp12

      - close database.fmp12 and remove it

      - close database_NEW.fmp12

      - rename database_NEW.fmp12 to database.fmp12

      - rename the "RC_Data[...]" folder in order to match the name of the database

      - reopen the database on the server



      This method works, but maybe some of you may have better solutions to do that.


      Thank you in advance for your comments.


      Best regards.

        • 1. Re: Methods to update a DB containing externally stored container on a Server

          Hi Benoit,


          Thanks for telling us your way of doing this.


          I have the same challenge with my solutions, and would also love to hear about an easier solution.


          Best regards

          • 2. Re: Methods to update a DB containing externally stored container on a Server

            We use RefreshFM since we are not only constantly modifying tables and fields, but have over 160 base tables in our data file.  Also, since we update anywhere from a dozen to hundreds of clients simultaneously every few weeks, RefreshFM allows us to do all of this in a batch process.


            Haven't had any issues so far with externally stored container fields.  We just have to make sure they are present with the data file while RefreshFM works.

            • 3. Re: Methods to update a DB containing externally stored container on a Server

              Thanks for explaining your methodology. I must say - I've been experiencing numerous challenges getting this to work properly.


              Oddly I still seem to get some images that display "Tampered ..." int he image placeholder. This seems to be related to the renaming of the RC_Data folder.


              I have only recently started using FMServer13 and am surprised that there is no way to remove and/or rename databases from within the Server Admin tool. Perhpas I'm missing something.

              • 4. Re: Methods to update a DB containing externally stored container on a Server

                I'm glad someone has opened this thread, as we don't believe FileMaker has got this right - yet!


                Our CRM system records all incoming/outgoing email attachments in externally stored container fields, so removing a file to add an updated version could require Gbs of data transfer. We've minimised the impact of this using the separation model, so most of the time we're only replacing the interface file, which doesn't contain any data, hence no files to transfer.


                However, many of our cloud customers do use single file solutions and the (disliked) procedure we use at the moment is to close and remove the database, which also removes the RC_Data_FMS folder (even if the separate Container folder option is used) to the Removed_by_FMS folder. We then upload the new file and open it, then close it again and replace the RC_Data_FMS folder with that previously removed. Reopening the database again has to date always worked.


                However, our cloud servers are all Windows 2008 R2 or Windows 2012 servers. I would assume that if you followed the above procedures on Mac OS X then after the potentially long data transfer the permissions would need to be sorted using chown -R, chgrp -R, etc. Again really slowing down the process.


                I believe FileMaker's advice is to save a copy as a self-contained copy, which brings all the external data back into the file's container field. This has to be carried out when the file is not hosted, as you can't save a copy when the file is running in FileMaker Server. I would not want to see the file size of some of our systems if we did this!


                We strongly believe FileMaker should provide an option to leave the RC_Data_FMS folder in place, albeit with a BIG warning, as I'm sure they would argue the risk of data corruption would be high. Perhaps they could provide a validation tool within FileMaker to check the links to the external files?


                As an aside, we're not including the external container files within the Filemaker backups. The storage overheads of keeping multiple copies of these is just too high. Therefore we're using the standard FMS backup for all data files and then using a third party application to duplicate the container folder (not located within the database folder) so only the most current contents are retained. Again, we believe FileMaker should offer an 'incremental' backup for external container files and will be requesting this on their 'feature request' web page.


                Please also note that the majority of container fields we're managing are open, not secure.


                FileMaker 13 has made good strides with containers compared to 12 and hopefully things will continue to get better with future releases.





                • 5. Re: Methods to update a DB containing externally stored container on a Server

                  If possible I would use a separate database with only the (externally stored) containers. Since this file only has the containers

                  there is no need to update this file, so we can leave this in place with the update process.


                  Hope that helps,


                  Best regards,


                  Ruben van den Boogaard

                  Infomatics Software


                  • 6. Re: Methods to update a DB containing externally stored container on a Server



                    This sounds like a great idea for my application, Because my images don't change much over time, just a handful of new images over the course of a year.


                    I'm going to test this out and see how it works.