11 Replies Latest reply on Jun 12, 2013 4:02 PM by msbertrand

    FM Server ADV 12V4 - Progressive Backup

    msbertrand

      Good day,

       

      We attempted 2 already to enable progressive backup (new in FM12) yet experienced performance issues and it appears to be caused by this feature.

       

      Anyone has more details/information on this process, and/or had any issues relating to it?

       

      On our last attempt, one Filemaker file got corrupted and shut down the system until we loaded a backup file. This was a first ever, I cannot attibute this failure to progressive backup, yet this happend while it was running. Anyone excperience this issue?

       

      Regards,
      MSB

        • 1. Re: FM Server ADV 12V4 - Progressive Backup
          wimdecorte

          What are the specs of your server and what is the interval that you set for the progressive backups?

          • 2. Re: FM Server ADV 12V4 - Progressive Backup
            msbertrand

            Tks for the update,

             

            Windows 2008 R2 Standard

            IMB Intel Xeon CPU E31230 @ 3.20GHz

            Ram 16 GB

            64 Bit

             

            The interval was left at default, 5 mins

             

            Regards, MSB

            • 3. Re: FM Server ADV 12V4 - Progressive Backup
              wimdecorte

              How big is the solution and how much free disk space is there?  Is this a virtual machine or a physical server?  Anything strange about the disk arrangement?

               

              Any AV or VSS active on the drives?

              • 4. Re: FM Server ADV 12V4 - Progressive Backup
                msbertrand

                The solution has 19 files, 2 that includes images are around 10GB which is being looked at for archiving and to reduce them.  The entire solution is around 50GB

                • 5. Re: FM Server ADV 12V4 - Progressive Backup
                  wimdecorte

                  how much free disk space?

                  virtual or physical?

                  anything special about the disk arrangement?

                   

                  What do the data changes look like?  Many new containers updated with big files?

                  • 6. Re: FM Server ADV 12V4 - Progressive Backup
                    msbertrand

                    900 GB, 450 GB Free.  Physical, no virtual server and server is dedicated to FM database only.

                     

                    No special arrangement, and no other partition on the disk (I believe).  I am not an IT person, but this server is kept simple.

                     

                    3 files includes images;

                     

                    - One 'Master' file where 'all' images are stored

                    - One IWP file where some images are duplicated (Master image file is IWP switched off in privilege)

                    - One 'transfer' file that receives images from iOS, and transfer simultaneously to IWP File and Master Image file.

                     

                    The transfer process from iOS is every 5 minutes and average 30 images (signature located in container fields for the majority).  This process pushed to both Master Image file and IWP file.

                     

                    I am currently using a '2 hour' backup and disabled both Integrity check and Clone, but this still take up to 40 minutes currently.

                     

                    I now have updated the IWP to keep only 90 days of images, and the iOS transfer file to keep only 30 days.  This will bring my server down to around 700 GB of free space once the backup copies are updated. 

                     

                    I also understand the 'hourly' backup are also progressive.  All my FM Database files are located on the same folder, but apparently they could be split among different folders and I could use selective backups, and backup folders instead either 'all' database, or only one file.  Am I correct to think this will not break any relationship?

                     

                    Finally, I was told to store my images differently and set my container to store a reference, not the image so it lighten my database.  Would this affect performance?

                     

                    Regards, and tks for the help.

                     

                    MSB

                    • 7. Re: FM Server ADV 12V4 - Progressive Backup
                      RubenVanDenBoogaard

                      use 'store container data externally' to store the images separated from the database.

                       

                      I have a solution of about 10 GB mostly images, which backups in seconds where the database is about 250 mb.

                       

                      I have not noticed a difference in performance, but handling 10 GB files are a pain, so I would not be supriced

                      if the performance was even better.

                       

                      Hope that helps,

                       

                      Best regards,

                       

                      Ruben van den Boogaard

                      Infomatics Software

                      ruben@infomatics.nl

                      • 8. Re: FM Server ADV 12V4 - Progressive Backup
                        wimdecorte

                        msbertrand wrote:

                         

                         

                        I also understand the 'hourly' backup are also progressive. 

                         

                        Not progressive (or incremental); hourly backups use hard-linking and don't use new hard disk space for files that have not changed since the last backup.  Your backup set will still contain the full set of files though.

                        In your case you are probably not benefiting from this as you only have 2 files and they all change inbetween backups.

                         

                        By switching to externally stored containers FMS will only need to use new disk space for new records so it will be much faster.

                         

                        As to the failure of the progressive backup feature: make sure that nothing is running that could lock the folders where the progressive backups are kept: no indexing, no AV, no user logged in with those folders open in Windows Explorer.

                        • 9. Re: FM Server ADV 12V4 - Progressive Backup
                          msbertrand

                          Hi Ruben,

                           

                          I have opted for this option and currently transferring my data.  I opt also to 'not' encrypt my storage as they are stored on the server hosting the database, and so they can be retrieved by the server administrator if needs to.

                           

                          I have 3 files storing containers; One 'main' file where 'all' images are collected, one 'iOS transfer' files that keeps data for a set period of time, than auto-delete and one IWP (used for both IWP and PHP) that keep container data for 90 days.

                           

                          Regarding IWP/PHP file, if I do the same and store my containers externally will they still be viewable via web, and for both IWP and PHP? If so I will do the same for my IWP/PHP file.

                           

                          My IWP file, which is 'one' file accessed via web from both IWP and PHP displays 'signature' by putting the container field directly on the web layout, and scanned document in IWP by 'open new window' and 'go to layout' and in PHP by 'downloading' the file interred in the container field.

                           

                          Tks for the suggestion, appreciated.

                           

                          Regards, MSB

                          • 10. Re: FM Server ADV 12V4 - Progressive Backup
                            RubenVanDenBoogaard

                            Hi MSB,

                             

                            I opted for the encrypted option due to troubles found by other users on the list (they might be solved by now)

                             

                            I would store all the (big) images externally, they are perfectly viewable in IWP (I have not tried PHP, but since FM is

                            handling the container there should be no problem)

                             

                            I do not completely understand what you are doing with the images in the IWP file, if you are combining the images by putting

                            2 images on top of each other, there may be a better way by using a plugin to merge the 2 images in 1, something like

                            the CNS Image plugin, or the free 360Works Scriptmaster.

                             

                            Hope this helps,

                             

                            Best regards,

                             

                            Ruben van den Boogaard

                            Infomatics Software

                            ruben@infomatics.nl

                            • 11. Re: FM Server ADV 12V4 - Progressive Backup
                              msbertrand

                              So it is done, and we moved the database to a Mac Server.  I have 3 files containing images that will be managed this way, and will see how it progress.

                               

                              Tks for the help, appreciated!