1 2 Previous Next 24 Replies Latest reply on Apr 5, 2012 9:05 AM by gmainis

    File size going Nuclear - Any clues?

    gmainis

      Hi All

       

      I have a client using a virtual system and 1 file has increased in size from 196Mb to 4.3Gb in a matter of hours, however the number of records has basically stayed the same.

       

      This Setup:

      Windows Server 2003

      Terminal Server

      Over a Virtual Network

      FileMaker Server 10

      FileMaker Pro 11.3

       

      Current Findings:

      When I recover the file, there is no size difference. When I Save As compressed there is only a 700mb reduction is side. If I export the fields to excel and reimport them I get a normal file size - not really what I want to do on critical data.

       

      The users where getting a disconnect error and the server log showed event ID 30. The FileMaker message from the terminal server is below:

       

      FMP.png

       

      HAS ANYONE SEEN ANYTHING LIKE THIS?

       

      Thanks

      Greg

      Simplified Software

      greg@simplifiedsoftware.net

        • 1. Re: File size going Nuclear - Any clues?
          psijmons

          Do you allow users to upload files in container fields?

          ... in a loop script?

           

          Do a Recovery on a backup of that file and run FMDiff (google that) to compare it with an older backup. This should give you a clou of the difference, otherwise your problem may be elsewhere on the OS.

          • 2. Re: File size going Nuclear - Any clues?
            gmainis

            Thanks for the info.

             

            There is a container field and I checked it, there is no info stored.  I have recovered, compressed and both do not decrease the size in any significant way.  The only thing that seems to work is exporting all the records to excel and importing back in.  This is critical data and I would do it only as a last resort.

             

            Greg

             

             

             

            Simplified Software

            simple solutions to complex problems

            • 3. Re: File size going Nuclear - Any clues?
              dhrowe

              Somebody doing a bunch of indexes on fields that have the "create index if necessary?  ( eg a bunch of finds on fields not expected to be searched or indexed on? )

               

              Other than Image fields, massive indexing is the only thing I know that can ballon a file like that.

               

              dhr

              • 4. Re: File size going Nuclear - Any clues?
                jameshea

                Hi,

                 

                Are there any audit fields or calculations that are tracking changes to the data. Sometimes these will expand quite quickly, especially if  you did an import without turning off the 'auto-enter' option.

                 

                Let us know what you find.

                 

                James

                • 5. Re: File size going Nuclear - Any clues?
                  gmainis

                  Thanks for the reply.

                   

                  I wouldn't be so upset if they were not getting disconnects all the time.  There is a fair amount of indexes however that was there prior to the file size increase.

                   

                  Greg

                   

                   

                  Simplified Software

                  simple solutions to complex problems

                  • 6. Re: File size going Nuclear - Any clues?
                    dhrowe

                    Might try 'pre' indexing any field that they can search on....  Then that is done, and you can see what the size is then.

                    • 7. Re: File size going Nuclear - Any clues?
                      gmainis

                      Tried removing un-needed indexing.

                       

                       

                       

                      Simplified Software

                      simple solutions to complex problems

                      • 8. Re: File size going Nuclear - Any clues?
                        dhrowe

                        But if you remove it, and don't want it to come back make sure you set the fields to not allow indexing.  The load of indexing good sized tables, especially if it is more than one person in different fields is a real 'chunker'.

                         

                        dhr

                        • 9. Re: File size going Nuclear - Any clues?
                          gmainis

                          Yes, there is a certain amount of indexing that is required and I did go though and removed as many as possible.  After saving the file again, I did not see any appreciable difference in the file size.  This definitely an "out of the norm" problem.

                           

                          Greg

                           

                           

                           

                          Simplified Software

                          simple solutions to complex problems

                          • 10. Re: File size going Nuclear - Any clues?
                            mardikennedy

                            Very recently Jeff Hough reported on Real Tech, his experiences with a bloated file:

                             

                            It appears to be calcs. Specifically 3 used to make XML (built before my time, now on my list to replace). The first calc returns more than 20k, the second more than 20k and the last 10bytes.

                            Now, multiply that by 231,837 records. I think that might be the issue.

                            I removed the 3 calcs from a copy and compressed the file. The new file is: 445.6mb (w/o indexes)

                            Is there any chance that a field calc was redefined/ added?

                             

                            Mardi Kennedy

                            • 11. Re: File size going Nuclear - Any clues?
                              mardikennedy

                              Very recently Jeff Hough reported on Real Tech, his experiences with a bloated file (*> 7 GB):

                               

                               

                              It appears to be calcs. Specifically 3 used to make XML (built before my time, now on my list to replace). The first calc returns more than 20k, the second more than 20k and the last 10bytes.

                              Now, multiply that by 231,837 records. I think that might be the issue.

                              I removed the 3 calcs from a copy and compressed the file. The new file is: 445.6mb (w/o indexes)

                               

                              Is there any chance that a field calc was redefined/ added?

                               

                              Mardi Kennedy

                              • 12. Re: File size going Nuclear - Any clues?
                                jameshea

                                If these were unstored calcs that triggered calcs into stored fields, then the indices would have grown, ballooning out the file.

                                • 13. Re: File size going Nuclear - Any clues?
                                  gmainis

                                  This file has been in production for a long time and this issue is only happening to one client.  I removed 100,000 records, re compressed and rebuilt the file.  Then I exported the records into a new file, the next morning when the clients started entering data the file size went back to 4.2 GB.  Currently the file is with FileMaker level 2 support.

                                   

                                   

                                  Simplified Software

                                  simple solutions to complex problems

                                  • 14. Re: File size going Nuclear - Any clues?
                                    AlanStirling

                                    Hi Greg

                                     

                                    Perhaps one of your clients has started using QuickFind?

                                     

                                    If one or more of their users has decided to use this feature and they run it on a busy layout with many fields showing, this could cause your file to bloat each time they run it (with a delay whilst FileMaker indexes almost all the fields in the file!).

                                     

                                    You can control which fields are included in a QuickFind search on each layout, but I would first ask them if they have been using it.

                                     

                                    I must say that I cannot think of any other good reason for a file to suddenly balloon in size.

                                     

                                    Hope this helps - Alan Stirling, London UK.

                                    1 2 Previous Next