10 Replies Latest reply on Nov 27, 2011 2:31 AM by jimpat

    Container fields taking mega space

    jimpat

      We have several thousand pdf's stored on our server with only a link to each file stored in a container field for the relevant record. With each new link, FM is gobbling up server storage. Currently, without links, the entire FM database is around 700mb. With only file links and no actual imagesor thumbnails stored in container fields, the database is over 20gb & growing daily as we add pdf's to our server.

      How can I store file links without taking up FM database space?

      Following is the right click view of the container field.

        • 1. Re: Container fields taking mega space
          psijmons

          Jimpat,

          have a look at the SuperContainer from 360works.com

           

          Files are stored outside the database in a folder structure that runs parallel with your tables and record ID's

          You can script the process to move all your legacy PDF's to the proper folders.

          It takes some effort to have it all installed but it is well worth the trouble (and the license).

          The example file will give you a good impression of what it can do and how to integrate it.

          • 2. Re: Container fields taking mega space
            rothdavid

            My first comment would be that the database is storing the actual PDF's in the container field.  If it is storing the path only, the database should not grow in size to that degree.  Embedding the PDF itself could certainly explain the large file size.

            • 3. Re: Container fields taking mega space
              RayCologon

              jimpat wrote:

              We have several thousand pdf's stored on our server with only a link to each file stored in a container field for the relevant record.

               

              Hi jimpat,

               

              How do you know that the pdf files are stored only as a link? As rothdavid says, the file size issue seems to indicate otherwise, and if users are placing the pdfs into the container field directly, whether scripted or not, you're nevertheless leaving it to the user's discretion as to whether the "Store only a reference to the file" is checked when they dismiss the SF dialog.

               

              Regardless, I'd suggest you make a slight change so that the path is stored in a text field rather than a container field, then change the container to an unstored calculation that references the text field where the path is stored. Then you will be certain that only paths (as text) are stored.

               

              To work that way, you'll need to provide a scripted upload process that prompts users to insert a pdf (with the insert step targeting a temporary global container field), then your script should grab the path of the file the user inserts (by enclosing the global container field within the GetAsText( ) function and parsing out the path), placing it into the text field. In the event that no valid path is returned - probably because the user unchecked the "Store only a reference to the file" option - your script can then return an error and prompt the user to try again.

               

              If you make this change, you will prevent possible future problems from occurring and you'll likely solve your current file size issue as well.

               

              Regards,

              Ray

              ------------------------------------------------

              R J Cologon, Ph.D.

              FileMaker Certified Developer

              Author, FileMaker Pro 10 Bible

              NightWing Enterprises, Melbourne, Australia

              http://www.nightwingenterprises.com

              ------------------------------------------------

              • 4. Re: Container fields taking mega space
                Stephen Huston

                We've had a similar problem which began with our upgrading to FM 11 last year.

                 

                We had been storing small thumbnails of huge PDFs, and file size was growing slowly and as expected. Then we upgraded to FM 11 and  file size on that one file began to grow multiples!

                 

                It now stores the PDF when we import, no matter what we set it to store, and file size grows by GIGs instead of by K.

                 

                We have to keep FM 10 running on one client machine in order to handle the record importing for this script or we get the entire PDF in the container rather than just capturing file info.

                 

                This bug was reported to FMI months ago, but no response from them was received even acknowledging the problem, so maybe it's a Version 11 "Feature" after all.

                1 of 1 people found this helpful
                • 5. Re: Container fields taking mega space
                  datastride

                  SuperContainer!

                   

                  Best investment in FileMaker-related software I’ve ever made. Wouldn’t even think of starting a FileMaker project without it. SO many uses. So much time savings. Pays for itself in short order. All these kinds of problems (and many more) disappear and a whole new world of additional capabilities is available to the developer …

                   

                  Peace, love & brown rice,

                  Morgan Jones

                   

                  FileMaker + Web:  Design, Develop & Deploy

                  Certifications: FileMaker 9, 10 & 11

                  One Part Harmony 

                  Austin, Texas • USA

                  512-422-0611

                  1 of 1 people found this helpful
                  • 6. Re: Container fields taking mega space
                    jimpat

                    Thank you all for your help.

                    Since all of our container fields require our users to add files via a script that only stores a link to the file with no thumbnails, it sounds like this is probably related to our upgrade to FM11. We are going to follow the advice from Morgan and psijmons and add SuperContainer from 360Works. I think this will be the simplest solution to our exploding FM database file size.

                    • 7. Re: Container fields taking mega space
                      datastride

                      I’m pretty sure you’ll never regret you’re investment in SuperContainer.

                       

                      One thing I’ve found very useful is that I can always access my stored images, PDFs, spreadsheets, Word documents, whatever, via a URL. So I can show them in a web viewer (where HTML emails display just great), and I can email links to them to by staff and customers. And I can add a couple of parameters to the URL to automatically create a thumbnail or a fixed-size image. And I’ve now implemented solutions in both IWP and CWP that use (and rely) on SuperContainer. I just have so much more flexibility than ever before. And because my apps only have to download little thumbnail files (created on the fly by SuperContainer), screens display much faster, especially for remote users. Then with the click of a button I can display an enlarged image (just a slight tweak to the parameters) in a web viewer on demand. And my database backups now take a fraction of the time required when all these files and images were stored in the db.

                       

                      So happy SuperContaining … And write if you need any help.

                       

                      Oh, one important detail ... I find that I get very quick and very good responses when I contact 360Works technical support with questions or problems. So keep them in mind as one of your best resources, especially when you are installing and getting something working for the first time. The entire crew at 360Works are a pleasure to work with.

                       

                      Peace, love & brown rice,

                      Morgan Jones

                       

                      FileMaker + Web:  Design, Develop & Deploy

                      Certifications: FileMaker 9, 10 & 11

                      One Part Harmony 

                      Austin, Texas • USA

                      512-422-0611

                      • 8. Re: Container fields taking mega space
                        jimpat

                        Excellent comments Morgan. It sounds like a great addition. We’ll see very quickly.

                         

                        Are you an independent contractor or part of a larger group?

                         

                        Jim

                        • 9. Re: Container fields taking mega space
                          datastride

                          Jim,

                           

                          I'm an independent contractor … but I'm happy to share some of the things I've learned with other developers like yourself without charge, as I have a lot of "paying it forward" to do as a result of all the help I've been given over the years.  

                           

                          Peace, love & brown rice,

                          Morgan Jones

                           

                          FileMaker + Web:  Design, Develop & Deploy

                          Certifications: FileMaker 9, 10 & 11

                          One Part Harmony 

                          Austin, Texas • USA

                          512-422-0611

                          • 10. Re: Container fields taking mega space
                            jimpat

                            Thanks Morgan. Have a nice weekend.

                             

                             

                             

                            jim