11 Replies Latest reply on May 12, 2012 3:07 AM by Vaughan

    Deleting Container Data

    slemson

      I have two questions about FMPro 12's new container functionality.

       

      1) I noticed that when I delete a record, the external data stored for that record in the container field is not deleted. Is this changeable? It seems like it will leave quite a bit of data on my external storage device that is orphaned. How are others handling this?

       

      2) I also noted that even though I go remove the file manually, the next time I drag it in to that record, it increments a number after it - even though it's the only one in there. For example, I drag a file called: sample_data.xml into it. It shows up externally. I delete the record, then go to the storage and remove the file (would prefer not to have to do this). If I drag that same file into a new record, an external file called "sample_data_2.xml" shows up. Is there a way to control this?

       

      Thanks,

       

      Sean

        • 1. Re: Deleting Container Data
          pennyruddell

          After seeing your post I thought I'd do a bit of testing...

           

          If you have inserted exactly the same image with the same name into the same container field on more than one record it appears to reside only once in the container folder; this surprised me. 

          In this case I tested using a FileMaker Template file with the container field using Open Storage.

           

          This one instance of the image appears to only be removed when the container field from the last record containing the image is cleared; perhaps this is what is happening to you.

           

          To resolve this issue I would recommend using the container field options to automatically generate a unique subfolder for each record using the record primary key and image name.

           

          When I inserted a different image with the same name as above, into a different record, FileMaker recognised this and added the new graphic file, appending _1 to the filename.

           

          There are a number of great resources available on the web regarding the new FM 12 Container feature, below are a couple I've enjoyed.

           

          I hope this helps.

           

          Penny Ruddell

           

          http://www.dbservices.com/articles/filemaker-12-managing-container-data

          http://fmforums.com/forum/blog/13/entry-313-filemaker%C2%AE-server-12-bold-new-steps/

          • 2. Re: Deleting Container Data
            jrenfrew

            AFAIK

            Its because of something called hard links which Penny alludes to

            A way of referencing a file more than once like an alias but having the filing system think it is a 'real' file

             

            The idea of FileMaker managing the storage is that IT manages the storage not you. All the best practice so far seems to point to NOT MESSING with the files that you put into containers with managed storage. Just because you can see it in your folder tree doesnt mean you should manually play with it. Either let FileMaker do it all or you do it all. Here be dragons.

            • 3. Re: Deleting Container Data
              Stephen Huston

              jrenfrew is correct about not messing around in external managed storage areas. Getting at those files and modifying them via the OS is almost as bad opening up the code of an FMP12 file in a text application and trying to redefine a field by editing the text. Those storage folders and files are actual pieces of your database, not disconnected files.

              • 4. Re: Deleting Container Data
                slemson

                I appreciate the responses.  You all seem to be focusing on my second question, but the first question is the one I'm mostly curious about.  These are movies I'm storing.  They could be large.  I can't have them stay on the storage device if the database record that "contains" them is deleted.  I'll tell you what I'm trying to do (and it's pretty clear it can't be done but I'm open for suggestions).

                 

                I want to make a media library catalog.  So I have a field called "rating" which can be one to five stars.  What I did was create a dynamic open container data base location that looks like this:  "/Volumes/somedrive/collections/" & rating & " Star/".  If I set the rating to 4, then drag a movie into the container, it correctly places it in the "4 Star" folder.  When I delete that record, the movie stays in the 4 Star folder.  I wrote an applescript that moves the file to the trash when the container field is cleared.  That seemed to do the job.  But if I drag that file back into a new 4 star record, it now names it with a "_2" appended on it in storage even though it's the only copy of it in that location.  So Filemaker is still holding on to that file existing somewhere even though I deleted the record that contained it.  It's kind of spooky.  I understand hard links, but deleting the record, should delete any link I would think.  I guess I was hoping that FileMaker would do the iTunes equivelant of "Keep media library organized" but it doesn't.

                 

                As it turns out, there's an even bigger issue which is probably unsurmountable which is when I change the rating, I want the file to move to the new rating's folder.  FileMaker won't move the file even though, technically, the repository location should recalculate to the new location because the rating changed.  It doesn't seem to make sense that I can make the location dynamic if Filemaker won't move the file when the calc changes.  I thought I read in Filemaker documentation that I'd be responsible for moving the data so I wrote an applescript which does this and it works beautifully on the filesystem when I change the rating, but as soon as the file moves, Filemaker complains that it's broken even though the calculated location and the actual location match so there's something internal here that I'm not getting access to.

                 

                Would superContainer be a better solution for this?  I've never used it.

                • 5. Re: Deleting Container Data
                  Stephen Huston

                  Are you closing the database after deleting the record to allow the file to perform its own file reduction processes, or are you checking the storage while the data file is still open?

                   

                  FM12 doesn't do what your expectations suggest, so you will probably need a plug-in to accomplish some of this. I don't know if any plug-in will do all of what you want.

                   

                  External storage is created as the data is entered. The fact that FM gives you the option of knowing where that happens to be on your disk if you use Open storage, does not mean the location will change if you change a data field which determined where it was originally placed in the storage system. (But, since it's part of the file itself, it really shouldn't matter.) FMI's lead container engineer told us that they originally planned to offer only Secure storage so people would not be able to mess with the file pieces in  storage, but then changed their minds, allowing users into the guts of the storage system. However, it's still a look-but-don't-touch area. You may copy a file from the open storage structure as a source, if it's needed elsewhere outside FM, but it is not safe to edit, delete, or move anything inside the storage structure to modfiy it in any way.

                  • 6. Re: Deleting Container Data
                    BowdenData

                    Hi,

                     

                    Just reading your post a bit late in the conversation. My initial reaction though was that you have your container storage linked in an odd way. I am thinking of the container data as just another attribute like the star rating is.

                     

                    The rating and container are attributes of the main media record. If you go with open storage of the remote container, it should be linked to the media record, not to the star rating. Your path would be something like "/Volumes/somedrive/collections/" & mediaRecordPrimaryID & "/". In this scenario, your media file would never move, which to me, is how it should be. If you change the star rating or name, or anything else about the media, nothing about the container is effected.

                     

                    You mentioned SuperContainer. This is how it would basically work as well. It creates a separate folder for each resource and that folder is typically setup to be equal to a primary key field value.

                     

                    I can't comment on the deletion issue though. I haven't played around with remote containers much yet, but I would sure think that if the container is linked directly to the media record, it would be removed if you remove that record.

                     

                    HTH.

                     

                    Doug de Stwolinska

                    • 7. Re: Deleting Container Data
                      slemson

                      I have closed the file but I didn't wait for more than a minute.  I'm not sure if that process takes a while. I'll test again later today.

                       

                      Thanks for the response!

                      • 8. Re: Deleting Container Data
                        slemson

                        Doug,

                         

                        I've thought about that.  I chose star rating as the organizational method for two reasons:

                         

                        1. It's how I browse the collection on my hard drive now.  I suppose this is just me not thinking of what it will be like to browse it with the database.  Perhaps when that's the case, I won't care any longer.

                        2. There are literally thousands of files.  I was trying to keep the hard drive organized by some attribute.  But perhaps I can find one that's not so connected to my opinion.

                         

                        Thanks for the response!

                        • 9. Re: Deleting Container Data
                          ch0c0halic

                          Sean may be stacking the files too high,

                           

                          Be granular with the folder naming calculation. If you put too many files into one folder it will overload the OS. Nothing FMS can do about it. Since you say there a thousands of files your Star Rating may not be granular enough to keep the number of files in a single folder below the OS limits. I'd use a different method since as you said the rating can change but, the storage doesn't. You should think of the storage location as 'Permanent'. While it is possible to change a files storage location it is not advisable, see Delete file comment below.

                           

                          Note of caution. Never change a file in the FMS external storages. Don't delete it or move it or 'touch' it with another app. Any change to the file modifies the File's "Check Sum" and FMS will no longer recognize the file.

                           

                          'Delete file comment'

                          You are wrong that a file is not deleted by FMS. You just don't understand why its still in place. FMS will delete the file(s) when there are no references to it. AND the Hard Link established from a backup is still there. Each Backup of your file which is referencing the Container contents will keep the file in place until all backups with a hard link to the file are deleted. Another reason to let FMS handle the file maintenance.

                          • 10. Re: Deleting Container Data
                            slemson

                            Well to be clear, this project is in design mode (early stages).  I don't have thousands of records (yet).  My system currently has a single record that I'm playing with but your advice is noted.

                             

                            As far as references go, this is not hosted (probably never will be) and there are no backups of the file to cause what you're describing.  I did some further testing and now the file is deleting when I delete the record.  Must have been something odd from last night.  That was my main concern.  Thanks everyone!

                            • 11. Re: Deleting Container Data
                              Vaughan

                              ch0c0halic wrote:

                               

                              FMS will delete the file(s) when there are no references to it. AND the Hard Link established from a backup is still there. Each Backup of your file which is referencing the Container contents will keep the file in place until all backups with a hard link to the file are deleted. Another reason to let FMS handle the file maintenance.

                               

                               

                              Could you explain that a bit for me.

                               

                              You are saying that the file(s) will NOT be deleted if there is a reference to it(s) from somewhere in the database. (That sounds fair.) AND the file will not be deleted if there is a backup for the file?

                               

                              This confuses me becase I expected that the backup process would make a *copy* of the file in the Backups folder. Subsequent backups will be hard-linked to this copy if the file is unchanged between backups.

                               

                              Thanks for resolving my confusion.