1 2 Previous Next 15 Replies Latest reply on Mar 20, 2016 4:44 PM by rickenbacker360

    Questions: Nuances of Managed Storage Container Fields [Long]

    rickenbacker360

      First, some background: I intend to use managed storage using the External (Open) method at the default [Hosted Location] on FM server. I have written code requiring user be a guest of FM Server to Insert File, Open, Email, and Delete the Document Management record containing the inserted file. The Open and Email functions use Get(Temporary Path) and there is liberal messaging telling users that they are opening a COPY of the file and that if they make changes they should save a copy of the file to a PERMANENT location and from there, re-insert the newly-saved file into the container field so the new file is again on the FM Server. I am trying to make the solution as bullet-proof as practical, but am also considering the 90/10 rule.

       

      Questions:

      1. When the solution file is hosted using FM Server, I notice the “Save A Copy As…” command is grayed out. This is good. However, If FMS is brought down (or the files are closed) and the user opens them using FM Pro, he/she will be able to use the “self-contained copy (single file)” option, thus altering the container field’s option to no longer use managed storage. I understand this method of saving literally deselects the “Store container data externally” parameter on the field definition’s “Storage” tab. This overall process embeds the externally (FMS) stored data in the self-contained copy. Assuming the user properly renames the file (as part of a multi-file solution) and replaces the originating file with this self-contained copy in the served directory back on FMS:
        1. Does this self-contained copy process remove the formerly externally stored files from the “RC_Data_FMS” folder hierarchy at the moment the file is saved? Or are the files still in place on FM Server but no longer "attached" to the solution file?
        2. Is there a script step which can turn on (reselect) the “Store container data externally” parameter? And, of course, the “Open Storage > [Hosted Location]” parameter to make the self contained copy return to its intended behavior? (Remember, the file does not have full access.) As you know, we do have a related script, “Set Next Serial Value,” allowing us to reset serial numbers inside fields when we do not have full access to field definitions.
        3. Failing that, it would seem that FMI breaks my solution. (I will forgo any editorial on whether that was a good idea.) Is my fallback method to use Custom Menus somehow to disable the “Save A Copy As…” command entirely? (I do already have a script to save a compressed copy for rebuilding indexes and making the file smaller.)
        4. Failing that, is my only option to force the user to import all data into an intact (unbroken) copy and redeploy same on FMS?
        5. Is there a “get” function to know if a file is now a self-contained copy? At least I could alert the user that the document management feature no longer works correctly, lock them out of the feature, and provide instructions what to do. At this point I only see huge file size increases and failure.
      2. If User 1 attaches a file using the Insert File from their Computer 1 and User 2 on Computer 2 inserts a different file to a different document record, FMS now has the files in the “RC_Data_FMS” folder hierarchy. This is good; all guest machines have access to the files without regard to their originating source location(s) on Computers 1 and 2. In my solution, the same file can be attached to a found set of parents by duplicating the desired child record and populating the foreign key with the parent’s primary key. “FileName A” can thus be in 50 document records with only a single iteration present in the “RC_Data_FMS” folder. Let’s say that parent (with serial number 23) attaches the first iteration in a child doc mgmt. record. If my directory structure on FMS includes the sub directory “/DOCS/DocContainer/23”, Filename A resides in it. When the user duplicates 50 iterations, the single Filename A remains, but hard links are made to it (inside the 23 sub directory). Now, in a beautiful design, FMI allows the user to delete the child document record for parent 23 and not delete the contents of folder 23 (because other child records have the same FileName A attached). In fact, the user can delete 49 of 50 child records and FileName A remains in sub directory 23. When the user deletes the last document record having the iteration of “FileName A,” the sub directory 23 and Filename A are deleted on FMS.
        1. Is there a way to store the originating directory from when FileName A from Computer 1 was attached by user 1 and FileName B from computer 2 was attached from Computer 2? Keep in mind that two different computers might have the same directory structure (Macintosh HD/Users/Name/Documents/Filename A for example).
        2. Is there a way to inform the user that they are about to delete the last iteration of a document record having FileName A attached/inserted? Right now, I provide general instructions to first open FileName A, save a copy to a permanent location for safe keeping; the presumption being that users who attach files will believe that things are all now being taken care of by FM Server and possibly delete FileName A at the originating source (where it resided before being inserted/attached).
        3. Is the serial number sub directory value [ Example: (23) ] of any worth? Originally, I though it would be helpful to help the user locate the file on the FM Server, but I’ve since learned this is extremely bad practice. As stated, I now use Get(TemporaryPath) to open or email a file. And, given my scenario, a user would likely be unable to locate the file anyway for the several reasons outlined.
      3. My solution has two deployment versions. The FMS hosted “mothership” version and the “light” version used by external field personnel to gather information such as courthouse document scans, etc. The mothership version is what I’m focused on here. However, I have a method of importing from the light version like this: The light version has dedicated “XP” files with complex structure (layouts, scripts, etc.) which Import from the corresponding live data files. These XP files are sent to the mothership site where FM Server closes the normally hosted files. While files are closed the XP files are placed into the live data (normally served) directory, replacing files of same name, and the solution is opened on FM Server using FileMaker Pro (not server) where the contents of the XP files are now imported into the mothership.
        1. Is there a way for the light version users to insert files into a self-contained copy, import into a self-contained set of XP files and then have that set of XP files import into the managed storage mothership files, where the desired outcome is that all container contents are then automatically placed into the RC_Data_FMS directory?

       

      Again, sorry for the length. I tried to be concise but give enough background for thoughtful and informed answers.

       

      Thanks for any help.

        • 1. Re: Questions: Nuances of Managed Storage Container Fields [Long]
          wimdecorte

          rickenbacker360 wrote:

           

            1. Does this self-contained copy process remove the formerly externally stored files from the “RC_Data_FMS” folder hierarchy at the moment the file is saved? Or are the files still in place on FM Server but no longer "attached" to the solution file?

           

           

          The files on the server are left intact.  In fact even the FM file is left on the server.

          Once you download a FM file through the admin console.

           

          To remove the RC data files (and the FM file), you'd have to manually move everything to somewhere else, then do a save-as-self-contained.  But it is not the "save as" that removes the files, it's you removing the files in order to do "save as" in the first place.

          • 2. Re: Questions: Nuances of Managed Storage Container Fields [Long]
            wimdecorte

            rickenbacker360 wrote:

             

              1. Failing that, it would seem that FMI breaks my solution. (I will forgo any editorial on whether that was a good idea.) Is my fallback method to use Custom Menus somehow to disable the “Save A Copy As…” command entirely? (I do already have a script to save a compressed copy for rebuilding indexes and making the file smaller.)
              2. Failing that, is my only option to force the user to import all data into an intact (unbroken) copy and redeploy same on FMS?

             

            I'm not really sure what you are after here... you don't need to disable "save as" at all.  Host the file and protect access to the server and the admin console: nobody will be able to use it...

            • 3. Re: Questions: Nuances of Managed Storage Container Fields [Long]
              wimdecorte

              rickenbacker360 wrote:

               

               

                1. Is there a “get” function to know if a file is now a self-contained copy? At least I could alert the user that the document management feature no longer works correctly, lock them out of the feature, and provide instructions what to do. At this point I only see huge file size increases and failure.

               

              GetContainerAttribute

               

              Again: not sure what your fear is and why you think you need to jump through these hoops.

               

              Whether file is embedded or stored externally: all your features for how you let users interact with the container will work exactly the same.  For the scripts etc it does not matter how FMS manages where the actual the data is stored.  A container is a container is a container when it comes to functionality.

              • 4. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                wimdecorte

                rickenbacker360 wrote:

                 

                 

                  1. Is there a way to store the originating directory from when FileName A from Computer 1 was attached by user 1 and FileName B from computer 2 was attached from Computer 2? Keep in mind that two different computers might have the same directory structure (Macintosh HD/Users/Name/Documents/Filename A for example).

                 

                You can make the actual path of the RC as variable as you want, by including the account name etc.  But why?  Only FM/FMS will manage that data, what do you expect to get out of including user/machine info in the actual path?

                • 5. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                  wimdecorte

                  rickenbacker360 wrote:

                   

                   

                    1. Is there a way to inform the user that they are about to delete the last iteration of a document record having FileName A attached/inserted? Right now, I provide general instructions to first open FileName A, save a copy to a permanent location for safe keeping; the presumption being that users who attach files will believe that things are all now being taken care of by FM Server and possibly delete FileName A at the originating source (where it resided before being inserted/attached).

                   

                   

                   

                  No.  And it does not matter... you need to catch that earlier in the workflow when you let the user modify the record or that field or not.   The fact that an RC data file may be the last hard-linked copy of that file shouldn't make a difference.  If the user has a valid reason to modify the container data or delete the record then they should be allowed to do so.

                  • 6. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                    wimdecorte

                    rickenbacker360 wrote:

                     

                    1. My solution has two deployment versions. The FMS hosted “mothership” version and the “light” version used by external field personnel to gather information such as courthouse document scans, etc. The mothership version is what I’m focused on here. However, I have a method of importing from the light version like this: The light version has dedicated “XP” files with complex structure (layouts, scripts, etc.) which Import from the corresponding live data files. These XP files are sent to the mothership site where FM Server closes the normally hosted files. While files are closed the XP files are placed into the live data (normally served) directory, replacing files of same name, and the solution is opened on FM Server using FileMaker Pro (not server) where the contents of the XP files are now imported into the mothership.
                      1. Is there a way for the light version users to insert files into a self-contained copy, import into a self-contained set of XP files and then have that set of XP files import into the managed storage mothership files, where the desired outcome is that all container contents are then automatically placed into the RC_Data_FMS directory?

                     

                     

                     

                    I would only use embedded storage for the XP files because you do not want the hassle of moving a FM file and its RC folders around.  The feature is meant to make storage (and backups) efficient, not to make portability efficient.

                     

                    If you have not tried it yet: create a small test file that uses RC open storage.  Host it on FMS, create some records with container data.  Then use the admin console to download the file.  Leave the hosted file closed and open the local copy: check to see if you can actually see the container data.

                    What I'm after: the local path to where the RC data is stored is slightly different than when it is hosted.  You do not want the hassle of moving the RC folder just so that a local copy of your file can see the container data.

                    • 7. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                      wimdecorte

                      rickenbacker360 wrote:

                       

                      When the user duplicates 50 iterations, the single Filename A remains, but hard links are made to it (inside the 23 sub directory).

                       

                      Did you test that?

                       

                      I'm not seeing that: no hard links are added either at the directory level or at the file level when you duplicate a record that contains the container.

                      • 8. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                        wimdecorte

                        rickenbacker360 wrote:

                         

                        Is the serial number sub directory value [ Example: (23) ] of any worth? Originally, I though it would be helpful to help the user locate the file on the FM Server, but I’ve since learned this is extremely bad practice. As stated, I now use Get(TemporaryPath) to open or email a file. And, given my scenario, a user would likely be unable to locate the file anyway for the several reasons outlined.

                         

                        It still sounds like you are too focused on the actual file aspect of the storage.  You really should not care how FM stores the files.  You only interact with them through the FM records and container field.

                         

                        As to the ID: it can be a good idea to include something like the ID in the path, not so that you can find it, but to limit the # of files that FMS will dump into one folder.  That's not a problem when you use secure storage but with open storage you can slow down FMS by having it have more files in one folder than the OS can efficiently handle.

                        • 9. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                          rickenbacker360

                          I may have been inarticulate using the term "hard link" inside directory 23. The rest of my statement has been double-checked.

                           

                          I created a child record for parent number 23. I inserted a file into its container field. I checked on Filemaker server and saw that a new directory 23 had been created and inside that directory was the file.

                           

                          I then duplicated that child record of parent number 23 in a looping script routine attaching each particular different parent ID 49 more iterations. (The parent records had already existed before hand and had very in primary keys, one of which was 100.) The result was that I had A total of 50 child records, each with its container containing the apparent same file. At that moment, directory 23 on FMS contained a single iteration of the file.— and no other directories or files were placed on FileMaker Server.

                           

                          Next, I deleted the originating first-created child record for parent 23. FMS still contained directory 23 and the file within.

                           

                          I then deleted 48 of the remaining 49 child records and the results on FMS we're still the same: directory 23 remained and so did the file.

                           

                          Finally, when I deleted the last remaining child record, having parent ID 100, for example, the directory 23 and its enclosed file were also deleted.

                           

                          Perhaps I should not have used the term "hard link" but the behavior is/was as written.

                           

                           

                           

                           

                          ••• ••• ••• •••

                          Scott Key

                          ••• ••• ••• •••

                          • 10. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                            wimdecorte

                            rickenbacker360 wrote:

                             

                            Perhaps I should not have used the term "hard link" but the behavior is/was as written.

                             

                             

                            Fair enough.

                            The behaviour is exactly as you describe and that works fine.

                             

                            I got thrown by the "hard link" reference since I did not recall hard-linking to be involved and when I retested I did not see it.

                            The hard-link count in the OS does not go up; it's FM/FMS that keeps track of what is linked where.

                            • 11. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                              rickenbacker360

                              I will keep the parent ID as part of the path, thanks to your answer—to help keep directory contents smaller and more manageable by the OS.

                              • 12. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                                rickenbacker360

                                My users are not sophistical power users and I do require high level administrator create and delete privileges for anything in the document management area of the solution.

                                 

                                I'm ready to follow your advice but do wish, for the record, to state that a death certificate for John Smith, for example, might be attached inadvertently to the wrong parent oil lease record. My user might then believe that all is well and delete the original death certificate file from his client machine. This would leave only a single copy on the entire network, that being on FileMaker server.

                                 

                                Eventually the same user, or a different one, might discover that the death certificate for John Smith is not applicable to the given lease parent and delete it, thus hoping to fix the lease management problem.

                                 

                                When that user attempts to find the Death certificate file for attachment to the correct lease parent, they will fail in my scenario. That is all I am (or was) trying to avoid. I think I will throw up my hands and just tell them not to do stupid stuff and provide general warnings to be careful as already in place.

                                • 13. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                                  wimdecorte

                                  I hear what you are saying but it is not something you can solve by tweaking the RC data storage side of things.  There is not a single bit of document mgmt built into it; its entirely a back-end "make storage efficient" function.

                                   

                                  Preventing data from being lost is something that you'd need to handle on the FM record/field level.  For instance you could build in a function to ask the user if they want a local backup of the certificate before deleting it...

                                   

                                  A good backup strategy may also help in protecting against lost data.

                                  • 14. Re: Questions: Nuances of Managed Storage Container Fields [Long]
                                    rickenbacker360

                                    Since my solution uses the portable XP files mentioned elsewhere, users at the FM Server need to bring FMS down (or more accurately, close the files using the FMSAC). Then they replace the XP files with the latest files from the field (light) version. I am looking for a method, while server is down, to have them avoid saving a self contained copy of the mothership (not light) version which is normally hosted by server. Having read wimdecorte's other responses, I'm sure I'm overthinking things. What I could see, since they've opened the mothership solution using FM PRO on the FM server machine, is that they don't screw things up by saving a self-contained solution, and then replacing the live data files with it. Trust me, I love my users but they've done this kind of thing. In fact, they've been trained to occasionally save a copy as (COMPRESSED) to rebuild indexes and make file size smaller. I can see them thinking there's no difference and save the self contained.

                                     

                                    I'm probably over thinking what can go wrong. I'm not sending someone to Mars with my solution, so...

                                     

                                    BTW, if you're wondering why they need to take the server down (close files and open with FMP, it's because extensive use is made of GLOBALS during the import process and those functions fail on a hosted multi-file solution. Special dispensation, please, this method has been in place for 20+ years in this solution.

                                    1 2 Previous Next