Stephen Huston

FM12 External Container Storage: Secure vs Open

Discussion created by Stephen Huston on Apr 13, 2012
Latest reply on Aug 30, 2012 by Stephen Huston

At FMI Headquarters last night, Oleg Zaydman, lead engineer on new FM12 Container Storeage made a presentation which hit on some issues we had encountered when transfering container data to external storage in converted files.

 

The original intent in FM12 was to offer only Secure storage, where FM12 manages the placement and naming of all storage directories in a heirarchy.

 

When the decision to also offer Open storage was reached, it was not well documented how to deal with users who might have many thousands of container records, as this requires a well-structured heirarchy of directories. The default filepath offered in the FM12 transfer dialog WILL NOT WORK for transfering large numbers of container files/images. It places them all on one directory.

 

In our case, that meant over 60,000 container-content files being written into one OS-level directory. The OS itself fights this kind of overload of a single directory, and the process of transfer comes to a near standstill after first few hundred records.

 

To use Open Storage, one must devise a calculated filepath and enter it in placed of the default for creating a sub-directory heirarchy that won't overload any single directory. The example shown last night required using values from multiple record fields to group and name the directory/sub-directory levels to avoid a situation where thousands of transfered files get dumped into a common directory. It was not clear that even this method was fully scalable to tens of thousands of records.

 

For server-based storage, the only option that is simple, error-free, and scalable to large record counts appears to be Secure Storage.

 

Stephen Huston

Outcomes