I say don't do it. Manage the images internally in their own table and have only thumbnails to send to layouts in a different table. Very fast and reliable. I have a production client with 10-13 large images per record stored and it is a lot of image Storage.
Downside is backup time. Upside images are backed up. SSD storage and progressive backup makes it run just fine.
bigtom, could you elaborate on why you would not store images externally? I will admit my experience, a bad one, was with a Server5.5 and FMP 6 system that held images internally. We had tremendous file bloat problems, painfully slow backups and sometimes retrieval, & the files just got twitchy and needed lots of maintenance.
With the advent of external storage I've utilized the encrypted method to store ten's of thousands of medical images without a hitch, the code file remained small, and everything ran nice and fast. I felt that disk based files would be inherently more secure/safe under the OS rather than in one file that if corrupted could loose the whole lot.
I know FMP can handle very large file sizes now but using external storage just seems to make sense. How am I thinking wrong about this? Thanks!
Is the issue here the "store a reference only" option or a container with external storage ("Store container data externally")?
I personally would steer away from the "Store a reference only" option as much as possible. If it's part of a workflow that the files need to be stored elsewhere, and there is no other way around it then perhaps.
"Store a reference only" has the risk of
- a file system / folder name change breaking the reference, it's not very good data integrity
- requires another backup method outside of FileMaker servers built in backup.
A container with "store container data externally" turned on is fine IMO.
William OKeefe wrote:
bigtom, could you elaborate on why you would not store images externally?
Trouble with externally stored containers This is one type of situation I want to avoid.
With storing tens of thousands of images full backups take under 10 minutes to complete. You need really fast storage for this to be viable. I have never had an issue with embedded containers. There is a plan in place to archive old images when the backups start to reach 10 minutes.
If you have full control of the server in your hands then RC storage is an option. If a client has access on their own server I do not want it unless absolutely needed.
breeanne: "Is there a way to preserve the file structure I have already created? What would be the best way to do this?"
I've pondered over this one for quite a while now. Like the previous posters I am not a fan of using a path as a reference (the FMP method) because it is too easy to break.
Putting files in FMP container fields works, but then the problem is that users lose the standard file folder metaphor, and users like the standard 40-year-old file folder metaphor. A dev can write a FMP-equivalent of the folder structure interface from MacOS Finder or Windows File Manager but that's a lot of work to get back to break-even. And I've never seen a good one.
I've been experimenting with a solution: I put a unique ID into each file. This link ID also goes into the FMP database. Spotlight or File Manager can instantly find the file by this link ID even if the file structure has been reorganized. To retrieve a file, FMP simply copies the link ID to the clipboard and then AppleScript opens Spotlight, inserts the copied string, and searches. It's two lines of code.
This works well but has one glitch. Neither Windows nor OSX creates a UUID when creating new files (so odd, but that's a discussion for another forum). Neither OS even has a good place to store a UUID (that I can find), which is also very odd given that file metadata will store all sorts of arcane stuff. FMP can generate a UUID or a logical ID (e.g., client number + project number + document number). If your workflow can suffer the extra step of putting an ID in each image or document file then this methods works very well. Note that a file subfolder can be tagged with an ID so the FMP link can have Spotlight open a subfolder directly.
Thanks everyone for all your insight and advice! It is much appreciated... I definitely have some thinking to do.
I will probably avoid the store only as a reference, based on comments and the fact that it is on the deprecated list.
BMyers: That's a really interesting solution. I will certainly give it some thought.
I don’t use the FM External Storage functionality because it is problematic in a different way than what has been described here. For example, for local files, under certain conditions (File Options, Log in using Guest Account, and with a custom Privilege set), it will not properly Save A Copy As, Self-Contained Copy (single file), meaning embed files in containers will not occur. I need this before transferring to FM Go.
That being said, I do my own external storage using scripts. Then can manage, using the scripts, to flip back and forth between Insert Reference files to containers or Insert / Embed files to containers as needed.
The process looks like this:
Using a script Create a folder, can be at same location as the DB files folder, or the Document Folder, or … Then Save the path and the folder name to global fields.
This maybe extremely annoying if there are thousands of photos. However logic can used in the scripts to minimize the time for the script to run.
1) Every time the FM file is opened, a Script Trigger, Exports every container with new data in every table to the above created / saved path and folder.
Then follows by Inserts all files as Reference to all containers in all tables; from above created / saved path and folder. Again logic can used in the scripts to minimize the time for the script to run.
2) Then have 2 scripts, 1) to insert into containers as Reference, and 2) to insert into containers as Insert, all from above created / saved path and folder. Use these scripts as needed / desired.
Thanks for the info! Quick question, when enable store container data externally, Filemaker backs up the containers contents as well?