Thank you for your post.
I have forwarded your post and sample file to our Development and Testing departments for review. When I receive any feedback, I will let you know.
According to Development and Testing, this is not considered an issue. Some users want MD5 of the compressed file. Suggestions include performing the MD5 on the uncompressed file and then throwing away the storage of the uncompressed data. Or, have the calculation work on another Container where the file is saved as a reference, which would also give you the desired results without enlarging the file size much.
I would recommend entering this suggestion into the Feature Requests web form at:
The entries into this web form populate a database file that is hosted and monitored by Product Management and Development. All entries are read, discussed and considered for possible implementation in a future release. Although I could easily copy your post and paste it into the web form, there are a few contact questions asked on the form that only you can answer.
I disagree for several reasons :
1. If it's a feature it should have been documented, which is not, at the very least documentation should be modified
2. If it's a feature then we need to have a reliable way to tell wether or not if it's compressed. Filesize differences alone is not reliable (cause sometimes compressed can be > than uncompressed).
3. MD5 is not the main problem, Base64encode is. With compressed containers, the base64 output is a compressed archive in an unknown filemaker format that you can't do much with it. So if you plan to use Base64encode you can say goodbye to compressed containers, and as dev can change mind as needs evolved, you can end up with thousands of file stored compressed for years, that you won't be able to use later with base64decode.
So, If some user rely on something undocumented (the curent behavior), for some reasons, at the very least we should have container functions that applies on the compressed data or the non compressed one at developper will. But, modifying the functions and have the dev to think about it, and do some testing in his code to determine ≠ behavior according to the compressed state or not, seems a lot of work both for you and for us, for a feature (the compression) which should be completely transparent.
Since the export container function, always output the uncompressed file (which is good), at leaf all output function (like base64 decode) should work by default upon the uncompressed file.
Suggestions include performing the MD5 on the uncompressed file and then throwing away the storage of the uncompressed data. Or, have the calculation work on another Container where the file is saved as a reference, which would also give you the desired results without enlarging the file size much.
Both suggestions doesn't work container that are already in filemaker or requires a complex export / reimport process which is slow and not live, and doesn't work in server side scripts
As documented, Base64Encode is to return the contents of the specified Container field as text in Base64 format, and Base64Decode returns Container content from text encoded in Base64 format. As it stands now, FileMaker Pro will not uncompress a compressed file for these functions. That is why I recommended entering this as a Feature Request.
The manager of Product Documentation has been notified about these functions giving different results for files inserted into Container fields with and without the Compress option.
Please also enter a suggestion into the Feature Requests web form to display if a file is compressed in a Container field.
As documented, Base64Encode is to return the contents of the specified Container field as text in Base64 format, and Base64Decode returns Container content from text encoded in Base64 format.
See, that's where the crux of the problem is, for users (end solution user AND us, FMP dev), the contents of a container is what we've put inside it, which the file in its unarchived way. So the container contents is a file, not an archive of it.
I understand that for FMI engineers it's more straightfoward the way it is, but this it's not the useful way, and it's not at all the obvious way from users standpoint.
The compressed file is (a cool, and much needed) mean of storage, it's not the contents, the uncompressed file is. Be sure to clear up that contents definition in updated doc (if it should sadly stay the way it is).
Moreover, beside that logic argument, the way it is as serious usage issues and lots of unnecessary work for devs :
You can't make sure you don't have duplicates of files because you can have them in both compressed and uncompressed form an their checksum will be different. So, juts like that, one usage of the MD5, de-duplication is gone.
That means that to use MD5 for deduplication, you must obsoletely forbid users to choose compression or not, so everywhere you've a container you have to script it's insertion. That's a lot of dev work that could be avoided.
Worse, if you have a lots of container that already exists, and have been inserted in some uncontrolled ways, since 12's release, you won't be able to know for sure, even in your updated solution that you don't have duplicates.
Finally, even if we would have a way, or we get a way, to know, reliably, if the container was compressed or not, it will be no use for MD5 in reduplication, because you won't be able to get the uncompressed MD5 from a compressed container to compare.
If you insert a jpoeg file and choose compression, then it will be compressed (but be more heavy), and Base64decode will translate in a unvievwable file, do that you will not be able to embed it. So base64 embedding is now dead.
Furthermore, even if we get to know if that file is a compressed container, even though that's useful to avoid to display gibberish, then we're basically toasted, because we can't do anything about it to get the real base 64 which "depicts" the real file.
So that current implementation, even though makes sense from an engineer standpoint, destroys 2 usefull usages (usage that may be part of the reasoning of those 2 functions existence). That low level thinking would be ok in C++ environment, but we're in Filemaker, a RAD, made for simplicity.
Possibles ways to fix this
- Container MD5, both compressed and uncompressed should be both computed and stored in filemaker when inserting or updating the container. Because it makes no sense from a performance standpoint to re-compute them on the fly. So for files, the GetContainerAttributeMD5 should be read and not computed (maybe that's already the case today, I don't know).
Of course, getting the MD5 on non files containers, like fields, on the fly computing should stay. I use it heavily
- a new attribute should be added to the GetContainerAttribute function : MD5_Compressed. It will return the compressed MD5 that have stored when the file was inserted, updated.
- a inserted_method attribute that would tell if file was inserted with compression on or off (faster than comparing the 2 MD5/MD5_compressed)
- To avoid previous code breaking, Base64EncodeUncompresed function should be created, it will uncompress on the file and create the Base64Encode from the uncompressed file. That function should work the same on compressed or uncompressed containers, so it will save the dev to have to test for compression or not, because otherwise every dev in the world will have to do this tests and that will be moe code and less efficient. Maybe it should be called Base64EncodeContents.
Other container needed features
Exporting and inserting containers need to be server side compatible. Containers are just files, and there's no reasons server side scripting couldn't do it. And that would be very useful (for instance mails with attachments)
Thank you for the additional comments.
Although I have attached your comments to the original issue, be sure to also these suggestions into the Feature Requests Web Form, as it will be visible by Product Management and key people in Development.
has it been fixed ?